Image processing method, medium, program product, and electronic device

Abstract

The application relates to an image processing method, a medium, a program product and an electronic device. The method comprises: obtaining at least one filling element from at least one material image; generating a first interface according to a background image and the at least one filling element; wherein the first interface comprises the background image and at least one card located above the background image, the at least one card covers a partial area of the background image, and the at least one card is filled with the at least one filling element; wherein a first gravity center is a gravity center of a picture formed by the background image and the at least one card in the first interface; and a distance between the first gravity center and a geometric center of the first interface is less than a first distance threshold. According to the position relationship between the first gravity center of the picture formed by the cards and the background image in the first interface and the geometric center of the first interface, the application selects the filling elements, so that the layout of the picture formed by the cards and the background image is reasonable, and user experience is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminals, and in particular to an image processing method, a medium, a program product and an electronic device. BACKGROUND

[0002] With the development of terminal technology, various intelligent electronic devices, such as mobile phones or notebook computers, play an increasingly important role in people's daily life. For example, users chat, watch videos, shop, play games, work, etc. through electronic devices.

[0003] In some cases, the electronic device will involve the layout of an image, and the rendered display of the laid-out picture. For example, a user will set different themes for an electronic device according to his own preferences. Taking a mobile phone as an example, when a user needs to set a theme for the mobile phone, the user can select a favorite theme according to the content of the theme, the style of the theme, etc. In some cases, the mobile phone desktop displays cards of some applications (a "card" is a kind of service card embedded in an application), and after the user selects a certain theme, the mobile phone will use the material images in the theme package to fill the service cards of the desktop. SUMMARY

[0004] Therefore, the embodiments of the present application provide an image processing method, a medium, a program product and an electronic device. According to the position relationship between the first barycenter of the picture formed by the background image and the cards in the first interface generated by the electronic device and the geometric center of the first interface, the filling elements used to fill the cards in the first interface are selected from the material images. The layout of the picture formed by the cards and the background image filled with the above-mentioned selected filling elements in the first interface generated by the electronic device is more reasonable, and the effect of the picture display is better, thereby helping to improve the user experience.

[0005] In a first aspect, the embodiments of the present application provide an image processing method applied to an electronic device, comprising:

[0006] obtaining at least one filling element from at least one material image;

[0007] generating a first interface according to the background image and the at least one filling element;

[0008] The first interface includes a background image and at least one card located above the background image, the at least one card covers part of the area of the background image, and the at least one card is filled with the at least one filling element; one of the at least one filling element is used to fill one of the at least one card;

[0009] The first center of gravity is a center of gravity of a picture formed by the background image and the at least one card in the first interface; and a distance between the first center of gravity and a geometric center of the first interface is less than a first distance threshold.

[0010] For example, the material image is a plurality of material images in a theme package downloaded by the electronic device. For another example, the material image is an image stored locally or downloaded by the electronic device. In addition, when the number of material images is large, the filling elements can be selected from different material images and filled into corresponding cards respectively; and when the number of material images is small and the number of cards is large, for example, there is only one material image, a plurality of filling elements need to be selected from the material image and filled into the cards respectively.

[0011] In addition, the filling element can also be referred to as a filling image, that is, an image of at least a partial region selected from the material image.

[0012] It should be understood that the filling elements selected from the material image to be filled into the cards in the first interface are selected according to the position relationship between the first center of gravity of the picture formed by the cards and the background image in the first interface generated by the electronic device and the geometric center of the first interface. The layout of the picture formed by the cards and the background image filled with the above-mentioned selected filling elements in the first interface generated by the electronic device is reasonable, and the effect of the picture display is good, thereby helping to improve the user experience.

[0013] In a possible implementation of the first aspect, the symmetric point of the center of gravity of the background image with respect to the geometric center of the first interface is a symmetric center of gravity of the background image, and a distance between the symmetric center of gravity of the background image and the center of gravity of the at least one card is less than a second distance threshold.

[0014] The center of gravity of the at least one card refers to the overall center of gravity of the picture formed by the cards to be filled in the first interface. It should be understood that in some cases, when there are multiple filling elements that meet the screening condition that the distance between the first center of gravity and the geometric center of the first interface is less than the first distance threshold, the filling elements in the material image can be further screened by the position relationship between the symmetric center of gravity of the background image and the overall center of gravity of the picture formed by the cards to be filled in the first interface.

[0015] It should be understood that in some cases, the filling elements in the material image can be screened by using both the position relationship between the first center of gravity and the geometric center of the first interface and the position relationship between the symmetric center of gravity of the background image and the center of gravity of the at least one card.

[0016] It should be understood that, based on the positional relationship between the first centroid of the image formed by the cards and background image in the first interface to be generated by the electronic device and the geometric center of the first interface, as well as the positional relationship between the symmetrical centroid of the background image and the centroid of at least one card, fill elements are selected from the source images to fill the cards in the first interface. This ensures that the layout of the image formed by the cards and background image after being filled with the selected fill elements in the first interface generated by the electronic device is more reasonable, and the image display effect is better, thereby helping to improve the user experience.

[0017] In one possible implementation of the first aspect above, each of the at least one fill element is filled into the target fill area of ​​the card corresponding to each fill element in the first interface.

[0018] The second center of gravity is the center of gravity of the image formed by each adjustment card and the background image corresponding to each card;

[0019] The target filling area for each card is determined based on the positional relationship between the second centroid and the geometric center of the first interface.

[0020] Each adjustment card is obtained by assigning multiple pixel values ​​to each card.

[0021] For example, adjusting cards can be done by dividing the card into a 3x3 grid, then assigning different pixel values ​​to the pixels in each grid, with all pixels in the same grid having the same grayscale value. For instance, to simplify calculations, the pixel values ​​in each grid can be represented only by 0 and 1, resulting in all pixels in the same grid having either all 0 or all 1 grayscale values. Determining the target fill area in each card by the positional relationship between the second centroid of the image formed by the adjustment cards and the background image and the geometric center of the first interface allows for a more reasonable layout of the image after the fill elements selected from the source image are filled into the target fill area of ​​each card, thus improving the user experience.

[0022] In one possible implementation of the first aspect mentioned above, the third center of gravity is the overall center of gravity of each adjustment card, and the overall center of gravity of each adjustment card is the center of gravity of the image composed of each adjustment card.

[0023] The target filling area for each card is determined based on the positional relationship between the third centroid and the symmetrical centroid of the background image.

[0024] In one possible implementation of the first aspect above, it further includes: determining each fill element in at least one source image in the following manner:

[0025] moving the card corresponding to each filling element in the material image at each scaling ratio within the target scaling range corresponding to each filling element;

[0026] determining each filling element according to the overlapping of the card corresponding to each filling element and the material image at each scaling ratio corresponding to each filling element;

[0027] The target scaling range corresponding to each filling element is determined according to the target filling area of the card corresponding to each filling element.

[0028] It should be understood that the determination of each filling element according to the overlapping of the card corresponding to each filling element and the material image at each scaling ratio corresponding to each filling element makes the layout of the picture composed of the filled cards and the background image reasonable after filling the determined filling element into each card, thereby helping to improve the user experience.

[0029] In a possible implementation of the first aspect, the coordinates of the center of gravity of the picture are obtained by the following formula:

[0030]

[0031] wherein x c is the coordinate of the center of gravity in the horizontal direction, y c is the coordinate of the center of gravity in the vertical direction, p i is the gray value of the i-th pixel point in the picture, x i is the coordinate of the i-th pixel point in the picture in the horizontal direction, y i is the coordinate of the i-th pixel point in the picture in the vertical direction.

[0032] In a possible implementation of the first aspect, the at least one material image is a material image satisfying a preset color screening condition, which is screened from a plurality of to-be-screened material images.

[0033] It should be understood that screening the material image in combination with the color screening condition can meet the preferences of different users for different colors and help to improve the user experience.

[0034] In a possible implementation of the first aspect, the at least one filling element obtained from the at least one material image is at least a part of at least one salient object in the at least one material image.

[0035] It should be understood that since the salient object in the material image is an object of interest of the user in the material image, using at least a part of the salient object in the material image as a filling element to fill into each card can make the visual experience of the user more rich.

[0036] In a possible implementation of the first aspect, the method further includes: in a case where it is determined that multiple filling elements are needed to be selected from the same material image to fill the multiple cards in the first interface, selecting different salient objects from the same material image as the filling elements for filling the multiple cards in the first interface, respectively.

[0037] It should be understood that when at least part of the image content in the same material image is used as a filling element to fill into each card of the first interface, if the image content of the same region of the material image is selected as the filling element to fill into each card, the content of the card is monotonous and repetitive, and the user experience is poor. Therefore, selecting the content of different regions in the same material image, for example, different salient objects, as the filling element to fill into each card can make the content of the card more abundant, and the user experience is better.

[0038] In a second aspect, the embodiments of the present application provide an image processing method applied to an electronic device, including: generating a preset target picture, the preset target picture including a background image and at least one to-be-filled element covering part of the background image;

[0039] adjusting pixel values of each pixel point of each to-be-filled element, and calculating the barycenter of the multiple preset target pictures obtained after the pixel value adjustment and the geometric center of the preset target picture;

[0040] According to the positional relationship between the barycenter of the multiple preset target pictures and the geometric center of the preset target picture, a barycenter meeting a preset condition is selected from the multiple barycenters;

[0041] Based on the preset target picture corresponding to the selected barycenter, the position of the target filling region of each to-be-filled element is determined;

[0042] Based on the position of the target filling region of each to-be-filled element determined, a filling image to be filled into the target filling region of each to-be-filled element is determined;

[0043] Each target filling region of the to-be-filled element is filled with the corresponding filling image, and a target picture is obtained.

[0044] In a possible implementation of the second aspect, the adjusting the pixel values of each pixel point of each to-be-filled element, and calculating the barycenter of the multiple preset target pictures obtained after the pixel value adjustment and the geometric center of the preset target picture includes:

[0045] Each to-be-filled element is divided into multiple sub-regions, wherein each sub-region includes multiple pixel points, respectively;

[0046] The pixel values of each sub-region of each to-be-filled element are respectively taken in a preset pixel value range to obtain a plurality of preset target pictures after pixel value adjustment, wherein the pixel values of each pixel point in the same sub-region are the same;

[0047] Based on the pixel value of each sub-region of each to-be-filled element obtained after pixel value adjustment and the position coordinates of each sub-region, the center of gravity of the plurality of preset target pictures obtained after pixel value adjustment is calculated, and

[0048] Based on the position coordinates of each pixel point of the preset target picture, the geometric center of the preset target picture is calculated.

[0049] In a possible implementation of the second aspect, the position of the target filling region of each to-be-filled element is determined based on the preset target picture corresponding to the selected center of gravity, including:

[0050] In a case where it is determined that the preset target picture corresponding to the selected center of gravity is one, the pixel value of each pixel point of each to-be-filled element in the one preset target picture is determined,

[0051] Based on the determined pixel value of each pixel point of each to-be-filled element, the position of the target filling region of each to-be-filled element is determined;

[0052] In a case where it is determined that the preset target picture corresponding to the selected center of gravity is a plurality of, the center of gravity of the background image of each preset target picture about the geometric center of each preset target picture and the overall center of gravity of a plurality of to-be-filled elements in each preset target picture are determined,

[0053] Based on the positional relationship between the symmetry points and the overall centers of gravity of the plurality of preset target pictures, an overall center of gravity meeting a preset condition is selected from the plurality of overall centers of gravity,

[0054] Based on the pixel value of each pixel point of each to-be-filled element in the preset target picture corresponding to the selected overall center of gravity, the position of the target filling region of each to-be-filled element is determined.

[0055] In a possible implementation of the second aspect, the filling image to be filled into the target filling region of each to-be-filled element is determined based on the determined position of the target filling region of each to-be-filled element, including:

[0056] Based on the determined position of the target filling region of each to-be-filled element, a target scaling range of a material image corresponding to the filling image to be filled into the target filling region of each to-be-filled element is determined;

[0057] determine, according to the preset scoring rule, an overlapping condition of each of the to-be-filled elements and the material image at each zooming scale in the corresponding target zooming range when each of the to-be-filled elements is moved in the material image at each zooming scale in the corresponding target zooming range;

[0058] determine, according to the overlapping condition, a target zooming scale of each of the material images corresponding to each of the to-be-filled elements, and a filling image of each of the material images to be filled into a target filling region of the corresponding to-be-filled element at the target zooming scale.

[0059] In a possible implementation of the second aspect, the filling image in each of the material images includes at least a part of a salient object in the material image.

[0060] In a possible implementation of the second aspect, the coordinates of the center of gravity of the picture are obtained by the following formula:

[0061]

[0062] where x c is the coordinate of the center of gravity of the picture in the horizontal direction, y c is the coordinate of the center of gravity of the picture in the vertical direction, p i is a gray value of the i th pixel in the picture, x i is the coordinate of the i th pixel in the picture in the horizontal direction, y i is the coordinate of the i th pixel in the picture in the vertical direction.

[0063] In a possible implementation of the second aspect, the method further includes:

[0064] obtaining a plurality of material images to be selected and a color screening condition;

[0065] determining, from the plurality of material images to be selected, a material image that satisfies the color screening condition.

[0066] In a possible implementation of the second aspect, the method further includes:

[0067] In a case where it is determined that the filling images need to be selected from the same material image to fill the plurality of to-be-filled elements in the target picture, different salient objects are selected from the same material image as the filling images for filling the plurality of to-be-filled elements in the target picture.

[0068] In a possible implementation of the second aspect, the target picture is a desktop of an electronic device,

[0069] The to-be-filled elements are service cards of the desktop of the electronic device.

[0070] In a third aspect, the embodiments of the present application provide a computer readable storage medium, having stored thereon instructions which, when executed on an electronic device, cause the electronic device to perform any of the image processing methods in the first aspect and various possible implementations of the first aspect, or in the second aspect and various possible implementations of the second aspect.

[0071] In a fourth aspect, the embodiments of the present application provide a computer program product, comprising instructions for implementing any of the image processing methods in the first aspect and various possible implementations of the first aspect, or in the second aspect and various possible implementations of the second aspect.

[0072] In a fifth aspect, the embodiments of the present application provide an electronic device, comprising:

[0073] a memory configured to store instructions, and

[0074] one or more processors configured to execute the instructions stored in the memory, when the instructions are executed by the one or more processors, cause the electronic device to perform any of the image processing methods in the first aspect and various possible implementations of the first aspect, or in the second aspect and various possible implementations of the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 According to some embodiments of the present application, an application scenario to which the technical solution of the present application is applicable is shown;

[0076] Figure 2A According to some embodiments of the present application, a schematic diagram of a card of a mobile phone desktop service is shown;

[0077] Figure 2B According to some embodiments of the present application, a schematic diagram of a user mobile phone theme setting interface is shown;

[0078] Figure 2C According to some embodiments of the present application, a user mobile phone theme setting interface is shown; Figure 2A material images used for filling the background image 13 and the card 11 and the card 12 in the mobile phone desktop shown in FIG. 13;

[0079] Figure 2D According to some embodiments of the present application, images after gray processing of the various material images shown in FIG. 14 are shown; Figure 2C

[0080] Figure 2E According to some embodiments of the present application, various pictures composed of the background image and the various cards are shown;

[0081] ​Figure 2F According to some embodiments of the present application, a schematic diagram of a mobile phone desktop after filling of each card of the mobile phone desktop according to the technical solution of the present application is shown;

[0082] Figure 3 According to some embodiments of the present application, a mobile phone desktop is shown; Figure 1 A hardware structure diagram of the electronic device 100 in the application scenario shown is shown;

[0083] Figure 4 According to some embodiments of the present application, a flowchart of an image processing method is shown;

[0084] Figure 5A and Figure 5B According to some embodiments of the present application, some theme download interfaces of the mobile phone 100 are shown;

[0085] Figure 6A According to some embodiments of the present application, a maximum scaling ratio and a minimum scaling ratio of a material image to be filled in the card 11 are shown; Figure 2A

[0086] Figure 6B According to some embodiments of the present application, a maximum scaling ratio and a minimum scaling ratio of a material image to be filled in the card 12 are shown; Figure 2A

[0087] Figure 6C According to some embodiments of the present application, a schematic diagram of the card 11 sliding in the image P23 is shown; Figure 6A

[0088] Figure 6D According to some embodiments of the present application, a schematic diagram of the card 11 sliding in the image P22 is shown; Figure 6A

[0089] Figure 7 According to some embodiments of the present application, a flowchart of another image processing method is shown;

[0090] Figure 8A According to some embodiments of the present application, a picture H5 composed of the filled card 11, the filled card 12 and the background image P10 is shown;

[0091] Figure 8B According to some embodiments of the present application, the mobile phone 100 displays the picture H5 as a desktop of the mobile phone 100 is shown; Figure 8A

[0092] Figure 9 ​​​​​According to some embodiments of the present application, a flowchart of another image processing method is shown. DETAILED DESCRIPTION

[0093] Illustrative embodiments of the present application include, but are not limited to, an image processing method, a medium, a program product, and an electronic device.

[0094] In order to facilitate the understanding of the technical solutions of the present application, first, some concepts involved in the present application are explained and described.

[0095] The gravity center of the picture: used to indicate the average position of the pixel value distribution of each pixel point in the picture.

[0096] The geometric center of the picture: used to indicate the center position of the picture.

[0097] The salient region: used to indicate the region where the salient object in the image / picture / card is located. It can be understood that only the card filled with the material image has a salient region, and the salient region of the card is the region in the card filled with the image (i.e. the filling material) cut out from the material image.

[0098] The salient object refers to a relatively salient object included in the image. When a user observes an image, the user's attention (or point of interest) will focus on the salient object. For example, the salient object in the image can be a person, an animal, a building, food, etc. In the following description, "the main object of the image" and "the salient object of the image" have the same meaning.

[0099] In order to make the purpose, technical solutions and advantages of the present application clearer and more apparent, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0100] In order to facilitate understanding, the following will take a mobile phone applicable to the technical solutions of the present application as an example to describe the technical solutions of the present application in detail.

[0101] First, an application scenario in which a user sets a theme for a mobile phone is introduced, and the method for the mobile phone to display a theme by the technical solutions provided by the present application is described in detail in combination with the application scenario.

[0102] Figure 1 According to some embodiments of the present application, a scenario diagram in which a user sets a theme for a mobile phone is shown. As shown in FIG. 1, a user sets a theme for a mobile phone, and the mobile phone displays the theme according to the technical solutions provided by the present application. Figure 1As shown, this includes a mobile phone 100 and a server 200. The mobile phone 100 has a theme app installed, and the server 200 can be an application server that provides services to the theme app on the mobile phone 100. The mobile phone 100 and the server 200 can establish a communication connection through Bluetooth (BT), Wireless Fidelity (Wi-Fi) networks, etc.

[0103] In some embodiments, a user can input theme filtering criteria through the theme app on mobile phone 100. Mobile phone 100 sends a theme download request carrying the theme filtering criteria input by the user to server 200. Server 200 can respond to the received theme download request, filter out themes that meet the above filtering criteria, and return them to mobile phone 100.

[0104] In this context, the theme requested by mobile phone 100 from server 200 is a pre-stored theme installation file (also known as a theme package) containing multiple source images. These source images can be used as background images for mobile phone 100's desktop or to fill various cards on the desktop. For example, in some embodiments, mobile phone 100 uses the source images from the theme package downloaded from server 200 as... Figure 2A The desktop background image 13 shown is filled with images from the theme pack, such as... Figure 2A Cards 11 and 12 are shown. It is understood that the source image used as the background image and the source image used to fill each card can be the same source image or different source images. It is understood that using source images to fill the various cards on the mobile phone 100 desktop means using at least a portion of the image content (i.e., the filling material) of the source image to fill each card.

[0105] In some embodiments, the above-mentioned theme filtering criteria can be theme style, content, color, etc. For example, users can select themes such as... Figure 2B In the theme settings interface of the shown mobile phone 100, you can select a theme with minimalist, Chinese style, or fresh style under the style category; you can also select... Figure 2B The content categories shown include themes featuring cartoons, cutting-edge technology, and natural themes; you can also select... Figure 2B The color categories shown feature themes with a single color as the main tone, such as black, green, or white.

[0106] As described in the background, in some cases, the phone 100 desktop display has some application cards, after the user selects a certain theme, the phone 100 fills the desktop with the material image in the theme package. In order to make the layout of the picture composed of the filled card and the background image more reasonable, the phone 100 can first determine the material image as the desktop background image and the material image used to fill each to-be-filled card by executing the technical solution provided in the present application. Then determine the target filling area in each to-be-filled card. Then according to the preset cropping method, determine how to crop the material image, so as to fill the cropped image as the filling material into the target filling area in each to-be-filled card.

[0107] In this way, when the phone 100 displays the desktop, the layout of the picture composed of the filled card and the background image is more reasonable, and the effect of the picture display is better, thereby helping to improve the user experience.

[0108] The following will first combine each embodiment shown in Figures 2A to 2F to exemplarily describe the above technical solution, so as to facilitate understanding of the technical solution of the present application.

[0109] For example, in some embodiments, it is assumed that the phone 100 shown in Figure 1 downloads the theme package from the server 200, which includes the material image P10, the material image P20 and the material image P30 shown in Figure 2C . The material image P10 is used as the background image of the phone 100 desktop, and the material image P20 and the material image P30 are used as the material images used by the to-be-filled card 11 and the to-be-filled card 12 shown in Figure 2A .

[0110] After determining the above-mentioned to-be-used material image, it is necessary to first determine the target filling area in the to-be-filled card 11 and the card 12. For example, in some embodiments, in order to facilitate the calculation of the target filling area in each to-be-filled card, first, the gray value of each pixel point of the material image P10 to P30 is binarized, for example, the gray value of the pixel point whose gray value exceeds the gray value threshold is set to 1, and the gray value of the pixel point whose gray value does not exceed the gray value threshold is set to 0. Thus, the image P11, P21 and P31 as shown in Figure 2D is obtained. Then, the image P11, the to-be-filled card 11 and the to-be-filled card 12 are combined into a picture, for example, combined into a picture H1 as shown in Figure 2E . The gray value threshold can be set as needed, for example, the gray value threshold can be set to 127, which is not limited in the present application.

[0111] To determine the target filling areas in cards 11 and 12, different grayscale values ​​can be assigned to each pixel in cards 11 and 12, resulting in various grayscale value assignments for each card. For example, if card 11 has 64 pixels, and each pixel has a grayscale value of either 0 or 1, then the resulting grayscale values ​​for card 11 are 2... 64 This describes the result of assigning grayscale values. For example, if card 12 has 32 pixels, and each pixel's grayscale value is either 0 or 1, then the result is a grayscale value of 2 for card 12. 32 The grayscale values ​​are assigned to various values. Then, the center of gravity and the geometric center of the image composed of the cards 11 and 12 to be filled and the background image with various grayscale values ​​are calculated. The cards 11 and 12 to be filled with the grayscale values ​​that are closest to the center of gravity and the geometric center of the image are selected.

[0112] For example, in some embodiments, to reduce computational load, the cards 11 and 12 to be filled are divided into multiple regions of the same size, such as dividing the cards 11 and 12 to be filled into... Figure 2E The diagram shows a nine-square grid. Within each square, all pixels have the same grayscale value, meaning all pixels in the same square have a grayscale value of either 0 or 1. Then, the grayscale value of each pixel in each square of the card to be filled 11 is assigned. For example, all pixels in a portion of the squares of the card to be filled 11 have a grayscale value of 1, while all pixels in the remaining portion of the squares have a grayscale value of 0, resulting in 2. 9 Card 11, with a combination of grayscale values, is to be filled. Card 12, with the same nine-grid grayscale values ​​as card 11, is then assigned the same values, resulting in 2. 9 Card 12 with different combinations of grayscale values. Then, various possible images composed of cards 11 and 12 with different combinations of grayscale values ​​and a background image are determined. For example, [the image is shown here]. Figure 2E The pixel values ​​of the four cells in the lower right corner of card 11 are set to 1, and the pixel values ​​of the four cells in the upper right corner of card 12 are set to 1, resulting in the following: Figure 2E The image shown is H2. For example, setting all pixel values ​​of a cell in the lower right corner of card 11 to 1, and setting all pixel values ​​of a cell in the upper right corner of card 12 to 1, results in the following: Figure 2E The image shown is H3.

[0113] Then, the geometric center and centroid of each of the above images are calculated, and the image whose centroid and geometric center are closest is selected. For example, ... Figure 2EThe geometric center C of image H2 is relatively close to the centroid B1 of image H2. However, the geometric center C of image H3 is much farther from the centroid B2 of image H3. Therefore, it can be determined that the target filling area in the card 11 to be filled is approximately... Figure 2E The four grid areas in the upper right corner of card 11 in the shown image H2; the target filling area in card 12 to be filled is approximately... Figure 2E The four grid areas in the upper right corner of card 12 in the shown image H2.

[0114] To determine the specific method for cropping the source images P20 and P30, and to fill the target filling areas in the cards 11 and 12 accordingly, so that the layout of the image composed of the filled cards 11 and 12 and the background image P10 is more reasonable, in some embodiments, the scaling ratio of source images P20 and P30 can be determined according to a preset cropping method, for example, determining to reduce source image P20 by a factor of 1.5 or by a factor of 3. Furthermore, after reducing source image P20 by a factor of 1.5, the image content of building area 1 in image P20 is cropped and filled into the target areas. Figure 2E In the four-square grid at the bottom right corner of card 11 shown; after reducing the source image P30 by a factor of 3, crop out the image content of cake area 2 in image P30 and fill it into... Figure 2E In the four-square grid in the upper right corner of card 12 shown, thus obtaining the following... Figure 2E The image shown is H4. Then, phone 100 renders image H4 as its desktop background, displaying the image as shown below. Figure 2F The interface shown. It's not hard to see that, Figure 2F In the desktop of the mobile phone 100 shown, the overall layout of the background image P10 and the filled cards 11 and 12 is relatively reasonable, and the display effect is good, which helps to improve the user experience. The preset cropping method will be described in detail below with reference to flowcharts provided in some embodiments of this application, and will not be elaborated here.

[0115] In addition, it is understandable that when filling in individual cards, in order to enrich the content of the filled cards, at least a portion of the prominent objects in the source image is usually used as the cropping target (i.e., the filling element), and then cropped and filled into the card to improve the user's visual experience.

[0116] In addition, in some embodiments, when the theme package includes multiple material images, the mobile phone 100 can also combine the background image and the material images with relevant filtering conditions such as color and contrast to select one or more material images that meet the conditions from the theme package, so as to use them as material images to fill each card.

[0117] In some embodiments, if multiple material images are included in the theme package, in order to make the picture effect of the desktop of the mobile phone 100 better, different images can be selected as the background image and to fill the cards respectively. That is, the images used by the background image and the cards are not repeated.

[0118] In some embodiments, if the number of material images in the theme package is insufficient, for example, the number of material images is less than the sum of the number of background images and the number of cards, that is, when the material image and the background image to be filled in each card are the same image, in order to make the picture effect of the desktop of the mobile phone 100 better, different filling elements in the same image can be selected to fill different cards.

[0119] In addition, it can be understood that, Figure 1 The mobile phone 100 in the scene shown can be any electronic device with a display screen, including but not limited to a mobile phone, a laptop computer, a desktop computer, a tablet computer, a wearable device, a head-mounted display, a portable game console, a portable music player, a reader device, a television with one or more processors embedded or coupled therein, or other electronic devices capable of accessing a network, etc.

[0120] In addition, it can be understood that, Figure 1 The server 200 in the scene shown can be any electronic device with strong computing and / or storage capabilities, including but not limited to a server, a laptop computer, a desktop computer, a tablet computer, other electronic devices capable of accessing a network, etc.

[0121] In addition, it can be understood that the theme package download and installation scene described above is only an example of an application scenario of the technical solution of the present application, and the technical solution of the present application can be applied to any scenario involving image layout. For example, the technical solution of the present application can also be applied to the layout of the covers of books in the bookshelf of a reading application. The present application does not limit this.

[0122] The following will be described in detail Figure 3 The technical solution of the present application can be executed by the mobile phone 100 in the scene shown. Figure 1 The hardware structure of the mobile phone 100 capable of executing the technical solution of the present application in the scene shown will be described in detail. Figure 3 According to an embodiment of the present application, a hardware structure schematic diagram of a mobile phone 100 is shown.

[0123] The mobile phone 100 can execute the technical solution provided by the present application. In Figure 3 In the drawings, similar components have the same reference numerals. As Figure 3As shown, the mobile phone 100 can include a processor 110, a power module 140, a memory 180, a camera 170, a mobile communication module 130, a wireless communication module 120, a sensor module 190, an audio module 150, an interface module 160, a display screen 102, and the like.

[0124] The processor 110 can include one or more processing units, for example, can include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro-programmed control unit (MCU), an artificial intelligence (AI) processor, or a programmable logic device (FPGA) processing module or processing circuit, etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors. For example, in some examples of the present application, the processor 110 can be used to determine the target filling area in each card, the scaling ratio of the material image to be filled, and the cropping range of each material image under the corresponding scaling ratio.

[0125] The memory 180 can be used to store data, software programs and modules, and can be a volatile memory (Volatile Memory), such as a random access memory (RAM); or a non-volatile memory (Non-Volatile Memory), such as a read-only memory (ROM), a flash memory (Flash Memory), a hard disk (HDD) or a solid state disk (SSD); or a combination of the above types of memory, or can also be a removable storage medium, such as a secure digital (SD) memory card. Specifically, the memory 180 can include a program storage area (not shown) and a data storage area (not shown). The program storage area can store program codes for causing the processor 110 to execute the image processing method provided by the embodiments of the present application by executing the program codes. In the embodiments of the application, the data storage area can be used to store material images.

[0126] The power module 140 can include a power source, a power management component, etc. The power source can be a battery. The power management component is used to manage charging of the power source and power supply of the power source to other modules. The charging management module is used to receive charging input from a charger; the power management module is used to connect the power source, the charging management module and the processor 110.

[0127] The mobile communication module 130 can include, but is not limited to, an antenna, a power amplifier, a filter, a low noise amplifier (LNA), etc. The mobile communication module 130 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the mobile phone 10. The mobile communication module 130 can receive electromagnetic waves by the antenna, and perform filtering, amplification, etc. on the received electromagnetic waves, and transmit to the modem processor for demodulation. The mobile communication module 130 can also amplify the signal modulated by the modem processor, and convert it into electromagnetic waves radiated by the antenna. In some embodiments, at least part of the functional modules of the mobile communication module 130 can be arranged in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 130 can be arranged in the same device as at least part of the modules of the processor 110.

[0128] The wireless communication module 120 can include an antenna, and realize the transceiving of electromagnetic waves via the antenna. The wireless communication module 120 can provide a solution for wireless communication including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. applied to the mobile phone 10. The mobile phone 10 can communicate with the network and other devices through the wireless communication technology.

[0129] In some embodiments, the mobile communication module 130 and the wireless communication module 120 of the mobile phone 10 can also be located in the same module.

[0130] The camera 170 is configured to capture still images or videos. An object projects an optical image through a lens to a photosensitive element. The photosensitive element converts the optical signal into an electrical signal, which is then transferred to an ISP (Image Signal Processor) to convert into a digital image signal. The mobile phone 10 can implement the photographing function through the ISP, the camera 170, a video codec, a GPU (Graphic Processing Unit), the display screen 102, and an application processor.

[0131] The display screen 102 includes a display panel. The display panel can be a LCD (Liquid Crystal Display), an OLED (Organic Light-emitting Diode), an AMOLED (Active-matrix Organic Lght-emitting Diode), a FLED (Flex Light-emitting Diode), a Mini LED, a Micro LED, a Micro OLED, a QLED (Quantum Dot Light-emitting Diodes), or the like. For example, the display screen 102 is configured to display a picture composed of the filled card and the background image of the table top.

[0132] The sensor module 190 can include a proximity light sensor, a pressure sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, or the like.

[0133] The audio module 150 can convert digital audio information into an analog audio signal for output, or convert an analog audio input into a digital audio signal. The audio module 150 can also be configured to encode and decode audio signals. In some embodiments, the audio module 150 can be disposed in the processor 110, or some functional modules of the audio module 150 can be disposed in the processor 110. In some embodiments, the audio module 150 can include a speaker, a receiver, a microphone, and a headset jack.

[0134] The interface module 160 includes an external memory interface, a universal serial bus (USB) interface, and a subscriber identification module (SIM) card interface, etc. The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 10. The external memory card communicates with the processor 110 through the external memory interface to realize data storage functions. The universal serial bus interface is used for communication between the mobile phone 10 and other mobile phones. The subscriber identification module card interface is used to communicate with the SIM card installed in the mobile phone 10, such as reading the phone number stored in the SIM card, or writing the phone number into the SIM card.

[0135] In some embodiments, the mobile phone 10 also includes keys, a motor, and an indicator, etc. The keys can include volume keys, on / off keys, etc. The motor is used to generate a vibration effect for the mobile phone 10. The indicator can include a laser indicator, a radio frequency indicator, an LED indicator, etc.

[0136] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the mobile phone 100. In other embodiments of the present application, the mobile phone 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0137] In the following, the desktop of the mobile phone 100 is added with the card 11 and the card 12, and the card 13 is added to the card 12. Figure 1 After the mobile phone 100 downloads the theme package from the server 200 as shown in the figure, the mobile phone 100 processes the material images in the theme package according to the technical solutions provided in the present application, and respectively uses them as the background image of the mobile phone 100, fills the card 11 and the card 12, to introduce an image processing method provided in the present application.

[0138] It should be noted that, since the material images in the theme package stored in the server 200 are usually multiple, the mobile phone 100 will use a single material image as the background image, and use two different material images different from the background image to fill the card 11 and the card 12, respectively.

[0139] However, in some application scenarios, for example, a local image (such as the only image in the album) saved by the mobile phone 100 is used as the wallpaper of the mobile phone 100, so the mobile phone 100 will use the same material image as the background image of the wallpaper, and fill the card 11 and the card 12.

[0140] Therefore, the image processing method provided by the present application will be introduced below in two cases respectively, i.e. the case of sufficient material images (more material images) and the case of less material images (e.g. only one material image).

[0141] First, the following will be introduced in combination with the accompanying drawings. Figure 4 In the case that the theme package downloaded by the mobile phone 100 from the server 200 is sufficient in material images, the mobile phone 100 uses a single material image as the background image, and uses two different material images different from the background image as the material images corresponding to the card 11 and the card 12 respectively, and then carries out the processes of selecting the filling material, determining the target filling area in the card 11 and the card 12, etc. Figure 2A The processes of selecting the filling material, determining the target filling area in the card 11 and the card 12, etc. will be introduced in detail below. Specifically, as shown in Figure 4 The image processing method provided by the present application includes the following steps:

[0142] Step S401: The mobile phone 100 and the server 200 establish a connection.

[0143] For example, the mobile phone 100 and the server 200 establish a wireless communication connection, and the mobile phone 100 and the server 200 communicate through wireless communication modes such as Wi-Fi and Bluetooth.

[0144] Step S402: The mobile phone 100 sends a theme download request to the server 200.

[0145] In some embodiments, for example, the user selects the theme of “fresh” style in the interface as shown in Figure 2B The mobile phone 100 displays a plurality of themes to be selected as shown in Figure 5A The user clicks the icon 106 of the theme 1, and the mobile phone 100 enters the download interface as shown in Figure 5B The user clicks the “download and install” control 107, and the mobile phone 100 generates a corresponding theme download request and sends it to the server 200.

[0146] Step S403: The server 200 returns a theme package to the mobile phone 100.

[0147] Step S404: The mobile phone 100 determines the material image as the background image, and the material images used to fill the card 11 and the card 12 respectively.

[0148] For example, in some embodiments, the mobile phone 100 determines to use the material image P10 as shown in Figure 2C as the background image of the desktop of the mobile phone 100. The material image P20 as shown in Figure 2C is used as the material image corresponding to the card 11 as shown in Figure 2A The material image P30 as shown in Figure 2C is used to fill the card 12 as shown in Figure 2AThe shown card 12 corresponds to a material image. It can be understood that the background image in a theme package is predetermined. For example, the theme package contains a background image identifier, and the material image corresponding to the background image is determined through the background image identifier.

[0149] In some embodiments, the phone 100 can determine two of the material images other than the background image as the material images corresponding to the card 11 and the card 12, respectively.

[0150] Step S405: The phone 100 determines the background image and the salient objects in the material images used to fill the card 11 and the card 12.

[0151] In some embodiments, when there is only one salient object in an image, the only one salient object can be directly determined as the salient object of the image. In some embodiments, when there are multiple salient objects in an image, the salient object with the highest proportion in the image can be determined as the salient object of the image.

[0152] In some embodiments, the phone 100 can determine the salient region in the background image P10 as the region where the person is located, the salient region in the material image P20 as the region where the building is located, and the salient region in the material image P30 as the region where the cake is located, through a saliency detection algorithm, such as a saliency detection algorithm in OpenCV (a cross-platform computer vision and machine learning software library). That is, the salient object of the image P10 is determined as the person, the salient object of the image P20 is determined as the building, and the salient object of the image P30 is determined as the cake.

[0153] Step S406: The phone 100 determines the target filling regions in the card 11 and the card 12, respectively.

[0154] For example, in some embodiments, in order to determine the target filling regions in the card 11 and the card 12, the card 11 and the card 12 can be regarded as binary processed gray images, and then the pixel points in the card 11 and the card 12 are assigned different gray values, and multiple gray value combinations of the card 11 and the card 12 are determined, respectively.

[0155] For example, in some embodiments, in order to reduce the amount of calculation, the card 11 and the card 12 are divided into multiple regions of the same size, for example, the card 11 and the card 12 are divided into nine regions as shown in the figure. Figure 2E In the same nine-region grid, the pixel values of all pixel points are the same, that is, the pixel values of all pixel points in the same nine-region grid are all 0 or all 1.

[0156] Then, the pixel grayscale values ​​of each cell in card 11 are transformed. For example, the grayscale value of all pixels in a portion of the cells in card 11 is set to 1, while the grayscale value of all pixels in the remaining portion of the cells in card 11 is set to 0, resulting in 2. 9 Card 11 has a combination of grayscale values. Card 12 undergoes the same nine-grid grayscale transformation as card 11, resulting in 2. 9 Card 12 with different combinations of grayscale values. Then, various possible images composed of cards 11 and 12 with different combinations of grayscale values ​​and a background image are determined.

[0157] Then calculate the geometric center and centroid of each of the above images. For example, calculate the geometric center and centroid of each image. Figure 2E The geometric center of image H2 is point C, the centroid of image H2 is point B1, the centroid of the background image in image H2 is point A, the point symmetrical to the centroid A of the background image about the geometric center C is point E, and the centroid of all cards in image H2 (i.e., cards 11 and 12) is point D1. The geometric center of image H3 is point C, the centroid of image H3 is point B2, the centroid of the background image in image H2 is point A, the point symmetrical to the centroid A of the background image about the geometric center C is point E, and the centroid of all cards in image H2 (i.e., cards 11 and 12) is point D2.

[0158] Then, based on the calculated geometric centers of the aforementioned frames, and multiple centroids, the frame whose centroid and geometric center are closest is selected. For example, as... Figure 2E The geometric center C of image H2 is relatively close to the centroid B1 of image H2. However, the geometric center C of image H3 is much farther from the centroid B2 of image H3. Therefore, it can be determined that the target filling area in card 11 on the desktop of mobile phone 100 is approximately... Figure 2E In the image H2 shown, the four grid areas in the lower right corner of card 11, the target filling area in card 11 is approximately... Figure 2E The four grid areas in the upper right corner of card 12 in screen H2 are shown. This determines the target filling areas in each card on the mobile phone 100's desktop.

[0159] In some embodiments, the center of gravity of the above image can be calculated using the following formula (1):

[0160]

[0161] Where, x c Let y be the coordinate of the center of gravity of the image in the horizontal direction. c p is the coordinate of the image's center of gravity in the vertical direction. i Let x be the grayscale value of the i-th pixel. i Let y be the horizontal coordinate of the i-th pixel in the image.i Let be the vertical coordinates of the i-th pixel in the image.

[0162] It is understandable that the center of gravity of the background image in each picture, and the center of gravity of all cards in the picture (that is, the overall center of gravity of the picture composed of all cards) can also be calculated using the above formula (1).

[0163] It is understandable that the division of cards 11 and 12 into a 3x3 grid is only for the convenience of calculation. The specific way to divide cards 11 and 12 can be determined according to the actual situation, and this application does not limit it.

[0164] Furthermore, it is understandable that Figure 2E Cards 11 and 12 shown do not obscure prominent objects in the background image. However, in some cases, the card area on the phone 100's desktop may obscure prominent objects in the background image. In such cases, the area obscured by the cards in the background image can be disregarded when calculating the centroid of the background image.

[0165] Furthermore, in some embodiments, it is possible that among the various images obtained using the 3x3 grid transformation pixel values ​​in cards 11 and 12, there are multiple images where the distance between the center of gravity of the image and the geometric center of the image is within a first distance threshold. In this case, from the determined multiple images, images where the distance between the overall center of gravity of all cards in the image and the symmetrical point of the center of gravity of the background image in the image (i.e., the symmetrical center of gravity of the background image) is within the second distance threshold can be selected, thereby determining the target filling area in each card on the desktop of the mobile phone 100.

[0166] Step S407: The mobile phone 100 determines the target scaling ratio of the material images used to fill card 11 and card 12, and the cropping area corresponding to each material image under the determined target scaling ratio, thereby obtaining the filling elements corresponding to card 11 and card 12.

[0167] In some embodiments, the mobile phone 100 may be configured such that, when filling cards 11 and 12, the maximum scaling ratio of the source image must ensure that the short side of the salient object to be filled into the card in the source image does not exceed the card area. The minimum scaling ratio must ensure that, when filling the card with the salient object of the source image, the area in the card that can be filled with the salient object cannot be empty.

[0168] For example, for card 11, the maximum size that the source image P20 can be enlarged to is as follows: Figure 6A The image P22 shown is designed such that the shorter side d1 in image P22 does not extend beyond card 11. The minimum size of the source image P20 can be as follows: Figure 6AThe image P23 shown is such that when card 11 is right-aligned with image P23, and card 11 is slid up and down in image P23, the four grid areas in the lower right corner of card 11 can overlap with parts of the buildings in image P23.

[0169] For example, for card 12, the maximum size that the source image P30 can be enlarged to is as follows: Figure 6B The image P32 shown is designed such that the shorter side d2 in image P32 does not exceed card 12. The minimum size of the source image P30 can be as follows: Figure 6B The image P33 shown is such that when card 12 is right-aligned with image P33, and card 12 is slid up and down in image P33, the four grid areas in the upper right corner of card 12 can overlap with the cake area in image P33.

[0170] After determining the scaling range of source images P20 and P30, the specific scaling ratios of source images P20 and P30, and how to crop source images P20 and P30 at the determined scaling ratios, are then determined to obtain the filling elements corresponding to cards 11 and 12, which are used to fill cards 11 and 12 respectively. In some embodiments, according to a preset cropping method, for source image P20, images obtained at all scaling ratios between the maximum and minimum scaling ranges are processed by moving card 11 from top to bottom and from left to right in each image, moving one grid square at a time. Then, a score is awarded based on the overlap between card 11 and the image at different positions in each image. Finally, all scores obtained at all scaling ratios of source image P20 are sorted.

[0171] For example, take card 11 from Figure 6C Starting from the upper left of image P23 (the image obtained by reducing source image P20 to its smallest scale), the card 11 moves downwards in one-square increments until it reaches the bottom of image P23. During this process, the overlap between the nine squares in card 11 and image P23 is scored. In some embodiments, in the four squares of the target filling area in card 11, squares not overlapping with image P23 are scored as 0 points, and squares overlapping with image P23 are scored as 1 point; while in the five squares of the non-target filling area in card 11, squares not overlapping with image P23 are scored as 1 point, and squares overlapping with image P23 are scored as 0 points. In some embodiments, if the area overlapping with image P23 in a square of card 11 exceeds a set scale threshold, then that square is determined to overlap with image P23. Therefore, if... Figure 6CAs shown, according to the calculation method, it can be determined that when the card 11 is located at the initial position (i.e., at the upper left of the image P23), the calculated score is 7; when the card 11 is slid down by one grid, the calculated score is 9, and other scores are not listed one by one.

[0172] For example, the card 11 is slid from the initial position to the right by one grid, and the score of the overlapping condition of the card 11 and the image P23 at the right side of the image P23 is calculated according to the calculation method. Figure 6D As shown, according to the calculation method, it can be determined that when the card 11 is located at the initial position (i.e., at the upper left of the image P23), the calculated score is 7; when the card 11 is slid down by one grid, the calculated score is 9, and other scores are not listed one by one. Figure 6D As shown, according to the calculation method, it can be determined that when the card 11 is located at the initial position (i.e., at the upper left of the image P23), the calculated score is 7; when the card 11 is slid down by one grid, the calculated score is 9, and other scores are not listed one by one.

[0173] And the calculation method of the score of the overlapping condition of the card 12 and the image P30 at different scaling ratios can also use the above calculation method, so as to obtain the score result of the overlapping condition of the card 12 and the image P30 at different scaling ratios. Here, the introduction is not expanded.

[0174] It can be understood that the calculation method of the score of the overlapping condition of the card 11 and the image P23, the image P22 is only an exemplary calculation method, and the specific calculation method can be determined according to the actual situation, and the application does not limit this.

[0175] In some embodiments, after obtaining the score result of the overlapping condition of the card 11 and the image P20 at different scaling ratios, and the score result of the overlapping condition of the card 12 and the image P30 at different scaling ratios, according to the overlapping condition of the card 11 at the highest score of the image P20 at a certain scaling ratio, the scaling ratio of the image P20 when filling into the card 11 is determined, and the overlapping condition of the card 11 at the scaling ratio is determined, so as to determine how to cut the image P20 at the scaling ratio to fill into the card 11. Similarly, according to the overlapping condition of the card 12 at the highest score of the image P30 at a certain scaling ratio, the scaling ratio of the image P30 when filling into the card 12 is determined, and the overlapping condition of the card 12 at the scaling ratio is determined, so as to determine how to cut the image P30 at the scaling ratio to fill into the card 12.

[0176] Step S408: The mobile phone 100 displays the screen composed of the background image and the filled card 11 and the filled card 12, generating the first interface.

[0177] In some embodiments, after determining the proportions of the images to be filled in card 11 and card 12 and the image areas to be cropped, the mobile phone 100 can crop the material images corresponding to card 11 and card 12 to obtain the filling elements corresponding to card 11 and card 12, and use the cropped filling elements to fill the background of card 11 and card 12.

[0178] For example, after determining that the source image P20 is reduced by a factor of 1.5, the image content of building area 1 within it is filled into... Figure 2E In the four-square grid at the bottom right corner of card 11 shown; after reducing the source image P30 by a factor of 3, fill the image content of area 2 of the cake area with it. Figure 2E In the four-square grid in the upper right corner of card 12 shown, thus obtaining the following... Figure 2E The image shown is H4. Then, phone 100 renders image H4 as its desktop background, displaying the image as shown below. Figure 2F The interface shown. It's not hard to see that, Figure 2F The background image and cards 11 and 12 on the desktop of the mobile phone 100 shown have a good overall picture effect, which helps to improve the user experience.

[0179] The following will be combined with the appendix Figure 7 This document details the process of using the same image as the background to fill cards 11 and 12 when the locally stored image on mobile phone 100 is insufficient. Mobile phone 100 is the primary entity executing each step. Specifically, as follows... Figure 7 As shown, the image processing method provided in this application includes the following steps:

[0180] Step S701: Identify the prominent objects in the source images that serve as both the background image and the material images used to fill cards 11 and 12.

[0181] For example, in some embodiments, the photo album of mobile phone 100 stores... Figure 2C The image P10 shown is set as wallpaper by the user. In response, the mobile phone 100 determines image P10 as the desktop background image and uses it to fill cards 11 and 12. To achieve a more aesthetically pleasing final display, the mobile phone 100 uses different prominent objects from image P10 to fill cards 11 and 12 respectively. For example, the mobile phone 100 determines that... Figure 2CThe main object 1 (a person) in the image P10 shown is used to fill card 11, and the main object 2 (a dolphin-shaped balloon) is used to fill card 12.

[0182] Step S702: Determine the target filling areas in cards 11 and 12 that can be filled with the corresponding filling elements.

[0183] Can be used and Figure 4 The same method used in step S406 involves dividing cards 11 and 12 into a nine-grid layout. Then, based on the distance between the centroid and geometric center of the different frames composed of the background image after pixel value transformation of each nine-grid, the areas in cards 11 and 12 that can be filled with material images (i.e., target filling areas) are determined. For details, please refer to the relevant description of step S406 above, which will not be repeated here.

[0184] Step S703: Determine the scaling ratio of the material image when filling card 11 and card 12, and how to crop the material image under the determined scaling ratio, so as to obtain the filling elements corresponding to filling card 11 and card 12 respectively, and fill card 11 and card 12.

[0185] For example, in some embodiments, employing and Figure 4 Using the same calculation method in step S407, the scores for various overlap situations of the main object 1 in the image P10 at different scaling ratios and the scores for various overlap situations of the main object 2 in the image P10 at different scaling ratios are calculated for the card 11. Based on the highest-scoring overlap situation of the card 11 at a certain scaling ratio of the image P10, the scaling ratio of the image P10 when filling the card 11, and the overlap situation with the card 11 at that scaling ratio, are determined. This determines how to crop the image P20 at that scaling ratio to obtain the corresponding filling element, which is then filled into the card 11. Similarly, based on the highest-scoring overlap situation of the card 12 at a certain scaling ratio of the image P10, the scaling ratio of the image P10 when filling the card 12, and the overlap situation with the card 12 at that scaling ratio, are determined. This determines how to crop the image P10 at that scaling ratio to obtain the corresponding filling element, which is then filled into the card 12. The specific scoring calculation method can be found in the relevant description of step S407 above, and will not be repeated here.

[0186] Step S704: Display the screen composed of the background image and the filled card 11 and the filled card 12 to generate the first interface.

[0187] In some embodiments, after determining the proportion of the same material image to be filled in card 11 and card 12 and the cropped image area, the mobile phone 100 can crop the image to obtain the filling element and fill it into card 11 and card 12.

[0188] For example, after determining that the source image P10 will be reduced by a factor of 1.5, the mobile phone 100 will fill in part of the content of the main object 1 within it. Figure 2E In the four-square grid at the bottom right corner of card 11 shown; after reducing the source image P10 by a factor of 1.5, fill the grid with the image content of the main object 2. Figure 2E The four-square grid in the upper right corner of card 12 is shown. Furthermore, to improve the overall visual effect of the card and background image after filling, the orientation of the main objects 1 and 2 can be adjusted. For example, both main objects 1 and 2 can be flipped, and then part of the content of main object 1 can be filled into card 11, and main object 2 can be filled into card 12, thus obtaining the following result: Figure 8A The H5 screen shown is then rendered by phone 100 as its desktop background, displaying the following: Figure 8B The interface shown. It's not hard to see that, Figure 8B The background image and cards 11 and 12 on the desktop of the mobile phone 100 shown have a good overall picture effect, which helps to improve the user experience.

[0189] The following will be combined with the appendix Figure 9 This application provides a detailed description of an image processing method based on some embodiments. The execution subject of each step can be the aforementioned mobile phone 100, or any electronic device with image processing and display functions. For ease of description, the following examples will use the mobile phone 100 as the execution subject for each step. Figure 9 The flowchart of the image processing method shown is introduced, such as... Figure 9 As shown, the image processing method provided in this application includes:

[0190] Step S901: Obtain at least one fill element from at least one source image.

[0191] In some embodiments, multiple screen elements constituting the interface to be displayed, and material images used to fill each screen element, can be determined first. For example, the interface to be displayed is the desktop of mobile phone 100, and the screen elements are... Figure 2A The cards 11 and 12, and the desktop background 13 shown are examples. The source images can be multiple source images from the installation package downloaded by the mobile phone 100 from the server 200, or one or more images stored locally on the mobile phone 100.

[0192] Then, by changing the pixel values ​​of each of the aforementioned first screen elements, multiple first screens are obtained. The first screen elements can be cards other than the desktop background. For example, in some embodiments, the first screen elements are cards 11 and 12. In some embodiments, to simplify calculations, each first screen element can be divided into a 3x3 grid. By changing the pixel values ​​of each grid, multiple first screens composed of the various first screen elements obtained after pixel value changes and a screen background are obtained. For example, Figure 2E The images shown are H2 and H3.

[0193] Then, based on the geometric center and centroid of each first screen, the target filling area of ​​each first screen element is determined. The same method as in step S406 above can be used to determine the target filling area of ​​each first screen element; for example, the target filling area of ​​card 11 is determined as follows. Figure 2E The four grid areas in the lower right corner of card 11 shown, and the target filling area of ​​card 12 are... Figure 2E The four squares in the upper right corner of card 12 are shown. For the specific calculation method, please refer to the relevant description in S406, which will not be repeated here.

[0194] Then, based on preset filtering rules, the target scaling ratio for filling each material image into the corresponding first screen element is selected, as well as the target content (i.e., the filling element) for each material image to be filled into the first screen element corresponding to each material image under the target scaling ratio. For example, using and Figure 4 The same calculation method is used in step S407. Based on the highest overlap score of card 12 with image P10 at a certain scaling ratio, the scaling ratio of image P10 when filling card 12 is determined, as well as the overlap with card 12 at that scaling ratio. This determines how to crop image P10 at that scaling ratio, thereby obtaining the corresponding filling element and filling it into card 12. The specific scoring calculation method can be found in the relevant description of step S407 above, and will not be repeated here.

[0195] Then, the target content of each source image at the target scaling ratio is filled into each first screen element to obtain multiple second screen elements. For example, in some embodiments, the mobile phone 100 determines that after scaling down the source image P20 by a factor of 1.5, the image content of the building area 1 therein is filled into... Figure 2E In the four-square grid at the bottom right corner of card 11 shown; after reducing the source image P30 by a factor of 3, crop out the image content of area 2 (the cake area) and use it as the fill element to fill the grid. Figure 2E In the four-square grid in the upper right corner of card 12 shown, thus obtaining the following... Figure 2EThe filled card 11 and the card 12 are shown.

[0196] Step S902: generating a first interface according to the background image and the at least one filling element; wherein the first interface includes the background image and the at least one card located above the background image, the at least one card covers a partial area of the background image, and the at least one card is filled with the at least one filling element; one of the at least one filling element is used to fill one of the at least one card; wherein the first barycenter is the barycenter of a picture formed by the background image and the at least one card in the first interface; and a distance between the first barycenter and a geometric center of the first interface is less than the first distance threshold.

[0197] For example, in some embodiments, the mobile phone 100 displays a picture H4 composed of the filled card 11, the card 12, and the background image P10.

[0198] The embodiments of the present application further provide a computer readable storage medium, which includes computer instructions, when the computer instructions are run on the above-mentioned electronic device, cause the electronic device to perform each function or step executed by the mobile phone 100 in the above-mentioned method embodiments.

[0199] The embodiments of the present application further provide a computer program product, when the computer program product is run on a computer, cause the computer to perform each function or step executed by the mobile phone 100 in the above-mentioned method embodiments.

[0200] Embodiments of the mechanisms disclosed herein can be implemented in hardware, software, firmware, or any combination thereof. Embodiments of the mechanisms disclosed herein can be implemented as computer programs or program code on programmable systems that can include at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0201] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices, in known fashion. For purposes of this application, a processing system includes any system that has a processor, such as, for example; a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0202] The program code can be implemented in a high level of programming language or a object-oriented programming language to communicate with a processing system. In the event that it is desired, the program code can be implemented using an assembly language or a machine language. In fact, the mechanisms described in this application are not limited in scope to any particular programming language. In any case, the language can be a compiled or interpreted language.

[0203] In some cases, the disclosed embodiments can be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments can also be implemented as instructions carried by or stored on a transitory or non-transitory machine-readable (e.g., computer-readable) medium, which can be read and executed by one or more processors. For example, the instructions can be distributed over the network or by other computer readable media. Thus, a machine-readable medium can include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including without limitation floppy disks, optical disks, optical disks, CD-ROMs, magnetic cassettes, read-only memories (ROMs), random access memories (RAMs), erasable programmable read only memories (EPROMs), electrically erasable programmable read only memories (EEPROMs), magnetic or optical cards, flash memory, or any other suitable device for storing or transmitting this information in a form that can be read by a machine. Thus, a machine-readable medium includes any suitable medium incident to a machine (e.g., a computer) that is suitable for storing or transmitting this information in a form that can be read by a machine.

[0204] In the drawings, some of the structural or methodological features can be shown in particular arrangements and / or orders. However, it should be understood that such particular arrangements and / or orders can not be required. Instead, these features can be arranged in a different manner and / or order than shown in the illustrative drawings, in some embodiments. Additionally, inclusion of structural or methodological features in a particular figure is not meant to imply that such features are required in all embodiments, and these features can not be included or can be combined with other features in some embodiments.

[0205] It should be noted that each unit / module mentioned in each device embodiment of the present application is a logical unit / module, and in physical, one logical unit / module can be a physical unit / module, or a part of a physical unit / module, or be realized in a combination of multiple physical unit / modules, and the physical realization of the logical unit / module itself is not the most important, and the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce the units / modules that are not closely related to solving the technical problems proposed in the present application, which does not mean that the above-mentioned device embodiments do not have other units / modules.

[0206] It should be noted that in the examples and descriptions of the present patent, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including one" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0207] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it should be understood by those skilled in the art that various changes in form and details can be made therein without departing from the spirit and scope of the present application.

Claims

1. An image processing method applied to electronic devices, characterized in that, include: Obtain at least one fill element from at least one source image; A first interface is generated based on the background image and the at least one fill element; The first interface includes the background image and at least one card located on the background image. The at least one card covers a portion of the background image and is filled with at least one fill element. One of the at least one fill element is used to fill one of the at least one cards. Wherein, the first center of gravity is the center of gravity of the image formed by the background image and the at least one card in the first interface; the distance between the first center of gravity and the geometric center of the first interface is less than a first distance threshold.

2. The method according to claim 1, characterized in that, The point symmetrical about the geometric center of the background image about the first interface is the centroid of the background image. The distance between the centroid of the background image and the centroid of the at least one card is less than a second distance threshold.

3. The method according to claim 1, characterized in that, Each of the at least one fill element is filled in the target fill area of ​​the card corresponding to each fill element in the first interface; The second center of gravity is the center of gravity of the image formed by each adjustment card corresponding to each card and the background image; The target filling area of ​​each card is determined based on the positional relationship between the second centroid and the geometric center of the first interface. The adjustment cards are obtained by assigning various pixel values ​​to each card.

4. The method according to claim 3, characterized in that, The third center of gravity is the overall center of gravity of each adjustment card. The overall center of gravity of each adjustment card is the center of gravity of the image composed of each adjustment card. The center of gravity of the image composed of each adjustment card represents the average position of the pixel value distribution of each pixel in the image composed of each adjustment card. The target filling area of ​​each card is determined based on the positional relationship between the third centroid and the symmetrical centroid of the background image.

5. The method according to claim 3, characterized in that, Each fill element in the at least one source image is determined in the following manner: Move the cards corresponding to each fill element within the target scaling range of each fill element in the source image at each scaling ratio; The filling elements are determined based on the overlap between the cards corresponding to each filling element and the source images at each scaling ratio corresponding to each filling element. The target scaling range for each fill element is determined based on the target fill area of ​​the card corresponding to each fill element.

6. The method according to any one of claims 1 to 5, characterized in that, The coordinates of the center of gravity of the image can be obtained using the following formula: Where, x c Let y be the coordinate of the center of gravity in the horizontal direction. c Let p be the coordinate of the centroid in the vertical direction. i Let x be the grayscale value of the i-th pixel in the image. i Let y be the horizontal coordinate of the i-th pixel in the image. i Let be the vertical coordinates of the i-th pixel in the image.

7. The method according to any one of claims 1 to 5, characterized in that, The at least one source image is a source image selected from multiple source images that meet the preset color filtering conditions.

8. The method according to any one of claims 1 to 5, characterized in that, The at least one fill element obtained from the at least one source image is at least a portion of at least one salient object in the at least one source image, the salient object being used to represent an object in the source image that attracts the user's focus of interest.

9. The method according to claim 8, characterized in that, The method further includes: If it is determined that multiple fill elements need to be selected from the same source image to fill multiple cards in the first interface, different prominent objects are selected from the same source image and used as fill elements to fill multiple cards in the first interface.

10. An image processing method applied to an electronic device, characterized in that, include: Generate a preset target image, the preset target image including a background image and at least one element to be filled that covers a portion of the background image; Adjust the pixel values ​​of each pixel of each element to be filled, and calculate the centroid of multiple preset target images and the geometric center of the preset target images after the pixel values ​​are adjusted; Based on the positional relationship between the centroids of the plurality of preset target images and the geometric center of the preset target images, select the centroid that satisfies the preset conditions from the plurality of centroids; Based on the preset target image corresponding to the selected center of gravity, determine the position of the target filling area for each element to be filled. Based on the determined position of the target filling region of each element to be filled, the filling image to be filled into the target filling region of each element to be filled is determined; The target area of ​​each element to be filled is filled using the corresponding fill image to obtain the target image.

11. The method according to claim 10, characterized in that, The step of adjusting the pixel values ​​of each pixel of each element to be filled, and calculating the centroid of multiple preset target images obtained after pixel value adjustment and the geometric center of the preset target images, includes: Each element to be filled is divided into multiple sub-regions, where each sub-region contains multiple pixels. Each sub-region of each element to be filled is selected with pixel value within a preset pixel value range to obtain multiple preset target images after pixel value adjustment, wherein the pixel values ​​of each pixel in the same sub-region are the same. Based on the pixel values ​​of each sub-region of each element to be filled after pixel value adjustment, and the position coordinates of each sub-region, the center of gravity of multiple preset target images obtained after pixel value adjustment is calculated, and Based on the position coordinates of each pixel in the preset target image, the geometric center of the preset target image is calculated.

12. The method according to claim 10, characterized in that, The step of determining the position of the target filling area for each element to be filled based on the preset target image corresponding to the selected center of gravity includes: If it is determined that there is only one preset target image corresponding to the selected center of gravity, then the pixel value of each pixel of each element to be filled in the preset target image is determined. Based on the pixel values ​​of each pixel of each element to be filled, the position of the target filling area of ​​each element to be filled is determined. When it is determined that there are multiple preset target images corresponding to the selected centroids, the symmetrical point of the centroid of the background image of each preset target image with respect to the geometric center of each preset target image is determined, as well as the overall centroid of multiple elements to be filled in each preset target image. The overall centroid of the multiple elements to be filled represents the average position of the pixel value distribution of each pixel in the multiple elements to be filled. Based on the positional relationship between the symmetrical points and the overall center of gravity of the multiple preset target images, an overall center of gravity that meets preset conditions is selected from the multiple overall centers of gravity. Based on the pixel values ​​of each pixel of each element to be filled in the preset target image corresponding to the selected overall center of gravity, the position of the target filling area of ​​each element to be filled is determined.

13. The method according to claim 10, characterized in that, The step of determining the fill image to be filled into the target fill area of ​​each element based on the determined position of the target fill area includes: Based on the determined position of the target filling region of each element to be filled, determine the target scaling range of the source image corresponding to the filling image to be filled into the target filling region of each element to be filled. According to the preset scoring rules, the overlap between each element to be filled and the material images at each scaling ratio within the corresponding target scaling range is determined when each element to be filled is moved within the corresponding target scaling range. Based on the overlap, the target scaling ratio of the source images corresponding to each element to be filled is determined, and the filling image to be filled into the target filling area of ​​the corresponding element to be filled at the target scaling ratio is determined.

14. The method according to claim 13, characterized in that, The fill image in each source image includes at least a portion of the salient object in each source image.

15. The method according to claim 13, characterized in that, The coordinates of the center of gravity of the image can be obtained using the following formula: Where, x c Let y be the coordinate of the center of gravity of the image in the horizontal direction. c Let p be the coordinate of the center of gravity of the image in the vertical direction. i Let x be the grayscale value of the i-th pixel in the image. i Let y be the horizontal coordinate of the i-th pixel in the image. i Let be the vertical coordinates of the i-th pixel in the image.

16. The method according to any one of claims 10 to 15, characterized in that, Also includes: Obtain multiple source images to be selected and color filtering criteria; Select the image that meets the color filtering criteria from the multiple images to be selected.

17. The method according to claim 16, characterized in that, Also includes: When it is determined that multiple elements to be filled in the target image need to be selected from the same source image, different prominent objects are selected from the same source image and used as fill images to fill the multiple elements to be filled in the target image.

18. The method according to any one of claims 10 to 15, characterized in that, The target screen is the desktop of the electronic device. The element to be filled is the service card on the desktop of the electronic device.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the image processing method according to any one of claims 1-18.

20. A computer program product, characterized in that, The computer program product includes instructions for implementing the image processing method as described in any one of claims 1-18.

21. An electronic device, characterized in that, include: Memory, used to store instructions, and One or more processors are configured to execute instructions stored in the memory, which, when executed by the one or more processors, cause the electronic device to perform the image processing method as described in any one of claims 1-18.

Images