Depth-of-field wallpaper generation method, electronic equipment and computer readable storage medium

Abstract

The invention relates to a depth-of-field wallpaper generation method, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring a wallpaper image and a control; determining a foreground main body area in the wallpaper image; and in response to the condition that the ratio of the overlapping area of the foreground main body area and the control to the area of the control does not meet the field depth wallpaper generation condition, moving the control to a target position in the wallpaper image in a first direction of pointing a first centroid of the foreground main body area to a second centroid of the control or in a second direction of pointing the second centroid to the first centroid, non-depth-of-field wallpaper is generated, and the target position is the position of the control in the wallpaper image when the ratio meets the depth-of-field wallpaper generation condition; and superposing the foreground main body corresponding to the foreground main body area on the non-depth-of-field wallpaper to generate the depth-of-field wallpaper. Through the method and the device, the technical problem that in order to realize the depth-of-field wallpaper, a user needs to continuously adjust the wallpaper image until the generation condition of the depth-of-field wallpaper is met, and the user operation is increased in the related technology is solved.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a method for generating a depth of field wallpaper, an electronic device, and a computer-readable storage medium. Background Art

[0002] Depth of field wallpaper adds a special visual effect to ordinary static wallpaper, which makes it show depth and layering when the field of view or device angle changes. This wallpaper can simulate a three-dimensional visual experience, making the content on the screen look more vivid and three-dimensional. However, there are some problems in the existing depth of field wallpaper implementation. For example, the wallpaper image subject selected by the user is strongly related to the location of the control, and the depth of field wallpaper generation conditions must be met before the depth of field wallpaper can be generated. In actual applications, users need to constantly adjust the image (crop or slide the image position, etc.) until the depth of field wallpaper generation conditions are met before the depth of field wallpaper can be generated, which increases user operations. Summary of the invention

[0003] According to one aspect of an embodiment of the present application, a method for generating a depth of field wallpaper is provided, including: acquiring a wallpaper image and a control; determining a foreground subject area in the wallpaper image; in response to a ratio of an overlapping area of ​​the foreground subject area and the control to an area of ​​the control not satisfying a depth of field wallpaper generation condition, moving the control to a target position in the wallpaper image in a first direction from a first centroid of the foreground subject area to a second centroid of the control or in a second direction from the second centroid to the first centroid, to generate a non-depth of field wallpaper, wherein the target position is a position of the control in the wallpaper image when the ratio satisfies the depth of field wallpaper generation condition; and superimposing the foreground subject corresponding to the foreground subject area on the non-depth of field wallpaper to generate a depth of field wallpaper.

[0004] Optionally, the depth of field wallpaper generation condition is that the ratio is greater than or equal to a second ratio threshold and less than or equal to a first ratio threshold, wherein the first ratio threshold is the sum of a specified ratio threshold and an error value, and the second ratio threshold is the difference between the specified ratio threshold and the error value.

[0005] Optionally, determining the foreground main area in the wallpaper image includes: inputting the processed wallpaper image into a trained machine learning model to obtain the probability of each pixel in the wallpaper image belonging to a category; for each pixel, taking the category corresponding to the maximum probability among the probabilities of the category to which the pixel belongs as the category of the pixel; and determining the foreground main area in the wallpaper image based on the category of each pixel.

[0006] Optionally, obtaining the first center of mass of the foreground main area includes: calculating the binary image moment of the mask area corresponding to the foreground main area; calculating the first center of mass based on the binary image moment; obtaining the second center of mass of the control includes: based on the geometric shape of the control, determining the center position of the geometric shape as the second center of mass.

[0007] Optionally, the control is moved to a target position in the wallpaper image in a first direction toward the first centroid of the foreground main area pointing to the second centroid of the control or in a second direction toward the second centroid pointing to the first centroid, including: when the ratio is greater than the first ratio threshold, moving the control to the target position in the wallpaper image in the first direction; when the ratio is less than the second ratio threshold, moving the control to the target position in the wallpaper image in the second direction.

[0008] Optionally, moving the control in the first direction to a target position in the wallpaper image, or moving the control in the second direction to a target position in the wallpaper image, includes: repeatedly performing the following steps: obtaining an initial position of the control, moving the control with a moving step length of half the distance between the first center of mass and the second center of mass, and calculating the ratio of the control after moving from the initial position to the current position; in response to the ratio after the movement not satisfying the depth of field wallpaper generation condition, setting the current position to the initial position and updating the position of the second center of mass after the movement; or in response to the ratio after the movement satisfying the depth of field wallpaper generation condition, ending the loop step.

[0009] Optionally, moving the control in the first direction to a target position in the wallpaper image, or moving the control in the second direction to a target position in the wallpaper image, includes: when the ratio is greater than the first ratio threshold, repeatedly performing the following steps: obtaining the initial position of the control, moving the control in the first direction with the distance between the first center of mass and the second center of mass as the moving step of the control, and calculating the ratio of the control after moving from the initial position to the current position; in response to the ratio after the movement being greater than the first ratio threshold, setting the current position after moving in the first direction to the initial position, and updating the position of the second center of mass after the movement; or, in response to the ratio after the movement being less than the second ratio threshold, obtaining the current position of the control, moving the control in the second direction with half the distance between the first center of mass and the second center of mass as the moving step of the control, setting the current position after moving in the second direction to the initial position, and updating the position of the second center of mass after the movement; or, in response to the ratio after the movement satisfying the depth of field wallpaper generation condition, ending the loop step.

[0010] Optionally, moving the control in the first direction to a target position in the wallpaper image, or moving the control in the second direction to a target position in the wallpaper image, includes: when the ratio is less than the second ratio threshold, repeatedly performing the following steps: obtaining the initial position of the control, moving the control in the second direction with half of the distance between the first center of mass and the second center of mass as the moving step of the control, and calculating the ratio of the control after moving from the initial position to the current position; in response to the ratio after the movement being less than the second ratio threshold, setting the current position after moving in the second direction as the initial position, and updating the position of the second center of mass after the movement; or, in response to the ratio after the movement being greater than the first ratio threshold, obtaining the current position of the control, moving the control in the first direction with half of the distance between the first center of mass and the second center of mass as the moving step of the control, setting the current position after moving in the first direction as the initial position, and updating the position of the second center of mass after the movement; or, in response to the ratio after the movement satisfying the depth of field wallpaper generation condition, ending the loop step.

[0011] According to another aspect of the embodiments of the present application, an electronic device is provided, including: a processor, and a memory storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the method in any of the above embodiments.

[0012] According to another aspect of the embodiments of the present application, a non-transitory machine-readable medium storing computer instructions is also provided, where the computer instructions are used to enable a computer to execute the method in any of the above embodiments.

[0013] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, which, when executed by a processor of a computer, causes the computer to execute the method in any of the above embodiments.

[0014] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work.

[0016] Figure 1 is a flow chart of a method for generating a depth of field wallpaper according to an embodiment of the present application;

[0017] Figure 2 is a flow chart of a method for acquiring a foreground subject area according to an embodiment of the present application;

[0018] Figure 3a is a schematic diagram of the positions of a control and a foreground subject area according to an embodiment of the present application;

[0019] Figure 3b is a schematic diagram of the positions of another control and a foreground subject area according to an embodiment of the present application;

[0020] Figure 3c is a schematic diagram of the positions of another control and a foreground main area according to an embodiment of the present application;

[0021] Figure 4a is a schematic diagram of control movement according to an embodiment of the present application;

[0022] Figure 4b is another schematic diagram of control movement according to an embodiment of the present application;

[0023] Figure 4c is another schematic diagram of control movement according to an embodiment of the present application;

[0024] Figure 5 is a schematic structural diagram of a device for generating a depth of field wallpaper according to an embodiment of the present application;

[0025] Figure 6 Schematic diagram of the structure of the electronic device of this embodiment. DETAILED DESCRIPTION

[0026] Embodiments of the present embodiment will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present embodiment are shown in the accompanying drawings, it should be understood that the present embodiment can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, which are instead provided for a more thorough and complete understanding of the present embodiment. It should be understood that the drawings and embodiments of the present embodiment are only for exemplary purposes and are not intended to limit the scope of protection of the present embodiment.

[0027] In existing wallpapers, controls (such as date, time, etc.) are generally overlaid on the wallpaper image. In order to achieve the depth of field effect of controls on the wallpaper image (i.e., controls are "inserted" between the main image and the image background), the deep learning model reasoning method can be used to obtain the main area of ​​the image, and then the depth of field wallpaper can be generated based on the overlap between the control area and the main image area, and the ratio of the overlap area to the control area meets certain threshold requirements (for example, the ratio of the area of ​​the overlap area to the area of ​​the control area does not exceed 20% or other appropriate ratios, because too much coverage of the image main body on the control will cause the user to be unable to recognize the information to be displayed by the control).

[0028] In actual applications, users need to constantly adjust images (crop or slide image positions, etc.) or controls until the aforementioned depth of field wallpaper generation conditions are met before depth of field wallpapers can be generated, which increases user operations. In order to solve this problem, a relevant solution is provided in the embodiments of the present application, which is described in detail below.

[0029] The application scenarios of the embodiments of the present application include but are not limited to the following scenarios:

[0030] 1. Smart devices. (1) Smartphones and tablets. Dynamic wallpapers: Dynamic wallpapers with depth effects change as the device is tilted and moved, providing a more vivid visual experience. Lock screen interface: Make the lock screen wallpaper three-dimensional to enhance the user experience. (2) Desktop computers and laptops. Desktop wallpapers: Display wallpapers with dynamic depth effects on large screens to add visual layers. Screensavers: Create dynamic screensavers with depth effects, especially suitable for displaying company brands or artwork.

[0031] 2. Interactive display. (1) Museums and exhibitions. Interactive display screens: Through the depth of field effect, the exhibits are more vivid, guiding the audience to interact and immerse themselves in the display content. (2) Retail and shopping malls. Digital billboards: Advertisements with depth of field effects can attract more attention and provide more attractive advertising displays.

[0032] 3. Games and VR. (1) Game interface. Background and scene: Use depth of field effect in games to enhance the realism and immersion of scenes and improve the player experience. UI design: Apply depth of field effect in game menu and interface design to make the interactive interface more vivid and realistic. (2) VR and AR. Immersive experience: In VR / AR applications, depth of field effect can significantly increase the realism and interactivity of the environment. Navigation and instructions: Use depth of field effect to create navigation paths or annotations to enhance the user's spatial perception ability.

[0033] 4. Education and training. (1) Online education platform. Virtual classroom: The depth of field effect makes the virtual classroom more realistic, increasing students' participation and concentration. Educational games: Applying the depth of field effect in educational games makes the learning process more interesting and interactive. (2) Professional training. Simulated training environment: For example, flight and driving simulators can enhance the realism of the training environment through the depth of field effect, thereby improving learning efficiency and safety.

[0034] 5. Medical and health. (1) Telemedicine. Virtual consultation room: Use depth of field effects to create a more realistic virtual consultation environment and improve the communication effect between doctors and patients. (2) Health applications. Relaxation training: Use wallpapers or audio-visual content with depth of field effects to help users relax and meditate, and improve mental health.

[0035] 6. Art and design. (1) Digital artworks. Interactive art exhibitions: Make artworks more layered and interactive through depth of field effects. Creative design: Help designers create works with greater visual impact and depth. (2) Architecture and interior design. Virtual model homes: Apply depth of field wallpapers in virtual model homes to help customers better experience the space layout and design effects.

[0036] For example, an embodiment of the present application provides a method for generating a depth of field wallpaper in the above application scenario. Figure 1 is a flow chart of a method for generating a depth of field wallpaper according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0037] Step S102, obtaining wallpaper images and controls.

[0038] It should be noted that the size of the wallpaper image is usually the same as the size of the electronic device's window (i.e., the window that the electronic device has display capabilities). For example, if the size of an electronic device is 1920*1080, then the size of the wallpaper image displayed on the electronic device is at least 1920*1080. It can be considered that when the wallpaper image is displayed in the window of the electronic device, it completely covers the window area of ​​the electronic device. The above-mentioned control can be a transparent image, such as an image that only includes the foreground subject but not the background. When it is covered on the wallpaper image to form a wallpaper, it is a non-depth of field wallpaper. The position attribute of the control (i.e., its placement area on the wallpaper image) is known, so the relative position relationship between the wallpaper image and the control can be determined by the height attribute of the wallpaper image (e.g., 1080), the width attribute (e.g., 1920), and the coordinates of the control in the window coordinate system (e.g., the upper left corner coordinates and the control length and width, or the upper left corner coordinates and the lower right corner coordinates).

[0039] It can be understood that the above-mentioned control can be a button that provides one or more functions for the electronic device to enable / disable / select, or a visual pattern that is related to the function but does not provide touch access, such as a control that displays the date and time, or a non-function-related visual pattern, such as a person, etc. Optionally, the above-mentioned control types that provide one or more functions for the electronic device to enable / disable / select can also include but are not limited to: basic controls (enable / disable switch (allowing users to quickly enable or disable the depth of field wallpaper function), select wallpaper (provide a gallery or browser from which users can select preset depth of field wallpapers or upload custom pictures)), interactive controls (sensitivity adjustment, touch interaction settings), visual effect controls (depth effect adjustment, light and shadow effects), performance and battery optimization space (refresh rate, low power mode), custom and advanced controls (layer management, color and filter, animation effects), sound effect controls (background sound effects, sound effect synchronization), save and share controls (save and backup, share functions), etc.

[0040] Step S104, determining the foreground subject area in the wallpaper image.

[0041] For example, the processed wallpaper image is input into the trained machine learning model to obtain the probability of each pixel in the wallpaper image belonging to a category; for each pixel, the category corresponding to the maximum probability among the probabilities of the category to which the pixel belongs is taken as the category of the pixel; based on the category of each pixel, the foreground subject area in the wallpaper image is determined.

[0042] Specifically, the foreground main area of ​​the wallpaper image can be calculated according to the inference result of the model by using a deep learning model inference method. The deep learning model used can be an image segmentation model. For example, in the actual implementation process, image segmentation models such as u-net, SegNet, RefineNet, PSPNet, etc. can be selected. Figure 2 is a flow chart of a method for obtaining a foreground subject area according to an embodiment of the present application. Figure 2 As shown, including:

[0043] Step S201, obtain the wallpaper image. In the Android mobile system, the wallpaper image usually obtained is in a bitmap format (for example, the wallpaper image is obtained by selecting from the album, or by converting video stream data, etc.). Step S202, wallpaper image pre-processing. The wallpaper image is pre-processed to adapt to the input of the model. Generally, the size of the wallpaper image is adjusted first (the resolution of the adjusted image is k*k), and then the adjusted wallpaper image is quantized and converted into a tensor that adapts to the model input. Step S203, model reasoning. The tensor obtained by quantizing the wallpaper image in step S202 is sent to the model for forward reasoning calculation to obtain the model reasoning result. Step S204, image post-processing. The image is post-processed according to the output result of the model. Usually the output tensor dimension of the image segmentation model is n*k*k*c, where n is the number of input images (generally 1, in model training and batch tasks, n>1). c is the number of subject categories supported by the image segmentation model + 1 (+1 because unsupported categories are considered background). For example, the image segmentation model supports recognition of 5 categories, including date (further including time), person, dog, car, and tree, so c is 6. Each specific value of dimension c in the output tensor of n*k*k*c represents the probability of each pixel in each image (adjusted image) belonging to a category. In image post-processing, that is, in an image (adjusted image), each pixel corresponds to the probability of c categories, and the category corresponding to the maximum value of the c category probabilities is taken as the category of this pixel. Further, according to the category of each pixel, it is judged whether the pixel belongs to the foreground subject or the background. For example, if the category corresponding to the pixel is a recognizable category (such as date), the pixel is used as the foreground subject pixel; for example, if the category corresponding to the pixel is an unrecognizable category, the pixel is a background area pixel. Thus, the foreground subject area of ​​the wallpaper image can be calculated through the categories of all pixels. For example, in actual implementation, it can be represented by a mask. The mask often uses an 8-bit bitmap. The pixel value of the subject area is 255, which is displayed as white, and the pixel value of the background area is 0, which is displayed as black.

[0044] In another example, the wallpaper image can also be segmented using a color histogram or brightness value. For example, if the foreground main area and the background of the wallpaper image have significant differences in color or brightness, algorithms such as Canny edge detection can be used to find the significant difference edges in the image to distinguish the foreground from the background. By selecting one or more seed points (pixels inside the foreground main area), and then expanding and connecting these areas based on color or texture similarity, a complete foreground main area is formed.

[0045] Step S106, in response to the ratio of the overlapping area of ​​the foreground subject area and the control to the area of ​​the control not satisfying the depth of field wallpaper generation condition, the control is moved to a target position in the wallpaper image in a first direction toward the first center of mass of the foreground subject area pointing to the second center of mass of the control or in a second direction toward the second center of mass pointing to the first center of mass, to generate a non-depth of field wallpaper, wherein the target position is the position of the control in the wallpaper image when the ratio satisfies the depth of field wallpaper generation condition.

[0046] It should be noted that, in the embodiment of the present application, the depth of field wallpaper generation condition may be that the ratio is greater than or equal to the second ratio threshold, and less than or equal to the first ratio threshold, wherein the first ratio threshold is the sum of the specified ratio threshold and the error value, and the second ratio threshold is the difference between the specified ratio threshold and the error value. For example, the ratio is denoted as λ, the allowable error is denoted as ε, and the specified ratio threshold is γ, then the second ratio threshold is γ-ε, and the first ratio threshold is ε+γ. That is, the depth of field wallpaper generation condition may be γ-ε≤λ≤ε+γ. Optionally, in the embodiment of the present application, the γ may be 20%, and the ε may be 1%. Therefore, there are two types of conditions that do not meet the depth of field wallpaper generation conditions, namely, λ<γ-ε, or λ>ε+γ. For example Figure 3a It means that the control has no overlap with the foreground main area. At this time, λ is 0, and it can be considered that λ<γ-ε. Figure 3b It means that the control overlaps slightly with the foreground main area, and λ<γ-ε. Figure 3c It means that the control overlaps with the foreground main area in a large area, and λ>ε+γ.

[0047] It can be understood that the above-mentioned specified ratio threshold γ can also be other suitable values, such as 15% or 25% or other, and the above-mentioned allowable error ε can also be other suitable values, such as 0.5% or other. Exemplarily, when the depth of field wallpaper generation condition is not met, the embodiment of the present application proposes a control movement method, wherein the control movement method may include:

[0048] S11, when the ratio is greater than the first ratio threshold, move the control toward the first direction to the target position in the wallpaper image; S12, when the ratio is less than the second ratio threshold, move the control toward the second direction to the target position in the wallpaper image.

[0049] Optionally, in the embodiment of the present application, the method of moving the control toward the first direction to the target position in the wallpaper image, or the method of moving the control toward the second direction to the target position in the wallpaper image may include:

[0050] Directly use the binary search method to calculate the adaptive position of the control;

[0051] When the initial distance between the first centroid and the second centroid is too close, the control is first set to face the first direction away from the foreground main area, and then the binary search method is used to calculate the adaptive position of the control; when the initial distance between the first centroid and the second centroid is too far, the binary search method is directly used to calculate the adaptive position of the control.

[0052] Step S108, superimposing the foreground subject corresponding to the foreground subject area on the non-depth-of-field wallpaper to generate a depth-of-field wallpaper.

[0053] Through the above steps S102 to S106, a wallpaper image and a control are obtained; a foreground subject area in the wallpaper image is determined; in response to the ratio of the overlapping area of ​​the foreground subject area and the control to the area of ​​the control not meeting the depth of field wallpaper generation condition, the control is moved to a target position in the wallpaper image in a first direction from the first centroid of the foreground subject area to the second centroid of the control or in a second direction from the second centroid to the first centroid, to generate a non-depth of field wallpaper, wherein the target position is the position of the control in the wallpaper image when the ratio meets the depth of field wallpaper generation condition; and a foreground subject corresponding to the foreground subject area is superimposed on the non-depth of field wallpaper to generate a depth of field wallpaper. That is, in the embodiment of the present application, after the user selects the wallpaper image, the control can be adaptively moved according to the position of the foreground subject area in the wallpaper image, and a depth of field wallpaper is formed when the depth of field wallpaper generation condition is met, without the user having to adjust the wallpaper image, thereby reducing user operations, thereby solving the technical problem in the related art that in order to achieve the depth of field wallpaper, the user needs to continuously adjust the wallpaper image until the depth of field wallpaper generation condition is met, which increases the user operations.

[0054] In a possible implementation, obtaining the first centroid of the foreground subject area may include:

[0055] S21, calculating the binary image moment of the mask area corresponding to the foreground main area;

[0056] S22, calculating the first centroid according to the binary image moment. Specifically, the image mask itself is a single-channel binary image, and the binary image moment of the image mask can be calculated by the moments() function of openCV, and then the centroid of the image mask, that is, the first centroid, is calculated using the binary image moment.

[0057] In a possible implementation, obtaining the second centroid of the control includes:

[0058] S23, according to the geometric shape of the control, determine the center position of the geometric shape as the second centroid. For example, assuming that the geometric shape of the control is a rectangle, the center position of the rectangular area can be used as the second centroid.

[0059] Through the above steps S21 to S23, the first center of mass and the second center of mass can be accurately located, thereby ensuring the correct offset during the movement of the control.

[0060] At least two methods of calculating the adaptive position of controls are mentioned above. The following describes these two methods in detail.

[0061] Regarding directly using the binary search method to calculate the adaptive position of the control, the embodiment of the present application proposes moving the control toward the first direction to the target position in the wallpaper image, or moving the control toward the second direction to the target position in the wallpaper image, including:

[0062] S31, repeatedly perform the following steps: obtain the initial position of the control, move the control with half the distance between the first center of mass and the second center of mass as the moving step of the control, and calculate the above ratio after the control moves from the initial position to the current position; in response to the above ratio after the movement not meeting the depth of field wallpaper generation condition, set the current position to the above initial position, update the position of the second center of mass after the movement; or in response to the above ratio after the movement meeting the depth of field wallpaper generation condition, end the loop step. Through the above step S31, the binary search method is used to calculate the adaptive position of the control, which reduces the number of calculations of the above ratio and further improves the calculation efficiency.

[0063] The following is an example of directly using the binary search method to calculate the adaptive position of the control, which can include the following steps:

[0064] Step S41, calculating the ratio λ of the overlapping area between the control and the foreground main area to the control area.

[0065] Step S42, determining whether the ratio λ in step S41 satisfies the depth of field wallpaper generation condition, that is, γ-ε≤λ≤

[0066] ε+γ, where ε is the allowable error, γ is the specified ratio threshold, and the values ​​of ε and γ can refer to the above description. If satisfied, proceed to step S46. If not satisfied, proceed to step S43.

[0067] Step S43: If the ratio λ satisfies λ<γ-ε, then the process proceeds to step S44. If the ratio λ satisfies λ>ε+γ, then the process proceeds to step S45.

[0068] Step S44, corresponding to sub-graph (4a) or sub-graph (4b) in FIG4, the control will move along the direction (corresponding to the above-mentioned second direction) from the center of mass of the control (corresponding to the above-mentioned second center of mass) toward the center of mass of the foreground main area (corresponding to the above-mentioned first center of mass), and the moving length is half of the distance between the center of mass of the control and the center of mass of the foreground main area, and then the position of the center of mass of the control is updated to the position of the center of mass of the control after the movement. For example, a horizontal offset can be added to the horizontal coordinate of the coordinate corresponding to the center of mass of the control before the movement, and a vertical offset can be added to the vertical coordinate, and then step S41 is entered.

[0069] Step S45, corresponding to the sub-figure (4c) in Figure 4, the control will move in the direction (corresponding to the above-mentioned first direction) from the centroid of the foreground main area (corresponding to the above-mentioned first centroid) to the centroid of the control (corresponding to the above-mentioned second centroid), and the moving length is half of the distance between the centroid of the foreground main area and the centroid of the control. Then the position of the centroid of the control is updated to the position of the centroid of the control after the move. For example, a horizontal offset can be added to the horizontal coordinate of the coordinate corresponding to the centroid of the control before the move, and a vertical offset can be added to the vertical coordinate, and then step S41 is entered.

[0070] Step S46, recording the position of the current control. For example, using the coordinates of the centroid of the current control and the size information of the current control (including length and width), calculate the coordinates of the upper left corner of the window of the electronic device.

[0071] When the initial distance between the first centroid and the second centroid is too close, the control is first set to move away from the foreground main area in the first direction, and then the binary search method is used to calculate the adaptive position of the control. When the initial distance between the first centroid and the second centroid is too far, the binary search method is directly used to calculate the adaptive position of the control. The embodiment of the present application proposes moving the control in the first direction to the target position in the wallpaper image, or moving the control in the second direction to the target position in the wallpaper image, including: S51, when the above ratio is greater than the above first ratio threshold, repeatedly performing the following steps: obtaining the initial position of the control, taking the distance between the first centroid and the second centroid as the moving step length of the control Move the control in the first direction, calculate the ratio after the control moves from the initial position to the current position; in response to the ratio after the movement being greater than the first ratio threshold, set the current position after moving toward the first direction to the initial position, and update the position of the second center of mass after the movement; or, in response to the ratio after the movement being less than the second ratio threshold, obtain the current position of the control, move the control in the second direction with half of the distance between the first center of mass and the second center of mass as the moving step of the control, set the current position after moving toward the second direction to the initial position, and update the position of the second center of mass after the movement; or, in response to the ratio after the movement satisfying the depth of field wallpaper generation condition, end the loop step.

[0072] S61, when the above ratio is less than the above second ratio threshold, repeat the following steps: obtain the initial position of the control, move the control toward the above second direction with half of the distance between the above first center of mass and the above second center of mass as the moving step of the control, and calculate the above ratio after the control moves from the initial position to the current position; in response to the ratio after the movement being less than the above second ratio threshold, set the current position after moving toward the second direction as the above initial position, and update the position of the second center of mass after the movement; or, in response to the ratio after the movement being greater than the above first ratio threshold, obtain the current position of the control, move the control toward the first direction with half of the distance between the above first center of mass and the above second center of mass as the moving step of the control, set the current position after moving toward the first direction as the initial position, and update the position of the second center of mass after the movement; or, in response to the ratio after the movement satisfying the depth of field wallpaper generation condition, end the loop step. The above process is illustrated below with reference to a specific example, which may include the following steps:

[0073] Step S71, calculating the ratio λ of the overlapping area between the control and the foreground main area to the control area.

[0074] Step S72: If λ in step S71 satisfies γ-ε≤λ≤ε+γ, where ε is the allowable error, γ is the specified ratio threshold, and the values ​​of ε and γ can be found in the above description, then proceed to step S75. If λ>ε+γ is satisfied, then proceed to step S73. If λ<γ-ε is satisfied, then proceed to step S74.

[0075] Step S73, corresponding to the sub-picture (4c) in FIG4, the control will move in the direction (corresponding to the first direction) from the centroid of the foreground main area (corresponding to the first centroid) to the centroid of the control (corresponding to the second centroid), and the moving length is the distance between the centroid of the foreground main area and the centroid of the control. Calculate the above ratio λ, if λ satisfies λ>ε+γ, continue to execute step S73 until λ does not satisfy λ>ε+γ, and then enter step S71.

[0076] Step S74, corresponding to sub-graph (4a) or sub-graph (4b) in FIG4, can use binary search to obtain the position of the control that meets the depth of field wallpaper generation condition. This step may include the following sub-steps, wherein, in this example, the center of mass of the current control (corresponding to the second center of mass) position may be marked as p c , the centroid position of the foreground main area (corresponding to the first centroid mentioned above) is marked as p o , the centroid position of the control that meets the depth of field wallpaper generation conditions is p d (It is understandable that d for the above target location):

[0077] Step S741, set the above p d For the above p c and the above p o It can be understood that in this step, the control is moved toward the second direction with half of the distance between the centroid of the foreground main area and the centroid of the control as the moving step length of the control.

[0078] Step S742, calculate the center of mass of the control at p d If γ-ε≤λ≤ε+γ is satisfied, then the process proceeds to step S75. If λ<γ-ε is satisfied, then the process proceeds to step S743. If λ>ε+γ is satisfied, then the process proceeds to step S744.

[0079] Step S743, set p c Equal to p d It can be understood that, in this step, the control center position is updated to the control center position after the movement, and then step S741 is continued.

[0080] Step S744, set p o Equal to p dIt can be understood that, in this step, the center of mass of the control is moved toward the first direction, and then step S741 is continued.

[0081] Step S75, recording the position of the current control. For example, using the coordinates of the centroid of the current control and the size information of the current control (including length and width), calculate the coordinates of the upper left corner of the window of the electronic device.

[0082] It should be noted that in the embodiment of the present application, during the adaptive adjustment of the control position, the coordinates of the control's center of mass (corresponding to the second center of mass mentioned above) can be used to calculate the movement process without actually moving the control. When the control's target position that meets the depth of field wallpaper generation conditions is determined, the control is moved to the target position to generate a non-depth of field wallpaper, and then the foreground subject corresponding to the foreground subject area is superimposed on the non-depth of field wallpaper to form a depth of field wallpaper.

[0083] In summary, the above-mentioned depth of field wallpaper generation method proposed in the embodiment of the present application is that after the user selects a wallpaper image, the control can be adaptively moved according to the position of the foreground main area in the wallpaper image, and when the depth of field wallpaper generation conditions are met, the depth of field wallpaper is formed to avoid manual adjustment by the user. In addition, the method proposed in the embodiment of the present application can make the overlapping area of ​​the control and the foreground main area account for the proportion of the control area close to the threshold, so as to better reflect the depth of field effect. In addition, in the process of adaptive movement of the control position, binary search is used to reduce the number of times the ratio is calculated in the adaptive movement, thereby improving the calculation efficiency.

[0084] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a method for generating a depth of field wallpaper. Figure 5 The figure is a structural block diagram of a device for generating a depth of field wallpaper according to an embodiment of the present application, including:

[0085] An acquisition module 52, used for acquiring wallpaper images and controls;

[0086] A determination module 54, used to determine a foreground subject area in the wallpaper image;

[0087] A moving module 56 is used for moving the control to a target position in the wallpaper image in a first direction from the first centroid of the foreground main area to the second centroid of the control or in a second direction from the second centroid to the first centroid in response to the ratio of the overlapping area of ​​the foreground main area and the control to the area of ​​the control not satisfying the depth of field wallpaper generation condition, so as to generate a non-depth of field wallpaper, wherein the target position is the position of the control in the wallpaper image when the ratio satisfies the depth of field wallpaper generation condition;

[0088] The generating module 58 is used to superimpose the foreground subject corresponding to the foreground subject area on the non-depth-of-field wallpaper to generate a depth-of-field wallpaper.

[0089] pass Figure 5 The device shown acquires a wallpaper image and a control; determines a foreground subject area in the wallpaper image; in response to the ratio of the overlapping area of ​​the foreground subject area and the control to the area of ​​the control not satisfying the depth of field wallpaper generation condition, moves the control to a target position in the wallpaper image in a first direction from the first centroid of the foreground subject area to the second centroid of the control or in a second direction from the second centroid to the first centroid, to generate a non-depth of field wallpaper, wherein the target position is the position of the control in the wallpaper image when the ratio satisfies the depth of field wallpaper generation condition; and superimposes the foreground subject corresponding to the foreground subject area on the non-depth of field wallpaper to generate a depth of field wallpaper. That is, in the embodiment of the present application, after the user selects the wallpaper image, the control can be adaptively moved according to the position of the foreground subject area in the wallpaper image, and when the depth of field wallpaper generation condition is met, a depth of field wallpaper is formed, without the user having to adjust the wallpaper image, thereby reducing user operations, thereby solving the technical problem in the related art that in order to realize the depth of field wallpaper, the user needs to continuously adjust the wallpaper image until the depth of field wallpaper generation condition is met, which increases user operations.

[0090] In one possible implementation, the depth of field wallpaper generation condition is that the ratio is greater than or equal to a second ratio threshold and less than or equal to a first ratio threshold, wherein the first ratio threshold is the sum of a specified ratio threshold and an error value, and the second ratio threshold is the difference between the specified ratio threshold and the error value.

[0091] The determination module 54 includes: a first determination unit, which is used to input the processed wallpaper image into the trained machine learning model to obtain the probability of each pixel in the wallpaper image belonging to a category; a setting unit, which is used to, for each pixel, take the category corresponding to the maximum probability among the probabilities of the category to which the pixel belongs as the category of the pixel; and a second determination unit, which is used to determine the foreground subject area in the wallpaper image based on the category of each pixel.

[0092] Optionally, the above-mentioned device is also used to obtain the first center of mass of the foreground main area, and to obtain the second center of mass of the control, wherein obtaining the first center of mass of the foreground main area includes: calculating the binary image moment of the mask area corresponding to the foreground main area; calculating the first center of mass based on the binary image moment; obtaining the second center of mass of the control includes: based on the geometric shape of the control, determining the center position of the geometric shape as the second center of mass.

[0093] The above-mentioned moving module 56 includes: a first moving unit, used to move the control in the first direction to the target position in the wallpaper image when the ratio is greater than the first ratio threshold; a second moving unit, used to move the control in the second direction to the target position in the wallpaper image when the ratio is less than the second ratio threshold.

[0094] Optionally, the first moving unit or the second moving unit is also used to repeatedly perform the following steps: obtain the initial position of the control, move the control with a moving step length of half the distance between the first center of mass and the second center of mass, and calculate the ratio of the control after moving from the initial position to the current position; in response to the ratio after the movement not satisfying the depth of field wallpaper generation condition, setting the current position to the initial position and updating the position of the second center of mass after the movement; or in response to the ratio after the movement satisfying the depth of field wallpaper generation condition, ending the loop step.

[0095] Optionally, the above-mentioned first moving unit or second moving unit is also used to repeat the following steps when the ratio is greater than the first ratio threshold: obtain the initial position of the control, move the control in the first direction with the distance between the first center of mass and the second center of mass as the moving step of the control, and calculate the ratio after the control moves from the initial position to the current position; in response to the ratio after the movement being greater than the first ratio threshold, set the current position after moving in the first direction to the initial position, and update the position of the second center of mass after moving; or, in response to the ratio after the movement being less than the second ratio threshold, obtain the current position of the control, move the control in the second direction with half of the distance between the first center of mass and the second center of mass as the moving step of the control, set the current position after moving in the second direction to the initial position, and update the position of the second center of mass after moving; or, in response to the ratio after the movement satisfying the depth of field wallpaper generation condition, end the loop step.

[0096] Optionally, the above-mentioned first moving unit or second moving unit is also used to repeat the following steps when the ratio is less than the second ratio threshold: obtain the initial position of the control, move the control in the second direction with half of the distance between the first center of mass and the second center of mass as the moving step of the control, and calculate the ratio after the control moves from the initial position to the current position; in response to the ratio after the movement being less than the second ratio threshold, set the current position after moving in the second direction as the initial position, and update the position of the second center of mass after moving; or, in response to the ratio after the movement being greater than the first ratio threshold, obtain the current position of the control, move the control in the first direction with half of the distance between the first center of mass and the second center of mass as the moving step of the control, set the current position after moving in the first direction as the initial position, and update the position of the second center of mass after moving; or, in response to the ratio after the movement satisfying the depth of field wallpaper generation condition, end the loop step.

[0097] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0098] The embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program executable by the at least one processor, and the computer program is used to enable the electronic device to perform the method of the embodiment of the present application when executed by the at least one processor.

[0099] An embodiment of the present application also provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present application.

[0100] The embodiment of the present application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present application.

[0101] refer to Figure 6, the structural block diagram of the electronic device that can be used as the server or client of the embodiment of the present application will now be described, which is an example of the hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0102] like Figure 6 As shown, the electronic device includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 606. An input / output (I / O) interface 605 is also connected to the bus 606.

[0103] Multiple components in the electronic device are connected to the I / O interface 605, including: an input unit 606, an output unit 606, a storage unit 608, and a communication unit 609. The input unit 606 can be any type of device that can input information to the electronic device, and the input unit 606 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 606 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 608 can include but is not limited to a disk, an optical disk. The communication unit 609 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0104] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present application may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via a ROM 602 and / or a communication unit 609. In some embodiments, the computing unit 601 may be configured to perform the above method in any other appropriate manner (e.g., by means of firmware).

[0105] The computer program for implementing the method of the embodiment of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0106] In the context of the present application embodiment, machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable signal medium can include but is not limited to electronic, magnetic, optical, electromagnetic or infrared systems, devices or equipment, or any suitable combination of the above. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0107] It should be noted that the term "including" and its variations used in the embodiments of the present application are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present application are illustrative and not restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0108] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0109] The various steps described in the method implementation methods provided in the embodiments of the present application can be performed in different orders and / or in parallel. In addition, the method implementation methods may include additional steps and / or omit the steps shown. The scope of protection of the present application is not limited in this respect.

[0110] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0111] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for generating a depth of field wallpaper, comprising: Get wallpaper images and controls; Determining a foreground subject area in the wallpaper image; In response to the ratio of the overlapping area of ​​the foreground main body area and the control to the area of ​​the control not satisfying the depth of field wallpaper generation condition, the control is moved to a target position in the wallpaper image in a first direction from the first mass center of the foreground main body area to the second mass center of the control or in a second direction from the second mass center to the first mass center, to generate a non-depth of field wallpaper, wherein the target position is the position of the control in the wallpaper image when the ratio satisfies the depth of field wallpaper generation condition; The foreground subject corresponding to the foreground subject area is superimposed on the non-depth-of-field wallpaper to generate a depth-of-field wallpaper.

2. The method for generating a depth of field wallpaper according to claim 1, wherein: The depth of field wallpaper generation condition is that the ratio is greater than or equal to a second ratio threshold and less than or equal to a first ratio threshold, wherein the first ratio threshold is the sum of a specified ratio threshold and an error value, and the second ratio threshold is the difference between the specified ratio threshold and the error value.

3. The method for generating a depth of field wallpaper according to claim 1, wherein: Determining a foreground subject area in the wallpaper image includes: Inputting the processed wallpaper image into the trained machine learning model to obtain the probability of each pixel in the wallpaper image belonging to a category; For each pixel, the category corresponding to the maximum probability among the probabilities of the category to which the pixel belongs is taken as the category of the pixel; Based on the category of each pixel, a foreground subject area in the wallpaper image is determined.

4. The method for generating a depth of field wallpaper according to claim 1, wherein: Acquiring the first centroid of the foreground main area includes: calculating the binary image moment of the mask area corresponding to the foreground main area; calculating the first centroid according to the binary image moment; Obtaining the second centroid of the control includes: according to the geometric shape of the control, determining the center position of the geometric shape as the second centroid.

5. The method for generating a depth of field wallpaper according to claim 2, wherein: Moving the control to a target position in the wallpaper image in a first direction toward a first centroid of the foreground main area pointing to a second centroid of the control or in a second direction toward the second centroid pointing to the first centroid, comprises: When the ratio is greater than the first ratio threshold, moving the control toward the first direction to a target position in the wallpaper image; When the ratio is less than the second ratio threshold, the control is moved toward the second direction to a target position in the wallpaper image.

6. The method for generating a depth of field wallpaper according to claim 5, wherein: Moving the control toward the first direction to a target position in the wallpaper image, or moving the control toward the second direction to a target position in the wallpaper image, includes: Repeat the following steps: Acquire the initial position of the control, move the control with half the distance between the first center of mass and the second center of mass as the moving step of the control, and calculate the ratio of the control after moving from the initial position to the current position; In response to the ratio after the movement not satisfying the depth of field wallpaper generation condition, setting the current position to the initial position, and updating the position of the second mass center after the movement; or In response to the ratio after the shift satisfying the depth of field wallpaper generation condition, the loop step ends.

7. The method for generating a depth of field wallpaper according to claim 5, wherein: Moving the control toward the first direction to a target position in the wallpaper image, or moving the control toward the second direction to a target position in the wallpaper image, includes: When the ratio is greater than the first ratio threshold, repeatedly performing the following steps: Acquire the initial position of the control, move the control in the first direction with the distance between the first center of mass and the second center of mass as the moving step length of the control, and calculate the ratio of the control after moving from the initial position to the current position; In response to the ratio after movement being greater than the first ratio threshold, the current position after moving in the first direction is set as the initial position, and the position of the second center of mass after moving is updated; or, in response to the ratio after movement being less than the second ratio threshold, the current position of the control is obtained, the control is moved in the second direction with half of the distance between the first center of mass and the second center of mass as the moving step of the control, the current position after moving in the second direction is set as the initial position, and the position of the second center of mass after moving is updated; or, in response to the ratio after movement satisfying the depth of field wallpaper generation condition, the loop step is ended.

8. The method for generating a depth of field wallpaper according to claim 5, wherein: Moving the control toward the first direction to a target position in the wallpaper image, or moving the control toward the second direction to a target position in the wallpaper image, includes: When the ratio is less than the second ratio threshold, repeatedly performing the following steps: Obtaining an initial position of the control, moving the control in the second direction with a moving step length of half the distance between the first center of mass and the second center of mass, and calculating the ratio of the control after moving from the initial position to the current position; In response to the ratio after the movement being less than the second ratio threshold, the current position after moving in the second direction is set as the initial position, and the position of the second center of mass after the movement is updated; or, in response to the ratio after the movement being greater than the first ratio threshold, the current position of the control is obtained, and the control is moved in the first direction with half of the distance between the first center of mass and the second center of mass as the moving step of the control, the current position after moving in the first direction is set as the initial position, and the position of the second center of mass after the movement is updated; or, in response to the ratio after the movement satisfying the depth of field wallpaper generation condition, the loop step is ended.

9. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images