Dynamic wallpaper generation method and device and image processing model generation method and device

Abstract

The invention provides a dynamic wallpaper generation method and device and an image processing model generation method and device, and relates to the technical field of computers. The method comprises the steps of firstly obtaining a first image and a reference video corresponding to a target page element to be fused, then determining a reference image, and then processing the reference video based on the reference image by utilizing a preset image processing model to obtain a main body image sequence, a background image sequence and a mask image sequence corresponding to the reference video, and finally, based on each mask image in the mask image sequence, fusing the first image with a corresponding background image in the background image sequence and a corresponding main body image in the main body image sequence to obtain the dynamic wallpaper containing the target page element. Therefore, the effect and quality of the generated dynamic wallpaper are improved, the reliability of dynamic wallpaper generation is improved, and the user experience is improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method for generating dynamic wallpapers, a method for generating image processing models, and an apparatus. Background Technology

[0002] Existing technologies typically generate live wallpapers based on video foreground extraction algorithms. However, this method cannot take into account the advantages of image segmentation and image matting, resulting in poor edge precision of the matting results, limited support for matting subject types, and poor inter-frame continuity and consistency, leading to poor quality and effect of the generated live wallpapers. Summary of the Invention

[0003] This disclosure aims to at least partially address one of the technical problems in the related art.

[0004] The first aspect of this disclosure provides a method for generating a dynamic wallpaper, comprising:

[0005] Obtain the first image and reference video corresponding to the target page element to be merged;

[0006] Identify a reference image;

[0007] Based on the reference image, the reference video is processed using a preset image processing model to obtain the main image sequence, background image sequence, and mask image sequence corresponding to the reference video.

[0008] Based on each mask image in the mask image sequence, the first image is fused with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence to obtain a dynamic wallpaper containing the target page elements.

[0009] A second aspect of this disclosure provides a method for generating an image processing model, comprising:

[0010] Acquire a first dataset, a second dataset, a third dataset, a fourth dataset, and a first reference image. The first dataset includes multiple sets of first original images and corresponding first matting results. The second dataset includes multiple sets of second original images and corresponding first segmentation results. The third dataset includes multiple sets of first videos and corresponding second matting results. The fourth dataset includes multiple sets of second videos and corresponding second segmentation results.

[0011] Based on the first reference image, the initial image processing model is trained using the first dataset and the second dataset to obtain a pre-trained model;

[0012] Based on the first reference image, the pre-trained model is trained using the first dataset, the second dataset, the third dataset, and the fourth dataset to obtain an image processing model.

[0013] A third aspect of this disclosure provides an apparatus for generating dynamic wallpapers, comprising:

[0014] The first acquisition module is used to acquire the first image and reference video corresponding to the target page element to be merged;

[0015] The determination module is used to determine the reference image;

[0016] The processing module is used to process the reference video based on the reference image using a preset image processing model to obtain the main image sequence, background image sequence and mask image sequence corresponding to the reference video;

[0017] The fusion module is used to fuse the first image with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence based on each mask image in the mask image sequence, to obtain a dynamic wallpaper containing the target page elements.

[0018] A fourth aspect of this disclosure provides an apparatus for generating an image processing model, comprising:

[0019] The second acquisition module is used to acquire a first dataset, a second dataset, a third dataset, a fourth dataset, and a first reference image. The first dataset includes multiple sets of first original images and corresponding first matting results. The second dataset includes multiple sets of second original images and corresponding first segmentation results. The third dataset includes multiple sets of first videos and corresponding second matting results. The fourth dataset includes multiple sets of second videos and corresponding second segmentation results.

[0020] The first training module is used to train the initial image processing model based on the first reference image, using the first dataset and the second dataset, to obtain a pre-trained model.

[0021] The second training module is used to train the pre-trained model based on the first reference image, using the first dataset, the second dataset, the third dataset, and the fourth dataset, to obtain an image processing model.

[0022] A fifth aspect of this disclosure provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method for generating a dynamic wallpaper as proposed in the first aspect of this disclosure and the method for generating an image processing model as proposed in the second aspect of this disclosure.

[0023] A sixth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for generating a dynamic wallpaper as proposed in the first aspect of this disclosure and the method for generating an image processing model as proposed in the second aspect of this disclosure.

[0024] The method for generating dynamic wallpapers, the method for generating image processing models, and the apparatus disclosed herein have the following beneficial effects:

[0025] In this embodiment, a first image and a reference video corresponding to the target page element to be fused are first obtained. Then, a reference image is determined. Based on the reference image, the reference video is processed using a preset image processing model to obtain a main image sequence, a background image sequence, and a mask image sequence corresponding to the reference video. Finally, based on each mask image in the mask image sequence, the first image is fused with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence to obtain a dynamic wallpaper containing the target page element. Thus, by extracting the main subject from the reference video based on the reference image, determining its corresponding main image sequence, background image sequence, and mask image sequence, and fusing the image corresponding to the target page element with the corresponding background image in the background image sequence and the main image in the main image sequence based on each mask image in the mask image sequence to generate a dynamic wallpaper, the effect and quality of the generated dynamic wallpaper are improved, the reliability of dynamic wallpaper generation is enhanced, and the user experience is improved.

[0026] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0027] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0028] Figure 1 A schematic flowchart illustrating a method for generating a dynamic wallpaper according to an embodiment of this disclosure;

[0029] Figure 2 A schematic diagram of the architecture of the image processing model provided in the embodiments of this disclosure;

[0030] Figure 3 A flowchart illustrating a method for generating a dynamic wallpaper according to another embodiment of this disclosure;

[0031] Figure 4 A flowchart illustrating a method for generating a dynamic wallpaper according to another embodiment of this disclosure;

[0032] Figure 5 A schematic diagram of the video foreground extraction process provided in this embodiment of the disclosure;

[0033] Figure 6 A flowchart illustrating a method for generating a dynamic wallpaper according to another embodiment of this disclosure;

[0034] Figure 7 This is a schematic diagram illustrating the effect of the generated live wallpaper provided in the embodiments of this disclosure;

[0035] Figure 8 This is a schematic flowchart illustrating a method for generating an image processing model according to another embodiment of the present disclosure.

[0036] Figure 9 This is a schematic flowchart illustrating a method for generating an image processing model according to another embodiment of the present disclosure.

[0037] Figure 10 This is a schematic diagram illustrating the process of modifying a model in a method for generating an image processing model according to another embodiment of this disclosure.

[0038] Figure 11 This is a schematic flowchart illustrating a method for generating an image processing model according to another embodiment of the present disclosure.

[0039] Figure 12 A schematic diagram of the structure of a dynamic wallpaper generation apparatus provided in another embodiment of this disclosure;

[0040] Figure 13 This is a schematic diagram of the structure of an image processing model generation apparatus provided in another embodiment of the present disclosure;

[0041] Figure 14 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0042] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0043] The following describes, with reference to the accompanying drawings, a method for generating dynamic wallpapers, a method for generating image processing models, and an apparatus for generating images according to embodiments of the present disclosure.

[0044] Figure 1 This is a schematic flowchart illustrating a method for generating a dynamic wallpaper according to an embodiment of the present disclosure.

[0045] like Figure 1As shown, the method for generating this live wallpaper may include the following steps:

[0046] Step 101: Obtain the first image and reference video corresponding to the target page element to be merged.

[0047] The target page element can be the lock screen control of an electronic device, that is, a user interface (UI) element, which can be pre-set, and this disclosure does not limit it.

[0048] The first image can be the image displayed on the lock screen for the target page element.

[0049] The reference video can be a video used to generate the live wallpaper. Its content can be pre-set or determined according to actual needs. This disclosure does not limit it in this regard.

[0050] In this disclosure, in order to improve the quality and effect of the generated live wallpaper, a first image and reference video corresponding to the target page element to be merged can be obtained before generating the live wallpaper.

[0051] Optionally, when obtaining the first image corresponding to the target page element to be merged, the target page element in the current lock screen interface of the electronic device can be determined first based on the current settings information of the electronic device, and then the first image corresponding to the target page element can be determined based on the display parameters associated with the target page element.

[0052] The display parameters associated with the target page element may include parameters such as the display position and display size of the target page element, which are not limited in this disclosure.

[0053] It should be noted that the specific type of electronic device can be pre-set or determined according to actual needs. For example, electronic devices can be mobile phones, tablets, etc., and this disclosure does not limit this.

[0054] It should be noted that, since the display parameters of the same target page element may be different on different electronic devices, the first image corresponding to the same target page element may be different on different electronic devices, and this disclosure does not limit this.

[0055] Step 102: Determine the reference image.

[0056] The reference image can be a reference mask image used for subject extraction from the reference video, also known as an alpha image.

[0057] In this disclosure, after obtaining the first image and reference video corresponding to the target page element to be merged, in order to improve the accuracy of subject extraction from the reference video, the reference image can be determined first.

[0058] Optionally, when all types of subjects correspond to a reference image, a preset second image can be determined as the reference image. For example, a second image in which the alpha value of all pixels is 0, i.e., a completely black image, can be determined as the reference image. This disclosure does not limit this.

[0059] The second image can be a pre-set mask image, or it can be a mask image determined according to actual needs. This disclosure does not limit the specific mask image.

[0060] Optionally, when different types of subjects correspond to different reference images, for example, the distribution of alpha values ​​in the reference image corresponding to a person is different from the distribution of alpha values ​​in the reference image corresponding to a vehicle, the image associated with the type of subject contained in the reference video can be determined as the reference image. This disclosure does not limit this.

[0061] Step 103: Based on the reference image, the reference video is processed using a preset image processing model to obtain the main image sequence, background image sequence, and mask image sequence corresponding to the reference video.

[0062] The image processing model can be a model used for subject extraction from a reference video, and it can be of any type and structure. For example, using... Figure 2 For example, Figure 2 This is a schematic diagram of the architecture of the image processing model provided in the embodiments of this disclosure, such as... Figure 2 As shown, the image processing model may include an image encoder, a subject extraction decoder, a classification branch, and a difficulty branch, which are not limited in this disclosure.

[0063] The image encoder, which is a convolutional neural network, can be used to extract high-dimensional semantic feature representations of images. It can be composed of multiple convolutional layers and pooling layers to extract low-dimensional to high-dimensional features of the image layer by layer. This allows us to obtain important features such as object boundaries, textures and structures in the image. This disclosure does not limit these features.

[0064] The subject extraction decoder can upsample the feature map extracted by the encoder layer by layer to restore the spatial size, and fuse it with the feature map corresponding to the encoder. Finally, it outputs a matting result or segmentation result with the same size as the original input image. It can also remove background noise and restore a clean mask map containing only the subject region. This disclosure does not limit this.

[0065] Among them, the classification branch can be used to identify the type of subject; the difficulty branch can be used to assess and score the difficulty of cutting out images from a video, and this disclosure does not limit it.

[0066] The mask image sequence can be a sequence of alpha images corresponding to the main subject in each frame of the reference video, i.e., an opaque mask image.

[0067] In this disclosure, after determining the reference image, the reference video can be processed based on the reference image using a preset image processing model to obtain the main image sequence, background image sequence, and mask image sequence corresponding to the reference video, thereby providing a data foundation for generating dynamic wallpapers.

[0068] It should be noted that when processing the reference video using the image processing model, in order to ensure that the extracted subject image has stable and smooth edges, the image processing model will focus on the edge sharpness of the subject image to avoid blurring or incompleteness. This disclosure does not limit this.

[0069] It should be noted that when the image processing model processes the reference video, it can ensure that the extraction of the subject foreground between consecutive frames is stable and consistent based on the motion changes of the subject in the video, avoiding jitter or discontinuity. This disclosure does not limit this.

[0070] Optionally, before processing the reference video using a preset image processing model, in order to improve processing efficiency and effect, the reference video can be preprocessed, and the specifications such as resolution, frame rate and duration of the reference video can be adjusted. For example, the higher the resolution of the reference video, the greater the computational load of the image processing model. To reduce the computational burden and perform calculations within a reasonable resource range, while balancing the clarity of the reference video and computational efficiency, reference videos with resolutions exceeding a preset resolution threshold, such as 2K, can be compressed to adjust their resolution to the threshold, such as 2K. When the frame rate of the reference video is higher, the smoother the video, but the computational load of the image processing model increases significantly. To balance processing efficiency and video smoothness, the frame rate of the reference video can be limited to a preset frame rate threshold or below, such as 60FPS or below. When the frame rate of the reference video is greater than 60FPS, automatic frame extraction can be used to reduce the frame rate to 60FPS. When the duration of the reference video is long, it will consume more system computing and storage resources. To ensure the processing efficiency of the image processing model and improve user experience, when the duration of the reference video exceeds a preset duration threshold, such as 6 seconds, the user can be prompted to edit the video and select a video segment with a duration of less than or equal to 6 seconds for processing. This disclosure does not impose any limitations on this.

[0071] FPS stands for Frames Per Second.

[0072] Step 104: Based on each mask image in the mask image sequence, merge the first image with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence to obtain a dynamic wallpaper containing the target page elements.

[0073] In this disclosure, after obtaining the main image sequence, background image sequence, and mask image sequence corresponding to the reference video, the first image, the corresponding background image in the background image sequence, and the corresponding main image in the main image sequence can be fused based on each mask image in the mask image sequence to obtain a dynamic wallpaper containing the target page elements. This ensures the quality and effect of the generated dynamic wallpaper and improves the reliability of the dynamic wallpaper generation method.

[0074] It should be noted that when fusing the first image with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence, the background image, the first image, and the main image can be superimposed and fused in the order of the background image, the first image, and the main image, so that the generated dynamic wallpaper can obtain a depth effect. This disclosure does not limit this.

[0075] In this embodiment, a first image and a reference video corresponding to the target page element to be fused are first obtained. Then, a reference image is determined. Based on the reference image, the reference video is processed using a preset image processing model to obtain a main image sequence, a background image sequence, and a mask image sequence corresponding to the reference video. Finally, based on each mask image in the mask image sequence, the first image is fused with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence to obtain a dynamic wallpaper containing the target page element. Thus, by extracting the main subject from the reference video based on the reference image, determining its corresponding main image sequence, background image sequence, and mask image sequence, and fusing the image corresponding to the target page element with the corresponding background image in the background image sequence and the main image in the main image sequence based on each mask image in the mask image sequence to generate a dynamic wallpaper, the effect and quality of the generated dynamic wallpaper are improved, the reliability of dynamic wallpaper generation is enhanced, and the user experience is improved.

[0076] Figure 3 This is a schematic flowchart illustrating a method for generating a dynamic wallpaper according to another embodiment of this disclosure.

[0077] like Figure 3 As shown, the method for generating this live wallpaper may include the following steps:

[0078] Step 301: Obtain the first image and reference video corresponding to the target page element to be merged.

[0079] Step 302: Determine the reference image.

[0080] Step 303: Based on the reference image, the reference video is processed using a preset image processing model to obtain the main image sequence, background image sequence, and mask image sequence corresponding to the reference video.

[0081] The specific implementation of steps 301 to 303 can be found in the detailed description of other embodiments of this disclosure, and will not be repeated here.

[0082] Step 304: Determine the type of subject contained in the reference video.

[0083] It should be noted that the subjects included in the reference video can be of any type. For example, they can be people, vehicles, cats, dogs, etc. This disclosure does not limit them.

[0084] In this disclosure, after obtaining the subject image sequence, background image sequence, and mask image sequence corresponding to the reference video, in order to ensure the effect and quality of the generated dynamic wallpaper, the type of the subject contained in the reference video can be determined first, thereby accurately identifying the objects and background in the video and improving the accuracy of image processing.

[0085] It should be noted that when determining the type of subject contained in the reference video, multi-frame analysis can be used to comprehensively judge the type of subject in the reference video, and this disclosure does not limit this.

[0086] It should be noted that the type of the subject in the reference video can be determined by the classification branch module integrated in the image processing model architecture. After the encoder extracts features from the image, the classification branch module is connected to the high-dimensional feature layer of the model. It can be composed of convolutional and fully connected layers. It can generate predicted probabilities for each type based on the extracted features, and then normalize these probabilities to ensure that the sum of the probabilities of all types is 1. Finally, the type with the highest probability is selected as the type of the subject. This disclosure does not limit this process.

[0087] It should be noted that the classification branch can determine the type of the subject through the image encoder, thereby enabling the classification task to utilize the contextual information of the matting or segmentation task to obtain more accurate subject type prediction results. This disclosure does not limit this.

[0088] Step 305: If the type of the main body is a preset type, based on each mask image in the mask image sequence, the first image is fused with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence to obtain a dynamic wallpaper containing the target page elements.

[0089] The preset type can be any type that is set in advance. For example, the preset type can be a person, a vehicle, a cat, a dog, etc., and this disclosure does not limit it.

[0090] In this disclosure, after determining the type of the subject contained in the reference video, if the type of the subject is a preset type, it can be determined that a live wallpaper can be generated based on the reference video. At this time, based on each mask image in the mask image sequence, the first image can be fused with the corresponding background image in the background image sequence and the corresponding subject image in the subject image sequence to obtain a live wallpaper containing the target page elements, thereby improving the efficiency and accuracy of generating live wallpapers.

[0091] Optionally, if the type of the subject is not a preset type, a fusion failure message can also be returned to the user, indicating that the subject type of the reference video is not within the supported type range. This disclosure does not limit this.

[0092] In this embodiment, firstly, a first image and a reference video corresponding to the target page element to be fused are obtained, and a reference image is determined. Then, based on the reference image, the reference video is processed using a preset image processing model to obtain a subject image sequence, a background image sequence, and a mask image sequence corresponding to the reference video. Finally, the type of the subject contained in the reference video is determined. If the subject type is a preset type, the first image is fused with the corresponding background image in the background image sequence and the corresponding subject image in the subject image sequence based on each mask image in the mask image sequence to obtain a dynamic wallpaper containing the target page element. Therefore, after obtaining the subject image sequence, background image sequence, and mask image sequence corresponding to the reference video, the type of the subject contained in the reference video is determined. If the subject type is a preset type, the image, background image, and subject image corresponding to the target page element are fused based on the mask image to generate a dynamic wallpaper, thereby improving the efficiency and reliability of dynamic wallpaper generation.

[0093] Figure 4 This is a schematic flowchart illustrating a method for generating a dynamic wallpaper according to another embodiment of this disclosure.

[0094] like Figure 4 As shown, the method for generating this live wallpaper may include the following steps:

[0095] Step 401: Obtain the first image and reference video corresponding to the target page element to be merged.

[0096] Step 402: Determine the reference image.

[0097] The specific implementation of steps 401 to 402 can be found in the detailed description of other embodiments of this disclosure, and will not be repeated here.

[0098] Step 403: Based on the reference image, process the reference video using a preset image processing model to obtain the first main image, the first background image, and the first mask image corresponding to the first image frame in the reference video.

[0099] In this disclosure, after determining the reference image, the reference video can first be processed using a preset image processing model based on the reference image to obtain the first main image, the first background image, and the first mask image corresponding to the first image frame in the reference video, thereby providing a data foundation for obtaining the main image sequence, background image sequence, and mask image sequence of the reference video.

[0100] Optionally, before processing the reference video based on the image processing model, the size of the 3-channel color image in the reference video can be normalized and adjusted, and combined with a reference image of the same size in a single channel to form a 4-channel input data (e.g., size 512*512*4). Then, the input data is sent to the image encoder in the image processing model to extract high-dimensional features, and then through the subject extraction decoder to obtain the subject image, background image and mask image. This disclosure does not limit this.

[0101] Step 404: Based on the first mask image, return to the step of processing the reference video using a preset image processing model until the main image sequence, background image sequence and mask image sequence corresponding to the reference video are obtained.

[0102] In this disclosure, after obtaining the first mask image corresponding to the first image frame in the reference video, the process can return to the step of processing the reference video using a preset image processing model based on the first mask image until the main image sequence, background image sequence and mask image sequence corresponding to the reference video are obtained, thereby improving the accuracy of the obtained main image sequence and background image sequence.

[0103] For example, the process of processing a reference video using a pre-defined image processing model to obtain a sequence of mask images corresponding to the reference video can be as follows: Figure 5 As shown, Figure 5 The flowchart of video foreground extraction provided in this embodiment is as follows: First, the reference image is set to a completely black image. After being synthesized with the RGB image of the first frame of the reference video, it is input into the subject extraction decoder in the image processing model to obtain the first mask image. Then, the first mask image is synthesized with the RGB image of the second frame and input into the subject extraction decoder to obtain the second image. This process is repeated to obtain the mask image corresponding to each frame of the reference video. Here, T = 0, 1, and 2, which are the first frame, the second frame, and the third frame, respectively. RGB is the abbreviation for the three primary colors red, green, and blue (RGB), which is not limited in this disclosure.

[0104] Step 405: Based on each mask image in the mask image sequence, merge the first image with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence to obtain a dynamic wallpaper containing the target page elements.

[0105] The specific implementation of step 405 can be found in the detailed description of other embodiments of this disclosure, and will not be repeated here.

[0106] In this embodiment, the first image and reference video corresponding to the target page element to be fused are first obtained, and the reference image is determined. Then, based on the reference image, the reference video is processed using a preset image processing model to obtain the first main image, the first background image, and the first mask image corresponding to the first image frame in the reference video. Then, based on the first mask image, the step of processing the reference video using the preset image processing model is returned to be executed until the main image sequence, the background image sequence, and the mask image sequence corresponding to the reference video are obtained. Finally, based on each mask image in the mask image sequence, the first image is fused with the background image corresponding to the background image sequence and the main image corresponding to the main image sequence to obtain a dynamic wallpaper containing the target page element. Therefore, after obtaining the image, reference video, and reference image corresponding to the target page element, the first main image, first background image, and first mask image corresponding to the first image frame in the reference video are determined based on the reference image and the image processing model. Based on the first mask image, the steps of processing the reference video using the image processing model are returned to execute until the main image sequence, background image sequence, and mask image sequence of the reference video are determined. Finally, the image corresponding to the target page element, the main image in the main image sequence, and the background image in the background image sequence are merged to generate a dynamic wallpaper, thereby improving the quality and reliability of the generated dynamic wallpaper.

[0107] Figure 6 This is a schematic flowchart illustrating a method for generating a dynamic wallpaper according to another embodiment of this disclosure.

[0108] like Figure 6 As shown, the method for generating this live wallpaper may include the following steps:

[0109] Step 601: Obtain the first image and reference video corresponding to the target page element to be merged.

[0110] Step 602: Determine the reference image.

[0111] Step 603: Based on the reference image, the reference video is processed using a preset image processing model to obtain the main image sequence, background image sequence, and mask image sequence corresponding to the reference video.

[0112] Optionally, after processing the reference video using a preset image processing model, the type of the subject contained in the reference video can be determined. If the type of the subject is not a preset type, a first indication message is returned. The first indication message is used to indicate that the type of the subject in the reference video is abnormal. This disclosure does not limit this.

[0113] Optionally, after processing the reference video using a preset image processing model, the difficulty score of the reference video can be determined. If the difficulty score is greater than a threshold, a second indication is returned. The second indication is used to indicate that the reference video does not meet the preset requirements, allowing the user to reselect a video that meets the preset requirements. This not only improves the efficiency of user operation but also avoids generating poor-quality live wallpapers, thus optimizing the user experience.

[0114] The threshold can be a critical value for judging whether a reference video meets the preset requirements. It can be preset, and this disclosure does not limit it.

[0115] The preset requirements can be any pre-set requirements. For example, preset requirements can be information such as whether the subject of the reference video is clear, the complexity of the scene, etc., which meet the requirements of the image processing model. This disclosure does not limit these requirements.

[0116] In this disclosure, for situations where the subject in the reference video is unclear, or where the difficulty of matting in certain complex scenes is too high, the image processing model may not be able to perform fine-grained matting well. In such cases, a difficulty branch module in the image processing model can be used to generate a difficulty score for the matting task of the reference video. The score ranges from 0 to 1. The higher the score, the easier it is to distinguish the subject in the reference video and the lower the difficulty of matting. Conversely, the lower the score, the higher the complexity of matting the reference video and the higher the difficulty for the image processing model to extract the subject. This allows the image processing model to more comprehensively understand and cope with various video scenes and has good generalization ability. This disclosure does not limit this aspect.

[0117] The specific implementation of steps 601 to 603 can be found in the detailed description of other embodiments of this disclosure, and will not be repeated here.

[0118] Step 604: Based on the pixel value at position i in the mask image, determine the first weight of the first pixel at position i in the corresponding background image and the second weight of the second pixel at position i in the corresponding main image, where i is a natural number.

[0119] The first weight can be the fusion weight of the first pixel at position i in the background image.

[0120] The second weight can be the fusion weight of the second pixel at position i in the main image.

[0121] It should be noted that the first weight may be different from the second weight, or it may be the same as the second weight; this disclosure does not limit this.

[0122] In this disclosure, after obtaining the main image sequence, background image sequence, and mask image sequence corresponding to the reference video, in order to ensure the quality and effect of the generated dynamic wallpaper, the first weight of the first pixel at position i in the corresponding background image and the second weight of the second pixel at position i in the corresponding main image can be determined first based on the pixel value of the mask image at position i.

[0123] Step 605: Based on the first weight and the second weight, the first pixel value of the first pixel point, the second pixel value of the second pixel point, and the third pixel value of the third pixel point at position i in the first image are fused to obtain an image frame in the dynamic wallpaper.

[0124] In this disclosure, after determining the first weight and the second weight, the first pixel value of the first pixel, the second pixel value of the second pixel, and the third pixel value of the third pixel at position i in the first image can be fused based on the first weight and the second weight to obtain an image frame in the dynamic wallpaper. This process is repeated until each image frame in the dynamic wallpaper is obtained. This ensures the effectiveness of the generated dynamic wallpaper.

[0125] For example, if the first weight and the second weight are both 1, the corresponding image frame is determined based on the preset fusion formula (1). Formula (1) is shown below. Formula (1) is only an example and is not limited in this disclosure.

[0126] C i =α i F i +U i +(1-α i B i , formula (1)

[0127] In formula (1), C i , where is the pixel value at position i in the final obtained image frame; F i , where is the pixel value of the main image at position i; U i , where is the pixel value of the first image at position i; B i α is the pixel value of the background image at position i; i , is the pixel value of the mask at position i, in the range [0, 1].

[0128] The main image and the mask image are blended using formula (1), and the first image is superimposed on top. The background image is then blended using 1-α. The final dynamic wallpaper effect is as follows: Figure 7 As shown, Figure 7 This is a schematic diagram illustrating the effect of the generated live wallpaper provided in the embodiments of this disclosure. Figure 7 As can be seen, the main image, the first image, and the background image of the generated live wallpaper blend naturally, the edges of the main image are smooth, and the generated live wallpaper has good quality and effect. This disclosure does not limit this.

[0129] In this embodiment, firstly, a first image and a reference video corresponding to the target page element to be fused are obtained, and the reference image is determined. Then, based on the reference image, the reference video is processed using a preset image processing model to obtain the main image sequence, background image sequence, and mask image sequence corresponding to the reference video. Next, based on the pixel value of the mask image at position i, the first weight of the first pixel at position i in the corresponding background image and the second weight of the second pixel at position i in the corresponding main image are determined. Finally, based on the first weight and the second weight, the first pixel value of the first pixel, the second pixel value of the second pixel, and the third pixel value of the third pixel at position i in the first image are fused to obtain an image frame in the dynamic wallpaper. Therefore, by using reference images and image processing models, the main image sequence, background image sequence, and mask image sequence of the reference video are determined. Based on the pixel values ​​of the mask image at each position, the weights corresponding to the pixels of the background image at each position and the main image at each position are determined. Based on these weights, the pixel values ​​of the background image at each position, the pixel values ​​of the main image at each position, and the pixel values ​​of the target page element at each position are fused to obtain each image frame of the dynamic wallpaper, thereby improving the quality and effect of the generated dynamic wallpaper.

[0130] Figure 8 This is a flowchart illustrating a method for generating an image processing model according to another embodiment of the present disclosure.

[0131] like Figure 8 As shown, the method for generating this image processing model may include the following steps:

[0132] Step 801: Obtain the first dataset, the second dataset, the third dataset, the fourth dataset, and the first reference image.

[0133] The first dataset includes multiple sets of first original images and corresponding first matting results; the second dataset includes multiple sets of second original images and corresponding first segmentation results; the third dataset includes multiple sets of first videos and corresponding second matting results; and the fourth dataset includes multiple sets of second videos and corresponding second segmentation results.

[0134] The first reference image can be a preset mask image, also known as an alpha image.

[0135] In this disclosure, in order to avoid overfitting the image processing model on the matting dataset and to improve the accuracy and reliability of the generated image processing model, the first dataset, the second dataset, the third dataset, the fourth dataset and the first reference image can be obtained first during the training process of the image processing model to improve the semantic understanding ability of the image processing model.

[0136] It should be noted that because image matting can separate the subject from the background, it reduces the interference of background noise on the image processing model and improves the model's ability to process edges and details. Furthermore, segmenting the dataset can make the training and testing data of the model more representative, thereby improving the generalization ability of the image processing model.

[0137] Step 802: Based on the first reference image, the initial image processing model is trained using the first dataset and the second dataset to obtain a pre-trained model.

[0138] In this disclosure, after obtaining the first dataset, the second dataset, and the first reference image, an initial image processing model can be trained based on the first reference image, using the first dataset and the second dataset to obtain a pre-trained model, thereby improving the semantic understanding and edge processing capabilities of the pre-trained model.

[0139] It should be noted that, in order to increase the model's continuous video processing capability, all inputs to the initial image processing model are the original image plus the reference image, and this disclosure does not limit this.

[0140] It should be noted that, in order to improve the semantic understanding and image edge processing capabilities of the image processing model, the first reference image can be set to a completely black image, and this disclosure does not limit this.

[0141] It should be noted that when training the initial image processing model, the matting dataset and the segmentation dataset can be used alternately. For example, the training method can be to first train on the first dataset for one training hour (epoch) and use the first matting result for supervision, and then use two training hours to train twice on the second dataset and use the first segmentation result for supervision. This stage has a total of 15 training hours, thereby improving the semantic understanding ability and edge processing ability of the pre-trained model. This disclosure does not limit this.

[0142] It should be noted that when training the initial image processing model, since there are many types of data augmentation and the intensity of changes is high, it is necessary to simulate various scenarios as much as possible to enhance the generalization of the model. For example, it can include general data augmentation such as affine transformation, scaling, rotation, cropping, brightness, contrast, saturation, hue, etc., and can also include image perturbation such as additive Gaussian noise, Gaussian blur, motion blur, JPEG compression, etc. This disclosure does not limit this.

[0143] JPEG, short for Joint Photographic Experts Group, is an image compression technology standard.

[0144] It should be noted that when training the initial image processing model, the Adam optimizer can be used, and the learning rate can be set as needed, for example, it can be set to 5e. -4 This disclosure does not impose any limitations on this matter.

[0145] Adam is short for Adaptive Moment Estimation.

[0146] Step 803: Based on the first reference image, the pre-trained model is trained using the first dataset, the second dataset, the third dataset, and the fourth dataset to obtain the image processing model.

[0147] In this disclosure, after training an initial image processing model to obtain a pre-trained model, the pre-trained model can be trained based on a first reference image using a first dataset, a second dataset, a third dataset, and a fourth dataset to obtain an image processing model. Training with image data and video data can ensure the temporal consistency and continuity of the image processing model in processing images.

[0148] It should be noted that when training the pre-trained model based on the first reference image and using the first dataset, the second dataset, the third dataset, and the fourth dataset, matting training and segmentation training can be performed alternately. When performing matting training or segmentation training, a mixture of image data and video data can be used, and this disclosure does not limit this.

[0149] It should be noted that, due to the differences between video data and image data, and the existence of a time series in video data, the corresponding reference image can be the real mask image corresponding to the current image frame's position in the video 5 to 15 frames before or after, i.e., the real alpha image. This allows the model to focus on the same area when processing the video, improving the temporal consistency and continuity of the video matting results. This disclosure does not limit this aspect.

[0150] It should be noted that when training the pre-trained model, the reference image corresponding to the image data used is no longer a completely black image. It can be a completely black image with a certain probability (e.g., 0.3) and an image after enhancement transformation with its own true alpha label with a certain probability (e.g., 0.7). This allows the image processing model to pay more attention to continuity. This disclosure does not limit this.

[0151] It should be noted that, when training the pre-trained model, in order to ensure the stability of the obtained image processing model output, data augmentation can be retained. This can include data augmentation of the reference image, such as thin plate spline interpolation, random affine transformation, opening and closing operations, random addition of small areas of pure white image, etc. In addition, image perturbation should be removed so that the training data can simulate video motion and approximate the characteristics of video data. This disclosure does not limit this.

[0152] It should be noted that the Adam optimizer can also be used when training the pre-trained model, and the learning rate can be set as needed, for example, it can be set to 1e. -4 This disclosure does not impose any limitations on this matter.

[0153] Optionally, after training the pre-trained model to obtain the image processing model, in order to improve the filtering and classification ability of the image processing model, the classification branch of the image processing model can be pre-trained. By utilizing the first reference image, the first dataset, the second dataset, the third dataset, the fourth dataset, and the preset classification dataset, the proportion of image data under each type is balanced. Then, training is carried out by alternating training methods of image matting and segmentation. This stage may include 5 training epochs, which is not limited in this disclosure.

[0154] It should be noted that when training the classification branch of the image processing model, the reference image and data augmentation methods are consistent with the settings described above for training the pre-trained model. The training supervision loss includes the cross-entropy loss of the classification branch (i.e., the classification loss). Since the cross-entropy loss can be used to measure the difference between the model's predicted type distribution and the true type label, minimizing the loss function allows the image processing model to accurately identify the type of the subject in the reference video. The Adam optimizer can also be used in this stage, and the learning rate can be set as needed, for example, to 1e. -5 This disclosure does not impose any limitations on this matter.

[0155] Optionally, after training the classification branch of the image processing model, a difficulty branch can be added to the image processing model architecture and trained in a self-supervised manner. This enables the image processing model to have quality assessment capabilities. In conjunction with a preset threshold for the difficulty branch, this stage allows the model to filter out videos with poor matting effects, improving the overall quality of matted videos. By utilizing a first reference image, a first dataset, a second dataset, a third dataset, a fourth dataset, and a preset difficulty judgment dataset, the proportion of images at each difficulty level is balanced, resulting in high discriminative power. Training is performed using alternating matting and segmentation techniques. This stage can include five training periods, but this disclosure does not limit the duration.

[0156] It should be noted that when training the difficulty branch of the image processing model, the reference image and data augmentation method can be consistent with the settings used when training the pre-trained model. The mean squared error loss function is used to supervise the difficulty score, thereby optimizing the difference between the true score and the predicted score. The predicted score is the score between 0 and 1 output by the difficulty branch, and the true score is the score calculated based on formula (2) between the true mask image and the predicted matting result. Formula (2) is shown below, and this disclosure does not limit its use.

[0157] real score=F(γ*iou(gt, pred)+(1-γ)*(1-L1(gt, pred))), formula (2)

[0158] in,

[0159] Among them, IOU can be used to supervise whether the semantics are correct; L1 supervises whether the edge details are complete; gt is the ground truth; pred is the predicted data; γ is a preset hyperparameter, whose value can be set as needed, and this disclosure does not limit it.

[0160] Since most of the data has been trained on the initial image model and the pre-trained model, the true scores may be concentrated in a certain range (e.g., 0.7-1.0). By setting formula (2.1), the calculated true scores are mapped to a larger range, thus widening the distance between different scores, and giving lower scores to difficult scores (e.g., they can be directly set to 0). This disclosure does not limit this.

[0161] It should be noted that when training the difficulty branch of the image processing model, a large number of completely black alpha images can be added to the difficulty judgment dataset. For videos with higher difficulty, the result calculated by formula (2.1) will be infinitely close to 0. Combined with the relatively simple data in the first, second, third, and fourth datasets, the calculated result is close to 1. Therefore, the difficulty separability of the difficulty branch can be significantly improved. In this stage, the Adam optimizer can also be used, and the learning rate can be set as needed, for example, it can be set to 1e. -5 This disclosure does not impose any limitations on this matter.

[0162] Optionally, without limiting the duration of the reference video, the image processing model can be trained using both short and long time sequences. For example, the model can be trained first with short time sequences, then with long time sequences, and finally with a hybrid training method to enhance the image processing model's ability to perform video matting under various sequences. This disclosure does not limit this.

[0163] In this embodiment, a first dataset, a second dataset, a third dataset, a fourth dataset, and a first reference image are first acquired. Then, based on the first reference image, an initial image processing model is trained using the first and second datasets to obtain a pre-trained model. Finally, based on the first reference image, the pre-trained model is trained using the first, second, third, and fourth datasets to obtain the final image processing model. Thus, by training the initial image processing model based on the reference image, the original image, and their corresponding matting and segmentation results to obtain a pre-trained model, and then using a mixture of image and video data to perform alternating matting and segmentation training on the pre-trained model, an image processing model is obtained. This improves the semantic understanding and edge processing capabilities of the image processing model, thereby enhancing its effectiveness and accuracy.

[0164] Figure 9 This is a flowchart illustrating a method for generating an image processing model according to another embodiment of the present disclosure.

[0165] like Figure 9 As shown, the method for generating this image processing model may include the following steps:

[0166] Step 901: Obtain the first dataset, the second dataset, the third dataset, the fourth dataset, and the first reference image.

[0167] The first dataset includes multiple sets of first original images and corresponding first matting results; the second dataset includes multiple sets of second original images and corresponding first segmentation results; the third dataset includes multiple sets of first videos and corresponding second matting results; and the fourth dataset includes multiple sets of second videos and corresponding second segmentation results.

[0168] The specific implementation of step 901 can be found in the detailed description of other embodiments of this disclosure, and will not be repeated here.

[0169] Step 902: Input the first reference image and the first original image in the first dataset into the initial image processing model to obtain the first processing result.

[0170] The first processing result can be the image matting result of the image processing model on the first original image.

[0171] In this disclosure, after obtaining the first reference image and the first dataset, the first reference image and the first original image in the first dataset can be input into the image processing model respectively to obtain the first processing result of the initial image processing model on the first original image, thereby training the image matting ability of the initial image processing model.

[0172] Step 903: Based on the first difference between the first processing result and the first matting result, the initial image processing model is corrected to obtain the first model.

[0173] In this disclosure, after obtaining the first processing result, a first difference between the first processing result and the first matting result can be determined first, and based on the first difference, the initial image processing model can be corrected to obtain the first model, thereby improving the accuracy of matting of the first model.

[0174] Step 904: Input the first reference image and the second original image from the second dataset into the first model to obtain the second processing result.

[0175] The second processing result can be the segmentation result of the first model on the second original image.

[0176] In this disclosure, after obtaining the first model, the first reference image and the second original image can be respectively input into the first model to obtain the second processing result, thereby training the image segmentation ability of the first model.

[0177] Step 905: Based on the second difference between the second processing result and the first segmentation result, the first model is corrected to obtain the second model.

[0178] In this disclosure, after obtaining the second processing result, a second difference between the second processing result and the first segmentation result can be determined first, and the first model can be corrected based on the second difference to obtain the second model, thereby improving the image segmentation capability of the second model.

[0179] Step 906: Based on the second model, return to execute the operation to obtain the second processing result until the first preset number of times is reached. Based on the obtained second model, return to execute the operation to obtain the first processing result until the pre-trained model is obtained.

[0180] The first preset number of iterations can be the number of training cycles during the segmentation training of the second model. It can be pre-set or determined according to actual needs. For example, the first preset number of iterations can be 2, etc., and this disclosure does not limit it in this way.

[0181] In this disclosure, after obtaining the second model, the operation of obtaining the second processing result can be returned to until a first preset number of times is reached. Then, based on the obtained second model, the operation of obtaining the first processing result is returned to, and the operation of obtaining the second processing result is performed again for the first preset number of times. This process is repeated to perform cyclic training until the number of cyclic training reaches a preset number (e.g., 5 times), and a pre-trained model is obtained. This improves the image matting and segmentation capabilities of the pre-trained model. This disclosure does not limit this aspect.

[0182] Step 907: Based on the first reference image, the pre-trained model is trained using the first dataset, the second dataset, the third dataset, and the fourth dataset to obtain the image processing model.

[0183] The specific implementation of step 907 can be found in the detailed description of other embodiments of this disclosure, and will not be repeated here.

[0184] In this embodiment, a first dataset, a second dataset, a third dataset, a fourth dataset, and a first reference image are first acquired. Then, the first reference image and the first original image in the first dataset are respectively input into an initial image processing model to obtain a first processing result. Based on the first difference between the first processing result and the first matting result, the initial image processing model is corrected to obtain a first model. Next, the first reference image and the second original image in the second dataset are respectively input into the first model to obtain a second processing result. Based on the second difference between the second processing result and the first segmentation result, the first model is corrected to obtain a second model. Finally, based on the second model, the operation of obtaining the second processing result is returned and executed until a first preset number of times is reached. Based on the obtained second model, the operation of obtaining the first processing result is returned and executed until a pre-trained model is obtained. Based on the first reference image, the pre-trained model is trained using the first dataset, the second dataset, the third dataset, and the fourth dataset to obtain an image processing model. Therefore, after obtaining the reference image, the original image, and their corresponding matting and segmentation results, the matting and segmentation capabilities of the initial image processing model are trained based on the original image and the reference image, respectively. Based on the differences between the processing results of the initial model and the matting and segmentation results, the initial model is corrected to obtain the corrected model. The corrected model is then trained alternately on the matting and segmentation capabilities to obtain the pre-trained model. Based on the mixture of image data and video data, the pre-trained model is trained on the matting and segmentation to obtain the image processing model. This improves the semantic understanding and image edge processing capabilities of the image processing model, thereby enhancing the overall performance of the image processing model.

[0185] Figure 10 This is a schematic diagram illustrating the process of modifying a model in a method for generating an image processing model according to another embodiment of this disclosure.

[0186] like Figure 10 As shown, the process of correcting the model based on the first or second processing result obtained from the above embodiments of this disclosure in the image processing model generation method may include the following steps:

[0187] Step 1001: Determine a first loss value based on the first distance between the main image and the labeled image in the labeled data, wherein the labeled data is the first matting result or the first segmentation result.

[0188] The first distance can be the distance between the model's processing result and the labeled data. It can be the first distance between the first processing result and the first matting result, or it can be the first distance between the second processing result and the first segmentation result. This disclosure does not limit it in this respect.

[0189] The first loss value can be the loss value between the first processing result and the first matting result, or it can be the loss value between the second processing result and the first segmentation result. This disclosure does not limit it in this respect.

[0190] It should be noted that the first processing result or the second processing result obtained through the above embodiments of this disclosure may include: the output of the decoder intermediate layer, the main image, and the probability that the main body belongs to a different type. This disclosure does not limit this.

[0191] The output of the intermediate layer of the decoder can be a matting mask or a segmentation mask, and this disclosure does not limit it.

[0192] It should be noted that the probability of subjects belonging to different types can be determined through the classification branch in the model architecture, and this disclosure does not limit this.

[0193] In this disclosure, after obtaining the first processing result or the second processing result, the first loss value of the model for matting training or the first loss value of the model for segmentation training can be determined based on the first processing result or the second processing result of the model on the subject image and the first distance between the first matting result or the first segmentation result.

[0194] It should be noted that when determining the first loss value based on the first processing result and the first matting result, the first loss value may also include the loss value of the first processing result and the real label, as well as the edge smoothness loss between the first processing result and the first matting result. For example, the first loss value when training the model for matting can be represented by formula (3), which is not limited in this disclosure:

[0195] L 第一损失值 =L 1(1) +λ1L lap +λ2L iou +λ3L tv +λ4L fgr , formula (3)

[0196] Among them, L 1(1) , which is the loss between the first processing result and the first matting result; Laplacian loss L lap IOU loss L iou All of these represent the loss between the first processing result and the true label; L tv L represents the edge smoothing loss between the first processing result and the real image; fgr λ1 represents the loss between the foreground image and the real foreground image in the first processing result, which can also be calculated using regression loss L1; λ1, λ2, λ3, and λ4 are all preset hyperparameters, whose values ​​can be set as needed, or can be preset, for example, the values ​​of λ1, λ2, λ3, and λ4 can be 1, 1, 1e, etc. -65. This disclosure does not limit this.

[0197] Here, Lap is short for Laplacian; IOU is short for Intersection Over Union (IOU); and tv is short for Total Variation Regularization.

[0198] It should be noted that when determining the first loss value based on the second processing result and the first segmentation result, the first loss value may also include the loss between the foreground image and the real foreground image in the second processing result. For example, the first loss value when performing segmentation training on the model can be expressed as formula (4), and this disclosure does not limit it:

[0199] L′ 第一损失值 =L bce(1) +λ5L fgr , formula (4)

[0200] Among them, L bce(1) λ is the loss between the second processing result and the first segmentation result; λ5 is a preset hyperparameter setting, for example, its value can be 5, and this disclosure does not limit it.

[0201] Here, bce is short for Binary Cross-Entropy (BCE).

[0202] Step 1002: Determine the second loss value based on the second distance between the output of the decoder intermediate layer and the labeled image.

[0203] The second distance can be the distance between the processing result output by the intermediate layer of the decoder and the labeled image. It can be the distance between the intermediate matting result and the labeled image, or it can be the distance between the intermediate segmentation result and the labeled image. This disclosure does not limit it in this respect.

[0204] The second loss value can be the loss value between the processing result (mating result or segmentation result) output by the intermediate layer of the decoder and the labeled image, and this disclosure does not limit it.

[0205] It should be noted that, to avoid overfitting the model on the training set, side outputs can be added to each layer of the decoder's intermediate layers, and these side outputs can be supervised. For example, let the decoder contain a total of 6 output layers. The first 5 layers (i.e., the intermediate layers) pass the features of each decoder layer through a linear transformation layer to obtain 2-channel mask images, which can represent matting masks or segmentation masks, respectively. These side output images are then upsampled to 512*512 for training supervision. The last layer outputs a 5*512*512 image, where, according to the channel division, 1*512*512 is the matting mask, 1*512*512 is the segmentation mask, and 3*512*512 is the predicted foreground image. This disclosure does not limit this.

[0206] It should be noted that when determining the second loss value based on the second distance, the output of the intermediate layer needs to be upsampled to the same size as the labeled image before determining the second loss value.

[0207] In this disclosure, after determining the first loss value, the second loss value for training the model for image matting or the second loss value for training the model for segmentation can be determined based on the second distance between the output of the decoder intermediate layer included in the first processing result or the second processing result and the labeled image. This disclosure does not limit this.

[0208] For example, when the decoder has 5 intermediate layers, the second loss value between the matting result output by the intermediate layers and the labeled data can be λ6L. 1(2-6) The second loss value between the segmentation result output by the intermediate layer of the decoder and the labeled data can be L. bce(2-6) , where (2-6) represents the intermediate layer output of the decoder, and λ6 can be a pre-set hyperparameter, such as its value can be 0.5, which is not limited in this disclosure.

[0209] Step 1003: Determine the prediction type of the subject based on the probability that the subject belongs to a different type.

[0210] In this disclosure, after determining the second loss value, the prediction type of the subject can be determined based on the probability that the subject belongs to different types contained in the first processing result or the second processing result, thereby improving the accuracy of the determined subject type.

[0211] It should be noted that when determining the prediction type of a subject, the prediction type can be determined based on the probability of the subject belonging to different types in multiple consecutive frames. For example, after obtaining the probability of the subject belonging to different types in a single frame, the probabilities of the subject belonging to different types in 10 consecutive frames can be obtained. When the probability of the subject belonging to a human is relatively high in more than 60% of these 10 frames, or when the probability of the subject belonging to a human is relatively high among the average probabilities of the subject belonging to different types in these 10 frames, the prediction type of the subject can be determined to be human. This disclosure does not limit this.

[0212] Step 1004: Determine the third loss value based on the matching degree between the predicted type and the annotation type in the labeled data.

[0213] The third loss value can be the classification loss between the predicted type and the labeled type of the subject. For example, the third loss value can be λ7L. cls It is indicated that λ7 can be a preset hyperparameter value, such as 0.1, and this disclosure does not limit it.

[0214] Here, cls is short for Classification.

[0215] In this disclosure, after determining the prediction type to which the subject belongs, a third loss value can be determined based on the matching degree between the prediction type and the labeling type in the labeled data, thereby providing conditions for improving the reliability of model correction.

[0216] Step 1005: Determine the target loss value based on the first loss value, the second loss value, and the third loss value.

[0217] The target loss value can be the final loss value corresponding to the first processing result or the second processing result.

[0218] In this disclosure, after determining the first loss value, the second loss value, and the third loss value corresponding to the first processing result or the second processing result, the target loss value corresponding to the second processing result can be determined based on the first loss value, the second loss value, and the third loss value, thereby improving the accuracy and reliability of the determined target loss value.

[0219] For example, when training a model by image matting, the target loss value L corresponding to the first processing result. mat As shown in formula (5), this disclosure does not limit the scope of the invention.

[0220] L mat =L 1(1) +λ6L 1(2-6) +λ1L lap +λ2L iou +λ3Ltv +λ4L fgr +λ7L cis , formula (5)

[0221] When training the model for segmentation, the target loss value corresponding to the second processing result can be as shown in formula (6), and this disclosure does not limit it:

[0222] L seg =L bce(1) +L bce(2-6) +λ5L fgr +λ7L cls , formula (6)

[0223] Here, mat stands for image matting; seg is short for segmentation.

[0224] It should be noted that when determining the target loss value, classification loss (L) is used. cls (Loss) and regression loss (L1 loss, L...) lap Loss, L iou The loss and other factors are jointly determined, and lateral supervision is introduced to enhance the model's ability to learn target boundary features, thereby improving the accuracy of the output of each layer of the decoder, better distinguishing different targets, and improving the accuracy and robustness of image matting and segmentation. This disclosure does not limit the scope of the invention.

[0225] Step 1006: Based on the target loss value, revise the model.

[0226] In this disclosure, after determining the target loss corresponding to the first processing result or the second processing result, the model can be modified based on the target loss value. For example, after determining the target loss value corresponding to the first processing result, the first model can be modified based on the target loss value to obtain the second model. Similarly, after determining the target loss value corresponding to the second processing result, the second model can be modified based on the target loss value, thereby improving the model's image matting and segmentation capabilities, as well as its semantic understanding and image edge processing capabilities.

[0227] In this embodiment, a first loss value is first determined based on a first distance between the subject image and the labeled image in the labeled data. Then, a second loss value is determined based on a second distance between the output of the decoder's intermediate layer and the labeled image. Next, the predicted type of the subject is determined based on the probability that the subject belongs to a different type. A third loss value is determined based on the matching degree between the predicted type and the labeled type in the labeled data. Finally, a target loss value is determined based on the first, second, and third loss values, and the model is corrected based on this target loss value. Thus, by determining the data contained in the first or second processing result, the target loss value for training the model for image matting and the target loss value for training the model for segmentation are determined respectively. Based on the target loss value, the model is corrected, thereby improving the semantic understanding and image processing capabilities of the corrected model.

[0228] Figure 11 This is a flowchart illustrating a method for generating an image processing model according to another embodiment of the present disclosure.

[0229] like Figure 11 As shown, the method for generating this image processing model may include the following steps:

[0230] Step 1101: Obtain the first dataset, the second dataset, the third dataset, the fourth dataset, and the first reference image.

[0231] The first dataset includes multiple sets of first original images and corresponding first matting results; the second dataset includes multiple sets of second original images and corresponding first segmentation results; the third dataset includes multiple sets of first videos and corresponding second matting results; and the fourth dataset includes multiple sets of second videos and corresponding second segmentation results.

[0232] Step 1102: Based on the first reference image, the initial image processing model is trained using the first dataset and the second dataset to obtain a pre-trained model.

[0233] The specific implementation of steps 1101 to 1102 can be found in the detailed description of other embodiments of this disclosure, and will not be repeated here.

[0234] Step 1103: Based on the first reference image and the first matting result, determine the second reference image corresponding to the first original image.

[0235] The second reference image can be the mask image corresponding to the first original image, i.e., the alpha image.

[0236] In this disclosure, after obtaining the pre-trained model, in order to train the model to ensure consistency and continuity of results when performing image processing, a second reference image corresponding to the first original image can be determined based on the first reference image and the first matting result.

[0237] Step 1104: Based on the first reference image and the first segmentation result, determine the third reference image corresponding to the second original image.

[0238] The third reference image can be the mask image corresponding to the second original image, i.e., the alpha image.

[0239] In this disclosure, after determining the second reference image corresponding to the first original image, in order to further improve the consistency and continuity of the model processing results, a third reference image corresponding to the second original image can be determined based on the first reference image and the first segmentation result.

[0240] Step 1105: Based on the second reference image, the third reference image, and the first reference image, the pre-trained model is trained using the first dataset, the second dataset, the third dataset, and the fourth dataset to obtain the image processing model.

[0241] It should be noted that when training the pre-trained model based on the second reference image, the third reference image, and the first reference image, and using the first dataset, the second dataset, the third dataset, and the fourth dataset, respectively, the model can be trained based on the second reference image using the first dataset, based on the third reference image using the second dataset, or based on the first reference image using the third and fourth datasets.

[0242] It should be noted that when processing non-first frames of a video, the mask image output by the model can be used as a reference image, and this disclosure does not limit this.

[0243] In this embodiment, a first dataset, a second dataset, a third dataset, a fourth dataset, and a first reference image are first acquired. Then, based on the first reference image, an initial image processing model is trained using the first and second datasets to obtain a pre-trained model. Next, based on the first reference image and a first matting result, a second reference image corresponding to the first original image is determined. Then, based on the first reference image and a first segmentation result, a third reference image corresponding to the second original image is determined. Finally, based on the second, third, and first reference images, the pre-trained model is trained using the first, second, third, and fourth datasets to obtain the image processing model. Therefore, after obtaining the pre-trained model, by determining the reference image corresponding to the original image and training the pre-trained model using the matting and segmentation datasets through image and video mixing, an image processing model is obtained, thereby improving the accuracy and reliability of the image processing model.

[0244] To implement the above embodiments, this disclosure also proposes a device for generating dynamic wallpapers.

[0245] Figure 12 This is a schematic diagram of the structure of a dynamic wallpaper generation apparatus provided in another embodiment of the present disclosure.

[0246] like Figure 12 As shown, the dynamic wallpaper generation device 1200 may include: a first acquisition module 1201, a determination module 1202, a processing module 1203, and a fusion module 1204.

[0247] The first acquisition module 1201 is used to acquire the first image and reference video corresponding to the target page element to be merged;

[0248] Module 1202 is used to determine the reference image;

[0249] The processing module 1203 is used to process the reference video based on the reference image using a preset image processing model to obtain the main image sequence, background image sequence and mask image sequence corresponding to the reference video.

[0250] The fusion module 1204 is used to fuse the first image with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence based on each mask image in the mask image sequence, to obtain a dynamic wallpaper containing the target page elements.

[0251] Optionally, the first acquisition module 1201 described above is specifically used for:

[0252] Based on the current settings of the electronic device, determine the target page element in the current lock screen of the electronic device;

[0253] Determine the first image corresponding to the target page element based on the display parameters associated with the target page element.

[0254] Optionally, the aforementioned determining module 1202 is specifically used for any of the following:

[0255] The preset second image is selected as the reference image;

[0256] Images associated with the type of subject contained in the reference video are identified as reference images.

[0257] Optionally, the aforementioned fusion module 1204 is specifically used for:

[0258] Determine the types of subjects contained in the reference video;

[0259] If the subject type is a preset type, the first image is merged with the corresponding background image in the background image sequence and the corresponding subject image in the subject image sequence.

[0260] Optionally, the above-mentioned processing module 1203 is specifically used for:

[0261] Based on the reference image, the reference video is processed using a preset image processing model to obtain the first main image, the first background image, and the first mask image corresponding to the first image frame in the reference video.

[0262] Based on the first mask image, return to the step of processing the reference video using a preset image processing model until the main image sequence, background image sequence, and mask image sequence corresponding to the reference video are obtained.

[0263] Optionally, the aforementioned fusion module 1204 is specifically used for:

[0264] Based on the pixel value at position i in the mask image, determine the first weight of the first pixel at position i in the corresponding background image and the second weight of the second pixel at position i in the corresponding main image, where i is a natural number;

[0265] Based on the first weight and the second weight, the first pixel value of the first pixel point, the second pixel value of the second pixel point, and the third pixel value of the third pixel point at position i in the first image are fused to obtain an image frame in the dynamic wallpaper.

[0266] Optionally, the above-mentioned processing module 1203 is further used for at least one of the following:

[0267] Determine the type of the subject contained in the reference video. If the type of the subject is not a preset type, return the first indication information, which is used to indicate that the subject type in the reference video is abnormal.

[0268] Determine the difficulty score of the reference video. If the difficulty score is greater than a threshold, return a second indication message, which indicates that the reference video does not meet the preset requirements.

[0269] The functions and specific implementation principles of the modules described in this embodiment can be found in the above method embodiments, and will not be repeated here.

[0270] The dynamic wallpaper generation apparatus of this disclosure first acquires a first image and a reference video corresponding to the target page element to be fused. Then, it determines a reference image. Next, based on the reference image, it processes the reference video using a preset image processing model to obtain a main image sequence, a background image sequence, and a mask image sequence corresponding to the reference video. Finally, based on each mask image in the mask image sequence, it fuses the first image with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence to obtain a dynamic wallpaper containing the target page element. Therefore, by extracting the main subject from the reference video based on the reference image, determining its corresponding main image sequence, background image sequence, and mask image sequence, and fusing the image corresponding to the target page element with the corresponding background image in the background image sequence and the main image in the main image sequence based on each mask image in the mask image sequence to generate a dynamic wallpaper, the effect and quality of the generated dynamic wallpaper are improved, the reliability of dynamic wallpaper generation is enhanced, and the user experience is improved.

[0271] Figure 13 This is a schematic diagram of the structure of an image processing model generation apparatus provided in another embodiment of the present disclosure.

[0272] like Figure 13 As shown, the image processing model generation device 1300 may include: a second acquisition module 1301, a first training module 1302, and a second training module 1303.

[0273] The second acquisition module 1301 is used to acquire a first dataset, a second dataset, a third dataset, a fourth dataset, and a first reference image. The first dataset includes multiple sets of first original images and corresponding first matting results. The second dataset includes multiple sets of second original images and corresponding first segmentation results. The third dataset includes multiple sets of first videos and corresponding second matting results. The fourth dataset includes multiple sets of second videos and corresponding second segmentation results.

[0274] The first training module 1302 is used to train the initial image processing model based on the first reference image, using the first dataset and the second dataset, to obtain a pre-trained model.

[0275] The second training module 1303 is used to train the pre-trained model based on the first reference image, using the first dataset, the second dataset, the third dataset, and the fourth dataset to obtain the image processing model.

[0276] Optionally, the first training module 1302 described above is specifically used for:

[0277] The first reference image and the first original image are respectively input into the initial image processing model to obtain the first processing result;

[0278] Based on the first difference between the first processing result and the first matting result, the initial image processing model is corrected to obtain the first model;

[0279] The first reference image and the second original image are respectively input into the first model to obtain the second processing result;

[0280] Based on the second difference between the second processing result and the first segmentation result, the first model is corrected to obtain the second model;

[0281] Based on the second model, return to execute the operation to obtain the second processing result until the first preset number of times is reached. Based on the obtained second model, return to execute the operation to obtain the first processing result until the pre-trained model is obtained.

[0282] Optionally, the first training module 1302 described above is also used for:

[0283] A first loss value is determined based on the first distance between the main image and the labeled image in the labeled data, wherein the labeled data is the first matting result or the first segmentation result;

[0284] The second loss value is determined based on the second distance between the output of the decoder's intermediate layer and the labeled image;

[0285] Based on the probability that the subject belongs to different types, determine the prediction type to which the subject belongs;

[0286] The third loss value is determined based on the matching degree between the predicted type and the labeling type in the labeled data;

[0287] The target loss value is determined based on the first loss value, the second loss value, and the third loss value.

[0288] The model is then modified based on the target loss value.

[0289] Optionally, the second training module 1303 mentioned above is specifically used for:

[0290] Based on the first reference image and the first matting result, a second reference image corresponding to the first original image is determined;

[0291] Based on the first reference image and the first segmentation result, a third reference image corresponding to the second original image is determined;

[0292] Based on the second reference image, the third reference image, and the first reference image, the pre-trained model is trained using the first dataset, the second dataset, the third dataset, and the fourth dataset to obtain the image processing model.

[0293] The functions and specific implementation principles of the modules described in this embodiment can be found in the above method embodiments, and will not be repeated here.

[0294] The image processing model generation apparatus of this disclosure first acquires a first dataset, a second dataset, a third dataset, a fourth dataset, and a first reference image. Then, based on the first reference image, it trains an initial image processing model using the first and second datasets to obtain a pre-trained model. Finally, based on the first reference image, it trains the pre-trained model using the first, second, third, and fourth datasets to obtain the final image processing model. Thus, by training the initial image processing model based on the reference image, the original image, and their corresponding matting and segmentation results to obtain a pre-trained model, and then using a mixture of image and video data to perform alternating matting and segmentation training on the pre-trained model, the final image processing model is obtained. This improves the semantic understanding and edge processing capabilities of the image processing model, thereby enhancing its effectiveness and accuracy.

[0295] To implement the above embodiments, this disclosure also proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the dynamic wallpaper generation method and the image processing model generation method proposed in the foregoing embodiments of this disclosure.

[0296] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the dynamic wallpaper generation method and the image processing model generation method proposed in the foregoing embodiments of this disclosure.

[0297] Figure 14 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Figure 14 The electronic device 1400 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0298] like Figure 14As shown, the electronic device 1400 is presented in the form of a general-purpose computing device. The components of the electronic device 1400 may include, but are not limited to: one or more processors or processing units 1416, system memory 1428, and bus 1418 connecting different system components (including system memory 1428 and processing unit 1416).

[0299] Bus 1418 represents one or more of several bus architectures, including memory buses or memory controllers, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. Examples of these architectures include, but are not limited to, Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses.

[0300] Electronic device 1400 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 1400, including volatile and non-volatile media, removable and non-removable media.

[0301] Memory 1428 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 1430 and / or cache memory 1432. Electronic device 1400 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 1434 may be used to read and write non-removable, non-volatile magnetic media (… Figure 14 Not shown; usually referred to as a "hard drive"). Although Figure 14Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media). In these cases, each drive may be connected to bus 1418 via one or more data media interfaces. Memory 1428 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0302] A program / utility 1440 having a set (at least one) of program modules 1442 may be stored, for example, in memory 1428. Such program modules 1442 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 1442 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0303] Electronic device 1400 can also communicate with one or more external devices 1414 (e.g., keyboard, pointing device, display 1424, etc.), and with one or more devices that enable a user to interact with electronic device 1400, and / or with any device that enables electronic device 1400 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 1422. Furthermore, electronic device 1400 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1420. As shown, network adapter 1420 communicates with other modules of electronic device 1400 via bus 1418. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0304] The processing unit 1416 executes various functional applications and data processing by running programs stored in the system memory 1428, such as implementing the methods mentioned in the foregoing embodiments.

[0305] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0306] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0307] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0308] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0309] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0310] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0311] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0312] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for generating a dynamic wallpaper, characterized in that, include: Obtain the first image and reference video corresponding to the target page element to be merged; Identify a reference image; Based on the reference image, the reference video is processed using a preset image processing model to obtain the main image sequence, background image sequence, and mask image sequence corresponding to the reference video. Based on each mask image in the mask image sequence, the first image is fused with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence to obtain a dynamic wallpaper containing the target page elements.

2. The method as described in claim 1, characterized in that, The step of obtaining the first image corresponding to the target page element to be merged includes: Based on the current settings of the electronic device, determine the target page element in the current lock screen interface of the electronic device; Based on the display parameters associated with the target page element, determine the first image corresponding to the target page element.

3. The method as described in claim 1, characterized in that, The determined reference image includes any one of the following: The preset second image is determined as the reference image; The image associated with the type of the subject contained in the reference video is determined as the reference image.

4. The method as described in claim 1, characterized in that, The step of fusing the first image with the corresponding background image in the background image sequence and the corresponding subject image in the subject image sequence includes: Determine the type of subject contained in the reference video; If the type of the subject is a preset type, the first image is fused with the background image corresponding to the background image sequence and the subject image corresponding to the subject image sequence.

5. The method as described in claim 1, characterized in that, The step of processing the reference video based on the reference image using a preset image processing model to obtain the main image sequence, background image sequence, and mask image sequence corresponding to the reference video includes: Based on the reference image, the reference video is processed using the preset image processing model to obtain the first main image, the first background image, and the first mask image corresponding to the first image frame in the reference video. Based on the first mask image, return to the step of processing the reference video using the preset image processing model until the main image sequence, background image sequence, and mask image sequence corresponding to the reference video are obtained.

6. The method according to any one of claims 1-5, characterized in that, The step of fusing the first image with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence based on each mask image in the mask image sequence to obtain a dynamic wallpaper containing the target page elements includes: Based on the pixel value at position i in the mask image, determine the first weight of the first pixel at position i in the corresponding background image and the second weight of the second pixel at position i in the corresponding main image, where i is a natural number; Based on the first weight and the second weight, the first pixel value of the first pixel point, the second pixel value of the second pixel point, and the third pixel value of the third pixel point at position i in the first image are fused to obtain an image frame in the dynamic wallpaper.

7. The method as described in claim 6, characterized in that, After processing the reference video using a preset image processing model, the process further includes at least one of the following: Determine the type of the subject contained in the reference video. If the type of the subject is not a preset type, return first indication information, wherein the first indication information is used to indicate that the subject type in the reference video is abnormal. The difficulty score of the reference video is determined. If the difficulty score is greater than a threshold, a second indication message is returned, wherein the second indication message is used to indicate that the reference video does not meet the preset requirements.

8. A method for generating an image processing model, characterized in that, include: Acquire a first dataset, a second dataset, a third dataset, a fourth dataset, and a first reference image. The first dataset includes multiple sets of first original images and corresponding first matting results. The second dataset includes multiple sets of second original images and corresponding first segmentation results. The third dataset includes multiple sets of first videos and corresponding second matting results. The fourth dataset includes multiple sets of second videos and corresponding second segmentation results. Based on the first reference image, the initial image processing model is trained using the first dataset and the second dataset to obtain a pre-trained model; Based on the first reference image, the pre-trained model is trained using the first dataset, the second dataset, the third dataset, and the fourth dataset to obtain an image processing model.

9. The method as described in claim 8, characterized in that, The step of training an initial image processing model based on the first reference image, using the first dataset and the second dataset, to obtain a pre-trained model includes: The first reference image and the first original image are respectively input into the initial image processing model to obtain the first processing result; Based on the first difference between the first processing result and the first matting result, the initial image processing model is corrected to obtain the first model; The first reference image and the second original image are respectively input into the first model to obtain the second processing result; Based on the second difference between the second processing result and the first segmentation result, the first model is corrected to obtain the second model; Based on the second model, return to execute the operation to obtain the second processing result until the first preset number of times is reached. Based on the obtained second model, return to execute the operation to obtain the first processing result until the pre-trained model is obtained.

10. The method as described in claim 9, characterized in that, The first processing result or the second processing result respectively includes: the output of the decoder intermediate layer, the subject image, and the probability that the subject belongs to a different type. The method further includes correcting the model based on the following process: A first loss value is determined based on the first distance between the main image and the labeled image in the labeled data, wherein the labeled data is the first matting result or the first segmentation result; The second loss value is determined based on the second distance between the output of the intermediate layer of the decoder and the labeled image; Based on the probability that the subject belongs to different types, determine the prediction type to which the subject belongs; A third loss value is determined based on the matching degree between the predicted type and the annotation type in the labeled data; Based on the first loss value, the second loss value, and the third loss value, the target loss value is determined; The model is then modified based on the target loss value.

11. The method according to any one of claims 8-10, characterized in that, The step of training the pre-trained model based on the first reference image, using the first dataset, the second dataset, the third dataset, and the fourth dataset to obtain an image processing model includes: Based on the first reference image and the first matting result, a second reference image corresponding to the first original image is determined; Based on the first reference image and the first segmentation result, a third reference image corresponding to the second original image is determined; The image processing model is obtained by training the pre-trained model using the first dataset, the second dataset, the third dataset, and the fourth dataset, based on the second reference image, the third reference image, and the first reference image, respectively.

12. A device for generating dynamic wallpapers, characterized in that, The device includes: The first acquisition module is used to acquire the first image and reference video corresponding to the target page element to be merged; The determination module is used to determine the reference image; The processing module is used to process the reference video based on the reference image using a preset image processing model to obtain the main image sequence, background image sequence and mask image sequence corresponding to the reference video; The fusion module is used to fuse the first image with the corresponding background image in the background image sequence and the corresponding main image in the main image sequence based on each mask image in the mask image sequence, to obtain a dynamic wallpaper containing the target page elements.

13. An apparatus for generating an image processing model, characterized in that, The device includes: The second acquisition module is used to acquire a first dataset, a second dataset, a third dataset, a fourth dataset, and a first reference image. The first dataset includes multiple sets of first original images and corresponding first matting results. The second dataset includes multiple sets of second original images and corresponding first segmentation results. The third dataset includes multiple sets of first videos and corresponding second matting results. The fourth dataset includes multiple sets of second videos and corresponding second segmentation results. The first training module is used to train the initial image processing model based on the first reference image, using the first dataset and the second dataset, to obtain a pre-trained model. The second training module is used to train the pre-trained model based on the first reference image, using the first dataset, the second dataset, the third dataset, and the fourth dataset, to obtain an image processing model.

14. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method for generating a dynamic wallpaper as described in any one of claims 1-7, or the method for generating an image processing model as described in any one of claims 8-11.

15. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for generating a dynamic wallpaper as described in any one of claims 1-7, or the method for generating an image processing model as described in any one of claims 8-11.

Images