Video generation method and electronic device

Abstract

Embodiments of the present application relate to the technical field of electronic devices, and provide a video generation method and an electronic device. The method enables an electronic device to automatically generate, on the basis of an instruction of a user, a video stream corresponding to a selected image, wherein the video stream can be used as a dynamic wallpaper. The method may comprise: receiving a first operation input by a user, the first operation corresponding to a first image, and the first operation being used for instructing an electronic device to generate a dynamic wallpaper; generating a first descriptive text on the basis of the image information of the first image; and generating a first video stream on the basis of the image information of the first image and the first descriptive text, the first video stream corresponding to the dynamic wallpaper of the first image.

Description

Video generation method and electronic device

[0001] The present application claims priority from the Chinese patent application No. 202411271265.5 filed on September 10, 2024, and entitled "Video generation method and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Embodiments of the present application relate to the technical field of electronic devices, and in particular to a video generation method and an electronic device. BACKGROUND

[0003] The wallpaper of an electronic device can generally be personalized by a user. Existing wallpapers can include static wallpapers and dynamic wallpapers.

[0004] A general dynamic wallpaper can be pre-made. This can limit the range of user selection of dynamic wallpapers. SUMMARY

[0005] The present application provides a video generation method and an electronic device, which can enable the electronic device to generate a video stream corresponding to a selected image according to a user's indication, and the video stream can be used as a dynamic wallpaper. Through the scheme provided by the embodiments of the present application, the generation of dynamic wallpapers can be more accurate and reasonable.

[0006] To achieve the above technical purposes, the present application adopts the following technical solutions:

[0007] In a first aspect, a video generation method is provided, which is applied to an electronic device. The method includes: receiving a first operation input by a user, the first operation corresponding to a first image, and the first operation being used to instruct the electronic device to generate a dynamic wallpaper. Generating a first description text according to image information of the first image. Generating a first video stream according to the image information of the first image and the first description text, the first video stream corresponding to a dynamic wallpaper of the first image.

[0008] In some implementations, the image information can further include a name of the first image.

[0009] Based on the scheme, the electronic device can automatically generate a corresponding video stream according to a frame of image corresponding to the first image in a case that the user triggers generation of a dynamic wallpaper corresponding to the first image. The video stream can correspond to the dynamic wallpaper of the first image. In the process of generating the video stream, the electronic device can generate the video stream in combination with the image information of the first image, the first description text corresponding to the first image, and thus the dynamic wallpaper corresponding to the first image can be ensured.

[0010] In some possible designs, before the first video stream is generated, the method further includes: obtaining first information and second information corresponding to the first image according to the image information of the first image. The first information includes information of a subject image in the first image, and the second information includes information processed by variational encoding corresponding to the first image.

[0011] In some possible designs, the generating the first video stream according to the image information of the first image and the first description text includes: generating the first video stream according to the first information, the second information, third information and the first description text. The third information includes information of noise images of a preset frame number, and the preset frame number is greater than or equal to 2.

[0012] In some possible designs, the noise image includes momentum Euler discretization noise (EDM noise).

[0013] In some possible designs, the first video stream includes images of a preset frame number.

[0014] Through the scheme, the input features for generating the first video stream are determined. According to the above scheme, since the information of the subject image (for example, an image obtained by subject cutout) of the first image is used when the first video stream is generated, each frame of image in the first video stream can be obtained by evolving the subject image. Therefore, the dynamic effect of the object in the first video stream can be for the subject image, rather than the background image, and the rationality of the first video stream and the corresponding dynamic wallpaper and the accurate correspondence with the first image are ensured.

[0015] In addition, in some designs, when the first video stream is generated, third information can also be configured. The third information is noise of multiple frames of images. In this way, by adjusting the frame number (for example, the preset frame number) of the noise in the third information, the frame number of the first video stream can be flexibly adjusted.

[0016] In the present application, the description text of the first image can also be referred to for generating the first video stream.

[0017] In some possible design, the electronic device is configured with a video generation diffusion model. The first video stream is generated according to the image information of the first image and the first description text, including: inputting the first information corresponding to the first image, the second information, the third information, and the first description text into the video generation diffusion model. The video generation diffusion model is controlled to output the first video stream.

[0018] In some possible design, before the first description text is generated according to the image information of the first image, the method further includes: determining a first label set corresponding to the first image according to the image information of the first image. The first label set includes at least one first label, and the first label is used to identify an object in the first image and / or an activity attribute of the object in the first image. In some implementation, the first label can also be referred to as a feature label of the first image.

[0019] In some possible design, the method further includes: determining the first description text according to the first label set.

[0020] In some possible design, the determining the first description text according to the first label set includes: determining a second label set according to the first label set. The second label set includes at least one second label, and any one of the second labels is also included in the first label set. The first description text is determined according to the second label set. In some implementation, the second label can also be referred to as a description label of the first image.

[0021] In some possible design, the electronic device is preconfigured with a third label set, and the third label set includes at least one third label, and each third label is used to identify an image of a corresponding scene. The determining the second label set according to the first label set includes: for each first label, if the first label is included in the third label set, determining that the first label is a second label in the second label set. In some implementation, the third label can also be referred to as a typical label.

[0022] Thus, some specific implementations of generating the first description text are provided. In which, each label in the first label set can completely identify each image feature (such as an object in the image, a behavior feature of each object, etc.) in the first image. The third label set can be preconfigured, and the third label set has a higher reference value. The higher reference value can be reflected in that, through the third label or the description text corresponding to the third label, the electronic device can accurately restore the training video. In this way, when the third label or the description text corresponding to the third label is used for inference of video stream generation, a more accurate video stream reflecting the current scene can be obtained.

[0023] Based on this, in this example, the second label set is obtained as the description label corresponding to the first image in combination with the first label set and the third label set. These description labels can accurately identify the first image on the one hand, and can also be used to accurately generate the corresponding video stream. Thus, the correspondence accuracy of the first description text and the first image can be ensured. Further, the first video stream generated according to the first description text can be more reasonably corresponded to the first image.

[0024] In some possible designs, the determining the first description text according to the second label set includes: determining the first description text according to the second label set and preset prompts.

[0025] In some possible designs, before the generating the first description text, the method further includes: determining, according to image information of the first image, that the first image satisfies a preset condition. The preset condition includes at least one of the following: a resolution of the first image is greater than a resolution threshold. A brightness of the first image is greater than a brightness threshold. A definition of the first image is greater than a definition threshold. Thus, in some implementations, a pre-judgment mechanism before generating the dynamic wallpaper is provided. In this implementation, in the case of passing the judgment of the preset condition, the selected first image can be used to generate a dynamic wallpaper of higher quality.

[0026] In some possible designs, after the generating the first video stream, the method further includes: storing the first video stream, and taking the first video stream as the dynamic wallpaper corresponding to the first image. In this example, the first video stream generated by the electronic device (such as the video stream output by the video generation diffusion model) can be directly stored as the dynamic wallpaper.

[0027] In some possible designs, after the generating the first video stream, the method further includes: performing at least one of the following processing on the first video stream to obtain a second video stream: resolution adjustment processing, frame number adjustment processing, and marking processing. Storing the second video stream, and taking the second video stream as the dynamic wallpaper corresponding to the first image. In this example, the electronic device can further perform subsequent processing on the generated first video stream, thereby obtaining a second video stream with better display effect as the dynamic wallpaper corresponding to the first image.

[0028] In a second aspect, the present application also provides an electronic device, which includes a memory and one or more processors. The memory and the processor are coupled. The memory is configured to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the technical solutions provided in the first aspect and any possible implementation thereof.

[0029] In a third aspect, the present application also provides a chip system, which is applied to an electronic device. The chip system can include one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected through a circuit. The interface circuit is configured to receive a signal from a memory of the electronic device and send the signal to the processor. The signal includes computer instructions stored in the memory. When the processor executes the computer instructions, the electronic device executes the technical solutions provided in the first aspect and any possible implementation thereof.

[0030] In a fourth aspect, the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the technical solutions provided in the first aspect and any possible implementation thereof.

[0031] In a fifth aspect, the present application also provides a computer program product. When the computer program product is executed on a computer, the computer executes the technical solutions provided in the first aspect and any possible implementation thereof.

[0032] It can be understood that the technical solutions provided in the second aspect to the fifth aspect of the present application can correspond to the first aspect and any possible implementation thereof, respectively. Therefore, the beneficial effects that can be achieved are similar, and thus will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0033] FIG. 1 is a schematic diagram of interface interaction according to an embodiment of the present application;

[0034] FIG. 2 is a schematic diagram of another interface interaction according to an embodiment of the present application;

[0035] FIG. 3 is a schematic diagram of another interface interaction according to an embodiment of the present application;

[0036] FIG. 4 is a logical diagram of a dynamic wallpaper according to an embodiment of the present application;

[0037] FIG. 5 is a schematic diagram of another interface interaction according to an embodiment of the present application;

[0038] FIG. 6 is a schematic diagram of an electronic device according to an embodiment of the present application;

[0039] FIG. 7 is a schematic diagram of a generation module according to an embodiment of the present application;

[0040] FIG. 8 is a logical diagram of interaction between modules according to an embodiment of the present application;

[0041] FIG. 9 is a working logic diagram of a video generation diffusion model according to an embodiment of the present application;

[0042] FIG. 10 is a schematic diagram of a pre-processing module according to an embodiment of the present application;

[0043] FIG. 11 is a schematic diagram of a post-processing module according to an embodiment of the present application;

[0044] FIG. 12 is a schematic diagram of an interaction between modules according to an embodiment of the present application;

[0045] FIG. 13 is a schematic diagram of another electronic device according to an embodiment of the present application;

[0046] FIG. 14 is a schematic diagram of another electronic device according to an embodiment of the present application;

[0047] FIG. 15 is a schematic diagram of a chip system according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] When using the electronic device, the user can set different wallpapers for the electronic device. The wallpaper can include static wallpaper and dynamic wallpaper. The static wallpaper can include one frame of image. The dynamic wallpaper can include a video stream composed of multiple frames of images.

[0049] Taking the user setting the dynamic wallpaper for the electronic device as an example.

[0050] Referring to FIG. 1, a schematic diagram of interface interaction is shown.

[0051] In this example, the electronic device can display icons of installed applications on the main interface (e.g., interface 01). For example, the icon C1 of the gallery application and the icon C2 of the theme application can be displayed on the interface 01.

[0052] The user can input a click operation on the icon C2 to instruct the electronic device to run the theme application.

[0053] In response to the click operation on the icon C2, the electronic device can run the theme application and switch to display the corresponding interface 02.

[0054] For example, the interface 02 can be the interface corresponding to the theme application. As shown in FIG. 1, the interface 02 can include options for personalization configuration of wallpaper, font, etc.

[0055] In the wallpaper configuration option, the user can be shown wallpaper resources on the local and / or cloud.

[0056] For example, the wallpaper resources can include static wallpaper resources, such as wallpaper 1, wallpaper 2, wallpaper 3, etc.

[0057] For another example, the wallpaper resources can also include dynamic wallpaper resources, such as dynamic wallpaper 1, dynamic wallpaper 2, etc.

[0058] It can be understood that the wallpaper resource displayed in the interface 02 can be pre-configured locally or in the cloud. In this way, after the theme application is running, the theme application can display the pre-configured wallpaper resource to the user on the interface 02, so that the user can select.

[0059] Taking the user-selected dynamic wallpaper 1 as an example.

[0060] The user can input a click operation on the dynamic wallpaper 1. In response to the click operation on the dynamic wallpaper 1, the electronic device can switch to display a detail interface (such as interface 03) of the dynamic wallpaper 1.

[0061] The interface 03 can include a preview image of the dynamic wallpaper 1 and other display content. In this example, the interface 03 can also include a control button B1 for setting the dynamic wallpaper 1 as a wallpaper.

[0062] When the user wants to select the dynamic wallpaper 1 as the wallpaper of the current electronic device, the user can input an operation (such as a click operation) on the control button B1. Correspondingly, the electronic device can set the dynamic wallpaper 1 as a lock screen wallpaper and / or a background wallpaper under the instruction of the user.

[0063] It can be understood that in the interaction process shown in FIG. 1, the wallpaper resource such as the dynamic wallpaper can be pre-configured. For example, the dynamic wallpaper can be pre-made and saved in the local of the electronic device or in the cloud server of the theme application. In this way, it can result in limited available resources of the dynamic wallpaper and the user cannot actively configure the dynamic wallpaper. In addition, the pre-configuration (such as generation, storage, etc.) of the dynamic wallpaper can also cause a large amount of human resources and storage space.

[0064] The technical scheme provided by the embodiments of the present application can enable the electronic device to generate a dynamic wallpaper according to the image selected by the user. The dynamic wallpaper generated by the scheme has no limitation on the image selected by the user, and the generated dynamic wallpaper corresponds to the image selected by the user, thereby achieving the active configuration of the dynamic wallpaper by the user and increasing the available range of the dynamic wallpaper. In addition, it can also save the human resources and storage space caused by the pre-configuration of the wallpaper resource.

[0065] The scheme provided by the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0066] Referring to FIG. 2, another schematic diagram of interface interaction is provided. In the case where the scheme provided by the embodiments of the present application is effective, the user can instruct the electronic device to generate a dynamic wallpaper through the interaction process shown in FIG. 2.

[0067] In the example of FIG. 2, taking the function control of the electronic device for displaying the automatically generated dynamic wallpaper to the user in the interface of the theme application as an example.

[0068] In combination with the interaction example in FIG. 1, the electronic device can run the theme application and switch to display interface 02 after the user inputs a click operation on icon C2.

[0069] As shown in the example of FIG. 2, interface 02 can include control button B2 in addition to the display content shown in FIG. 1. Control button B2 can provide a function entry for automatically generating a dynamic wallpaper.

[0070] For example, the user can click control button B2 to instruct the electronic device to automatically generate a dynamic wallpaper.

[0071] In response to the user operation on control button B2, the electronic device can switch to display interface 04. In interface 04, a detail page for automatically generating a dynamic wallpaper can be shown to the user.

[0072] In this example, interface 04 can include an image adding control. The image adding control can be used to guide the user to select a target image so that the electronic device can generate a dynamic wallpaper based on the target image.

[0073] In some implementations, the target image can include an image stored locally by the electronic device. For example, the target image can include an image in a gallery application.

[0074] In other implementations, the target image can also be selected through a network.

[0075] For example, the target image is an image in a gallery application.

[0076] As shown in FIG. 2, after the user clicks the image adding control on interface 04, the electronic device can display interface 05 of the gallery application. In interface 05, thumbnails of various images stored in the gallery application can be included.

[0077] For example, the gallery application stores image M1. In interface 05, multiple thumbnails including a thumbnail of image M1 can be shown to the user.

[0078] The user can select any image in the gallery application and instruct the electronic device to generate a dynamic wallpaper based on the image.

[0079] For example, the user selects image M1 and instructs the electronic device to generate a dynamic wallpaper based on image M1. In this way, image M1 is the target image.

[0080] The user can click the thumbnail of the image M1 on the interface 05 to select the image M1. Correspondingly, the electronic device can switch to display the interface 06. The interface 06 can correspond to the interface 04 described above. In the interface 06, the display region of the image adding control of the interface 04 can be replaced by a preview of the image M1 that has been selected.

[0081] After the user selects the target image, the electronic device can show the user a function button for triggering the generation of the dynamic wallpaper. For example, the electronic device can show the user the control button B3 on the interface 06. The control button B3 can be used to trigger the start of the generation of the dynamic wallpaper.

[0082] The user can click the control button B3 to instruct the electronic device to generate a corresponding dynamic wallpaper according to the selected target image (such as the image M1).

[0083] Correspondingly, the electronic device can generate a corresponding dynamic wallpaper according to the image M1. The specific process of generating the dynamic wallpaper will be described later.

[0084] After generating the dynamic wallpaper, the electronic device can also show the user the automatically generated dynamic wallpaper.

[0085] In some embodiments, the electronic device can display the newly generated dynamic wallpaper in the wallpaper resource display region of the theme application. Taking the newly generated dynamic wallpaper as an example, the dynamic wallpaper is LW1.

[0086] Referring to FIG. 3, after generating the dynamic wallpaper LW1 according to the image M1, the electronic device can show the user the wallpaper resource of the dynamic wallpaper LW1 on the interface 02 of the theme application.

[0087] When the user wants to use the dynamic wallpaper LW1, the user can click the dynamic wallpaper LW1 on the interface 02.

[0088] Correspondingly, the electronic device can switch to display the detail page of the dynamic wallpaper LW1. For example, the electronic device can switch to display the interface 08. The interface 08 can include a preview of the dynamic wallpaper LW1 and a control button B1 for setting the dynamic wallpaper LW1 as a wallpaper.

[0089] It can be understood that the dynamic wallpaper LW1 can include a video stream composed of multiple images corresponding to the image M1.

[0090] As an example, FIG. 4 provides a schematic diagram of the logical structure of the dynamic wallpaper LW1.

[0091] In this example, the dynamic wallpaper LW1 can include multiple frames of images. Each frame of image can be related to the image M1. For example, any frame of image can correspond to the effect of the subject image (e.g., a person) in the image M1 after micro-motion processing according to certain rules. In this way, the continuous display of the multiple frames of images can present the display effect of a video stream of the continuous activity of the subject image in the image M1.

[0092] As shown in FIG. 4, the dynamic wallpaper LW1 can include frame A, frame B, and frame C. The timing of frame A is earlier than that of frame B, and the timing of frame B is earlier than that of frame C.

[0093] In frame A, the electronic device can perform micro-motion processing on the subject image in the image M1, so that the person corresponding to the subject image is in position A in the entire image.

[0094] In frame B, the electronic device can perform micro-motion processing on the subject image in the image M1, so that the person corresponding to the subject image is in position B in the entire image.

[0095] In frame C, the electronic device can perform micro-motion processing on the subject image in the image M1, so that the person corresponding to the subject image is in position C in the entire image.

[0096] The positions A, B, and C can correspond to the positions of the person in the order of continuous activity, respectively. For example, position A can be close to the left edge of the image, position B can be close to the middle of the image, and position C can be close to the right edge of the image. In this way, when the images of frame A, frame B, and frame C are played continuously, the continuous activity effect of the person from left to right can be presented.

[0097] In some implementations, the posture of the person corresponding to the subject image can also be adjusted according to the different positions.

[0098] Continuing to refer to FIG. 3, in some implementations, the preview of the dynamic wallpaper LW1 in the interface 08 can play the video stream as shown in FIG. 4 in a loop. In this way, the user can conveniently determine the display effect of the dynamic wallpaper LW1.

[0099] When the user wants to use the dynamic wallpaper LW1, the user can input a click operation on the control button B1 in the interface 08. Correspondingly, the electronic device can set the dynamic wallpaper LW1 as the lock screen wallpaper or the background wallpaper after unlocking.

[0100] The above-mentioned interaction examples provided by FIG. 2 and FIG. 3 are examples of the electronic device automatically generating a dynamic wallpaper in the interface of the theme application according to the user's indication. In other embodiments, the electronic device can also provide other entrances for the user to operate the electronic device to automatically generate a dynamic wallpaper.

[0101] For example, referring to FIG. 5, an example is provided in which a user triggers automatic generation of a dynamic wallpaper in a gallery application.

[0102] In the example of FIG. 5, the home page (e.g., interface 01) of the electronic device displays the icon C1 of the gallery application, as in the example of FIG. 1.

[0103] In the example of FIG. 5, the home page (e.g., interface 01) of the electronic device displays the icon C1 of the gallery application, as in the example of FIG. 1.

[0104] In the example of FIG. 5, the home page (e.g., interface 01) of the electronic device displays the icon C1 of the gallery application, as in the example of FIG. 1.

[0105] The interface 09 can include images that have been captured or obtained from other locations (e.g., network locations). In some implementations, the interface 09 can display information about each image to the user in the form of a thumbnail.

[0106] For example, the interface 09 can include a thumbnail of the image M1.

[0107] The user can input a preset operation on the thumbnail of the image M1. For example, the preset operation can be operation OP1. In some examples, the operation OP1 can include a long press operation, a preset gesture, or the like.

[0108] In response to the operation OP1, the electronic device can display a drop-down menu 51 corresponding to the image M1 to the user. The drop-down menu 51 can include at least one shortcut to an operation related to the image M1. For example, the drop-down menu 51 can include a shortcut to a wallpaper setting operation related to the image M1.

[0109] As in the example of FIG. 5, the drop-down menu 51 can include a control such as option 52. The option 52 can correspond to a function of generating a dynamic wallpaper based on the image M1. In a case in which the user wants to generate a dynamic wallpaper based on the image M1, the user can input a corresponding operation (e.g., a click operation) on the option 52.

[0110] In response to the operation OP1, the electronic device can display a drop-down menu 51 corresponding to the image M1 to the user. The drop-down menu 51 can include at least one shortcut to an operation related to the image M1. For example, the drop-down menu 51 can include a shortcut to a wallpaper setting operation related to the image M1.

[0111] The embodiments of the present disclosure also provide a processing method that enables an electronic device to automatically and accurately generate a dynamic wallpaper based on a user-selected image (e.g., image M1).

[0112] The processing method provided by the embodiments of the present disclosure is described in detail below with reference to the accompanying drawings.

[0113] It should be noted that the technical solutions provided in the embodiments of the present application can be applied to an electronic device of a user. The electronic device can include a display screen. Through the processing method provided in the embodiments of the present application, the electronic device can generate a corresponding dynamic wallpaper according to the selected image under the indication of the user according to any one of the above interaction modes or other interaction modes.

[0114] In different embodiments, the specific implementation of the electronic device can be different.

[0115] For example, the electronic device can include at least one of a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device, or a smart city device. The embodiments of the present application do not specially limit the specific type of the electronic device.

[0116] In the embodiments of the present application, one or more software modules can be configured in the electronic device, and each software module cooperates with each other, so that the electronic device can generate a corresponding dynamic wallpaper according to the selected image.

[0117] As an example, referring to FIG. 6, a schematic diagram of the composition of an electronic device is provided. Through the configuration as shown in FIG. 6, the electronic device can generate a corresponding dynamic wallpaper according to the wallpaper selected by the user.

[0118] It should be noted that the various component modules of the electronic device as shown in FIG. 6 are only examples. The implementation of the scheme as shown in FIG. 6 does not constitute any limitation on the electronic device. In other embodiments, the electronic device can further include more or fewer modules. One or more modules can be configured by splitting or integrating.

[0119] As shown in FIG. 6, in this example, the electronic device can include a preprocessing module, a generation module, and a post-processing module.

[0120] The preprocessing module can be used to pre-process the target image selected by the user, so as to determine that the target image meets the preset requirements for generating a dynamic wallpaper.

[0121] The generation module can be configured to identify the target image, and generate a description text corresponding to the target image. The generation module can further generate a dynamic wallpaper corresponding to the target image according to the description text corresponding to the target image and a preset generation model.

[0122] The post-processing module can further process the generated dynamic wallpaper. For example, the post-processing module can perform aesthetic scoring, resolution adjustment, frame number adjustment, marking, and the like on the generated dynamic wallpaper. In this way, the dynamic wallpaper can achieve better display effect.

[0123] It should be noted that in some embodiments of the present application, the processing device of the electronic device can only be configured with part of the modules as shown in FIG. 6. For example, the electronic device can only be configured with the generation module.

[0124] In this way, after the user selects the target image, the corresponding dynamic wallpaper can be generated by the generation module.

[0125] For example, the electronic device is configured with the generation module.

[0126] Referring to FIG. 7, a schematic diagram of a generation module is provided.

[0127] As shown in FIG. 7, in this example, the generation module can include a plurality of functional units, and each functional unit can be configured to cooperate with each other to achieve the purpose of generating a dynamic wallpaper by itself.

[0128] As shown in FIG. 7, the generation module can include an identification unit, a matching unit, a description unit, and a video generation unit.

[0129] In the embodiments of the present application, the identification unit can be configured to perform image analysis on the target image to determine a feature tag set D of the image.

[0130] The matching unit can be configured to determine a description tag set F of the target image by matching and screening from the feature tag set D of the target image according to a preset tag set E. The description tag set F can be an intersection of the preset tag set and the feature tag set D. The description tag set F can be used to generate a description text H of the target image.

[0131] The description unit can be configured to determine the corresponding description text H according to the description tag set F of the target image.

[0132] The video generation unit can be configured to generate a video stream corresponding to the target image according to the image information of the target image and the description text H. The video stream can include a plurality of frames of images, each frame of image can correspond to the target image, and the video stream corresponding to the plurality of frames of images can correspond to the dynamic wallpaper corresponding to the target image.

[0133] Referring to FIG. 8, a schematic diagram of inter-module interaction of the generation module when generating a dynamic wallpaper is provided based on the composition as shown in FIG. 7.

[0134] As shown in FIG. 8, when the generation module is working, the electronic device can input image information of a target image to a recognition unit. The image information can include at least one of the following: information indicating the length and width of the target image, color information of each pixel point of the target image, and the like. The information indicating the length and width of the target image can specifically include the length and width dimensions of the target image, or the size dimensions of the target image, or the resolution information of the target image.

[0135] In some embodiments, the recognition unit can be pre-configured with a Recognize Anything Plus Model (RAM++). Through the RAM++, the target image can be analyzed to determine the label of the target image.

[0136] Correspondingly, the recognition unit can determine one or more labels corresponding to the target image according to the image information of the target image. The one or more labels corresponding to the target image are also referred to as a feature label set D corresponding to the target image.

[0137] In this application, the labels in the feature label set D are referred to as feature labels or first labels.

[0138] In some implementations, the feature labels can correspond to each image element or each object in the target image, respectively. As a possible implementation, the objects in the image can include independent individuals such as people, cats, dogs, chairs, windows, seas, and clouds.

[0139] In other implementations, the feature labels can include labels representing the activity attributes (or behavior features) of the objects in the target image. As a possible implementation, the activity attributes can include running, standing, swimming, and the like.

[0140] Taking the target image as the image M1 in the foregoing example, the image M1 can include elements such as people and running.

[0141] In the case where the image information of the image M1 is input to the recognition unit, the recognition unit can determine that the feature label set D of the target image (such as the image M1) can include a “person” label and a “motion” label. In this example, the label 1 is the “person” label, and the label 2 is the “motion” label.

[0142] Taking the target image as the image M2, the image M2 includes “a cat lying on a chair, and a window behind the chair” as an example.

[0143] In the case that the image information of the target image (e.g., image M2) is input to the recognition unit, the recognition unit can determine that the feature label set D of the target image (i.e., image M2) can include labels such as "cat", "window", "chair", and the like.

[0144] In the following examples, the target image is taken as image M2. Correspondingly, the recognition unit can determine that the feature label set corresponding to image M2 is denoted as D = {cat, window, chair}.

[0145] As shown in FIG. 8, the recognition unit can input the feature label set D to the matching unit for further processing.

[0146] In some embodiments, the matching unit can pre-store a preset label set E.

[0147] The label set E can include one or more labels.

[0148] For each label in the label set E, the label can accurately describe the current scene. Through the labels in the label set E, the electronic device can accurately correspond to the corresponding image scene.

[0149] In some embodiments, one or more labels in the label set E can be obtained through pre-training.

[0150] In some implementations, the training process can include: for different scene images (e.g., training images), manually or through an image recognition algorithm, determining the foreground / subject image in the training image corresponding scene, and determining the label of the foreground / subject image as the typical label of the training image corresponding to the current scene. For example, taking a training image including a seaside scene as an example, the corresponding typical label can include "sea" or "sea". For example, taking a training image including a sky scene image taken at the seaside as an example, the corresponding typical label can include "sea", "cloud", and the like. For example, taking a training image including an image M1 corresponding to a person running image as an example, the corresponding typical label can include "person" and "running". For example, taking a training image including an image M2 corresponding to a cat lying on a chair, and the chair including a window behind the cat as an example, the corresponding typical label can include "cat".

[0151] In some implementations, the training process can include training the video generation diffusion model based on preset video generation diffusion model and existing video streams as training videos. For example, assume that each frame of the training video stream includes a cat. The noisy image data of each frame of the training video stream can be input into the video generation diffusion model, and the possible typical labels (e.g., kitten, climbing, meowing, etc.) corresponding to the video stream can be input into the video generation diffusion model in sequence. The video generation diffusion model can generate a corresponding video stream based on the input noisy image data and possible typical labels. It can be understood that when the input typical labels are different, the generated video streams are also different.

[0152] Thus, by comparing the generated video stream and the training video, when the difference between the generated video stream and the training video stream is acceptable, the typical label input when generating the video stream is included in the label set corresponding to the training video stream.

[0153] In this way, by inputting different training videos and possible typical labels, the set of typical labels that can accurately restore the training video in each scene can be obtained.

[0154] In this way, the label set E described above can be obtained, which can include a set of typical labels corresponding to each scene. In some implementations, the label set E can also be referred to as a third label set. Any typical label in the third label set can also be referred to as a third label. In this way, each third label can be used to identify the corresponding image. That is, each third label can accurately represent the features in the corresponding image.

[0155] For example, the label set E can include cat, dog, sea, water, and cloud. In the following example, assume that the label set E = {cat, dog, dwa, water, cloud}.

[0156] It can be understood that through the above scheme, each label in the label set E can accurately identify the scene information of the corresponding image. In some cases, because the video generation model has weak generation capability for some scenes (e.g., human, leaves, flowers, etc.), each label in the label set E determined based on the above scheme can correspond to a dominant object in the corresponding scene. Through the dominant label, the image scene corresponding to the label can be accurately restored.

[0157] In this example, the matching unit can determine a description label set F that can accurately generate the target image based on the feature label set D and the preset label set E.

[0158] For example, taking the target image as image M2, the feature label set D = {cat, window, chair}, and the preset label set E = {cat, dog, dwa, water, cloud}. Thus, the matching unit can take the intersection of the feature label set D and the label set E to determine the description label set F = {cat} corresponding to the target image (e.g., image M2).

[0159] In this application, the description label set can also be referred to as a second label set. The second label set can include one or more second labels. Each second label can be included in both the feature label set and the typical label set (i.e., the label set E). Thus, each second label can accurately correspond to the scene of the target image. For example, in the case of the target image being image M2, the second label can include "cat" in combination with the above example.

[0160] As shown in FIG. 8, the matching unit can input the description label set F of the target image to the description unit for subsequent processing by the description unit.

[0161] For example, the description unit can be configured with a multi-modal large language model. For example, the multi-modal large language model can include the InternVL 13B model.

[0162] As an example, the multi-modal large language model can include a preset prompt language. The description unit can determine the description text H corresponding to the target image according to the prompt language and the labels included in the description label set F.

[0163] For example, the preset prompt language includes: "Please vividly describe the next possible movement of the object corresponding to the label F in image A based on image A". Wherein, "image A" can be replaced by the name or other information of the target image. The label F can be replaced by one or more labels in the description label set F.

[0164] For example, taking the target image as image M2 and the corresponding description label set F = {cat} as an example. The description unit can determine the description text H of the target image (i.e., image M2) as "Please vividly describe the next possible movement of the object corresponding to the cat in image M2 based on image M2".

[0165] For example, taking the target image as image M1 and the corresponding description label set F = {men} as an example. The description unit can determine the description text H of the target image (i.e., image M2) as "Please vividly describe the next possible movement of the object corresponding to the man in image M1 based on image M1".

[0166] Thus, the description unit can obtain the description text H corresponding to the target image. It can be understood that the description text H combines the preset prompt language and the label accurately expressing the scene where the target image is located, and jointly uses image and text information, so that the visual semantic consistency of the generated video result can be significantly improved.

[0167] As shown in FIG. 8, the description unit can transmit the obtained description text H to the video generation unit.

[0168] In this example, the video generation unit can generate a video stream corresponding to the target image according to the description text and the image information of the target image. The video stream is also the dynamic wallpaper corresponding to the target image.

[0169] In this application, the video generation unit can be configured with a video generation diffusion model.

[0170] In some embodiments, the video generation diffusion model can include a motion module and a diffusion module. The motion module and the diffusion module can generate a video stream composed of multiple frames corresponding to the input image information of the target image and the description text H.

[0171] As a specific example, referring to FIG. 9, a working logic diagram of a video generation diffusion model is provided.

[0172] As shown in FIG. 9, in this example, the video generation unit can determine input parameters according to the image information of the target image before inputting data into the video generation diffusion model.

[0173] The input parameters can include image information 1, image information 2, and image information 3.

[0174] The image information 1 can correspond to the image information of the subject cutout of the target image. For example, taking the target image as image M2 as an example. The image information 1 determined by the video generation unit according to the image information of the target image can include the image information of the image composed of the image elements of the cat in the image M2. In the case where the image M2 also includes background (such as green plants) and other image elements, the background and other image elements can be excluded from the image information of the subject cutout.

[0175] As a possible implementation, the video generation unit can perform subject cutout processing on the target image according to the labels in the description label set F in combination with a subject cutout algorithm (such as a salient object detection algorithm), to obtain a binary mask image M0 corresponding to the subject of the target image. The video generation unit can overlay the binary mask image M0 on the target image, to obtain the image information 1 including only the subject image elements.

[0176] For example, taking the target image as image M2, and taking the description label set F = {cat} as an example. The video generation unit can filter the image elements corresponding to the description label (such as "cat") from the image information of the target image according to a subject matting algorithm. Then, the target image is subjected to subject matting processing according to the image elements, and the subject matting result corresponding to "cat" is obtained.

[0177] The image information 2 can correspond to the information after the Variational Auto-Encoder (VAE) processing of the target image.

[0178] The image information 3 can include multiple frames of noise information. Each frame of image can include noise. For example, the number of frames of the image information 3 can be a preset number of frames. For example, the preset number of frames can be 25. In some implementations, the noise can include Euler Discretization with Momentum noise (EDM noise).

[0179] In this way, the video generation unit can input the image information 1, the image information 2, and the image information 3 into the video generation diffusion model respectively.

[0180] In addition, the video generation unit can also input the description text H of the target image into the video generation diffusion model.

[0181] Then, the video generation diffusion model can generate the video stream corresponding to the target image according to the image information 1, the image information 2, and the image information 3, in combination with the description text H.

[0182] For example, the video generation diffusion model can process the input parameters and the description text H through the moving component and the diffusion component respectively, thereby obtaining the video stream composed of multiple frames of images.

[0183] In some embodiments, the video generation unit can perform inter-frame optical flow calculation on the video stream generated by the video generation diffusion model. For example, the inter-frame optical flow calculation can include finding the correspondence between the previous frame and the current frame by using the change of pixels in the image sequence in the time domain and the correlation between adjacent frames, so as to calculate the motion information of objects between adjacent frames.

[0184] The video generation unit can adjust each frame of image in the generated video stream according to the inter-frame optical flow calculation result, so as to optimize the fluency and correlation between each frame of image in the generated video stream, and thus make the display of the video stream more smooth.

[0185] As the above example, when the video generation diffusion model is controlled to work, the video generation unit inputs the parameters into the video generation diffusion model, and the image information 1 and the image information 2 are both a frame of image corresponding to the target image. The image information 3 can be a plurality of frames of image with a preset frame number. In this way, the frame number of the video stream generated by the video generation diffusion model according to the input parameters and the description text H can be the same as the frame number of the image information 3. For example, the frame number of the video stream generated by the video generation diffusion model can also be 25 frames. That is, the frame number of the video stream generated by the video generation diffusion model is the same as the input noise frame number. In different implementations, the input noise frame number can be configured to be different preset frame numbers. Correspondingly, the frame number of the video stream generated by the video generation diffusion model can also be adjusted to be a preset frame number.

[0186] It can be understood that in the present application, in the process of automatically generating a video stream by the video generation diffusion model, the input parameters can include the above-mentioned image information 1. The image information 1 can only include the subject image of the target image.

[0187] In a real scene, there is a big difference in the motion amplitude and motion mode of the foreground subject and the background in the image. Therefore, in the implementation of the above-mentioned scheme, the image information (i.e. image information 1) of the subject cutout of the target image is input into the video generation diffusion model separately, which can enable the video generation diffusion model to generate a plurality of frames of image for the subject cutout, and then obtain the video stream corresponding to the target image. Thus, the situation that the background moves in each frame of image but the subject does not move is avoided. Through this scheme, the rationality and accuracy of the generated video stream can be significantly improved.

[0188] In the implementation of the scheme provided in the above-mentioned FIG. 9, the video generation unit can generate the video stream corresponding to the target image according to the pre-set video generation diffusion model.

[0189] In some embodiments, the parameters of the pre-set video generation diffusion model can be configured in advance.

[0190] In combination with the description in FIG. 9, the parameters of the video generation diffusion model can be obtained through a similar training mechanism.

[0191] For example, in the training process, a plurality of video streams can be input into the video generation diffusion model as training videos respectively.

[0192] Taking a training video 1 as an example.

[0193] The electronic device can perform subject cutout processing on the first frame of image of the training video 1, thereby obtaining the image information 1 corresponding to the training video 1.

[0194] The electronic device can also perform VAE processing on the first frame of image of the training video 1, thereby obtaining the corresponding image information 2.

[0195] Taking an example that the training video 1 includes n frames of images. Different from the example in FIG. 9, during the training, the electronic device can perform VAE processing on each frame of the n frames of images of the training video 1 respectively, and obtain n VAE processing results. The electronic device can further perform noise adding processing on the n VAE processing results, and thus obtain image information 3 including n frames of images. The image information 3 is also the image information 3 corresponding to the training video 1.

[0196] In this way, the electronic device can input the image information 1, the image information 2 and the image information 3 corresponding to the training video 1 into the video generation diffusion model.

[0197] In addition, the electronic device can also input the description text corresponding to the training video 1 into the video generation diffusion model.

[0198] In some implementations, the electronic device can process a frame of image (such as the first frame) in the training video 1 through the identification unit, the matching unit and the description unit in the foregoing examples, and thus obtain the description text of the training video 1.

[0199] In another implementation, the description text of the training video 1 can also be obtained in other forms. For example, the description text of the training video 1 can include the description text input by the developer into the electronic device when using the training video 1.

[0200] In this way, the image information 1 to the image information 3 of the training video 1 and the description text are repeatedly input into the video generation diffusion model, and the conformity of the video stream output in the video generation diffusion model to the training video 1 is compared. If the conformity is greater than a corresponding threshold, the parameters used by the current video generation diffusion model are the training results.

[0201] After that, other training videos (such as training video 2, training video 3, etc.) can be used to train the parameters of the video generation diffusion model in a similar manner. Different training videos can correspond to different scenes. For example, the training video 1 can be a landscape video, and the training video 2 can be a video with an animal as the main subject. In this way, the parameters of the video generation diffusion model obtained through training can be applicable to the generation of video streams of various scenes.

[0202] In some embodiments of the present application, the training of the parameters of the video generation diffusion model described above can be performed in the electronic device in real time, or the training results can be pre-configured in the electronic device, or the electronic device can periodically / real-time obtain the training results from a cloud server.

[0203] Thus, in combination with the above description of FIGS. 7 to 9, in the present application, the electronic device can accurately and reasonably generate a video stream corresponding to the target image by the cooperation of each unit in the generation module. The video stream is also a dynamic wallpaper corresponding to the target image.

[0204] In some embodiments of the present application, as shown in FIG. 6, the electronic device can also be provided with a preprocessing module and a post-processing module.

[0205] The preprocessing module can be used to preprocess the target image before generating the corresponding video stream after the user selects the target image.

[0206] For example, as shown in FIG. 10, the preprocessing module can be configured with a judgment unit. The judgment unit can be provided with one or more threshold values. The judgment unit can perform different dimensional judgment processing on the target image based on the one or more threshold values. In the case where the judgment conditions in each dimension are met, the preprocessing module can determine that the target image can be used to generate a dynamic wallpaper, and then transmit the image information of the target image to the generation module for subsequent processing.

[0207] In some embodiments, the different dimensional judgment processing can include one or more of the following: resolution judgment, brightness judgment, definition judgment, and preset rule judgment.

[0208] Taking the resolution judgment as an example, the judgment unit can be configured with a resolution threshold value corresponding to the resolution judgment. After the user selects the target image and triggers the generation of the dynamic wallpaper, the image information of the target image can be input to the preprocessing module. The judgment unit in the preprocessing module can determine whether the resolution of the target image is greater than the resolution threshold value according to the information indicating the image size (or image size, or image length and width, or resolution) in the image information of the target image. When the resolution of the target image is greater than the resolution threshold value, it is determined that the judgment condition is met.

[0209] Taking the brightness judgment as an example, the judgment unit can be configured with a brightness threshold value corresponding to the brightness judgment. Correspondingly, the judgment unit in the preprocessing module can determine the brightness information of the target image according to the image information of the target image. In the case where the brightness value indicated by the brightness information is greater than the brightness threshold value, or in the case where the brightness value indicated by the brightness information is within the range indicated by the brightness threshold value, the preprocessing module can determine that the judgment condition is met.

[0210] For example, the judgment processing includes a definition judgment. The definition threshold corresponding to the definition judgment can be configured in the judgment unit. Correspondingly, the judgment unit in the preprocessing module can determine the definition information of the target image according to the image information of the target image. In some embodiments, the preprocessing module can determine the definition information of the target image according to the image information of the target image and the Laplace function. If the definition value indicated by the definition information is greater than the definition threshold, the preprocessing module can determine that the judgment condition is met.

[0211] In some embodiments, the judgment processing of different dimensions can also include a preset rule judgment. For example, the preset rule can include an image safety related detection rule.

[0212] In some implementations of the present application, the judgment processing of each dimension described above can be configured in the preprocessing module. For example, the preprocessing module can be configured with resolution judgment, or brightness judgment, or definition judgment, or preset rule judgment. In this way, if the corresponding judgment condition is met, the preprocessing module can determine that the target image can be used to generate dynamic wallpaper. The preprocessing module can transmit the image information of the target image to the generation module, so that the generation module generates the corresponding video stream according to the scheme provided in FIGS. 7 to 9.

[0213] In some implementations of the present application, the judgment processing of each dimension described above can include multiple judgment conditions configured in the preprocessing module. In this way, the preprocessing module can judge the judgment conditions of each dimension respectively, and if all the judgment conditions are met, the preprocessing module can determine that the target image can be used to generate dynamic wallpaper.

[0214] In the example of FIG. 6, the electronic device can also be provided with a post-processing module.

[0215] In some embodiments of the present application, the post-processing module can be used to perform subsequent processing on the generated video stream.

[0216] For example, referring to FIG. 11, a schematic diagram of the logical composition of a post-processing module is provided.

[0217] In this example, the post-processing module can include a detection unit, a resolution adjustment unit, a frame number adjustment unit, and a marking unit, etc.

[0218] The detection unit can be used to provide the ability of aesthetic score.

[0219] In some embodiments, the detection unit can be provided with an aesthetic score model. For example, the aesthetic score model can include a personalized image aesthetics assessment module (PAM).

[0220] After obtaining the video stream corresponding to the target image, the detection unit can perform aesthetic scoring on one or more images in the video stream by using the aesthetic score model, thereby evaluating the aesthetic effect of the video stream.

[0221] For example, the detection unit can be configured to extract 4 images in the video stream at equal intervals. Taking 25 frames as the preset frame number, the detection unit can extract the 1st, 5th, 9th and 13th images for aesthetic scoring.

[0222] The detection unit can input the 4 images into the PAM respectively, thereby obtaining 4 aesthetic scores. The average of the 4 aesthetic scores is taken as the score result of the video stream.

[0223] When the score result of the video stream is greater than the corresponding score threshold, it is considered that the display effect of the video stream is better, and the video stream can be used as an automatically generated dynamic wallpaper.

[0224] On the contrary, when the score result of the video stream is greater than the corresponding score threshold, it is considered that the display effect of the video stream is poor. In some implementations, the electronic device can re-trigger the generation of the dynamic wallpaper for the target image, thereby re-generating the video stream corresponding to the target image according to the scheme provided in the foregoing examples. In other implementations, the electronic device can prompt the user through an interface that the dynamic wallpaper corresponding to the current target image is unavailable or has poor quality, and then perform the next step according to the user's indication. In other embodiments, the electronic device can store the video stream, upload the video stream and the corresponding score result to a cloud server, so that the cloud server adjusts the parameters of the video generation diffusion model accordingly.

[0225] In the present example, the resolution adjustment unit can be used for resolution expansion of the video stream.

[0226] In some embodiments, the resolution adjustment unit can be provided with a super-resolution algorithm model. For example, the super-resolution algorithm model can include a real-enhanced super-resolution generative adversarial network (real-ESRGAN).

[0227] In this way, the resolution adjustment unit can input each frame of the video stream into the super-resolution algorithm model for resolution adjustment processing. Thus, the super-resolution algorithm model outputs an image with higher resolution. The resolution of each frame of image adjusted by the super-resolution algorithm model can be less than or equal to the highest resolution supported by the electronic device. For example, the resolution of each frame of image adjusted by the super-resolution algorithm model can be less than or equal to 4K or 8K.

[0228] In the present example, the frame number adjustment unit can be configured to perform frame number expansion processing on the video stream.

[0229] In combination with the foregoing description, the frame number of the video stream generated by the generation module can be a preset frame number. In some implementations, in order to reduce the work overhead of the generation module and improve the working efficiency of the video generation diffusion model, the preset frame number can be small, for example, set to 25 frames.

[0230] However, for a dynamic wallpaper, the number of frames included needs to be able to be played continuously within 5s to 10s. Thus, if a video stream composed of 25 frames is directly played continuously within 5s to 10s, a stuttering situation will occur.

[0231] In the present example, the frame number adjustment unit can expand the frame number of the video stream generated by the generation module from the preset frame number to more frames by using an interpolation algorithm, thereby avoiding the occurrence of the above-mentioned stuttering problem.

[0232] Taking a dynamic wallpaper with a playing time of 5s as an example. The frame number adjustment unit can determine the number of frames that need to be displayed within 5s according to the display frame rate of the current display screen or the preset frame rate of the dynamic wallpaper. Then, according to the number of frames that need to be displayed within 5s, the video stream generated by the video generation diffusion model is interpolated to obtain a video stream with a corresponding frame number.

[0233] For example, taking the preset frame rate of the dynamic wallpaper as 25Fps as an example. Within a time length of 5s, the number of frames that need to be played can be 25*5=125 frames. Thus, the frame number adjustment unit can expand the 25-frame video stream output by the video generation diffusion model to a 125-frame video stream by using an interpolation algorithm.

[0234] In the example, the marking unit can be configured to mark the video stream. For example, the marking process can include adding a watermark or the like. In some embodiments, the marking unit can configure a preset field of the video stream data, so as to configure the preset field to correspond to the generation module. In this way, according to the configuration result of the preset field, it can be determined whether the video stream is generated based on the generation module as shown in FIG. 6. In different implementations, the marking processed watermark or the like can be displayed on part or all of the frame images included in the video stream to the user; or the marking processed watermark or the like can not be displayed on the display screen to the user.

[0235] It should be noted that the composition and functions of each unit in the post-processing module as shown in FIG. 11 are only examples. In specific implementations, one or more units can be flexibly selected and configured according to actual conditions, or other units can be configured to process the video stream output by the generation module. The embodiments of the present application do not limit this.

[0236] It can be understood that in the implementation of the scheme provided by the embodiments of the present application, the video stream corresponding to the target image can be generated through the mutual cooperation of each unit in the generation module in the above examples. In some implementations, the pre-processing module and / or post-processing module as shown in FIG. 6 can be optional components. The embodiments of the present application do not limit the configuration of the pre-processing module and / or post-processing module.

[0237] The above illustrates the implementation of the technical scheme provided by the embodiments of the present application from the perspective of modules and units. Referring to FIG. 12, a flowchart of inter-module interaction provided by the embodiments of the present application is shown. Based on the scheme implementation as shown in FIG. 12, the electronic device can automatically generate the corresponding dynamic wallpaper (i.e., video stream) after the user triggers the generation of the dynamic wallpaper. In the example of FIG. 12, the pre-processing module and the post-processing module are configured in the electronic device.

[0238] As shown in FIG. 12, the scheme can include:

[0239] S1201, the electronic device receives a first operation input by the user.

[0240] The first operation can be used to instruct the electronic device to generate a corresponding dynamic wallpaper according to a selected target image.

[0241] Taking the target image as the first image as an example. The first image can be image M1 in the foregoing example, or the first image can be image M2 in the foregoing example.

[0242] Taking the first image as image M1 as an example.

[0243] In some examples, the first operation can include a click operation on the control button B3 as shown in FIG. 2.

[0244] In some other examples, the first operation can include a click operation on the option 52 as shown in FIG. 5.

[0245] In some other examples, the first operation can also include a voice instruction, a preset gesture, or the like for triggering the electronic device to generate the dynamic wallpaper.

[0246] As a possible implementation, the first operation includes a click operation on the option 52 as shown in FIG. 5.

[0247] After the electronic device receives the first operation through the touch screen, the electronic device can generate an operation event corresponding to the first operation. The operation event can include an operation type of the first operation, a position on the display screen, and the like.

[0248] The electronic device can obtain the operation event of the first operation.

[0249] In this way, the electronic device can determine the target image corresponding to the first operation. For example, the electronic device can determine, according to the operation event of the first operation, an image (e.g., the image M1) corresponding to the position indicated by the operation event on the current display interface as the target image.

[0250] Further, the electronic device can obtain attribute information of the target image. For example, the attribute information of the target image can include a storage address of the target image, a name of the target image, and the like. The storage address of the target image can store image information of the target image. For example, the image information can include information indicating a size of the target image, color information of each pixel point of the target image, and the like.

[0251] As another possible implementation, the first operation includes a click operation on the control button B3 as shown in FIG. 2.

[0252] After the electronic device receives the click operation on the control button B3, the electronic device can obtain attribute information of the selected image. The selected image can be the image selected by the user in the interaction example of the interface 04 to the interface 05 to the interface 06 as shown in FIG. 2. For example, the image can be the image M1 as shown in FIG. 2.

[0253] In this example, the name of the target image is taken as the first image.

[0254] In this way, the electronic device can obtain the name of the first image and image information of the first image according to the attribute information of the image. In some implementations, the name of the first image can also be included in the image information of the first image.

[0255] In this example, the electronic device can send the image information of the first image to the preprocessing module for subsequent processing.

[0256] S1202, the preprocessing module obtains image information of the first image.

[0257] For example, the image information of the first image can include the name of the first image, information indicating the size of the first image, and information indicating the color of each pixel of the first image.

[0258] S1203, the preprocessing module determines that the first image meets the preset condition according to the image information of the first image.

[0259] In combination with the description in FIG. 10, the preset condition can include one or more of the following:

[0260] Resolution judgment, brightness judgment, definition judgment, and preset rule judgment, etc. The configuration and application of specific judgment conditions and thresholds can refer to the description in FIG. 10 above, which will not be repeated here.

[0261] In the case where the preprocessing module determines that the first image meets the preset condition, it can be determined that the first image can be used to generate a dynamic wallpaper. In this way, the electronic device can continue to perform the following S1204.

[0262] S1204, the preprocessing module sends the image information of the first image to the recognition unit.

[0263] S1205, the recognition unit determines a feature tag set of the first image.

[0264] For example, the recognition unit can determine the feature tag set of the first image according to the image analysis model (such as RAM++) set in the recognition unit. The feature tag set can include one or more tags. For example, the tag in the feature tag set is called a feature tag. In this way, the feature tag set can include one or more feature tags.

[0265] For example, the first image is image M2. The recognition unit can determine the feature tag set D = {cat, window, chair} of the first image.

[0266] S1206, the recognition unit sends the feature tag set of the first image to the matching unit.

[0267] S1207, the matching unit determines a description tag set of the first image.

[0268] In this example, the matching unit can pre-acquire a label set E. The label set E can include one or more typical labels, each of which can accurately describe the corresponding scene. In some implementations, the label set E can also be referred to as a typical label set. For example, take the label set E = {cat, dog, dwa, water, cloud} as an example.

[0269] In combination with the description in FIG. 8, the matching unit can determine the description label set of the first image according to the label set E and the feature label set of the first image.

[0270] For example, the matching unit can determine the labels in the feature label set that are included in the label set E as the description labels of the first image. The set of the description labels of the first image can constitute the description label set of the first image. The description label set of the first image can include one or more first label description labels.

[0271] In combination with the above example, when the first image is image M2, the labels in the corresponding description label set can be the intersection of the typical label set and the feature label set. For example, the description label set F = {cat}.

[0272] S1208, the matching unit sends the description label set of the first image to the description unit.

[0273] S1209, the description unit determines the description text of the first image.

[0274] For example, the description unit can determine the description text corresponding to the first image according to the preset prompt language and the labels in the description label set.

[0275] For example, the preset prompt language includes “Please vividly describe the motion that the object corresponding to [the label in the description label set] in [the name of the first image] may perform in the next stage in combination with [the name of the first image]”. The content in [] can be flexibly replaced according to the actual situation of the first image.

[0276] For example, the first image is image M2, and the corresponding [name of the first image] = image M2. The corresponding [label in the description label set] of the first image = cat, or the corresponding [label in the description label set] of the first image = cat.

[0277] In this way, the description text H corresponding to the first image (i.e., image M2) can be “Please vividly describe the motion that the object corresponding to the cat in image M2 may perform in the next stage in combination with image M2”.

[0278] It can be understood that in the embodiments of the present application, the description text is jointly constituted based on the preset suggestive language and the description label corresponding to the first image. The description label is a typical label that can accurately represent the scene corresponding to the first image after being screened and acquired. Therefore, the description text can be more accurately corresponded to the first image.

[0279] S1210, the description unit sends the description text of the first image to the video generation unit.

[0280] S1211, the video generation unit determines the video stream of the first image.

[0281] For example, the video generation unit can obtain the video generation diffusion model before performing S1211. For example, the video generation unit can obtain the video generation diffusion model by self-training. For another example, the video generation unit can obtain the video generation diffusion model from the cloud server through interaction between the electronic device and the cloud server. For another example, the video generation diffusion model can be pre-set in the video generation unit.

[0282] In this example, the video generation unit can obtain the corresponding video stream according to the image information of the first image and the description text of the first image through the video generation diffusion model. For specific processes, reference can be made to the description of the foregoing FIG. 9, which will not be described here.

[0283] It should be noted that in some embodiments, the video generation unit can obtain the image information of the first image from the preprocessing module. In other embodiments, the video generation unit can obtain the image information of the first image from the recognition unit or the matching unit or the description unit.

[0284] Therefore, the video generation unit can generate a video stream including multiple images corresponding to the first image. In some embodiments, the video stream can be used as the dynamic wallpaper of the first image.

[0285] In other embodiments, as shown in this example, the electronic device can further continue to perform other processing on the generated video stream, and use the processing result as the dynamic wallpaper of the first image.

[0286] S1212, the video generation unit sends the video stream of the first image to the post-processing module.

[0287] S1213, the post-processing module performs post-processing on the video stream of the first image to obtain the dynamic wallpaper of the first image.

[0288] Exemplarily, the post-processing performed by the post-processing module can include one or more of the following: an aesthetic score processing performed by the detection unit, a resolution expansion processing performed by the resolution adjustment unit, a frame number expansion processing performed by the frame number adjustment unit, a marking processing performed by the marking unit, and the like.

[0289] The specific implementation can refer to the example in FIG. 11.

[0290] Therefore, by implementing the flow as shown in FIG. 12, the electronic device can automatically generate a dynamic wallpaper corresponding to the first image after the user inputs the first operation. The dynamic wallpaper can include multiple frames of images. Each frame of image can correspond to the first image. In this application, due to the technical solution used in the process of generating the dynamic wallpaper, each frame of image in the dynamic wallpaper can accurately and clearly correspond to the first image, and an accurate generation result corresponding to the main image of the first image can be obtained. Therefore, the dynamic wallpaper can also have an accurate display effect.

[0291] In some embodiments, after obtaining the dynamic wallpaper of the first image, the electronic device can also present the dynamic wallpaper to the user so that the user can use the dynamic wallpaper. In other embodiments, the electronic device can also store the dynamic wallpaper, for example, in the storage area corresponding to the gallery application. The specific implementation can refer to the related description in FIG. 3, which will not be described here.

[0292] It can be understood that the electronic device provided by the embodiments of the present application contains the corresponding hardware structure and / or software module for executing each function in order to realize the above functions. Those skilled in the art should easily realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present application.

[0293] The embodiments of the present application can divide the functions of the above-mentioned electronic device into function modules according to the above-mentioned method examples. For example, each function module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be realized in the form of hardware or software function module. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical function division. Actual implementation can have another division method.

[0294] Exemplarily, referring to FIG. 13, another software composition of an electronic device is provided.

[0295] In this application, the software system of the electronic device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. Embodiments of this application take a layered architecture as an example to illustrate the software structure of the electronic device. The system is taken as an example.

[0296] The layered architecture divides the software into several layers, each of which has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, the application layer, the application framework layer, the Android runtime (ART) and the native C / C++ library, the hardware abstraction layer (HAL), and the kernel layer.

[0297] The application layer can include a series of application packages. The application layer can also be referred to as the application layer, or the APP layer.

[0298] As shown in FIG. 13, the application package can include gallery, calendar, map, WLAN, music, SMS, call, navigation, Bluetooth, video, and other applications.

[0299] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications of the application layer. The application framework layer includes some pre-defined functions.

[0300] As shown in FIG. 13, the application framework layer can include window manager, content provider, view system, resource manager, notification manager, activity manager, input manager, etc. The application framework layer can also be referred to as the framework layer, or the Framework layer, or Framework.

[0301] Corresponding to the scheme implementation as shown in FIG. 6, the generation module, the preprocessing module, and the post-processing module can be configured in the framework layer.

[0302] The window manager provides window management services (WMS), which can be used for window management, window animation management, surface management, and as an input system relay station.

[0303] The content provider is used to store and obtain data, and makes the data accessible to the application. The data can include video, image, audio, dialed and received calls, browsing history and bookmarks, phonebook, etc.

[0304] The view system includes visual controls, such as controls that display text, controls that display images, and the like. The view system can be used to build an application. A display interface can be composed of one or more views. For example, a display interface that includes a short message notification icon can include a view that displays text and a view that displays an image.

[0305] The resource manager provides various resources for an application, such as localized strings, icons, images, layout files, video files, and the like.

[0306] The notification manager enables an application to display notification information in a status bar, which can be used to convey a message of the notification type, which can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify a download completion, a message reminder, and the like. The notification manager can also be a notification that appears in the top status bar of the system in the form of a chart or a scrolling text, such as a notification of an application running in the background, and can also be a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is emitted, the electronic device vibrates, the indicator light flashes, and the like.

[0307] The activity manager can provide an activity management service (AMS), which can be used for the startup, switching, scheduling of system components (such as activities, services, content providers, broadcast receivers), and the management and scheduling of application processes.

[0308] The input manager can provide an input management service (IMS), which can be used to manage the input of the system, such as touch screen input, key input, sensor input, and the like. The IMS takes events from the input device node and distributes the events to the appropriate window through interaction with the WMS. For example, in the case of a first operation input by a user, the input manager can obtain an operation event of the first operation from the hardware.

[0309] The input manager can also cooperate with the activity manager to respond to the operation event of the first operation and resource scheduling. For example, the input manager can send the operation event of the first operation to the activity manager, and the activity manager can determine to trigger the flow shown in FIG. 12 according to the operation event of the first operation. The input manager or the activity manager can also send the operation event of the first operation to the preprocessing module, so that each related module in the framework layer can be implemented according to the scheme provided in FIG. 12 or any one of the preceding embodiments, to generate a corresponding dynamic wallpaper according to the first image.

[0310] The Android runtime includes core libraries and an Android runtime. The Android runtime is responsible for converting source code into machine code. The Android runtime mainly includes an ahead of time (AOT) compilation technique and a just in time (JIT) compilation technique.

[0311] The core libraries are mainly used to provide functions of basic Java class libraries, such as basic data structures, mathematics, IO, tools, databases, network libraries, and the like. The core libraries provide APIs for users to develop Android applications.

[0312] The native C / C++ libraries can include a plurality of functional modules. For example, a surface manager, a media framework, libc, OpenGL ES, SQLite, Webkit, and the like.

[0313] The surface manager is used to manage a display subsystem and provides fusion of 2D and 3D layers for a plurality of applications. The media framework supports playback and recording of a plurality of commonly used audio, video formats, and static image files, and the like. The media library can support a plurality of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, and the like. The OpenGL ES provides drawing and operation of 2D graphics and 3D graphics in an application. The SQLite provides a lightweight relational database for applications of an electronic device.

[0314] A hardware abstraction layer (HAL) runs in a user space, encapsulates kernel layer drivers, and provides a calling interface to an upper layer. For example, the hardware abstraction layer can include a display module, an audio module, a camera module, a Bluetooth module, and the like. The hardware abstraction layer can also be referred to as an abstraction layer.

[0315] A kernel layer is a layer between hardware and software. The kernel layer at least includes a display driver, a camera driver, an audio driver, and a Bluetooth driver.

[0316] It should be noted that the composition of the electronic device provided in each of the embodiments of the present application is only an example, and does not constitute any limitation on the electronic device. In other embodiments, the composition of the electronic device can also be different from any of the above examples.

[0317] Exemplarily, FIG. 14 shows a schematic diagram of a composition of an electronic device 1400. As shown in FIG. 14, the electronic device 1400 can include a processor 1401 and a memory 1402. The memory 1402 is configured to store computer-executable instructions. Exemplarily, in some embodiments, when the processor 1401 executes the instructions stored in the memory 1402, the electronic device 1400 can be caused to perform the method shown in any of the above embodiments.

[0318] It should be noted that all the related contents of the steps involved in the above method embodiments can be cited to the function description of the corresponding function modules, which will not be repeated here.

[0319] FIG. 15 shows a schematic diagram of a composition of a chip system 1500. The chip system 1500 can include a processor 1501 and a communication interface 1502, which are configured to support the related device to implement the functions involved in the above embodiments. In a possible design, the chip system further includes a memory configured to store the necessary program instructions and data of the electronic device. The chip system can be composed of a chip, or can include a chip and other discrete devices. It should be noted that in some implementations of the present application, the communication interface 1502 can also be referred to as an interface circuit.

[0320] It should be noted that all the related contents of the steps involved in the above method embodiments can be cited to the function description of the corresponding function modules, which will not be repeated here.

[0321] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a computer to implement the method flow related to the electronic device in any of the above method embodiments. Specifically, the computer can be the electronic device described above.

[0322] The embodiments of the present application also provide a computer program or a computer program product including the computer program. When the computer program is executed on a computer, the computer will implement the method flow related to the electronic device in any of the above method embodiments. Specifically, the computer can be the electronic device described above.

[0323] In the present application, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments, unless otherwise specified, the meaning of "multiple" is two or more.

[0324] The functions or actions or operations or steps in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the computer program instructions can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the whole or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or data storage device including one or more servers, data centers, etc. integrated with the medium. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0325] Although the present application is described in conjunction with specific features and embodiments thereof, it is evident that many alternatives, modifications and combinations of features will be apparent to those skilled in the art in view of the foregoing description. Accordingly, the description and drawings are to be regarded as illustrative in nature and not as restrictive. The scope of the application is indicated by the appended claims, and all changes which come within the meaning and range of equivalents are intended to be embraced therein. It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the scope or spirit of the application. Thus, it is intended that the present application cover the modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. A method of video generation, the method comprising: The method is applied to an electronic device, and the method comprises: receiving a first operation input by a user, the first operation corresponding to a first image, the first operation being used to instruct the electronic device to generate a dynamic wallpaper; generating a first description text according to image information of the first image; the image information comprises information indicating a size of the first image, and color information of each pixel point of the first image; generating a first video stream according to the image information of the first image and the first description text, the first video stream corresponding to a dynamic wallpaper of the first image.

2. The method of claim 1, wherein, before the generating the first video stream, the method further comprises: obtaining first information and second information corresponding to the first image according to the image information of the first image; wherein the first information comprises information of a subject image in the first image, and the second information comprises information processed by a variational encoding corresponding to the first image. The generating the first video stream according to the image information of the first image and the first description text comprises:

3. The method of claim 2, wherein, generating the first video stream according to the first information, the second information, third information and the first description text; the third information comprises information of noise images of a preset frame number, and the preset frame number is greater than or equal to 2.

4. The method of claim 3, wherein, the noise image comprises Euler Discretization of Momentum noise (EDMnoise). The electronic device is configured with a video generation diffusion model.

5. The method according to claim 3 or 4, characterized in that, The generating the first video stream according to the image information of the first image and the first description text comprises: inputting the first information, the second information, the third information and the first description text corresponding to the first image into the video generation diffusion model; controlling the video generation diffusion model to output the first video stream. The first video stream comprises images of a preset frame number.

6. The method of claim 5, wherein, 7. The method of any one of claims 1-6, wherein, before the generating the first description text according to the image information of the first image, the method further comprises: determining a first label set corresponding to the first image according to the image information of the first image; the first label set comprises at least one first label, and the first label is used to identify an object in the first image and / or an activity attribute of the object in the first image. The method further comprises: determining the first description text according to the first label set.

8. The method of claim 7, wherein, 9. The method of claim 8, wherein, the determining the first description text according to the first label set comprises: determining a second label set according to the first label set; the second label set comprises at least one second label, and any one of the second labels is also included in the first label set; determining the first description text according to the second label set. ​ ​ 10. The method of claim 9, wherein, The electronic device is pre-provided with a third label set, the third label set includes at least one third label, and each third label is used to identify the image of the corresponding scene; The second label set is determined according to the first label set, including: For each first label, if the first label is included in the third label set, the first label is determined as a second label in the second label set.

11. The method of claim 9 or 10, wherein, The first description text is determined according to the second label set and the preset prompt language prompts.

12. The method of any one of claims 1-11, wherein, Before the first description text is generated, the method further includes: According to the image information of the first image, it is determined that the first image satisfies a preset condition; The preset condition includes at least one of the following: The resolution of the first image is greater than a resolution threshold; The brightness of the first image is greater than a brightness threshold; The definition of the first image is greater than a definition threshold.

13. The method of any one of claims 1-12, wherein, After the first video stream is generated, the method further includes: Storing the first video stream, and taking the first video stream as the dynamic wallpaper corresponding to the first image.

14. The method of any one of claims 1-12, wherein, After the first video stream is generated, the method further includes: At least one of the following is performed on the first video stream to obtain a second video stream: Resolution adjustment processing, frame number adjustment processing, and label processing; Storing the second video stream, and taking the second video stream as the dynamic wallpaper corresponding to the first image. The electronic device includes a memory and one or more processors; the memory and the processor are coupled; 15. An electronic device, comprising: Wherein, the memory is used to store computer program code, the computer program code includes computer instructions, when the processor executes the computer instructions, makes the electronic device execute the method as claimed in any one of claims 1-14. The chip system is applied to an electronic device; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through a circuit; the interface circuit is used to receive a signal from the memory of the electronic device, and send the signal to the processor, the signal includes the computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the method as claimed in any one of claims 1-14.

16. A chip system, characterized by Including computer instructions, when the computer instructions run on an electronic device, so that the electronic device executes the method as claimed in any one of claims 1-14.

17. A computer-readable storage medium, characterized in that, ​

Images