Live picture generation method and device, electronic equipment and medium

Abstract

The invention provides a live picture generation method and device, electronic equipment and a medium, and relates to the technical field of data processing, in particular to the technical field of image processing and artificial intelligence. According to the specific implementation scheme, a to-be-processed picture is determined, then similar pictures of the to-be-processed picture are determined from all alternative pictures, and the to-be-processed picture and the alternative pictures are both non-live pictures. And carrying out frame insertion processing between the to-be-processed picture and the similar picture, wherein a frame insertion processing result is used for generating a live picture. In the generation process of the live picture, the user operation is simplified, and the playing effect of the live picture is smoother.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to the field of image processing and artificial intelligence technology. Background Art

[0002] Live photo (livp), also known as live picture or dynamic picture. When the shooting device has the live function turned on, a live picture can be taken when a shooting command is received. The live picture includes videos shot within a period of time before and after the shooting command is received. When the live picture is displayed, a video frame in the video included in the live picture is displayed by default. The video frame can be called the cover picture. When a dynamic display command is received, for example, when it is detected that the user long presses the live picture, the video included in the live picture is played, achieving a dynamic display effect of the picture. The live picture can also be set as the lock screen wallpaper of the smart device, so that the dynamic moment when the picture is taken can be reproduced even when the smart device is locked. Summary of the invention

[0003] The present disclosure provides a method, device, electronic device and medium for generating a live picture.

[0004] According to a first aspect of the embodiments of the present disclosure, a method for generating a live picture is provided, comprising:

[0005] Determine the pictures to be processed;

[0006] Determine, from each candidate picture, a similar picture to the picture to be processed, wherein both the picture to be processed and the candidate pictures are non-live pictures;

[0007] An interpolation process is performed between the picture to be processed and the similar picture, and the interpolation process result is used to generate a live picture.

[0008] A second aspect of the embodiments of the present disclosure provides a live picture generating device, including:

[0009] A determination module, used to determine the image to be processed;

[0010] A screening module, used to determine similar pictures of the picture to be processed from each candidate picture, wherein both the picture to be processed and the candidate pictures are non-live pictures;

[0011] The interpolation module is used to perform interpolation processing between the to-be-processed picture determined by the determination module and the similar picture obtained by the screening module, and the interpolation processing result is used to generate a live picture.

[0012] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method according to the first aspect.

[0016] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute a method according to any one of the first aspects.

[0017] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method according to any one of the first aspects is implemented.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0020] Figure 1 is a flow chart of a method for generating a live picture provided by an embodiment of the present disclosure;

[0021] Figure 2 is an exemplary schematic diagram of a recommendation page provided by an embodiment of the present disclosure;

[0022] Figure 3 is a flow chart of a frame insertion processing method provided by an embodiment of the present disclosure;

[0023] Figure 4 is a flow chart of another method for generating a live picture provided by an embodiment of the present disclosure;

[0024] Figure 5 is a structural schematic diagram of a live picture generating device provided by an embodiment of the present disclosure;

[0025] Figure 6 It is a block diagram of an electronic device used to implement the live picture generating method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] With the development of network technology, more and more users love to share and store pictures on the Internet, and there are a large number of non-live pictures among these pictures. For example, non-live pictures include: pictures taken by a camera that does not support the live function, and pictures taken when the camera does not have the live function turned on. If these non-live pictures can be converted into live pictures, it will bring better user experience.

[0028] In conventional live picture generation schemes, users are generally required to manually select the video to generate the live picture. When there is no video, users are required to select multiple pictures, and then play these pictures in sequence to simulate the display effect of the live picture. However, this method requires manual selection of pictures, and when there are many pictures to choose from, it is difficult for users to manually select. Moreover, there are certain differences between the multiple pictures selected manually, and the display effect is relatively stiff when they are played in sequence.

[0029] In order to solve the above problems, the embodiments of the present disclosure provide a method for generating a live picture, which is applied to an electronic device, for example, the electronic device can be a device with image processing capabilities such as a server or a desktop computer. Among them, the software module in the electronic device that executes the method for generating a live picture provided by the embodiments of the present disclosure can be called a cloud, for example, the cloud is a software platform built using application virtualization technology.

[0030] like Figure 1 As shown, the live picture generation method provided by the embodiment of the present disclosure includes the following steps:

[0031] S101, determining the picture to be processed.

[0032] The image to be processed may be an image selected by the user, or an image that the user is viewing, etc.

[0033] S102: Determine similar pictures to the picture to be processed from the candidate pictures.

[0034] Among them, the pictures to be processed and the alternative pictures are all non-live pictures.

[0035] The candidate images include: images that belong to the same account as the image to be processed. For example, the candidate images and the image to be processed are both images uploaded to the cloud by the same account.

[0036] The candidate images may also include other non-live images, such as non-live images in a public data set, which is not specifically limited in the embodiments of the present disclosure.

[0037] S103: performing frame interpolation processing between the image to be processed and a similar image, wherein the result of the frame interpolation processing is used to generate a live image.

[0038] Through the frame insertion process, a transition frame can be inserted between the image to be processed and the similar image, and when there are multiple similar images, a transition frame can be inserted between the similar images. The transition frame is a picture with similar content to the image to be processed and the similar image.

[0039] Through the above method, the disclosed embodiment can determine the to-be-processed picture and the similar picture of the to-be-processed picture from the non-live picture. Since the similar picture has similarity with the to-be-processed picture, it can be used to generate the live picture. This method does not require manual selection of pictures, thus simplifying the user operation. Moreover, the disclosed embodiment can also perform interpolation processing between the to-be-processed picture and the similar picture, so that the inserted transition frame plays a transition role between the to-be-processed picture and the similar picture, so that the playback effect of the live picture generated based on the interpolation result is smoother.

[0040] The following is a detailed description of the method for generating a live picture provided by the embodiment of the present disclosure:

[0041] In some embodiments of the present disclosure, the above-mentioned S101 cloud-based method of determining the picture to be processed can be implemented as follows: receiving a notification message sent by a terminal, wherein the notification message is used to indicate that the terminal is displaying the picture to be processed, and the notification message includes a picture identifier of the picture to be processed, and then the cloud determines that the picture corresponding to the picture identifier is the picture to be processed.

[0042] When the terminal displays a single picture, the notification message includes the picture ID of the picture. Alternatively, when the terminal displays multiple pictures, the notification message includes the picture ID of the picture with the largest display area among the multiple pictures.

[0043] The terminal may be a mobile phone, tablet computer, smart watch or other device that has the capability of synthesizing and displaying live images.

[0044] The image identifier of the image to be processed may be: an image name or an image identifier (IDentifier, ID), etc.

[0045] Through the above method, the embodiment of the present disclosure can determine the picture being displayed by the terminal as the picture to be processed, so as to subsequently generate a live picture based on the picture to be processed. This provides users with richer picture display functions when browsing pictures, enhances the fun of picture display, and improves the user experience.

[0046] In some embodiments of the present disclosure, Figure 1 Before the process shown, the cloud can first obtain the image analysis permission, where the image analysis permission indicates that the image uploaded to the cloud is allowed to be processed and analyzed.

[0047] For example, the terminal communicates with the cloud through the client installed on itself. When the terminal opens the client for the first time, the client displays the image analysis permission request window. When it detects that the user clicks the "Agree" button in the window, the client sends an image analysis permission confirmation message to the cloud, indicating that the image analysis permission has been granted by the client login account.

[0048] For another example, when the terminal opens the client for the first time, the client displays the usage agreement, which is used to request multiple permissions including image analysis permissions. When it detects that the user agrees to the usage agreement, the client sends a usage agreement confirmation message to the cloud, indicating that it has obtained multiple permissions including image analysis permissions granted by the client login account.

[0049] Alternatively, the cloud may also obtain image analysis permissions in other ways, which are not specifically limited in the embodiments of the present disclosure.

[0050] After obtaining the image analysis rights granted by the client's login account, when the account uploads an image to the cloud through the client, the cloud can analyze the similarity between the image and the candidate images, where the candidate images include other non-live images uploaded by the login account. The following takes the image uploaded by the terminal as an example of the image to be processed to illustrate the similarity analysis process in the cloud.

[0051] The cloud can identify the content information of the image to be processed through a content recognition model, and then determine the similarity between the image to be processed and the candidate images based on the metadata and content information of the image to be processed and the metadata and content information of the candidate images.

[0052] The content recognition model may be a large language model (LLM) or a convolutional neural network (CNN) or other model with image recognition capabilities.

[0053] Content information can also be called scene information, and includes: identification of people in the picture, posture of people in the picture, and names of objects, etc. The identification of people can be a person ID or a name marked by a user for the person, etc. For example, content information includes: person 1, sitting on a high stool, looking toward the window, white table, floor-to-ceiling windows, and a cup of coffee on the table.

[0054] In one implementation, the metadata and content information of the image to be processed can be converted into a vector, and the metadata and content information of each candidate image can be converted into a vector. Then, the similarity between the vector of the image to be processed and the vector of each candidate image can be calculated, for example, the cosine similarity between the vectors is calculated as the similarity between the image to be processed and each candidate image.

[0055] In another implementation method, the metadata and content information of the image to be processed, as well as the metadata and content information of each candidate image, can be filled into a preset first prompt template to obtain first prompt information, and then the first prompt information is input into the first largest language model to obtain the similarity between the image to be processed and each candidate image output by the first largest language model.

[0056] The first prompt template is used to indicate the similarity between the images. The first prompt template can be preset based on the business requirements in the actual application scenario.

[0057] For example, the first prompt template is:

[0058] "Please combine the metadata and content information of the image to be processed with the metadata and content information of the candidate images to determine the similarity between the image to be processed and each candidate image.

[0059] Metadata of the image to be processed:

[0060] Content information of the image to be processed:

[0061] Metadata of Alternative Image 1:

[0062] Content information of alternative image 1:

[0063] Metadata of alternative image 2:

[0064] Content information of alternative image 3:”

[0065] After filling in the first prompt template, the first prompt information obtained is:

[0066] "Please combine the metadata and content information of the image to be processed with the metadata and content information of the candidate images to determine the similarity between the image to be processed and each candidate image.

[0067] Metadata of the image to be processed: Time: Saturday, April 2, 2022, 13:00; Location: AA Building, Haidian District, Beijing; Latitude and longitude: xx.xx, yy.yy; Altitude: 53 meters; Weather: Sunny.

[0068] Content information of the image to be processed: Person 1, sitting on a high stool, looking toward the window, white table, floor-to-ceiling windows, a cup of coffee on the table.

[0069] Metadata of alternative image 1: Time: Saturday, April 2, 2022, 13:00; Location: AA Building, Haidian District, Beijing; Latitude and longitude: xx.xx, yy.yy; Altitude: 69 meters; Weather: Sunny.

[0070] Content information of alternative image 1: Person 1, sitting on a high stool holding coffee, looking at coffee, white table, floor-to-ceiling windows, coffee.

[0071] Metadata of alternative image 2: Time: Saturday, April 2, 2022, 13:01; Location: AA Building, Haidian District, Beijing; Latitude and longitude: xx.xx, yy.yy; Altitude: 69 meters; Weather: Sunny.

[0072] Content information of alternative image 2: Sitting on a high stool, coffee at mouth, drinking coffee, white table, floor-to-ceiling windows, coffee. ”

[0073] After the first prompt information is input into the first large language model, the output result of the large language model is: "The similarity between the image to be processed and the candidate image 1 is 95%, and the similarity between the image to be processed and the candidate image 2 is 92%."

[0074] Through the above method, the embodiment of the present disclosure can use the first language model to determine the similarity between pictures based on the metadata and content information of the pictures. That is, it realizes the use of artificial intelligence (AI) analysis capabilities to more comprehensively analyze the association between pictures from multiple angles such as time, location and content, so that the similarity between pictures is determined more accurately.

[0075] Moreover, the embodiment of the present disclosure analyzes the similarities between pictures in advance before generating the live picture, so that when generating the live picture, similar pictures can be directly screened based on the pre-analyzed similarities without having to determine the similarities between pictures in real time, thereby improving the generation efficiency of the live picture and improving the user experience.

[0076] In some embodiments of the present disclosure, the above-mentioned S102 cloud determines similar pictures of the picture to be processed from various candidate pictures, which can be implemented as: selecting candidate pictures whose similarity with the picture to be processed is higher than a preset threshold as similar pictures.

[0077] The preset threshold value may be preset according to the business requirements in the actual application scenario. For example, when the similarity value range is [0%, 100%], the preset threshold value may be 80%.

[0078] In the disclosed embodiment, the cloud can obtain the similarity between the predetermined image to be processed and each candidate image, and take each candidate image whose similarity with the image to be processed is higher than a preset threshold as a similar image.

[0079] Through the above method, the embodiment of the present disclosure can use alternative pictures whose similarity with the picture to be processed is higher than a preset threshold as similar pictures, so that the correlation between the similar pictures and the picture to be processed is stronger, and the playback effect of the live picture generated thereby is smoother.

[0080] In some embodiments of the present disclosure, after determining similar pictures, it is also possible to further determine whether a live picture can be generated based on the similar pictures. That is, before performing interpolation processing between the picture to be processed and the similar pictures in S103, the cloud can also determine whether the number of similar pictures reaches a preset number. The preset number can be set according to the business needs in the actual application scenario, for example, the preset number is 3.

[0081] If the number of similar pictures reaches a preset number, it means that there are a large number of similar pictures. At this time, a live picture with smooth playback effect can be generated based on the picture to be processed and the similar pictures. Therefore, the cloud can send a recommendation message to the terminal, where the recommendation message is used to indicate the recommendation to generate a live picture based on the picture to be processed.

[0082] After receiving the recommendation message, the terminal may display a recommendation page or a recommendation pop-up window for indicating the recommendation to generate a live picture based on the picture to be processed. Figure 2 As shown in the figure, the recommendation page includes: the image to be processed, the recommendation text, the start generation button and the cancel button. The recommendation text is "AI recognizes that this image can generate a live image, do you want to generate it?". It should be noted that Figure 2 The dotted box including the “picture to be processed” indicates a picture being displayed by the terminal. In actual application scenarios, a real picture can be displayed in the dotted box.

[0083] Afterwards, when the terminal detects that the user agrees to generate the live picture, for example, when the user clicks Figure 2 When the start generation button in is clicked, the terminal can send a confirmation message to the cloud, where the confirmation message is used to indicate confirmation of generating a live picture based on the picture to be processed. When the cloud receives the confirmation message sent by the terminal, it executes the above S103 to perform frame insertion processing between the picture to be processed and the similar picture.

[0084] If the number of similar pictures does not reach the preset number, it means that the number of similar pictures is too small. At this time, even if interpolation is performed between the picture to be processed and the similar pictures, the interpolation result is difficult to achieve the smooth effect of the actual video shooting. Therefore, the cloud can not send a recommendation message to the terminal at this time.

[0085] Since it is difficult to generate a live picture with smooth playback effect using the pictures to be processed and similar pictures when there are too few similar pictures, the cloud can not generate live pictures at this time. On the contrary, when the number of similar pictures is sufficient, the pictures to be processed and similar pictures can generate a live picture with smooth playback effect, so the cloud can generate live pictures at this time, thereby enriching the display effect of pictures in the terminal.

[0086] In the disclosed embodiment, the image to be processed and each candidate image have metadata, for example, the metadata includes: timestamp, location, longitude and latitude, altitude, weather and shooting parameters, etc. Among them, the timestamp indicates the time when the image is obtained, for example, the timestamp is the shooting time of the real-shot image, or the capture time of the screenshot, etc. The location is the name of the building, road or park where the terminal is located when the image is obtained. The longitude and latitude are the longitude and latitude where the terminal is located when the image is obtained. The altitude is the altitude where the terminal is located when the image is obtained. The weather is the weather at the location where the terminal is located when the terminal obtains the image. The shooting parameters are the hardware parameters of the terminal camera, as well as the sensitivity, focal length, exposure value, lens aperture value and shutter speed used by the terminal when taking pictures. For example, the hardware parameters of the terminal camera include the manufacturer name and model.

[0087] On this basis, before the cloud sends a recommendation message to the terminal, the cloud can also determine whether the image to be processed is a real shot image based on the metadata of the image to be processed.

[0088] Among them, the real shot pictures are pictures taken by a camera, and the non-real shot pictures are pictures not taken by a camera. For example, the non-real shot pictures include: screenshots and pictures generated by using picture generation tools.

[0089] The cloud can determine whether the metadata of the image to be processed includes shooting parameters. If so, it is determined that the image to be processed is a real shot image. If not, it is determined that the image to be processed is not a real shot image, that is, it is a non-real shot image.

[0090] If the image to be processed is a real-shot image, the step of sending a recommendation message to the terminal is executed.

[0091] If the image to be processed is not a real-shot image, no recommendation message is sent to the terminal.

[0092] After determining the image to be processed, the disclosed embodiment determines that the generation recommendation of the live image can be performed only when the image to be processed is a real-shot image, so that the live image is generated based on the real-shot image. Since the live image is usually obtained by actual shooting in actual application scenarios, the disclosed embodiment generates the live image based on the real-shot image, which can make the generated live image closer to the live image taken in the actual application scenario.

[0093] In some embodiments of the present disclosure, see Figure 3 The above-mentioned S103 cloud performs frame insertion processing between the image to be processed and the similar image, including the following steps:

[0094] S301: Based on the image to be processed and similar images, fill in a preset second prompt template to obtain second prompt information.

[0095] The cloud can fill the to-be-processed image and the similar images into the second prompt template in the order of their timestamps.

[0096] The second prompt template is used to indicate to perform interpolation processing between pictures.

[0097] For example, the second prompt template is:

[0098] "Please perform interpolation processing on the following images so that the interpolation processing results meet the specified requirements.

[0099] picture:

[0100] Specify requirements:

[0101] Among them, the specified requirements include frame rate and duration. The specified requirements can be fixed or can be pre-set by the logged-in user of the terminal. Among them, the duration represents the playback duration of the video corresponding to the interpolation processing result, and the frame rate represents the number of video frames (Frame Per Second, fps) included in the video per second. Therefore, the product of the duration and the frame rate represents: the number of pictures included in the interpolation processing result. For example, the frame rate is 30 fps and the duration is 3 seconds (Second, s), so the interpolation processing result needs to include 30×3=90 pictures. The specified requirements may also include other parameters, which are not specifically limited in the embodiments of the present disclosure.

[0102] After filling in the second prompt template, the second prompt information is:

[0103] "Please perform interpolation processing on the following images so that the interpolation processing results meet the specified requirements.

[0104] Pictures: Similar Picture 1, Pictures to be processed, Similar Picture 2, Similar Picture 3.

[0105] Specified requirements: 30 fps, 3s. ”

[0106] S302: Input the second prompt information into the second largest language model to obtain an output result of the second largest language model.

[0107] The output results include: the image to be processed, similar images and transition frames sorted in the playback order. For example, the output results include 90 images in total, namely: similar image 1, transition frames 1 to 30, image to be processed, transition frames 31 to 59, similar image 2, transition frames 60 to 86, and similar image 3.

[0108] The second largest language model may be a multimodal large language model (MM-LLMs) or other large language models with interpolation capabilities. The first largest language model and the second largest language model may be the same model or different models, which is not specifically limited in the embodiments of the present disclosure.

[0109] The interpolation process can make the transition between the image to be processed and the similar image smoother. For example, in the image to be processed and the similar image, in the first image, the hand of person 1 is on the table, and in the second image, the hand of person 1 is off the table and above the table. Several transition frames are inserted between the two images, and in each transition frame, the hand of person 1 moves away from the table and approaches the position of the hand of person 1 in the second image, so that when playing from the first image to the second image, the effect of person 1's hand gradually lifting from the table is achieved.

[0110] The disclosed embodiment utilizes the second largest language model to perform frame interpolation processing on the image to be processed and similar images, thereby inserting sufficient transition frames between the image to be processed and similar images, making the picture transition between the image to be processed and similar images more natural and smooth, thereby simulating the effect of continuous video frames actually shot, and improving the realism of the live picture obtained thereby.

[0111] After performing frame insertion processing between the to-be-processed picture and the similar picture in the above S103, the embodiment of the present disclosure can also generate a live picture in the following two ways.

[0112] Method 1: The cloud synthesizes the video according to the playback order of the to-be-processed pictures, similar pictures and transition frames, and then sends the video to the terminal, so that the terminal uses its own live picture synthesis function to generate a live picture based on the video.

[0113] The cloud can use a preset video synthesis tool to synthesize the video according to the playback order of the image to be processed, similar images and transition frames. The format of the video can be a preset video format, such as an audio and video file encapsulation format (QuickTime Movie, mov). The cloud then sends the video to the terminal, and the terminal can use the live picture synthesis function provided by its own operating system, or the live picture synthesis function provided by its own installed software, to generate a live picture based on the video, and use the image to be processed as the cover of the live picture. The format of the live picture can be a preset picture format, such as livp. Afterwards, the terminal can display the live picture for the user to save or share the live picture.

[0114] For example, when the terminal operating system is a mobile operating system (iPhone Operating System, IOS), since the audio-visual media framework (AVFoundation) included in IOS can provide a live picture synthesis function, the terminal can input the video and the picture to be processed into AVFoundation to obtain the live picture output by AVFoundation.

[0115] For another example, when the operating system of the terminal is Android, a multimedia processing tool that can provide a live picture synthesis function, such as a multimedia processing framework (FFmpeg), can be installed in the terminal, so that the terminal can input the video and the picture to be processed into the multimedia processing tool to obtain the live picture output by the multimedia processing tool.

[0116] Through the above method, the cloud can synthesize the interpolation results into a video, and use the terminal's live picture synthesis function to generate a live picture based on the video. This method realizes the generation of a live picture based on the current viewing picture when the user views the picture through the terminal, providing the user with a dynamic display effect of the current viewing picture, making the display effect of the picture richer and improving the user experience.

[0117] Method 2: The cloud sends the pictures to be processed, similar pictures and transition frames to the terminal, so that the terminal generates a video according to the playback order of the pictures to be processed, similar pictures and transition frames, and uses its own live picture synthesis function to generate a live picture based on the video.

[0118] Among them, the method of synthesizing video by the terminal is the same as that of synthesizing video by the cloud, and you can refer to the description in method one. The method of generating live pictures by the terminal can also refer to the description in method one, which will not be repeated here.

[0119] Through the above method, the cloud can send the to-be-processed pictures, similar pictures and transition frames to the terminal, so that the terminal can generate videos and live pictures based on them. This method realizes the generation of live pictures based on the currently viewed pictures when the user views pictures through the terminal, providing the user with a dynamic display effect of the currently viewed pictures, making the display effect of the pictures richer and improving the user experience.

[0120] Since the live picture actually taken may also include audio, the embodiment of the present disclosure may also determine the target audio, where the target audio is used to generate the live picture.

[0121] That is, the above S301 fills in the preset second prompt template based on the picture to be processed and similar pictures, which can be implemented as follows: obtaining keywords of multiple alternative audios, and then filling the keywords of the multiple alternative audios into the second prompt template, and filling the picture to be processed and similar pictures into the second prompt template in timestamp order.

[0122] The second prompt template is also used to indicate the selection of a target audio that matches the image to be processed and similar images from a plurality of candidate audios.

[0123] The cloud pre-stores multiple candidate audios, where the duration of each candidate audio is the duration of the video included in the live picture. The cloud can also pre-analyze the content, emotion, and adaptation scene of each candidate audio using a large language model, and output keywords for each candidate audio. For example, the keywords of a candidate audio for celebrating the New Year include: blessing, joy, festival, and New Year. The large language model can be the first large language model, or the second large language model, or other large language models.

[0124] Alternatively, the cloud can receive the keywords set by the staff for each candidate audio. Alternatively, the cloud can also determine the keywords for each candidate audio in other ways, which is not specifically limited in the embodiments of the present disclosure.

[0125] Correspondingly, after the second prompt information is input into the large language model, the second large language model can combine the keywords of the alternative audio and the input picture information to analyze the correlation between each alternative audio and the picture to be processed and similar pictures, thereby selecting the alternative audio with the highest correlation as the target audio.

[0126] The output result of the second language model also includes the audio identifier of the target audio. For example, the audio identifier is an audio ID or an audio name. Afterwards, the cloud can obtain the target audio corresponding to the audio identifier.

[0127] Since the disclosed embodiment can utilize a large language model and combine keywords of multiple candidate audios to select the target audio that best matches the image to be processed and similar images from multiple candidate audios, the target audio and the interpolation result are closer to those obtained simultaneously through actual shooting. Moreover, selecting the target audio that best matches the image to be processed and similar images can also improve the realism of the live image and reduce the sense of separation between the video screen and the audio in the live image.

[0128] In the case where the target audio is acquired, after performing frame insertion processing between the to-be-processed picture and the similar picture in the above S103, the embodiment of the present disclosure can also generate a live picture in the following two ways.

[0129] Method 1: synthesize the video according to the playback order of the to-be-processed picture, similar pictures and transition frames and in combination with the target audio, and then send the video to the terminal so that the terminal uses its own live picture synthesis function to generate a live picture based on the video.

[0130] The cloud can use a preset video synthesis tool to synthesize the video according to the playback order of the image to be processed, similar images and transition frames and in combination with the target audio. The specific method can be referred to the description in method 1 and will not be repeated here.

[0131] The method for the terminal to generate a live picture can be referred to the description in method 1, which will not be repeated here.

[0132] Through the above method, the cloud can synthesize the interpolation result and the target audio into a video, and use the terminal's live picture synthesis function to generate a live picture based on the video. This method realizes the generation of live pictures based on the current viewing picture when the user views the picture through the terminal, providing the user with a dynamic display effect of the current viewing picture with adapted audio, making the picture display effect richer and improving the user experience.

[0133] Method 2: Send the image to be processed, similar images, transition frames and target audio to the terminal, so that the terminal synthesizes the video in the playback order of the image to be processed, similar images and transition frames and combines the target audio, and uses its own live picture synthesis function to generate a live picture based on the video.

[0134] Among them, the method of synthesizing video by the terminal is the same as that of synthesizing video by the cloud, and you can refer to the description in method 1. The method of generating live pictures by the terminal can also refer to the description in method 1, which will not be repeated here.

[0135] Through the above method, the cloud can send the image to be processed, similar images, transition frames and target audio to the terminal, and synthesize them into a video through the terminal to generate a live image. This method realizes the generation of a live image based on the current viewing image when the user views the image through the terminal, providing the user with a dynamic display effect of the current viewing image with adapted audio, making the display effect of the image richer and improving the user experience.

[0136] See also Figure 4 The following describes the overall process of the live picture generation method provided by the embodiment of the present disclosure in combination with actual application scenarios:

[0137] The software modules related to generating live images in the cloud include: image analysis module, intelligent frame insertion module and video generation module. The software modules related to generating live images in the terminal include: live image synthesis module.

[0138] The user uploads pictures to the cloud through the client installed in the terminal. The cloud stores the uploaded pictures and uses the picture analysis module to determine the similarity between each uploaded picture and each candidate picture based on the metadata and content information of the picture and the metadata and content information of each candidate picture, where the candidate pictures are: other pictures belonging to the client's login account except the picture.

[0139] When the terminal displays a picture through the client, the picture is used as a picture to be processed, and a notification message is sent to the cloud. The cloud determines that the picture corresponding to the picture identifier included in the notification message is the picture to be processed. After that, the cloud selects alternative pictures whose similarity with the picture to be processed is higher than the preset threshold as similar pictures through the picture analysis module, and determines whether the number of similar pictures reaches the preset number. If so, a recommendation message is sent to the terminal. The terminal displays the recommendation page, and after detecting that the user clicks the start generation button on the recommendation page, it sends a confirmation message to the cloud. After receiving the confirmation message, the cloud fills in the preset second prompt template based on the picture to be processed and the similar picture through the intelligent interpolation module, obtains the second prompt information, and inputs the second prompt information into the multimodal large language model, so that the multimodal large language model performs interpolation processing, and outputs the picture to be processed, similar pictures and transition frames sorted in the order of playback.

[0140] Then, the cloud uses the video generation module to synthesize the video according to the playback order of the image to be processed, similar images and transition frames, and sends the video to the terminal. The terminal then uses its own live picture synthesis function to generate a live picture based on the video and displays the live picture to the user.

[0141] It can be seen that the embodiments of the present disclosure realize automatic analysis of similar pictures to the picture currently viewed by the user, and automatically perform frame insertion processing and generate live pictures, which reduces the dependence of the live picture generation process on manual work and reduces the complexity of manual operations. In addition, the generated live pictures are more realistic, the playback effect is smoother, and the user experience is improved.

[0142] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of the image information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0143] It should be noted that the large language model and the image in the embodiments of the present disclosure are not targeted at a specific user and cannot reflect the personal information of a specific user.

[0144] It should be noted that the images processed by the embodiments of the present disclosure may come from a public dataset.

[0145] Based on the same inventive concept, the present disclosure also provides a live picture generating device, see Figure 5 , the device includes: a determination module 501, a screening module 502 and a frame insertion module 503;

[0146] A determination module 501 is used to determine a picture to be processed;

[0147] A screening module 502 is used to determine similar pictures of the picture to be processed from the candidate pictures, wherein the picture to be processed and the candidate pictures are both non-live pictures;

[0148] The frame insertion module 503 is used to perform frame insertion processing between the to-be-processed picture determined by the determination module 501 and the similar picture obtained by the screening module 502, and the result of the frame insertion processing is used to generate a live picture.

[0149] In some embodiments of the present disclosure, the screening module 502 is specifically used to:

[0150] Select candidate images whose similarity with the image to be processed is higher than a preset threshold as similar images.

[0151] In some embodiments of the present disclosure, the device further comprises:

[0152] The recognition module is used to recognize content information of the image to be processed by using a content recognition model before determining the image to be processed;

[0153] The determination module 501 is further used to determine the similarity between the image to be processed and the candidate images based on the metadata and content information of the image to be processed and the metadata and content information of the candidate images.

[0154] In some embodiments of the present disclosure, the determination module 501 is specifically configured to:

[0155] Filling the metadata and content information of the image to be processed, as well as the metadata and content information of the candidate image, into a preset first prompt template to obtain first prompt information, where the first prompt template is used to indicate the similarity between the images;

[0156] The first prompt information is input into the first largest language model to obtain the similarity between the to-be-processed image and the candidate image output by the first largest language model.

[0157] In some embodiments of the present disclosure, the determination module 501 is specifically configured to:

[0158] A notification message sent by a receiving terminal is used to indicate that the terminal is displaying a picture to be processed, and the notification message includes a picture identifier of the picture to be processed;

[0159] It is determined that the picture corresponding to the picture identifier is the picture to be processed.

[0160] In some embodiments of the present disclosure, the device further comprises:

[0161] A judging module, used for judging whether the number of similar pictures reaches a preset number before performing frame insertion processing between the picture to be processed and the similar pictures;

[0162] A sending module, used for sending a recommendation message to the terminal if the judgment result of the judgment module is yes, the recommendation message is used to indicate a recommendation to generate a live picture based on the picture to be processed;

[0163] The frame insertion module 503 is specifically used for:

[0164] When a confirmation message sent by the terminal is received, interpolation processing is performed between the picture to be processed and the similar picture, and the confirmation message is used to indicate confirmation of generating a live picture based on the picture to be processed.

[0165] In some embodiments of the present disclosure, the determination module 501 is further configured to:

[0166] Before sending the recommendation message to the terminal, determining whether the image to be processed is a real shot image based on the metadata of the image to be processed;

[0167] If so, the sending module is called to execute the step of sending the recommendation message to the terminal.

[0168] In some embodiments of the present disclosure, the frame insertion module 503 is specifically configured to:

[0169] Based on the image to be processed and similar images, fill in a preset second prompt template to obtain second prompt information, where the second prompt template is used to indicate to perform interpolation processing between the images;

[0170] The second prompt information is input into the second largest language model to obtain an output result of the second largest language model, where the output result includes: the to-be-processed pictures, similar pictures and transition frames sorted in the playback order.

[0171] In some embodiments of the present disclosure, the frame insertion module 503 is specifically configured to:

[0172] Get keywords for multiple audio alternatives;

[0173] The keywords of multiple alternative audios are filled into the second prompt template, and the pictures to be processed and similar pictures are filled into the second prompt template in the order of timestamps. The second prompt template is also used to indicate the selection of target audio that matches the pictures to be processed and similar pictures from multiple alternative audios. The output result also includes the audio identifier of the target audio, and the target audio is used to generate a live picture.

[0174] In some embodiments of the present disclosure, the device further comprises:

[0175] A synthesis module, for synthesizing a video according to the play order of the to-be-processed picture, the similar picture and the transition frame after performing frame insertion processing between the to-be-processed picture and the similar picture;

[0176] The sending module is used to send the video to the terminal so that the terminal can generate a live picture based on the video by using its own live picture synthesis function.

[0177] In some embodiments of the present disclosure, the device further comprises:

[0178] The sending module is used to send the picture to be processed, the similar picture and the transition frame to the terminal after performing frame insertion processing between the picture to be processed and the similar picture, so that the terminal generates a video according to the playback order of the picture to be processed, the similar picture and the transition frame, and uses its own live picture synthesis function to generate a live picture based on the video.

[0179] In some embodiments of the present disclosure, the device further comprises:

[0180] A synthesis module, for synthesizing a video according to the play order of the to-be-processed picture, the similar picture and the transition frame and combining the target audio after performing frame insertion processing between the to-be-processed picture and the similar picture;

[0181] The sending module is used to send the video to the terminal so that the terminal can generate a live picture based on the video by using its own live picture synthesis function.

[0182] In some embodiments of the present disclosure, the device further comprises:

[0183] The sending module is used to send the picture to be processed, similar pictures, transition frames and target audio to the terminal after interpolating between the picture to be processed and the similar pictures, so that the terminal synthesizes the video in the playback order of the picture to be processed, similar pictures and transition frames and in combination with the target audio, and uses its own live picture synthesis function to generate a live picture based on the video.

[0184] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0185] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0186] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0187] Multiple components in the electronic device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0188] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as a live picture generation method. For example, in some embodiments, the live picture generation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the live picture generation method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the live picture generation method in any other appropriate manner (e.g., by means of firmware).

[0189] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0190] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0191] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0192] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0193] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0194] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0195] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0196] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating a live picture, comprising: Determine the pictures to be processed; Determine, from each candidate picture, a similar picture to the picture to be processed, wherein both the picture to be processed and the candidate pictures are non-live pictures; An interpolation process is performed between the picture to be processed and the similar picture, and the interpolation process result is used to generate a live picture.

2. The method according to claim 1, wherein: The step of determining a similar picture to the picture to be processed from the candidate pictures includes: A candidate picture whose similarity with the picture to be processed is higher than a preset threshold is selected as the similar picture.

3. The method according to claim 2, before determining the picture to be processed, further comprising: Identify the content information of the image to be processed by a content recognition model; Based on the metadata and content information of the picture to be processed and the metadata and content information of the candidate pictures, the similarity between the picture to be processed and the candidate pictures is determined.

4. The method according to claim 3, wherein: The determining the similarity between the image to be processed and the candidate images based on the metadata and content information of the image to be processed and the metadata and content information of the candidate images includes: Filling the metadata and content information of the to-be-processed picture and the metadata and content information of the candidate picture into a preset first prompt template to obtain first prompt information, wherein the first prompt template is used to indicate the similarity between the pictures; The first prompt information is input into a first large language model to obtain the similarity between the to-be-processed picture and the candidate picture output by the first large language model.

5. The method according to any one of claims 1 to 4, wherein: The step of determining the picture to be processed includes: receiving a notification message sent by a terminal, the notification message being used to indicate that the terminal is displaying the picture to be processed, the notification message including a picture identifier of the picture to be processed; Determine that the picture corresponding to the picture identifier is the picture to be processed.

6. The method according to claim 5, before performing frame interpolation processing between the to-be-processed picture and the similar picture, further comprising: Determine whether the number of similar pictures reaches a preset number; If yes, sending a recommendation message to the terminal, the recommendation message is used to indicate a recommendation to generate a live picture based on the picture to be processed; The interpolation process is performed between the to-be-processed picture and the similar picture, comprising: When a confirmation message sent by the terminal is received, interpolation processing is performed between the picture to be processed and the similar picture, and the confirmation message is used to indicate confirmation of generating a live picture based on the picture to be processed.

7. The method according to claim 6, before sending the recommendation message to the terminal, further comprising: Based on the metadata of the picture to be processed, determining whether the picture to be processed is a real shot picture; If yes, the step of sending the recommendation message to the terminal is performed.

8. The method according to any one of claims 1 to 7, wherein: The interpolation process is performed between the to-be-processed picture and the similar picture, comprising: Based on the picture to be processed and the similar picture, fill in a preset second prompt template to obtain second prompt information, where the second prompt template is used to indicate to perform interpolation processing between pictures; The second prompt information is input into a second largest language model to obtain an output result of the second largest language model, wherein the output result includes: the to-be-processed picture, the similar picture and the transition frame sorted in a playback order.

9. The method according to claim 8, wherein: The filling in a preset second prompt template based on the image to be processed and the similar image includes: Get keywords for multiple audio alternatives; Fill the keywords of the multiple alternative audios into the second prompt template, and fill the picture to be processed and the similar pictures into the second prompt template in timestamp order. The second prompt template is also used to indicate the selection of target audio that matches the picture to be processed and the similar pictures from the multiple alternative audios. The output result also includes the audio identifier of the target audio, and the target audio is used to generate the live picture.

10. The method according to claim 8 or 9, after performing frame interpolation processing between the to-be-processed picture and the similar picture, further comprising: Synthesize a video according to the play order of the to-be-processed picture, the similar picture and the transition frame; The video is sent to a terminal so that the terminal generates the live picture based on the video by using its own live picture synthesis function.

11. The method according to claim 8 or 9, after performing frame interpolation processing between the to-be-processed picture and the similar picture, further comprising: The picture to be processed, the similar picture and the transition frame are sent to the terminal, so that the terminal generates a video according to the playback order of the picture to be processed, the similar picture and the transition frame, and uses its own live picture synthesis function to generate the live picture based on the video.

12. The method according to claim 9, after performing frame interpolation processing between the to-be-processed picture and the similar picture, further comprising: Synthesize a video according to the play order of the to-be-processed picture, the similar picture and the transition frame and in combination with the target audio; The video is sent to a terminal so that the terminal generates the live picture based on the video by using its own live picture synthesis function.

13. The method according to claim 9, after performing frame interpolation processing between the to-be-processed picture and the similar picture, further comprising: The picture to be processed, the similar picture, the transition frame and the target audio are sent to the terminal, so that the terminal synthesizes the video according to the playback order of the picture to be processed, the similar picture and the transition frame and in combination with the target audio, and uses its own live picture synthesis function to generate the live picture based on the video.

14. A live picture generating device, comprising: A determination module, used to determine the image to be processed; A screening module, used to determine similar pictures of the picture to be processed from each candidate picture, wherein both the picture to be processed and the candidate pictures are non-live pictures; The interpolation module is used to perform interpolation processing between the to-be-processed picture determined by the determination module and the similar picture obtained by the screening module, and the interpolation processing result is used to generate a live picture.

15. The device according to claim 14, wherein: The screening module is specifically used for: A candidate picture whose similarity with the picture to be processed is higher than a preset threshold is selected as the similar picture.

16. The apparatus according to claim 15, further comprising: An identification module, used for identifying content information of the image to be processed by using a content identification model before determining the image to be processed; The determination module is further used to determine the similarity between the image to be processed and the candidate images based on the metadata and content information of the image to be processed and the metadata and content information of the candidate images.

17. The device according to claim 16, wherein: The determination module is specifically used for: Filling the metadata and content information of the to-be-processed picture and the metadata and content information of the candidate picture into a preset first prompt template to obtain first prompt information, wherein the first prompt template is used to indicate the similarity between the pictures; The first prompt information is input into a first large language model to obtain the similarity between the to-be-processed picture and the candidate picture output by the first large language model.

18. The device according to any one of claims 14 to 17, wherein: The determination module is specifically used for: receiving a notification message sent by a terminal, the notification message being used to indicate that the terminal is displaying the picture to be processed, the notification message including a picture identifier of the picture to be processed; Determine that the picture corresponding to the picture identifier is the picture to be processed.

19. The apparatus according to claim 18, further comprising: A judging module, used for judging whether the number of the similar pictures reaches a preset number before performing frame insertion processing between the picture to be processed and the similar pictures; A sending module, configured to send a recommendation message to the terminal if the judgment result of the judgment module is yes, wherein the recommendation message is used to indicate a recommendation to generate a live picture based on the picture to be processed; The interpolation module is specifically used for: When a confirmation message sent by the terminal is received, interpolation processing is performed between the picture to be processed and the similar picture, and the confirmation message is used to indicate confirmation of generating a live picture based on the picture to be processed.

20. The apparatus according to claim 19, wherein the determining module is further configured to: Before sending the recommendation message to the terminal, determining whether the picture to be processed is a real-shot picture based on the metadata of the picture to be processed; If so, the sending module is called to execute the step of sending the recommendation message to the terminal.

21. The device according to any one of claims 14 to 20, wherein: The interpolation module is specifically used for: Based on the picture to be processed and the similar picture, fill in a preset second prompt template to obtain second prompt information, where the second prompt template is used to indicate to perform interpolation processing between pictures; The second prompt information is input into a second largest language model to obtain an output result of the second largest language model, wherein the output result includes: the to-be-processed picture, the similar picture and the transition frame sorted in a playback order.

22. The device according to claim 21, wherein the frame insertion module is specifically configured to: Get keywords for multiple audio alternatives; Fill the keywords of the multiple alternative audios into the second prompt template, and fill the picture to be processed and the similar pictures into the second prompt template in timestamp order. The second prompt template is also used to indicate the selection of target audio that matches the picture to be processed and the similar pictures from the multiple alternative audios. The output result also includes the audio identifier of the target audio, and the target audio is used to generate the live picture.

23. The apparatus according to claim 21 or 22, further comprising: A synthesis module, configured to synthesize a video according to the play order of the picture to be processed, the similar picture and the transition frame after performing frame insertion processing between the picture to be processed and the similar picture; The sending module is used to send the video to the terminal, so that the terminal uses its own live picture synthesis function to generate the live picture based on the video.

24. The apparatus according to claim 21 or 22, further comprising: A sending module is used to send the picture to be processed, the similar picture and the transition frame to the terminal after performing frame insertion processing between the picture to be processed and the similar picture, so that the terminal generates a video according to the playback order of the picture to be processed, the similar picture and the transition frame, and uses its own live picture synthesis function to generate the live picture based on the video.

25. The apparatus of claim 22, further comprising: A synthesis module, configured to synthesize a video according to the play order of the picture to be processed, the similar picture and the transition frame and in combination with the target audio after performing frame insertion processing between the picture to be processed and the similar picture; The sending module is used to send the video to the terminal, so that the terminal uses its own live picture synthesis function to generate the live picture based on the video.

26. The apparatus of claim 22, further comprising: A sending module is used to send the picture to be processed, the similar picture, the transition frame and the target audio to the terminal after performing frame insertion processing between the picture to be processed and the similar picture, so that the terminal synthesizes the video in the playback order of the picture to be processed, the similar picture and the transition frame and in combination with the target audio, and uses its own live picture synthesis function to generate the live picture based on the video.

27. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 13.

28. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-13.

29. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 13.

Images