Data generation method and device, electronic equipment and computer readable medium

Abstract

The invention discloses a data generation method and device, electronic equipment and a computer readable medium, and relates to the technical field of data generation, and the method comprises the steps: obtaining to-be-processed data which comprises at least one of picture data and video data; obtaining first prompt information based on the features of the to-be-processed data and personalized information of a target user; performing semantic analysis on the first prompt information through a large language model to obtain first text information; generating target audio data based on the first text information; and generating target video data based on the to-be-processed data and the target audio data. And the first prompt information obtained based on the features of the to-be-processed data and the personalized information of the target user is combined with a large language model to generate the target video data, so that the target video data has more personalized characteristics.

Description

Technical Field

[0001] The present application relates to the technical field of data generation, and more specifically, to a data generation method, device, electronic device and computer-readable medium. Background Art

[0002] At present, with the development of electronic information technology, electronic devices can generate video data including audio data based on data input by users. However, the currently obtained video data cannot meet the personalized needs of users. Summary of the invention

[0003] The present application proposes a data generation method, device, electronic device and computer-readable medium.

[0004] In a first aspect, an embodiment of the present application provides a data generation method, comprising: obtaining data to be processed, wherein the data to be processed includes at least one of image data and video data; obtaining first prompt information based on characteristics of the data to be processed and personalized information of a target user; performing semantic analysis on the first prompt information through a large language model to obtain first text information; generating target audio data based on the first text information; and generating target video data based on the data to be processed and the target audio data.

[0005] In the second aspect, the embodiment of the present application also provides a data generating device, which includes: a first acquisition unit, a second acquisition unit, a third acquisition unit, a target audio data generating unit, and a target video data generating unit. The first acquisition unit is used to acquire the data to be processed, wherein the data to be processed includes at least one of image data and video data; the second acquisition unit is used to obtain the first prompt information based on the characteristics of the data to be processed and the personalized information of the target user; the third acquisition unit is used to perform semantic analysis on the first prompt information through a large language model to obtain the first text information; the target audio data generating unit is used to generate the target audio data based on the first text information; the target video data generating unit is used to generate the target video data based on the data to be processed and the target audio data.

[0006] In a third aspect, an embodiment of the present application also provides an electronic device, comprising: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the above method.

[0007] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the method described in the first aspect.

[0008] The data generation method, device, electronic device and computer-readable medium provided by the present application first obtain the data to be processed, wherein the data to be processed includes at least one of image data and video data; then obtain the first prompt information based on the characteristics of the data to be processed and the personalized information of the target user; then perform semantic analysis on the first prompt information through a large language model to obtain the first text information; generate target audio data based on the first text information; thereby generate target video data based on the data to be processed and the target audio data. In the scheme of the present application, the first prompt information is semantically analyzed based on the large language model to obtain the first text information, thereby subsequently combining the first text information and the data to be processed to generate the target video data, wherein the first prompt is obtained based on the characteristics of the data to be processed and the personalized information of the target user. Thus, the obtained target video data has the personalized information of the target user and the characteristics of the data to be processed, meeting the personalized needs of the user.

[0009] Other features and advantages of the embodiments of the present application will be described in the subsequent description, and partly become apparent from the description, or can be understood by practicing the embodiments of the present application. The purposes and other advantages of the embodiments of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A method flow chart of a data generation method provided by an embodiment of the present application is shown;

[0012] Figure 2 A method flow chart of a data generation method provided by another embodiment of the present application is shown;

[0013] Figure 3 A method flow chart of a data generation method provided by another embodiment of the present application is shown;

[0014] Figure 4A method flow chart of a data generation method provided by yet another embodiment of the present application is shown;

[0015] Figure 5 A method flow chart of a data generation method provided by another embodiment of the present application is shown;

[0016] Figure 6 A structural block diagram of a data generating device provided in an embodiment of the present application is shown;

[0017] Figure 7 A structural block diagram of an electronic device provided in an embodiment of the present application is shown;

[0018] Figure 8 A structural block diagram of a computer-readable storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] In order to make those skilled in the art better understand the present application scheme, the technical scheme in the present application embodiment will be clearly and completely described below in conjunction with the drawings in the present application embodiment. Obviously, the described embodiment is only a part of the present application embodiment, rather than all the embodiments. The components of the present application embodiment usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application for protection, but merely represents the selected embodiment of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0020] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0021] At present, with the development of electronic information technology, electronic devices can generate video data including audio data based on data input by users. However, the video data currently obtained cannot meet the personalized needs of users. How to make the obtained video data more meet the personalized needs of users is a problem to be solved urgently.

[0022] Currently, electronic devices can obtain material content, then generate corresponding background music based on the material content, and then combine the background music with the material content to generate video data including music.

[0023] However, the inventors found in their research that the current method of generating video data cannot meet the personalized needs of users.

[0024] Therefore, in order to overcome the above-mentioned defects, the embodiments of the present application provide a data generation method, device, electronic device and computer-readable medium to solve or partially solve the above-mentioned problems.

[0025] See also Figure 1 , Figure 1 A data generation method provided by an embodiment of the present application is shown, and the method can be applied to an electronic device, and the execution subject of the method can be a processor in the electronic device. Specifically, the method includes steps S110 to S150.

[0026] It should be noted that the data generation method provided in each embodiment of the present application can be applied to various scenarios that require generating video data with audio.

[0027] Step S110: Acquire data to be processed, wherein the data to be processed includes at least one of image data and video data.

[0028] The data to be processed may include at least one of picture data and video data, wherein the picture data may be one piece of picture data or multiple pieces of picture data, and the multiple pieces of picture data may also be referred to as a picture data set.

[0029] The data to be processed may be provided to the electronic device by the target user. Exemplarily, the electronic device may be configured with an image acquisition module, so that the target user can control the electronic device to acquire data content through the image acquisition module, and then use the data content as the data to be processed. The acquired data content may include at least one of picture data and video data.

[0030] In another exemplary embodiment, an electronic device can establish a communication connection with another electronic device, so that the target user can send at least one of the image data and video data stored on the other electronic device to the electronic device, so that the electronic device can use at least one of the acquired image data and video data as data to be processed. For example, the electronic device can be a smart phone, and the other electronic device can be a smart tablet, so that at least one of the image data and video data stored on the smart tablet can be sent to the smart phone.

[0031] In another exemplary embodiment, the electronic device may include a storage module, which can store at least one of image data and video data, so that the target user can control the electronic device to access the storage module and then select at least one of the stored image data and video data as data to be processed.

[0032] Step S120: obtaining first prompt information based on the characteristics of the data to be processed and the personalized information of the target user.

[0033] The characteristics of the data to be processed may include text information describing the characteristics of the data to be processed, and specifically, for example, the characteristics of the data to be processed may be described by second text information. For some embodiments, the characteristics of the data to be processed may include characteristics of the content in the data to be processed, and may also include characteristics of the element data corresponding to the data to be processed. For an introduction to obtaining the characteristics of the data to be processed, please refer to the subsequent embodiments.

[0034] In addition, for some embodiments, the personalized information of the target user may include information related to the personalization of the target user. For example, it may include the model parameters of the electronic device used by the target user, the active time period of the target user's social media activities, the target user's dietary preferences, the target user's sports health data, the target user's search, browsing and music listening history data or the target user's purchase history data, etc. For other embodiments, the personalized information of the target user may also include the shooting feature information corresponding to the data to be processed input by the target user. For example, the shooting feature information may include the geographical location information of the data to be processed, the weather information when the data to be processed was shot, or the shooting time information, etc.

[0035] The personalized information of the target user may be described by text information, for example, the personalized information of the target user may be described by third text information.

[0036] It should be noted that the specific examples of personalized information of the target user shown above are only for the purpose of introducing and explaining the personalized information, and do not constitute a specific limitation on the embodiments of the present application. In actual applications, they can be flexibly adjusted as needed.

[0037] Thus, the first prompt information can be obtained based on the characteristics of the data to be processed and the personalized information of the target user. For example, the second text information describing the characteristics of the data to be processed and the third text information describing the personalized information of the target user can be combined to obtain the first prompt information.

[0038] Optionally, the portrait label of the target user may also be obtained through the above-mentioned personalized information, so that the first prompt information can be obtained later based on the characteristics of the data to be processed and the portrait label.

[0039] Step S130: performing semantic analysis on the first prompt information through a large language model to obtain first text information.

[0040] After obtaining the first prompt information, the first prompt information may be semantically analyzed by a large language model to obtain first text information, wherein the first text information may include text information describing the target audio data.

[0041] In some embodiments, in order to obtain the first text information describing the target audio data, the second prompt information can be obtained, and the second prompt information and the first prompt information can be input into the large language model together as input, so as to perform semantic analysis on the first prompt information through the large language model to obtain the first text information.

[0042] The second prompt information may include a task description indicating the task that the large language model needs to perform, such as "You are an intelligent text transcription expert. Your task is to transcribe the input text into a prompt text suitable for the audio generation model."

[0043] In addition, the second prompt information may also include a description of the first prompt information, such as "the input text includes a feature description of the image data or video data, shooting time information, geographic location information, shooting weather information, and personalized information of the target user."

[0044] Optionally, the second prompt information may also include a sample task description and at least one sample output example, so that the large language model can learn the target relationship between the sample task description and the sample output example through an in-context learning algorithm (ICL), and then perform semantic analysis on the first prompt information in combination with the target relationship, thereby inferring the first text information. For a detailed introduction, please refer to the subsequent embodiments.

[0045] It should be noted that the first prompt information and the second prompt information can be used as input prompts of the large language model.

[0046] Step S140: Generate target audio data based on the first text information.

[0047] As can be seen from the above introduction, the first text information can be used to describe the target audio data, so the target audio data can be generated based on the first text information. The target audio corresponding to the target audio data is what needs to be included in the target video data generated subsequently.

[0048] In some embodiments, the target audio may be an audio file or a data stream of an audio file. The target audio data may be used to characterize a first vector of the target audio, and the first vector may be a feature vector of the target audio. That is, the feature vector of the target audio may be generated based on the first text information. Exemplarily, the first text information may be encoded to obtain the first vector of the target audio as the target audio data. For a detailed description, please refer to the subsequent embodiments.

[0049] Step S150: Generate target video data based on the data to be processed and the target audio data.

[0050] After obtaining the target audio data, the target video data can be generated in combination with the data to be processed. For some embodiments, in order to obtain the target video data containing the target audio data, the specified video data corresponding to the data to be processed can be first obtained, and the specified video data can be a video file or a video data stream. The specified video data can be generated based on the data to be processed, and the detailed method of obtaining the specified video data can be referred to in the subsequent embodiments.

[0051] Furthermore, the designated video data and the target audio data may be fused to generate the target video data. For example, the second vector corresponding to the designated video data may be determined, and then a third vector may be constructed based on the first vector and the second vector, so that the target video data may be obtained through the third vector, so that the target video data may include the content of the designated video data and the content of the target audio data. For a detailed description, please refer to the subsequent embodiments.

[0052] The data generation method provided by the present application first obtains the data to be processed, wherein the data to be processed includes at least one of image data and video data; then obtains the first prompt information based on the characteristics of the data to be processed and the personalized information of the target user; then performs semantic analysis on the first prompt information through a large language model to obtain the first text information; generates target audio data based on the first text information; and thus generates target video data based on the data to be processed and the target audio data. In the scheme of the present application, the first prompt information is semantically analyzed based on the large language model to obtain the first text information, and then the target video data is subsequently generated by combining the first text information and the data to be processed, wherein the first prompt is obtained based on the characteristics of the data to be processed and the personalized information of the target user. Thus, the obtained target video data has the personalized information of the target user and the characteristics of the data to be processed, meeting the personalized needs of the user.

[0053] See also Figure 2 , Figure 2A data generation method provided by an embodiment of the present application is shown, and the method can be applied to an electronic device, and the execution subject of the method can be a processor in the electronic device. Specifically, the method includes steps S210 to S270.

[0054] Step S210: Acquire data to be processed, wherein the data to be processed includes at least one of image data and video data.

[0055] Among them, step S210 has been introduced in detail in the above embodiment and will not be repeated here.

[0056] Step S220: Acquire second text information for describing the characteristics of the data to be processed.

[0057] It can be known from the foregoing introduction that the data to be processed may include at least one of picture data and video data, that is, the data to be processed may include picture data, may include video data, or may include both picture data and video data.

[0058] For some implementations, when the data to be processed includes picture data, step S220 may include step S221.

[0059] Step S221: performing a content understanding operation on the image data through an image understanding model to obtain fifth text information for describing features of the image data as the second text information.

[0060] Among them, the picture understanding model can be used to analyze the picture data and extract key feature information in the image data, such as objects, scenes, and contextual relationships in the image data. The input of the picture understanding model can be picture data, and the output can be text information corresponding to the picture data. In some embodiments, the picture recognition module can be constructed based on deep learning technology, such as based on image recognition algorithms such as convolutional neural networks (CNN).

[0061] Thus, when the data to be processed includes image data, the image understanding model can be used to perform content understanding operations on the image data to obtain fifth text information used to describe the characteristics of the image data as the second text information. The fifth text information is the text information output by the image understanding model for the input image data.

[0062] For example, the fifth text information may be "This is a bird's-eye view of a tropical seaside resort. The resort is gently embraced by the blue sea. The resort is filled with white and blue villas, as well as exotic thatched-roof huts."

[0063] For some other implementations, when the data to be processed includes multiple picture data, step S220 may further include step S222 and step S223.

[0064] Step S222: Perform content understanding operation on each of the picture data through the picture understanding model to obtain sixth text information corresponding to each of the picture data and used to describe the picture data.

[0065] Step S223: Arrange each of the sixth text information based on a specified order to obtain the second text information, wherein the specified order is the arrangement order of the plurality of image data.

[0066] Similar to step S221, the image understanding model can be used to perform content understanding operations on each of the image data to obtain sixth text information corresponding to each of the image data for describing the image data. That is, multiple sixth text information can be obtained, and the sixth text information corresponds to the image data one by one.

[0067] Furthermore, each sixth text information may be arranged based on a specified order to obtain the second text information, wherein the specified order is the arrangement order of the plurality of image data. That is to say, the arrangement order of the sixth text information is the same as the arrangement order of the plurality of image data, thereby ensuring that each sixth text information in the second text information corresponds to the image data one by one.

[0068] For some other implementations, when the data to be processed includes video data, step S220 may include step SS224.

[0069] Step S224: extracting metadata of the video data as the second text information.

[0070] If the data to be processed includes video data, metadata of the video data can be directly extracted as the second text information. Exemplarily, the metadata may include data such as video title, shooting time, encoding format, resolution, bit rate, frame rate, and bit depth.

[0071] For some other implementations, when the data to be processed includes picture data and video data, step S220 may include steps S225 to S227.

[0072] Step S225: Performing a content understanding operation on the image data through the image understanding model to obtain fifth text information for describing the characteristics of the image data.

[0073] Step S226: extracting metadata of the video data.

[0074] Step S227: taking the fifth text information and the metadata as the second text information.

[0075] Similar to the above steps, the image data may be subjected to content understanding operation by the image understanding model, thereby obtaining fifth text information for describing the characteristics of the image data. Then, metadata of the video data may be extracted, thereby using the fifth text information and the metadata as the second text information.

[0076] Step S230: Acquire third text information for describing personalized information of the target user.

[0077] In addition, third text information for describing personalized information of the target user may also be obtained. For some embodiments, the target user's habit of using the electronic device and data in various applications running on the electronic device may be collected through the electronic device, thereby comprehensively generating the third text information. For example, the target user's habit of using the electronic device and data in various applications may be collected in the electronic device by pre-embedded points.

[0078] In addition, the data of the user's life can also be collected through the collection module configured by the electronic device, such as the collection of the user's daily routine data. Furthermore, the electronic device can also establish a communication connection with the sensor device. For example, the sensor device can be a smart wearable device, such as a smart bracelet, a smart watch, etc., so that the data of the target user's life can be collected through the sensor device and sent to the electronic device. The electronic device performs a comprehensive analysis on the acquired data to obtain the third text information used to describe the personalized information of the target user.

[0079] Exemplarily, the third text information may include "the target user is an artistic youth who likes fitness, has a regular schedule, and often plays passionate and dynamic music."

[0080] It should be noted that the target user can set permissions for the information that the electronic device can collect. If the target user does not set permissions, the electronic device can collect information based on default permissions, thereby obtaining third text information for describing the target user's personalized information.

[0081] Step S240: obtaining the first prompt information based on the second text information and the third text information.

[0082] Further, after obtaining the second text information and the third text information, the first prompt information may be obtained based on the second text information and the third text information. In some implementations, the second text information and the third text information may be directly merged to obtain the first prompt information.

[0083] In some other implementations, the target user may also input fourth text information, so that the target user may describe the characteristics of the data to be processed or describe the audio usage preference through the input fourth text information. Step S240 may also include step S241.

[0084] Step S241: When the fourth text information input by the target user is detected, the first prompt information is obtained based on the second text information, the third text information and the fourth text information, wherein the fourth text information includes prompt information of the target user describing the characteristics of the data to be processed and prompt information describing the audio usage preference.

[0085] The fourth text information includes prompt information describing the characteristics of the data to be processed and prompt information describing the audio usage preference of the target user. Thus, the first prompt information can be obtained based on the second text information, the third text information and the fourth text information. For example, the second text information, the third text information and the fourth text information are combined to obtain the first prompt information. The prompt information of the audio usage preference can be used to characterize the target user's preference for the target audio data contained in the target video data generated subsequently. For example, the prompt information describing the audio usage preference can include "passionate and exciting audio". In addition, the prompt information of the audio usage preference can also be used to characterize whether the target user expects the target audio data to include sound effects, for example, the sound effects can include virtual sound effects, such as "birds singing, insects chirping, whistles, spaceship navigation sound effects, etc.". The prompt information describing the characteristics of the data to be processed by the target user can be information actively input by the target user, for example, it can include "This is a bird's-eye view of an ancient Asian city, with many blue and gray tile roofs in the city, and a magnificent temple that combines Chinese and Buddhist elements."

[0086] It can be understood that, when the audio usage preference in the fourth text information includes sound effects, the subsequently generated target video data may have the characteristics of the sound effects, and when the audio usage preference in the fourth text information does not include sound effects, the subsequently generated target video data may not include sound effects.

[0087] It should be noted that although the prompt information in the fourth text information that describes the characteristics of the data to be processed by the target user and the second text information are both prompt information describing the characteristics of the data to be processed, the acquisition methods are different. The fourth text information is actively input by the target user, while the second text information is obtained by performing content understanding operations on the image data through the image understanding model, or directly extracting the metadata of the video data. As a result, even though both are prompt information describing the characteristics of the data to be processed, the prompt information in the second text information and the fourth text information that describes the characteristics of the data to be processed by the target user is different.

[0088] In some embodiments, when the data to be processed is obtained, the electronic device may generate a reminder message. The reminder message may be used to prompt the target user to input a description of the data to be processed or a description of the audio usage preference, etc. Exemplarily, the electronic device may display the reminder message in the form of text through a configured display screen, or may play the reminder message in the form of voice through a configured speaker. For example, the reminder message may be "Please enter the video duration, audio style, keywords describing the audio, desired music type, and whether background sound effects are required."

[0089] Thus, after receiving the reminder information, the target user can feedback information to the electronic device, and the information fed back by the target user can be used as the fourth text information. For example, the target user can touch and interact with the display screen of the electronic device to input the fourth text information.

[0090] Therefore, when the fourth text information input by the target user is detected, the first prompt information can be obtained based on the second text information, the third text information and the fourth text information. Thus, the personalization degree of the target audio data in the target video data obtained subsequently can be further increased.

[0091] Step S250: performing semantic analysis on the first prompt information through a large language model to obtain first text information.

[0092] Step S260: Generate target audio data based on the first text information.

[0093] Step S270: Generate target video data based on the data to be processed and the target audio data.

[0094] Among them, step S250 to step S270 have been introduced in detail in the above embodiments and will not be repeated here.

[0095] In the embodiment provided by the present application, when the fourth text information input by the target user is detected, the first prompt information is obtained based on the second text information, the third text information and the fourth text information, so that the personalization of the target audio data in the target video data obtained subsequently can be further increased. In addition, from the perspective of the target user, the target user only needs to input prompt information describing the characteristics of the data to be processed and prompt information describing the audio usage preference, etc., without actively selecting the target audio data to be matched, which reduces the threshold for the target user to select the soundtrack.

[0096] See also Figure 3 , Figure 3 A data generation method provided by an embodiment of the present application is shown, and the method can be applied to an electronic device, and the execution subject of the method can be a processor in the electronic device. Specifically, the method includes steps S310 to S3100.

[0097] Step S310: Acquire data to be processed, wherein the data to be processed includes at least one of image data and video data.

[0098] Step S320: obtaining first prompt information based on the characteristics of the data to be processed and the personalized information of the target user.

[0099] Among them, step S310 and step S320 have been introduced in detail in the above embodiments and will not be repeated here.

[0100] Step S330: Determine a sample task description and at least one sample output example corresponding to the data to be processed.

[0101] Step S340: using the first prompt information, the sample task description and at least one sample example as inputs of a large language model, so that the large language model learns the target relationship between the sample task description and the sample output example through a scenario learning algorithm.

[0102] Step S350: inferring the first prompt information and the target relationship through the large language model to obtain first text information.

[0103] It is understandable that in the process of obtaining the first text information through the large language model, in order to improve the accuracy of the obtained first text information, a sample task description and at least one sample output example corresponding to the data to be processed can be obtained. Then the first prompt information, the sample task description and at least one sample example are used as inputs of the large language model, so that the large language model learns the target relationship between the sample task description and the sample output example through a scenario learning algorithm. Then, the first prompt information and the target relationship are inferred by the large language model to obtain the first text information.

[0104] Taking the data to be processed as image data as an example, a sample task description may include “an example is as follows:

[0105] Sample task description:

[0106] Feature description of the image data: This is an overhead photo of a European-style town. The town is surrounded by a river. There are many houses with orange and red roofs, as well as a medium-sized Romanesque-style church.

[0107] Personalized information: Photographed in the small town of Cesky Krumlov, Czech Republic, (date), sunny day. The user is a literary youth who often posts on WeChat Moments. Recently, he prefers light food and fruits. He has been checking European travel guides recently and has purchased items such as foreign phone cards.

[0108] Audio usage preference: Video soundtrack of about 20 seconds, classical music, European style, beautiful and lyrical, with slight sound effects of flowing water.

[0109] At least one sample output example may include "sample output example:

[0110] Output Audio Generation Prompt:

[0111] Aerial shot of the Czech town of Cesky Krumlov in autumn, surrounded by rivers, European-style buildings and churches, literary style, classical music, beautiful and lyrical, and sound effects of flowing water, length 20 seconds.

[0112] It is understandable that the above sample task description and at least one sample output example may be included in the second prompt information, and the second prompt information may also include a task description indicating the task that the large language model needs to perform. Optionally, the task description may also include "taking copyrighted audio data as an example, generating new audio data without copyright risk", thereby instructing the large language model to avoid music copyright issues.

[0113] The target relationship is the semantic and logical relationship between the sample task description learned by the large language model through the scenario learning algorithm and at least one sample output example.

[0114] The prompt information input to the large language model may include first prompt information and second prompt information. Exemplarily, taking the data to be processed as an image data as an example, the prompt information input to the large language model may include "You are an intelligent text transcription expert. Your task is to transcribe the input text into a prompt text suitable for the audio generation model. The input text includes a feature description of an image data, shooting time information, geographic location information, shooting weather information, and personalized information of the target user; it may include a description of the target user's own audio usage preferences. An example is as follows:

[0115] Sample task description:

[0116] Feature description of the image data: This is an overhead photo of a European-style town. The town is surrounded by a river and contains many orange and red-roofed houses, as well as a medium-sized Romanesque church.

[0117] Personalized information: Photographed in the Czech town of Cesky Krumlov, (date), sunny day, the user is a literary youth, often posts on WeChat Moments, recently prefers light food and fruits, recently often checks European travel guides, and has purchased foreign phone cards and other items;

[0118] Audio usage preference: Video soundtrack of about 20 seconds, classical music, European style, beautiful and lyrical, with slight sound effects of flowing water;

[0119] Example sample output:

[0120] Output Audio Generation Prompt:

[0121] An aerial shot of the Czech town of Cesky Krumlov in autumn, surrounded by rivers, European-style buildings and churches, literary style, classical music, beautiful and lyrical, and the sound of flowing water. The video is 20 seconds long.

[0122] Current input: (second text message, third text message, and fourth text message)."

[0123] Thus, the large language model may display “output: first text information”.

[0124] In another exemplary embodiment, the data to be processed is an image data. The prompt information input to the large language model may include multiple image data. The prompt information input to the large language model may include "You are an intelligent text writing expert. Your task is to transcribe the input text into a prompt text suitable for the audio generation model. The input text includes several descriptions of several image data, corresponding to the order of the image data, feature description, shooting time information, geographic location information, shooting weather information and personalized information of the target user; it may include a description of the target user's own audio usage preference. An example is as follows:

[0125] Sample task description:

[0126] Feature description of multiple image data: Figure 1 :xxx; Figure 2 :xxx;......

[0127] Personalized information: xxx;

[0128] Example sample output:

[0129] Output Audio Generation Prompt:

[0130] xxx;

[0131] Current input: (second text message, third text message, and fourth text message)."

[0132] Thus, the large language model may display “output: first text information”.

[0133] In another exemplary embodiment, taking the data to be processed as video data as an example, the sample task description may include "You are an intelligent text transcription expert. Your task is to transcribe the input text into a prompt text suitable for the audio generation model. The input text contains the time information, geographic location information, weather information of the shooting, and personalized information of the target user; it may also contain a description of the target user's own audio usage preferences. An example is as follows:

[0134] Sample task description:

[0135] Personalized information: xxxx;

[0136] Audio usage preferences: remove the audio of the original video, add background music, add sound effects such as running water and insects;

[0137] Example sample output:

[0138] xxx;

[0139] Current input: (second text message, third text message, and fourth text message)."

[0140] Thus, the large language model may display “output: first text information”.

[0141] It should be noted that the "xxx" in the above examples is used to represent that no specific text is limited. It can be understood that the above xxx can represent different texts.

[0142] Step S360: Perform an encoding operation on the first text information through a text encoder to obtain a first vector of the target audio as the target audio data.

[0143] In some implementations, the generated first text information can be input into a text encoder, so that the text encoder performs an encoding operation on the first text information to obtain a first vector of the target audio as the target audio data, wherein the first vector can be used to represent a feature vector of the target audio.

[0144] Among them, the text encoder can convert text information into a context feature vector representation. For example, the word vector of the input first text information can be obtained through a neural network model, and then the word vector is encoded in combination with the encoder to obtain a context feature vector, which is the first vector of the target audio, as the target audio data.

[0145] Step S370: Generate designated video data based on the data to be processed.

[0146] Furthermore, the specified video data may be generated based on the data to be processed. Specifically, the data to be processed may be described by classification as including video data or not including video data.

[0147] For some implementations, the data to be processed may include picture data, so step S370 may include step S371 and step S372.

[0148] Step S371: when seventh text information input by the target user is detected, generating designated video data based on the seventh text information and the picture data, wherein the seventh text information is used to describe the video configuration preference of the target user.

[0149] Step S372: when the seventh text information input by the target user is not detected, generating designated video data based on the default video configuration information and the picture data.

[0150] In some embodiments, the electronic device may further obtain seventh text information input by the target user, wherein the seventh text information is used to describe the target user's video configuration preference. For example, the seventh text information may be "display image data for 15 seconds". The method for obtaining the seventh text information is similar to the method for obtaining the fourth text information input by the target user in the aforementioned embodiment, and will not be described in detail here.

[0151] Thus, when the seventh text information input by the target user is detected, the specified video data can be generated based on the seventh text information and the picture data. For example, continuing to use the seventh text information in the above example as "display picture data for 15 seconds", the specified video data can be generated at this time, and the duration of the specified video data is 15 seconds, and specifically 15 seconds of picture data can be displayed.

[0152] In addition, the seventh text information input by the target user may not be detected, for example, the target user does not input. Thus, the specified video data can be generated based on the default video configuration information and the picture data. Exemplarily, for the case where the data to be processed is one picture data, the default video configuration information may be "a single picture data lasts for 15 seconds"; for another example, for the case where the data to be processed is multiple picture data, the default video configuration information may be "multiple picture data, each picture data pauses for 2 seconds". Then, the specified video data can be generated based on the default video configuration information and the picture data.

[0153] Among them, the picture data can be converted into specified video data without audio by using the frame insertion method.

[0154] In other implementations, the data to be processed may also include video data, so that the video data may be directly used as the designated video data.

[0155] The designated video data may include a video data file, that is, a video file that has been encoded and packaged, or may include a video data stream.

[0156] It should be noted that, in the case where the data to be processed includes image data and video data, the video data may also be directly used as the designated video data corresponding to the data to be processed.

[0157] Optionally, the audio usage preference in the fourth text information may also include whether to retain the audio data in the specified data, so that when the audio usage preference includes "retain the audio data in the specified data", the target video data generated subsequently may include not only the features of the target audio data but also the features of the audio data of the specified video data. That is, on the basis of retaining the original audio track of the specified video data, the effect of adding the target audio data to the specified video data is achieved.

[0158] Step S380: performing an encoding operation on the designated video data through a video encoder to obtain a second vector.

[0159] Furthermore, after acquiring the designated video data, the designated video data may be encoded, specifically, the designated video data may be encoded by a video encoder to obtain a second vector. That is, the input designated video data may be analyzed by a video encoder, and a feature vector describing the designated video data may be extracted, and the key characteristics or patterns in the designated video data may be characterized by the feature vector of the designated video data, which is the second vector.

[0160] It can be seen that the specified video data may be first encoded and generated, the specified video data being a video file, and then the video file is encoded by a video encoder to obtain the second vector. Alternatively, the video data stream may be first obtained, and then the video file is encoded by a video encoder to obtain the second vector.

[0161] Step S390: construct a third vector based on the first vector and the second vector.

[0162] Then, a third vector is constructed based on the first vector and the second vector. It can be understood that the third vector includes both the feature vector of the target audio data and the feature vector of the specified video data. Therefore, the target video data is subsequently generated based on the third vector, so that the target video data has the characteristics of the target audio data and the characteristics of the specified video data.

[0163] For some implementations, step S390 may include step S391 and step S392.

[0164] Step S391: input the first vector and the second vector into a diffusion model to perform feature fusion on the first vector and the second vector to obtain a fourth vector.

[0165] Step S392: forward diffuse and backward diffuse the fourth vector through the diffusion model to obtain an output third vector.

[0166] First, the first vector and the second vector can be input into the diffusion model, and the first vector and the second vector are subjected to feature fusion to obtain a fourth vector. Thus, the fourth feature vector is a vector that fuses the features of the target audio data and the features of the designated video data.

[0167] Furthermore, the fourth vector can also be forward diffused through the diffusion model. Forward diffusion is generally a process of gradually adding noise to the fourth vector through a diffusion model, which simulates the process of data gradually becoming blurred and unrecognizable. Then, the fourth vector that has been forward diffused is back-diffused, and the noise is reversely removed through the diffusion model until the data distribution related to the input fourth vector is restored. By forward diffusing and back-diffusion of the fourth vector, the characteristics of the target audio data and the characteristics of the specified video data in the fourth vector can be highlighted, and then the fourth vector after forward diffusion and back-diffusion is used as the third vector.

[0168] Step S3100: performing a decoding operation on the third vector through a decoder to obtain the target video data.

[0169] It should be noted that the third vector can represent the characteristic vector of the data stream, so that the data stream can be obtained through the third vector, and the target video data can be obtained through the data stream.

[0170] In some implementations, after obtaining the third vector, the third vector may be input to a decoder, and the decoder may decode the third vector to obtain the target video data. The target video data may be a data stream in a video coding format obtained by decoding a data stream obtained based on the third vector by an encoder, that is, the target video data may be a data stream in a video coding format.

[0171] Therefore, the obtained target video data includes the features of the designated video data and the features of the target audio data, and the video data with personalized audio is obtained.

[0172] In the embodiment of the present application, personalized information of the target user can be obtained, and the personalized information can also be input into the large language model as part of the prompt information, and the target video data can be obtained through artificial intelligence generated content (AIGC), so that the target video data generated subsequently has personalized characteristics. In addition, on the basis of retaining the original audio track of the specified video data, the effect of adding target audio data to the specified video data can be achieved, and the original audio track of the specified video data is integrated to make the sound of the target video data richer and more vivid.

[0173] Optionally, in combination with artificial intelligence generated content, video data with audio data can also be generated directly based on the fourth text information input by the target user and the third text information used to describe the personalized information of the target user.

[0174] See also Figure 4 , Figure 4 The method flow chart of the data generation method provided by the embodiment of the present application is shown. Figure 4 In the embodiment shown in FIG. , the data to be processed includes picture data, and the picture data may be at least one. Figure 4 The process includes steps S401 to S411.

[0175] Step S401: Obtain personalized information.

[0176] Step S402: Obtain target user input.

[0177] Step S403: Acquire image data.

[0178] First, personalized information, target user input and picture data can be obtained. For detailed introduction of personalized information, target user input and picture data, and methods for obtaining personalized information, target user input and picture data, please refer to the above embodiments, which will not be repeated here.

[0179] Step S404: whether the target user input is obtained.

[0180] Step S405: Obtain the seventh text information.

[0181] Step S406: Obtain fourth text information.

[0182] Step S407: Default video configuration information.

[0183] Step S408: Obtain designated video data.

[0184] In the above steps, the target user may not make any input, so it can be determined whether the target user input is obtained. It is understandable that the target user input may include the fourth text information and the seventh text information. Among them, the fourth text information may include the prompt information of the target user describing the characteristics of the data to be processed and the prompt information describing the audio usage preference; the seventh text information is used to describe the video configuration preference of the target user.

[0185] Therefore, when the target user input is not obtained, the specified video data can be obtained through the default video configuration information; and when the target user input is obtained, the specified video data can be obtained through the seventh text information.

[0186] Step S409: Obtain second text information through the image understanding model.

[0187] In addition, the second text information can also be obtained based on the acquired picture data through an image understanding model. In an embodiment of the present application, the second text information is information used to describe the characteristics of the picture data.

[0188] Step S410: input a large language model.

[0189] Step S411: Input to the video encoder.

[0190] Step S412: Input to the text encoder.

[0191] The personalized information and the second text information can be input into the large language model. In addition, when the target user input is obtained, the fourth text information can also be input into the large language model. Thus, the first text information output by the large language model can be input into the text encoder.

[0192] In addition, designated video data may also be input to the video encoder.

[0193] Step S413: Input to the diffusion model.

[0194] Step S414: Input to decoder.

[0195] Step S415: Obtain target video data.

[0196] Therefore, the second vector output by the video encoder and the first vector output by the text encoder are input into the diffusion model to obtain a third vector, and then the third vector is input into the decoder to obtain the target video data through the decoder.

[0197] In some implementations, the decoder, diffusion model, text encoder, and video encoder may constitute a video generation module.

[0198] For a detailed description of each of the above steps, please refer to the description of the above embodiments, which will not be repeated here.

[0199] See also Figure 5 , Figure 5 The method flow chart of the data generation method provided by the embodiment of the present application is shown. Figure 5 In the embodiment shown in FIG. , the data to be processed includes video data. Figure 5 The process includes steps S501 to S511.

[0200] Step S501: Obtain personalized information.

[0201] Step S502: Obtain target user input.

[0202] Step S503: Acquire video data.

[0203] First, personalized information, target user input and picture data can be obtained. For detailed introduction of personalized information, target user input and picture data, and methods for obtaining personalized information, target user input and picture data, please refer to the above embodiments, which will not be repeated here.

[0204] Step S504: Obtain fourth text information.

[0205] It should be noted that, similar to the aforementioned embodiment, the target user input may not be obtained here, so the fourth text information cannot be obtained. In this case, the fourth text information may not be input into the subsequent large language model.

[0206] Step S505: Obtain second text information.

[0207] The metadata of the video data may be extracted as the second text information.

[0208] Step S506: input a large language model.

[0209] Step S507: Input to the video encoder.

[0210] Step S508: Input to the text encoder.

[0211] The personalized information and the second text information can be input into the large language model. In addition, when the target user input is obtained, the fourth text information can also be input into the large language model. Thus, the first text information output by the large language model can be input into the text encoder.

[0212] In addition, the designated video data may be input into the video encoder. When the data to be processed is video data, the video data may be used as the designated video data.

[0213] Step S509: Input to the diffusion model.

[0214] Step S510: Input to decoder.

[0215] Step S511: Obtain target video data.

[0216] Therefore, the second vector output by the video encoder and the first vector output by the text encoder are input into the diffusion model to obtain a third vector, and then the third vector is input into the decoder to obtain the target video data through the decoder.

[0217] In some implementations, the decoder, diffusion model, text encoder, and video encoder may constitute a video generation module.

[0218] See also Figure 6, which shows a structural block diagram of a data generating device 600 provided in an embodiment of the present application, the device includes: a first acquisition unit 610, a second acquisition unit 620, a third acquisition unit 630, a target audio data generating unit 640 and a target video data generating unit 650.

[0219] The first acquisition unit 610 is used to acquire data to be processed, wherein the data to be processed includes at least one of picture data and video data.

[0220] The second acquisition unit 620 is configured to obtain first prompt information based on the characteristics of the data to be processed and the personalized information of the target user.

[0221] Optionally, the second acquisition unit 620 can also be used to acquire second text information for describing the characteristics of the data to be processed; acquire third text information for describing personalized information of the target user; and obtain the first prompt information based on the second text information and the third text information.

[0222] Optionally, the second acquisition unit 620 can also be used to obtain the first prompt information based on the second text information, the third text information and the fourth text information when fourth text information input by the target user is detected, wherein the fourth text information includes prompt information of the target user describing the characteristics of the data to be processed and prompt information describing the audio usage preference.

[0223] Optionally, the second acquisition unit 620 may also be configured to perform a content understanding operation on the image data through an image understanding model to obtain fifth text information for describing features of the image data as the second text information.

[0224] Optionally, the second acquisition unit 620 can also be used to perform content understanding operations on each of the image data through an image understanding model to obtain sixth text information corresponding to each of the image data for describing the image data; and arrange each of the sixth text information based on a specified order to obtain the second text information, wherein the specified order is the arrangement order of multiple image data.

[0225] Optionally, the second acquisition unit 620 may also be configured to extract metadata of the video data as the second text information.

[0226] Optionally, the second acquisition unit 620 can also be used to perform content understanding operations on the image data through an image understanding model to obtain fifth text information used to describe the characteristics of the image data; extract metadata of the video data; and use the fifth text information and the metadata as second text information.

[0227] The third acquisition unit 630 is used to perform semantic analysis on the first prompt information through a large language model to obtain first text information.

[0228] Optionally, the third acquisition unit 630 can also be used to perform content understanding operations on the image data through an image understanding model to obtain fifth text information used to describe the characteristics of the image data; extract metadata of the video data; and use the fifth text information and the metadata as second text information.

[0229] The target audio data generating unit 640 is configured to generate target audio data based on the first text information.

[0230] Optionally, the target audio data generating unit 640 may also be configured to perform an encoding operation on the first text information through a text encoder to obtain a first vector of the target audio as the target audio data.

[0231] The target video data generating unit 650 is configured to generate target video data based on the data to be processed and the target audio data.

[0232] Optionally, the target video data generating unit 650 can also be used to generate designated video data based on the data to be processed; encode the designated video data through a video encoder to obtain a second vector; construct a third vector based on the first vector and the second vector; and decode the third vector through a decoder to obtain the target video data. Optionally, the data to be processed includes video data, and the video data is used as the designated video data.

[0233] Optionally, the target video data generation unit 650 can also be used to input the first vector and the second vector into a diffusion model to perform feature fusion on the first vector and the second vector to obtain a fourth vector; and perform forward diffusion and backward diffusion on the fourth vector through the diffusion model to obtain an output third vector.

[0234] Optionally, the target video data generation unit 650 can also be used to generate specified video data based on the seventh text information and the picture data when the seventh text information input by the target user is detected, wherein the seventh text information is used to describe the video configuration preference of the target user; and to generate the specified video data based on default video configuration information and the picture data when the seventh text information input by the target user is not detected.

[0235] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0236] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0237] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0238] See also Figure 7 , which shows a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device 110 can be an electronic device such as a smart phone, a tablet computer, an e-book, etc. that can run applications. The electronic device 110 in the present application may include one or more of the following components: a processor 111, a memory 112, and one or more applications, wherein the processor 111 is connected to the memory 112. One or more applications may be stored in the memory 112 and configured to be executed by one or more processors 111, and one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0239] The processor 111 may include one or more processing cores. The processor 111 uses various interfaces and lines to connect various parts of the entire electronic device 110, and executes various functions and processes data of the electronic device 110 by running or executing instructions, programs, code sets or instruction sets stored in the memory 112, and calling data stored in the memory 112. Optionally, the processor 111 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 111 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 111, but may be implemented separately through a communication chip.

[0240] The memory 112 may include a random access memory (RAM) or a read-only memory (ROM). The memory 112 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 112 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the following various method embodiments, etc. The data storage area may also store data created by the electronic device 110 during use, etc.

[0241] Please refer to Figure 8 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 800 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0242] The computer readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer readable storage medium 800 has storage space for program code 810 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in an appropriate form.

[0243] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data generation method, characterized in that: include: Acquire data to be processed, wherein the data to be processed includes at least one of image data and video data; Obtaining first prompt information based on the characteristics of the data to be processed and the personalized information of the target user; Performing semantic analysis on the first prompt information by using a large language model to obtain first text information; Generate target audio data based on the first text information; Target video data is generated based on the data to be processed and the target audio data.

2. The method according to claim 1, characterized in that The obtaining of the first prompt information based on the characteristics of the data to be processed and the personalized information of the target user includes: Acquire second text information for describing characteristics of the data to be processed; Acquire third text information for describing personalized information of the target user; The first prompt information is obtained based on the second text information and the third text information.

3. The method according to claim 2, characterized in that The obtaining the first prompt information based on the second text information and the third text information includes: When fourth text information input by the target user is detected, the first prompt information is obtained based on the second text information, the third text information and the fourth text information, wherein the fourth text information includes prompt information of the target user describing characteristics of the data to be processed and prompt information describing audio usage preferences.

4. The method according to claim 2, characterized in that: The data to be processed includes picture data, and the obtaining of second text information for describing features of the data to be processed includes: A content understanding operation is performed on the image data through an image understanding model to obtain fifth text information for describing features of the image data as the second text information.

5. The method according to claim 4, characterized in that The picture data is multiple, and the picture understanding model is used to perform a content understanding operation on the picture data to obtain second text information for describing features of the picture data, including: Performing a content understanding operation on each of the image data by using the image understanding model to obtain sixth text information corresponding to each of the image data and used to describe the image data; Arrange each of the sixth text information based on a specified order to obtain the second text information, wherein the specified order is the arrangement order of the plurality of image data.

6. The method according to claim 2, characterized in that The data to be processed includes video data, and the obtaining of second text information for describing features of the data to be processed includes: The metadata of the video data is extracted as the second text information.

7. The method according to claim 2, characterized in that The data to be processed includes picture data and video data, and the obtaining of second text information for describing features of the data to be processed includes: Performing a content understanding operation on the image data through the image understanding model to obtain fifth text information for describing features of the image data; Extracting metadata of the video data; The fifth text information and the metadata are used as second text information.

8. The method according to claim 1, characterized in that The performing semantic analysis on the first prompt information by using a large language model to obtain first text information includes: Determine a sample task description and at least one sample output example corresponding to the data to be processed; Using the first prompt information, the sample task description, and at least one sample example as inputs to a large language model, so that the large language model learns the target relationship between the sample task description and the sample output example through a contextual learning algorithm; The first prompt information and the target relationship are inferred by the large language model to obtain first text information.

9. The method according to claim 1, characterized in that: The target audio data includes a first vector of a target audio, and the generating target video data based on the data to be processed and the target audio data includes: Generate specified video data based on the data to be processed; Performing an encoding operation on the designated video data by a video encoder to obtain a second vector; constructing a third vector based on the first vector and the second vector; The third vector is decoded by a decoder to obtain the target video data.

10. The method according to claim 9, characterized in that The constructing a third vector based on the first vector and the second vector includes: Inputting the first vector and the second vector into a diffusion model to perform feature fusion on the first vector and the second vector to obtain a fourth vector; The fourth vector is forward diffused and backward diffused by the diffusion model to obtain an output third vector.

11. The method according to claim 9, characterized in that The data to be processed includes picture data, and the step of generating designated video data based on the data to be processed includes: In the case of detecting seventh text information input by the target user, generating designated video data based on the seventh text information and the picture data, wherein the seventh text information is used to describe the video configuration preference of the target user; When the seventh text information input by the target user is not detected, the designated video data is generated based on the default video configuration information and the picture data.

12. The method according to claim 9, characterized in that The data to be processed includes video data, and the video data serves as the designated video data.

13. The method according to claim 1, characterized in that The generating target audio data based on the first text information includes: The first text information is encoded by a text encoder to obtain a first vector of the target audio as the target audio data.

14. A data generating device, characterized in that: include: A first acquisition unit, configured to acquire data to be processed, wherein the data to be processed includes at least one of image data and video data; A second acquisition unit, configured to obtain first prompt information based on the characteristics of the data to be processed and the personalized information of the target user; A third acquisition unit, configured to perform semantic analysis on the first prompt information through a large language model to obtain first text information; a target audio data generating unit, configured to generate target audio data based on the first text information; A target video data generating unit is used to generate target video data based on the data to be processed and the target audio data.

15. An electronic device, characterized in that: include: one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1-13.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, and the program code can be called by a processor to execute the method according to any one of claims 1 to 13.

Images