Data labeling and data generation method, device, equipment and storage medium

Abstract

The present disclosure provides a data labeling, data generation method, device, equipment and storage medium, data fragmentation processing is carried out for the first type of data, N data fragments corresponding to the first type of data are obtained, and the fragmentation characteristics of each data fragment in the N data fragments are extracted; from the data feature library corresponding to the second type of data, a data feature set whose similarity with the fragmentation characteristics of each data fragment in the N data fragments satisfies a preset condition is obtained, N data feature sets corresponding to the N data fragments are obtained; the data features in each data feature set in the N data feature sets are respectively counted, the occurrence times of the data features in the N data feature sets are counted, and the first type of data is labeled based on the second type of data corresponding to one or more data features with the most occurrence times.

Description

Technical Field

[0001] This disclosure relates to the field of machine learning technology, and in particular to data annotation, data generation methods, apparatus, devices and storage media. Background Technology

[0002] In the field of machine learning, data labeling is a crucial step, providing algorithms and models with the training data needed for supervised learning. Specifically, data labeling is the process of adding labels to data, which can be images, text files, videos, etc., and the labels represent specific attributes of the data.

[0003] Currently, data annotation still relies on manual work, requiring significant time and human resources. Furthermore, for data of a certain length, such as audio, video, or images, manual annotation typically only labels the entire data, resulting in low accuracy. Requiring fine-grained manual annotation is inefficient and costly. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides data annotation, data generation methods, apparatus, devices and storage media.

[0005] According to a first aspect of the embodiments of this specification, a data annotation method is provided, the method comprising: Data sharding is performed on the first type of data to obtain N data shards corresponding to the first type of data, and the sharding features of each data shard in the N data shards are extracted. From the data feature library corresponding to the second type of data, obtain a set of data features whose similarity to the fragmentation features of each of the N data fragments meets a preset condition, and obtain N sets of data features corresponding to the N data fragments; wherein, the data feature library includes data features extracted from several types of second type data; The number of times each data feature in each of the N data feature sets appears in the N data feature sets is counted. Based on the second type of data corresponding to the one or more data features with the most frequent occurrences, the first type of data is labeled.

[0006] According to a second aspect of the embodiments of this specification, a data generation method is provided, the method comprising: Obtain the second type of data used to generate the first type of data; The second type of data is input into a preset pre-trained model; wherein the training samples used to train the pre-trained model are the first type of data obtained by the data annotation method described in the first aspect; Obtain the first type of data generated by the pre-trained model based on the second type of input data.

[0007] According to a third aspect of the embodiments of this specification, a data annotation apparatus is provided, the apparatus comprising: The sharding module is used to: perform data sharding processing on the first type of data to obtain N data shards corresponding to the first type of data, and extract the sharding features of each data shard in the N data shards; The acquisition module is configured to: acquire, from the data feature library corresponding to the second type of data, a set of data features whose similarity to the segmentation features of each of the N data segments meets a preset condition, and obtain N data feature sets corresponding to the N data segments; wherein, the data feature library includes data features extracted from several types of second type of data; The annotation module is used to: count the number of times each data feature in each of the N data feature sets appears in the N data feature sets, and annotate the first type of data based on the second type of data corresponding to the one or more data features with the most frequent occurrences.

[0008] According to a fourth aspect of the embodiments of this specification, a data generation apparatus is provided, the apparatus comprising: The first acquisition module is used to: acquire the second type of data used to generate the first type of data; The input module is configured to: input the second type of data into a preset pre-trained model; wherein the training samples used to train the pre-trained model are the first type of data obtained by the data annotation method described in the first aspect; The second acquisition module is used to: acquire the first type of data generated by the pre-trained model based on the second type of input data.

[0009] According to a fifth aspect of the embodiments of this specification, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method embodiments described in the first or second aspect above.

[0010] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method embodiments described in the first or second aspect above.

[0011] The technical solutions provided in the embodiments of this specification may include the following beneficial effects: In this embodiment, the first type of data is segmented into multiple data segments, achieving refined processing of the first type of data. Segmentation features are extracted from each data segment, and using a data feature library corresponding to the second type of data, a set of data features similar to each segment feature is obtained by utilizing the similarity between the segmentation features and the data features. Finally, the frequency of occurrence of the statistical feature in the N data feature sets is analyzed; the higher the frequency, the more relevant the data feature is to the first type of data, and the higher the correlation between the second type of data corresponding to the data feature and the first type of data. Thus, this embodiment achieves refined automatic annotation of the first type of data through the similarity between segmentation features and data features, improving annotation efficiency and reducing annotation costs.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this specification and, together with the specification, serve to explain the principles of this disclosure.

[0014] Figure 1A This is a flowchart illustrating a data annotation method according to an exemplary embodiment of this specification.

[0015] Figure 1B This is a schematic diagram illustrating a data annotation according to an exemplary embodiment of this specification.

[0016] Figure 2 This is a flowchart illustrating a data generation method according to an exemplary embodiment of this specification.

[0017] Figure 3 This is a hardware structure diagram of a computer device containing a data generation apparatus, as illustrated in this specification according to an exemplary embodiment.

[0018] Figure 4 This is a block diagram illustrating a data annotation apparatus according to an exemplary embodiment of this specification.

[0019] Figure 5 This is a block diagram illustrating a data generation apparatus according to an exemplary embodiment of this specification. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0021] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0022] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0023] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0024] In the field of machine learning, deep learning models require a large amount of training data, which needs to be labeled to give the data a label. As mentioned earlier, data labeling itself requires costly manual work. For some data, the accuracy of manual labeling is insufficient. If fine-grained manual labeling is required, the cost and efficiency become even more unacceptable.

[0025] As an example, background music generation is one application scenario based on large-scale pre-trained models. Background music generation uses prompts / sentences as input and an AI music model to generate music that matches the meaning of the prompts. Training this AI music model requires massive amounts of data, consisting of a series of "music description text-music" matching pairs. The text acts as a label for the music, describing its type, setting, style, rhythm, and other information that matches the meaning of the music. However, the open-source training datasets in related technologies are mainly used for speech-to-text conversion; these datasets are not "music description text-music" matching pairs and therefore cannot be used to train AI music models.

[0026] Therefore, to train an intelligent music model, it is necessary to construct matching pairs of "music description text - music". However, due to the specialized nature of the music field, these matching pairs need to be annotated by professionals. Furthermore, since a piece of music has a certain length and typically includes multiple parts such as an introduction, verse, chorus, climax, or ending, different parts may express different emotions. When manually annotating, annotators can generally only annotate the music as a whole with classification information such as style or genre, but cannot provide segmented textual descriptions, resulting in poor annotation precision.

[0027] Similar issues exist in the video generation field, leading to problems with existing data annotation schemes such as poor precision and accuracy, low efficiency, and high cost. Therefore, this specification provides an efficient, low-cost data annotation scheme with high precision and accuracy. This embodiment performs data segmentation on the first type of data to obtain multiple data segments, achieving refined processing of the first type of data. Segment features are extracted from the data segments, and using a data feature library corresponding to the second type of data, the similarity between the segment features and data features is used to obtain a set of data features similar to each segment feature. Finally, the frequency of occurrence of the statistical feature in the N data feature sets is analyzed; the higher the frequency, the more relevant the data feature is to the first type of data, and the higher the correlation between the second type of data corresponding to the data feature and the first type of data. Thus, this embodiment achieves refined automatic annotation of the first type of data through the similarity between segment features and data features, improving annotation efficiency and reducing annotation costs. The following is a detailed description of this specification's embodiment.

[0028] like Figure 1A As shown, Figure 1A This is a flowchart illustrating a data annotation method according to an exemplary embodiment, which may include the following steps: Step 102: Perform data sharding on the first type of data to obtain N data shards corresponding to the first type of data, and extract the sharding features of each data shard in the N data shards.

[0029] Step 104: From the data feature library corresponding to the second type of data, obtain the set of data features whose similarity to the fragmentation features of each of the N data fragments meets the preset conditions, and obtain the N data feature sets corresponding to the N data fragments.

[0030] The data feature library includes data features extracted from several types of second-class data.

[0031] Step 106: Count the number of times each data feature in each of the N data feature sets appears in the N data feature sets, and label the first type of data based on the second type of data corresponding to the one or more data features that appear most frequently.

[0032] In this embodiment, the first type of data is the labeled data, and the second type of data serves as the label for the first type of data. The specific data can be flexibly configured as needed in practical applications, and this embodiment does not impose any limitations on this. As an example, the first type of data may include data of a certain length in a specific dimension, such as media files. Optionally, media files may include any of the following: audio, images, and videos. In practical applications, several sets of first-type data can be prepared in advance. The second type of data is used to describe the first type of data and may be text, etc. In practical applications, several sets of second-type data can be prepared in advance, and a data feature library corresponding to the second type of data can be constructed.

[0033] In some examples, the first type of data can be used as training samples to train a pre-trained model. Since the second type of data serves as the label for the first type of data, the pre-trained model can generate the first type of data based on the input second type of data. When the pre-trained model is trained using training samples, the labels for the first type of data can come from several second type of data points used to construct the feature library. When the trained pre-trained model is applied, the second type of data input to the model can be the same as or different from the several second type of data points used to construct the feature library. For example, new second type of data can be input into the pre-trained model, and the pre-trained model can generate the first type of data based on the input.

[0034] As an example, a pre-trained model could be a pre-trained model that generates media files based on the input text.

[0035] In step 102, the first type of data is processed by data segmentation. In practical applications, a suitable segmentation method can be selected according to specific needs and data characteristics. Furthermore, the specific value of N can also be configured as needed; this embodiment does not limit this. For example, segmentation methods may include segmentation by file size, that is, dividing a piece of data into N parts according to a set size. Or segmentation by fixed duration, dividing a piece of data into N parts according to a set duration, and so on. Thus, after segmentation, subsequent fine-grained annotation of the first type of data can be achieved. As an example, if the first type of data is an image, the image can be segmented into multiple parts; if the first type of data is audio or video, the audio or video can be segmented into multiple parts.

[0036] In this data sharding process, the first type of data is divided into N data shards. These data shards may or may not overlap. Overlapping shards refer to adjacent shards having some overlap. This method ensures data continuity and integrity, and may provide better contextual information in some cases. For example, in time series data analysis, if adjacent time windows overlap, dynamic changes in the data can be captured more effectively. Non-overlapping shards refer to adjacent shards having no overlap; each shard is independent. In practical applications, configuration can be tailored to specific circumstances, and this embodiment does not impose any limitations on this.

[0037] In some examples, the first type of data may be audio data, and the data segmentation process for the first type of data may include: dividing the audio data into multiple segments of the preset duration according to a preset duration.

[0038] In this embodiment, the preset duration of each segment can be determined, and can be selected based on specific needs and application scenarios. For example, the preset duration can be a fixed duration such as 10 seconds or 30 seconds; it can also be determined in conjunction with the duration of the audio data, with different preset durations for audio data of different durations. Based on the preset duration, the audio data is divided into multiple segments of preset duration, which can be achieved by cutting segments of fixed length. During the segmentation process, it can be determined whether there are overlapping parts between segments as needed. Through the above embodiment, data segmentation processing can be performed quickly, with high processing efficiency.

[0039] In other examples, different segmentation methods can be used. For instance, the durations of multiple segments of audio data can also differ. For example, if the audio data is specifically a music file, and the music includes multiple parts such as intro, verse, chorus, climax, or ending, these parts can be identified as multiple segments, each with a different duration. The identification method can be configured as needed. For example, multiple sample music files can be used to statistically analyze the total duration of the music files, and then the durations of the aforementioned parts within that total duration can be statistically analyzed to obtain the durations of the aforementioned parts for different total durations of music files. Thus, when segmenting the current music file is required, the total duration of the current music file is obtained, and based on the statistical analysis results, the durations of the aforementioned parts corresponding to the total duration of the current music file are found. Then, the current music file is divided into segments corresponding to each of these parts.

[0040] After data sharding, the first type of data is divided into N data shards. The sharding features of each data shard can be extracted, resulting in N shards and N shards with N shards. The extraction of sharding features can be achieved using existing techniques, such as using a feature extraction model to extract the sharding features of each shard separately.

[0041] In step 104, the second type of data corresponds to a data feature library. The construction of the data feature library can be achieved by pre-preparing several pieces of second-type data and extracting data features for each piece of second-type data. The extraction of data features can be implemented using existing technologies, such as using a feature extraction model to extract data features.

[0042] The similarity between the piecewise features and the data features can be represented by the vector distance between the feature vector corresponding to the piecewise features and the feature vector corresponding to the data features. Specifically, the vector distance can be cosine similarity or Euclidean distance, etc., and can be configured as needed in practical applications.

[0043] Optionally, for ease of computation, the feature vectors corresponding to the piecewise features and the feature vectors corresponding to the data features can have the same dimension. In implementation, different feature extraction models can be used for the extraction of piecewise features and data features. This may result in differences in the dimensions of the two types of features. This can be addressed by setting two models to extract feature vectors of the same dimension, or by performing a dimensionality reduction operation on the higher-dimensional feature vector to match it with the lower-dimensional feature vector.

[0044] Optionally, the same feature extraction method can be consistently selected, which allows for a more accurate measurement of the similarity between sharded features and data features. For example, sharded features are extracted from data shards using a feature extraction model, and data features are also extracted from the second type of data using the same feature extraction model. Thus, the feature extraction model in this embodiment is used to extract both sharded features from the first type of data and data features from the second type of data; using the same feature extraction model to extract features from both types of data allows the features of both types of data to be represented by feature vectors of the same dimension, facilitating subsequent comparison of the similarity between the features of the two types of data.

[0045] As an example, a feature extraction model can include two encoders: one encoder converts slices of data of the first class into fixed-length feature vectors, and the other encoder converts data of the second class into fixed-length feature vectors. This allows for accurate comparison of the similarity between sliced ​​features and data features.

[0046] Optionally, the feature extraction model can select an existing pre-trained model according to actual needs and conduct supervised training as required. As an example, the feature extraction model can be obtained through supervised training based on several labeled first-class data samples; the labels are one or more second-class data samples. In this embodiment, the first-class data samples can be pre-labeled, which can be done manually. Thus, the first-class data samples are paired with one or more second-class data samples, and each first-class data sample has a second-class data sample that can accurately describe it. Based on this, the feature extraction model can learn the correlation between the features of the first-class data and the features of the second-class data within a feature space.

[0047] In step 102, from the data feature library corresponding to the second type of data, a set of data features whose similarity to the fragmentation features of each of the N data fragments meets the preset conditions can be obtained, thus obtaining N data feature sets corresponding to the N data fragments. There are multiple ways to implement this.

[0048] For example, such as Figure 1B As shown, the first type of data is divided into N data fragments, including data fragment 1, data fragment 2, ..., data fragment N, and each of the above data fragments corresponds to fragment feature 1, fragment feature 2, ..., fragment feature N, respectively.

[0049] Each of the aforementioned segmented features has a corresponding set of data features; for example, segmented feature 1 corresponds to set 1, segmented feature 2 corresponds to set 2, ..., segmented feature N corresponds to set N. Each data feature set includes one or more data features. For any segmented feature i from segmented feature 1 to segmented feature N, the similarity between each data feature included in the data feature set i corresponding to segmented feature i and segmented feature i satisfies a preset condition. This preset condition can be a similarity threshold, etc., and can be flexibly configured as needed in practical applications; this embodiment does not limit this.

[0050] The data feature set i corresponding to the segment feature i can be obtained from the data feature library corresponding to the second type of data. As an example, the data feature library has M data features. The similarity between the segment feature i and each of the M data features is calculated, and the data feature set i that meets the preset conditions is obtained based on this. For each segment feature, M similarity values ​​need to be calculated, and the total computation is M*N.

[0051] Considering that the value of M in the data feature library may be large, in order to reduce the computational load, in some examples, obtaining the data feature set from the data feature library corresponding to the second type of data whose similarity to the fragment features of each of the N data fragments meets a preset condition may include: Calculate the average feature of the fragmentation features of each data fragment in the N data fragments; From the data feature library corresponding to the second type of data, obtain a set of candidate data features whose similarity to the average feature satisfies the first preset condition; Each candidate data feature set is selected from the candidate data feature set, and the similarity between the data feature set and the fragmentation feature set of each of the N data fragments satisfies the second preset condition.

[0052] In this embodiment, average features are used to reduce computational load. The average feature can be calculated by adding all the piecewise features together, then dividing the sum by the number of piecewise features.

[0053] Then, using the average feature, a set of candidate data features that meet the first preset condition of similarity with the average feature is obtained from the data feature library. Assuming that the number of candidate data feature sets is m, in general, the number of data features m that are similar to the average feature will be less than the total number of features M in the data feature library.

[0054] Next, the data feature set i corresponding to the segment feature i is obtained from the candidate data feature set. The candidate data feature set contains m data features. The similarity between the segment feature i and each of the m data features is calculated, and based on this, the data feature set i that satisfies the second preset condition is obtained. For each segment feature, m similarity values ​​need to be calculated, and the total computational cost is m*N. Since m is less than M, the computational cost can be reduced.

[0055] The first and second preset conditions can be the same or different, and this embodiment does not limit this. For example, the similarity threshold representing the first preset condition can be lower than the similarity threshold representing the second preset condition. In this way, some data features can be selected from the data feature library as a candidate data feature set by means of average features, and then the data feature set corresponding to each segment feature can be accurately selected by means of the second preset condition with a higher threshold.

[0056] For step 106, as follows Figure 1B As shown, in data feature set 1 to data feature set N, the data features included in each set may be the same or different. For example: Data feature set 1 includes: data feature 1, data feature 2, data feature 4, data feature 5, ..., data feature 58; Data feature set 2 includes: data feature 2, data feature 3, data feature 4, data feature 5, ..., data feature 99; ...

[0057] It can be seen that some data features appear in multiple sets of data features, while others appear in only one set. For example, if a data feature appears 10 times, it means that this data feature is related to the 10 data segments in the first category of data. Therefore, the more times it appears, the more related the data feature is to the first category of data, and the higher the correlation between the second category of data and the first category of data corresponding to the data feature. That is, the frequency of occurrence and the correlation between the second category of data and the first category of data corresponding to the data feature are positively correlated.

[0058] Therefore, by counting the number of times each data feature in each of the N data feature sets appears in the N data feature sets, the second type of data corresponding to the one or more data features that appear most frequently can be used as the label for the first type of data.

[0059] After counting the occurrences, the method for determining the one or more data features that appear most frequently can be configured as needed. For example, a frequency threshold can be set, and data features with an occurrence frequency greater than or equal to the threshold can be selected. Alternatively, the data can be sorted by frequency of occurrence, and the k most frequent data features can be selected, where k can be configured according to the actual application.

[0060] like Figure 1B As shown, the data features that appeared most frequently in the final selection were: data feature 3, data feature 4, data feature 5, data feature 58 and data feature 76. These five data features correspond to the second type of data, that is, the labels of the first type of data.

[0061] Based on this, the above embodiments can achieve automatic labeling of the first type of data. The labeled first type of data can be used as training samples to train a pre-trained model, which in turn generates the first type of data based on the input second type of data. For example, the second type of data used to generate the first type of data can be obtained and input as a prompt to the pre-trained model, which then generates the first type of data based on the prompt.

[0062] As an example, the first type of data can be audio, and the second type of data can be text. The audio is labeled using the above embodiment, with the label being the text describing the audio. The labeled audio is used as training samples to train a pre-trained model that generates audio based on the input text. Once the model is trained, audio can be automatically generated as follows: obtain the text used to generate the audio, input the obtained text into the model, and the model will generate audio based on the input text. The pre-trained model can be Google's MusicLM model or Noise2Music model, etc. In practical applications, an existing pre-trained model can be selected as needed, or a pre-built model can be constructed.

[0063] In other examples, the first type of data can be video, and the second type of data can be text. Using the above embodiments, videos are labeled with text describing them. These labeled videos serve as training samples to train a pre-trained model that generates videos based on the input text. Once the model is trained, videos can be automatically generated as follows: the text used to generate the video is obtained, and this obtained text is input into the model, which then generates the video based on the input text.

[0064] In other examples, the first-class data obtained using the annotations in this embodiment can also have other applications. For instance, if there is a third-class data whose label is also a second-class data, then the first-class and third-class data can be paired based on the second-class data. For example, given an annotated first-class database and an annotated third-class database, where the first-class data has one or more labels and the third-class data has one or more labels, the similarity between the labels of the first-class data and the labels of the third-class data can be calculated. If the similarity meets a preset condition, then the first-class data and the third-class data can be paired. The paired first-class and third-class data can be used as training samples for training a pre-trained model. Specifically, the first-class data can be used as the label for the third-class data, allowing the model to generate third-class data based on the first-class data; alternatively, the third-class data can be used as the label for the first-class data, allowing the model to generate first-class data based on the third-class data.

[0065] As an example, the scheme of this embodiment can be used to annotate text for audio data, and the same scheme can be used to annotate text for video data. In this way, audio and video data can be paired using text. For example, each audio data entry in the audio database has a tag text; each video data entry in the video database also has a tag text, and both use the same text library. To establish a pairing between audio data in the audio database and video data in the video database, for a video data entry, the text features of its tag text are compared with the text features of the tag texts of each audio data entry in the audio database to calculate the similarity. Since a video data entry may have multiple tag texts, the average feature of the text features of multiple tag texts can be calculated. Similarly, the average feature of the tag texts of the audio data is also calculated, and then the similarity between the two average features is calculated. Finally, one or more audio data entries that meet preset conditions are paired with the video data.

[0066] The following example uses a music generation scenario. In this embodiment, a text library and a music library can be prepared in advance. The text library includes descriptive text for multiple pieces of music, and the music library includes multiple music files; the data in the two databases are not yet linked.

[0067] The purpose of this embodiment is to pair data between two databases; that is, for each piece of music in the music database, select one or more text prompts from the text database that match the meaning of the music. The data annotation process may include: (1) Using a feature extraction model (Feature Model, FM, which can be used to extract both music features and text features). Optionally, text features of each text in the Text Corpus can be extracted in advance, and a Text Feature Corpus (TFC) is established.

[0068] For example, the text corpus includes j texts, text features are extracted for each text to obtain the text feature corpus, which includes j text features TF.

[0069] (2) Selecting a piece of music from the music library as the target music.

[0070] (3) Slicing a target music into a plurality of Music Clips (MC) according to Tc length.

[0071] Optionally, the Tc length can be set; there may be various slicing methods, for example, the target music is evenly sliced to obtain n MCs.

[0072] (4) Using the feature extraction model FM to extract the Clip Feature (CF) of each music clip to obtain CFs of n music clips MC.

[0073] (5) Calculating an Average Feature (AF) by using all CFs, which is the average feature of the target piece of music.

[0074] (6) Using the average feature AF to screen a candidate data feature set whose similarity with AF meets a first preset condition from TFC; the candidate data feature set includes K features TF, and each feature TF has high similarity with the average feature AF.

[0075] (7) Calculating the similarity between each CF of the target piece of music and each of the K TFs, selecting K' (K'<K) TFs with the highest similarity to obtain a data feature set for each CF, each data feature set includes K' data features; counting the occurrence times of each data feature in each data feature set.

[0076] (8) Selecting k texts with the largest occurrence times as the texts that best match the target music, and using them as the target annotation result.

[0077] Wherein, k can be configured as required, for example, it can be equal to K', which is not limited in the present embodiment.

[0078] (9) Returning to perform steps (2) to (8) until every piece of music in the music library is assigned a tag.

[0079] Through the data annotation in the above embodiments, each piece of music in the music library is tagged, which can then be used to train an intelligent music model (AI Music Model). After training, a prompt word is input into the AI ​​Music Model, and the model can generate music based on this prompt word.

[0080] like Figure 2 The diagram shown is a flowchart illustrating a data generation method according to an exemplary embodiment of this specification. The method may include the following steps: Step 202: Obtain the second type of data used to generate the first type of data.

[0081] Step 204: Input the second type of data into the preset pre-trained model.

[0082] The training samples used to train the pre-trained model are the first type of data obtained through the aforementioned data annotation method embodiment.

[0083] Step 206: Obtain the first type of data generated by the pre-trained model based on the second type of input data.

[0084] The specific implementation of this embodiment can be referred to the description of the foregoing embodiments, and will not be repeated here.

[0085] Corresponding to the aforementioned embodiments of the data annotation method / data generation method, this specification also provides embodiments of the data annotation apparatus / data generation apparatus and the computer equipment used therein.

[0086] The embodiments of the data annotation device / data generation device described in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by its processor reading the corresponding computer program instructions from non-volatile memory into memory and executing them. From a hardware perspective, such as... Figure 3 The diagram shown is a hardware structure diagram of a computer device containing the data annotation device / data generation device described in this specification. (Except for...) Figure 3 In addition to the processor 310, memory 330, network interface 320, and non-volatile memory 340 shown, the computer device in which the data annotation device / data generation device 331 is located in the embodiment may also include other hardware depending on the actual function of the computer device, which will not be described in detail here.

[0087] like Figure 4 As shown, Figure 4 This is a block diagram illustrating a data annotation apparatus according to an exemplary embodiment of this specification, the apparatus including: The sharding module 41 performs data sharding processing on the first type of data to obtain N data shards corresponding to the first type of data, and extracts the sharding features of each data shard in the N data shards. The acquisition module 42 acquires a set of data features from the data feature library corresponding to the second type of data, whose similarity to the fragmentation features of each of the N data fragments meets a preset condition, to obtain N data feature sets corresponding to the N data fragments; wherein, the data feature library includes data features extracted from several types of second type data; The annotation module 43 counts the frequency of occurrence of each data feature in each of the N data feature sets, and annotates the first type of data based on the second type of data corresponding to the most frequently occurring data feature. The first type of data is used as training samples to train a pre-trained model that generates the first type of data based on the input second type of data.

[0088] In some examples, the first type of data is used as training samples to train a pre-trained model; the pre-trained model is used to generate the first type of data based on the input second type of data.

[0089] In some examples, the acquisition module 42 acquires a set of data features from the data feature library corresponding to the second type of data, which shows a similarity to the fragment features of each of the N data fragments that meets a preset condition. This set may include: Calculate the average feature of the fragmentation features of each data fragment in the N data fragments; From the data feature library corresponding to the second type of data, obtain a set of candidate data features whose similarity to the average feature satisfies the first preset condition; Each candidate data feature set is selected from the candidate data feature set, and the similarity between the data feature set and the fragmentation feature set of each of the N data fragments satisfies the second preset condition.

[0090] In some examples, the similarity is represented by the vector distance between the feature vector corresponding to the segment feature and the feature vector corresponding to the data feature.

[0091] In some examples, the first type of data is audio data, and the segmentation module 41 performs data segmentation processing on the first type of data, including: The audio data is divided into multiple data segments of preset duration.

[0092] In some examples, the sharding features are extracted from the data shards using a feature extraction model, and the data features are extracted from the second type of data using the same feature extraction model.

[0093] In some examples, the feature extraction model is obtained through supervised training based on several first-class data samples carrying labels; the labels are one or more second-class data samples.

[0094] In some examples, the first type of data is media files, and the second type of data is text; the pre-trained model is a pre-trained model that generates media files based on the input text.

[0095] In some examples, the media file includes any of the following: audio, images, and video.

[0096] like Figure 5 As shown, Figure 5 This is a block diagram illustrating another data generation apparatus according to an exemplary embodiment of this specification, the apparatus comprising: The first acquisition module 51 acquires the second type of data used to generate the first type of data; Input module 52 inputs the second type of data into a preset pre-trained model; wherein, the training samples used to train the pre-trained model are the first type of data obtained through the aforementioned data annotation method embodiment; The second acquisition module 53 acquires the first type of data generated by the pre-trained model based on the second type of input data.

[0097] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0098] Accordingly, embodiments of this specification also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned data annotation method / data generation method embodiments.

[0099] Accordingly, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the data annotation method / data generation method embodiment.

[0100] Accordingly, embodiments of this specification also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the data annotation method / data generation method embodiments.

[0101] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0102] The above embodiments can be applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. The hardware of the computer device includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0103] The computer device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0104] The computer equipment may also include network equipment and / or user equipment. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0105] The network in which the computer device is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0106] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0107] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0108] The terms "specific example" or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with the embodiments or examples, which are included in at least one embodiment or example of this specification. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0109] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this specification are indicated by the following claims.

[0110] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.

[0111] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A data annotation method, the method comprising: Data sharding is performed on the first type of data to obtain N data shards corresponding to the first type of data, and the sharding features of each data shard in the N data shards are extracted. Calculate the average feature of the fragmentation features of each data fragment in the N data fragments; From the data feature library corresponding to the second type of data, obtain a set of candidate data features whose similarity to the average feature satisfies the first preset condition; Each candidate data feature set is obtained from the candidate data feature set, and the similarity of the fragmentation features of each of the N data fragments with the fragmentation features of each of the N data fragments satisfies a second preset condition, thereby obtaining N data feature sets corresponding to the N data fragments; wherein, the data feature library includes data features extracted from several second-class data; The number of times each data feature in each of the N data feature sets appears in the N data feature sets is counted, and the first type of data is labeled based on the second type of data corresponding to the one or more data features with the most frequent occurrences. The first type of data is media files, which include any one of audio, images, and videos; the second type of data is text.

2. The method according to claim 1, wherein the first type of data is used as training samples to train a pre-trained model; the pre-trained model is used to generate the first type of data based on the input second type of data.

3. The method according to claim 1, wherein the similarity is characterized by the vector distance between the feature vector corresponding to the segment feature and the feature vector corresponding to the data feature.

4. The method according to claim 1, wherein the first type of data is audio data, and the data segmentation process for the first type of data includes: The audio data is divided into multiple data segments of preset duration.

5. The method according to claim 1, wherein the segmentation features are extracted from the data segments by a feature extraction model, and the data features are extracted from the second type of data by the feature extraction model.

6. The method according to claim 5, wherein the feature extraction model is obtained by supervised training based on a plurality of first-class data samples carrying labels; the label is one or more second-class data samples.

7. The method according to claim 2, wherein the pre-trained model is used to generate media files based on the input text.

8. A data generation method, the method comprising: Obtain the second type of data used to generate the first type of data; The second type of data is input into a preset pre-trained model; wherein the training samples used to train the pre-trained model are the first type of data obtained by any of the data annotation methods described in claims 1 to 7; Obtain the first type of data generated by the pre-trained model based on the input second type of data; The first type of data is media files, which include any one of audio, images, and videos; the second type of data is text.

9. A data annotation device, the device comprising: The sharding module performs data sharding processing on the first type of data to obtain N data shards corresponding to the first type of data, and extracts the sharding features of each data shard in the N data shards; The acquisition module calculates the average feature of the fragmentation features of each data fragment in the N data fragments; From the data feature library corresponding to the second type of data, a set of candidate data features whose similarity to the average feature satisfies a first preset condition is obtained; from the candidate data feature sets, a set of data features whose similarity to the segmentation features of each of the N data segments satisfies a second preset condition is obtained, thus obtaining N data feature sets corresponding to the N data segments; wherein, the data feature library includes data features extracted from several types of second type data; The annotation module counts the number of times each data feature in each of the N data feature sets appears in the N data feature sets, and annotates the first type of data based on the second type of data corresponding to the one or more data features with the most frequent occurrences. The first type of data is media files, which include any one of audio, images, and videos; the second type of data is text.

10. A data generation apparatus, the apparatus comprising: The first acquisition module acquires the second type of data used to generate the first type of data; The input module inputs the second type of data into a preset pre-trained model; wherein the training samples used to train the pre-trained model are the first type of data obtained by any of the data annotation methods described in claims 1 to 7; The second acquisition module acquires the first type of data generated by the pre-trained model based on the input second type of data; The first type of data is media files, which include any one of audio, images, and videos; the second type of data is text.

11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any one of claims 1 to 8.

Images