Application processing method for content characterization, computer equipment, readable storage medium and program product

Abstract

The invention relates to a content representation application processing method, computer equipment, a computer readable storage medium and a computer program product, and relates to the technical field of artificial intelligence. According to the invention, the application processing efficiency of content characterization can be improved. The method comprises the steps that multi-layer content characterization obtained by a content characterization large model according to content data is obtained, the multi-layer content characterization is input into a trained characterization compression model suitable for a target task, and the characterization compression model is used for compressing the number of layers of the multi-layer content characterization; target content representation obtained by compressing the multiple layers of content representation through the trained representation compression model is obtained, the layer number of the target content representation meets the layer number condition of the target task, and the target content representation is applied to the target task.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an application processing method for content representation, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of artificial intelligence technology, big model technology has rapidly emerged and developed, allowing users to ask questions to big models (big language models), and the big models can quickly return high-quality answers, eliminating the need to dig out valid information from search results one by one.

[0003] The smooth communication between the big model and users is due to the big model's ability to understand data. Specifically, the content representation big model can efficiently and accurately represent, learn, and understand various forms of content (such as text, images, audio, etc.).

[0004] The architecture of the content representation model can include Transformers (a general term referring to various models or related technologies derived from the Transformer architecture). The content representation output by the content representation model can convert content data (such as text, images, audio, etc.) into a vector space representation that can be understood by computers. This content representation can be applied to various downstream tasks (content classification, question answering, translation, etc.).

[0005] However, the content representation output by the large content representation model usually includes multiple layers of content representation. According to current technology, it is necessary to try these multiple layers of content representation layer by layer to obtain the target content representation for downstream tasks. There is a technical problem of low application processing efficiency of content representation. Summary of the Invention

[0006] Based on this, it is necessary to provide an application processing method for content representation, a computer device, a computer-readable storage medium, and a computer program product to address the above technical issues.

[0007] In a first aspect, the present application provides a method for processing an application of content representation, comprising:

[0008] Obtaining a multi-layer content representation obtained by a content representation model based on content data;

[0009] Inputting the multi-layer content representation into a trained representation compression model suitable for a target task; the representation compression model is used to compress the number of layers of the multi-layer content representation;

[0010] Obtaining a target content representation obtained by compressing the multi-layer content representation with the trained representation compression model; the number of layers of the target content representation meets the layer number condition of the target task;

[0011] The target content representation is applied to the target task.

[0012] In a second aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0013] Obtain a multi-layer content representation obtained by a content representation large model based on content data; input the multi-layer content representation into a trained representation compression model suitable for a target task; the representation compression model is used to compress the number of layers of the multi-layer content representation; obtain a target content representation obtained by compressing the multi-layer content representation by the trained representation compression model; the number of layers of the target content representation meets the layer number condition of the target task; and apply the target content representation to the target task.

[0014] In a third aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0015] Obtain a multi-layer content representation obtained by a content representation large model based on content data; input the multi-layer content representation into a trained representation compression model suitable for a target task; the representation compression model is used to compress the number of layers of the multi-layer content representation; obtain a target content representation obtained by compressing the multi-layer content representation by the trained representation compression model; the number of layers of the target content representation meets the layer number condition of the target task; and apply the target content representation to the target task.

[0016] In a fourth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0017] Obtain a multi-layer content representation obtained by a content representation large model based on content data; input the multi-layer content representation into a trained representation compression model suitable for a target task; the representation compression model is used to compress the number of layers of the multi-layer content representation; obtain a target content representation obtained by compressing the multi-layer content representation by the trained representation compression model; the number of layers of the target content representation meets the layer number condition of the target task; and apply the target content representation to the target task.

[0018] The aforementioned content representation application processing method, computer device, computer-readable storage medium, and computer program product obtain a multi-layer content representation obtained by a large content representation model based on content data, input the multi-layer content representation into a trained representation compression model suitable for the target task, and compress the multi-layer content representation. The representation compression model is used to compress the number of layers of the multi-layer content representation, obtain a target content representation obtained by compressing the multi-layer content representation using the trained representation compression model, and the number of layers of the target content representation meets the layer number conditions of the target task. The target content representation is then applied to the target task. This solution can compress the multi-layer content representation obtained by the large content representation model based on content data into a target content representation using a trained representation compression model suitable for the target task, thereby enabling the target content representation to be applied to the target task without the need to screen the multi-layer content representation layer by layer, thereby improving the efficiency of content representation application processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 An application environment diagram of a method for processing content representation applications in one embodiment;

[0021] Figure 2 A diagram of an application processing framework for content representation in related technologies;

[0022] Figure 3 A flowchart of a method for processing content representation applications in one embodiment is shown;

[0023] Figure 4 1 is a schematic diagram of a process for training a representation compression model in one embodiment;

[0024] Figure 5 A schematic diagram of a process for training a representation compression model in another embodiment;

[0025] Figure 6 A schematic diagram of a process for constructing a model loss in one embodiment;

[0026] Figure 7 A diagram of an application processing framework for content representation in one embodiment;

[0027] Figure 8 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0029] The content representation application processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the application environment may include a terminal and a server, and the terminal and the server may communicate through a network. The application environment may also include a data storage system, which may store data that the server needs to process. The data storage system may be integrated on the server, or placed on a cloud or other network server. The application processing method represented by the content of this application may be applied to Figure 1 The terminals shown can also be applied to Figure 1 The server shown can also be applied to a system including a terminal and a server.

[0030] Terminals include, but are not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices include smart speakers, smart TVs, smart air conditioners, smart car devices, and projectors. Portable wearable devices include smart watches, smart bracelets, and head-mounted devices. Head-mounted devices can include virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Servers can be standalone physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services.

[0031] There are many large content representation models, such as the EncodecMAE model, the MERT model, and the MusicFM model. These models have their own strengths in different aspects, but usually adopt the Transformers architecture. Their output content representation usually includes multi-layer content representation. The multi-layer content representation can be a matrix composed of multi-layer vectors, and each layer of vectors is a layer of content representation. When used in specific downstream tasks, according to current technology, the multi-layer content representation must be tried layer by layer to obtain the target content representation for downstream tasks. There is a technical problem of low application processing efficiency of content representation.

[0032] like Figure 2As shown, the large content representation model can be an Encoder MAE model. A trained Encoder MAE model can output a multi-layer content representation of Transformers based on the input content data. Assume that the Encoder MAE model has 20 layers of content representation, each with 1024 dimensions. For tasks with some prior experience, some layers (such as the first five) can be extracted and tested on downstream tasks, gradually identifying the target content representation. For tasks with less prior experience, all 20 layers of content representation need to be tested, and the i-th layer (i ranges from 1 to 20, representing the number of layers of the content representation output by the Encoder MAE model) is extracted and fed into the downstream task's task processing model (or task processing network, which can be a neural network, etc.). This task processing model can then be trained and tested on the task processing model's test set to determine the performance of the i-th layer on the task. As can be seen, as the complexity of downstream tasks increases, the amount of training data becomes increasingly large. This layer-by-layer traversal approach becomes extremely time-consuming and impractical, and suffers from the technical issue of low efficiency in the application and processing of content representations.

[0033] In this regard, the application processing method of content representation provided in the embodiment of the present application can compress the multi-layer content representation obtained by the content representation large model based on the content data into a target content representation through a trained representation compression model suitable for the target task, so that the target content representation can be applied to the target task without the need to screen the multi-layer content representation layer by layer, thereby improving the application processing efficiency of the content representation.

[0034] The application processing method of the content representation of this application is described below in conjunction with various embodiments and drawings.

[0035] In an exemplary embodiment, Figure 3 As shown, a content representation application processing method is provided, which can be performed as follows Figure 1 The terminal or server shown in the figure may execute the method, which may include the following steps:

[0036] Step S301: obtaining a multi-layer content representation obtained by a content representation macro model based on content data.

[0037] The large content representation model is a deep learning-based AI model that can efficiently and accurately learn representations of various types of content (such as text, images, and audio) to capture the inherent semantics and structure of the content. Trained on massive amounts of multi-source content data, the large content representation model possesses powerful feature extraction capabilities. It typically utilizes a Transformer architecture to transform content data into multi-layered representations.

[0038] The content data can be used as input data for the content representation model, and its types may include text, images, audio, etc. Specifically, the content data may include text data such as articles, paragraphs, and sentences, image data such as images and videos, and audio data such as audio files.

[0039] Content representation is the result of processing content data using a large content representation model. It converts content data into a form that computers can understand and process, typically as a vector representation. This vector representation contains information about the semantics and structure of the content data and can be used for various downstream tasks, including but not limited to text classification, image recognition, and information retrieval.

[0040] Among them, the content representation output by the content representation large model based on the content data usually includes multiple layers of content representation, and the dimensions of each layer of content representation are the same. For example, the EncodecMAE model can output 20 layers of content representation, and the content dimension of each layer is 1024 dimensions.

[0041] In this step, the content data may be input into a trained content representation model, and the trained content representation model may obtain and output multi-layer content representations based on the content data, so that the terminal or server may obtain multi-layer content representations.

[0042] Step S302: Input the multi-layer content representation into a trained representation compression model suitable for the target task.

[0043] Among them, the representation compression model can be used to compress the number of layers of the multi-layer content representation output by the content representation large model. The representation compression model is a trained representation compression model suitable for the target task. The representation compression model can be trained to obtain a target content representation suitable for the target task based on the multi-layer content representation compression, thereby facilitating subsequent application in the target task. For example, it can be used as input data for downstream target task classifiers, regressors, and other models, thereby improving the processing efficiency and performance of the target task.

[0044] Among them, the target task can be a task that needs to be completed downstream of the content representation model, such as image recognition tasks, sentiment classification tasks, speech-to-text tasks, audio classification tasks, etc.

[0045] In this step, after obtaining the multi-layer content representation, the terminal or server may input the multi-layer content representation into a trained representation compression model suitable for the target task.

[0046] Step S303: Obtain a target content representation obtained by compressing the multiple content representations using the trained representation compression model.

[0047] The trained representation compression model can compress the input multi-layer content representation to obtain a target content representation, where the number of layers of the target content representation meets the layer condition of the target task. The layer condition of the target task can indicate the required number of layers of the target content representation. As an example, the layer condition of the target task can indicate that the required number of layers of the target content representation is 1 layer, that is, the trained representation compression model can compress, for example, 20 layers of content representation into 1 layer of target content representation. The dimension of the target content representation can be the same as the dimension of each layer of the content representation in the 20 layers, such as 1024 dimensions.

[0048] In this step, the trained representation compression model compresses the multi-layer content representation to obtain and output a target content representation whose number of layers meets the layer number conditions of the target task, and the terminal or server obtains the target content representation output by the trained representation compression model.

[0049] Step S304: Apply the target content representation to the target task.

[0050] In this step, after obtaining the target content representation, the terminal or server can apply the target content representation to the target task. This can provide an accurate data foundation for the execution of the target task through the efficiently obtained target content representation, so as to facilitate the execution of the target task. For example, the target content representation of text data can be used to accurately classify text, the target content representation of image data can be used to accurately identify objects in images, and the target content representation of audio data can be used to classify audio genres.

[0051] The content representation application processing method of this embodiment obtains a multi-layer content representation obtained by a large content representation model based on content data, inputs the multi-layer content representation into a trained representation compression model suitable for the target task, and compresses the number of layers of the multi-layer content representation. The trained representation compression model is used to compress the number of layers of the multi-layer content representation, obtains a target content representation obtained by compressing the multi-layer content representation using the trained representation compression model, and the number of layers of the target content representation meets the layer number requirement of the target task, and then applies the target content representation to the target task. This solution can compress the multi-layer content representation obtained by the large content representation model based on content data into a target content representation using a trained representation compression model suitable for the target task, thereby allowing the target content representation to be applied to the target task without having to screen the multi-layer content representation layer by layer, thereby improving the efficiency of content representation application processing.

[0052] In an exemplary embodiment, Figure 4 As shown, before inputting the multi-layer content representation into the trained representation compression model suitable for the target task in step S302, the method may further include the following steps:

[0053] Step S401: Acquire multiple content data samples of a target task.

[0054] The content data samples are samples of content data used to train the representation compression model to be trained, and there are multiple content data samples.

[0055] In this step, the terminal or server may obtain multiple content data samples for the target task. These content data samples are applicable to the target task and may include training data and label values. For example, content data samples applicable to an audio genre classification task may include audio training data and a label for the genre.

[0056] Step S402: Acquire multi-layer content representation samples of each content data sample.

[0057] Each content data sample has multiple layers of content representation samples, which are obtained by a large content representation model based on the content data sample.

[0058] In this step, for each content data sample, the terminal or server can input the content data sample into a trained content representation model, and the content representation model obtains and outputs a multi-layer content representation sample of the content data sample based on the content data sample, thereby obtaining a multi-layer content representation sample of each content data sample.

[0059] Step S403 : training the representation compression model to be trained according to the multi-layer content representation samples of each content data sample to obtain a trained representation compression model.

[0060] In this step, the target task can be combined with the corresponding loss (loss function) to construct the representation compression model to be trained based on the multi-layer content representation samples of each content data sample, so that the trained representation compression model is suitable for the target task, thereby obtaining a trained representation compression model.

[0061] The solution of this embodiment can train a representation compression model to be trained based on multi-layer content representation samples of content data samples to obtain a trained representation compression model suitable for a target task.

[0062] In an exemplary embodiment, the training of the representation compression model to be trained based on the multi-layer content representation samples of each content data sample in step S403 to obtain the trained representation compression model may include:

[0063] The multi-layer content representation samples of each content data sample are input into the representation compression model to be trained; the representation compression model to be trained is trained according to the multi-layer content representation samples of each content data sample, the target content representation sample and the multi-layer reconstructed content representation to obtain a trained representation compression model.

[0064] The representation compression model to be trained may include an encoder and a decoder. For example, the representation compression model may employ a convolutional autoencoder (CAE) network, which includes an encoder and a decoder. The encoder compresses the input data layers, while the decoder reconstructs the input data layers.

[0065] In this embodiment, the terminal or server can input the multi-layer content representation sample of each content data sample into the representation compression model to be trained. The encoder of the representation compression model to be trained can compress the multi-layer content representation sample of each content data sample to obtain a target content representation sample that meets the layer number condition of the target task (e.g., 1 layer). The decoder of the representation compression model to be trained can reconstruct the target content representation sample of each content data sample into a multi-layer reconstructed content representation. The multi-layer reconstructed content representation corresponds to the number of layers of the multi-layer content representation sample. For example, if the multi-layer content representation sample has 20 layers, the multi-layer reconstructed content representation also has 20 layers. Thus, the terminal or server can construct a corresponding loss based on the multi-layer content representation sample of each content data sample, the target content representation sample, and the multi-layer reconstructed content representation, in combination with the target task, to train the representation compression model to be trained, thereby obtaining a trained representation compression model.

[0066] In an exemplary embodiment, Figure 5 As shown, the above-mentioned training of the representation compression model to be trained based on the multi-layer content representation samples, the target content representation samples and the multi-layer reconstructed content representation of each content data sample to obtain the trained representation compression model may include:

[0067] Step S501 : constructing a first model loss according to the similarity between the multi-layer content representation sample and the multi-layer reconstructed content representation of each content data sample.

[0068] In this step, the terminal or server can construct a first model loss based on the similarity between the multi-layer content representation sample and the multi-layer reconstructed content representation of each content data sample. The first model loss can be used to ensure that the representation compression model compresses the target content representation without losing the main characteristics of the original multi-layer content representation.

[0069] Step S502 : constructing a second model loss applicable to the target task according to the target content representation sample of each content data sample.

[0070] In this step, the terminal or server can construct a second model loss applicable to the target task based on the target content representation sample of each content data sample. The second model loss can be used to ensure that the compressed target content representation is applicable to the target task. For example, the second model loss can be constructed by combining the labels contained in the content data samples and utilizing the similarities and differences between the target content representation samples of each content data sample to ensure that the compressed target content representation is applicable to the target task.

[0071] Step S503 : Training the representation compression model to be trained according to the first model loss and the second model loss to obtain a trained representation compression model.

[0072] In this step, the terminal or server uses the sum of the first and second model losses as the total loss, and trains the representation compression model to be trained based on the total loss. After the total loss converges, the trained representation compression model can be obtained. This preserves the characteristics of the content representation itself during the compression process and adapts to downstream target tasks.

[0073] In one exemplary embodiment, the target task includes a content classification task, which includes multiple content classifications; constructing a second model loss applicable to the target task based on the target content representation sample of each content data sample in step S502 may include:

[0074] A second model loss is constructed based on the similarity between the target content representation sample of each content data sample and the target content representation samples of content data samples of the same content classification.

[0075] A content classification task refers to a task of categorizing content. This task may include multiple content categories, each of which indicates a specific type of content. For example, an audio genre classification task may classify the genre of music in an audio file. This task may include multiple genres.

[0076] In this embodiment, a second model loss can be constructed based on the similarity between the target content representation sample of each content data sample and the target content representation samples of content data samples of the same content classification. The distance between vectors can be used to represent the similarity or commonality between samples. The smaller the distance, the higher the similarity or commonality. This second model loss can be constructed to ensure that the compressed target content representation is suitable for content classification tasks.

[0077] In another exemplary embodiment, the target task includes a content classification task, which includes multiple content classifications; constructing a second model loss applicable to the target task based on the target content representation sample of each content data sample in step S502 may include:

[0078] A second model loss is constructed based on the similarity between the target content representation sample of each content data sample and the target content representation sample of the content data sample of the same content classification, and the difference between the target content representation sample of each content data sample and the target content representation sample of the content data sample of a different content classification.

[0079] In this embodiment, in addition to similarity or commonality, a second model loss can also be constructed by combining the differences between the target content representation sample of each content data sample and the target content representation samples of content data samples of different content classifications. Differences can also be represented by the distance between vectors; the greater the distance, the greater the difference. This allows the second model loss to be constructed. For example, a ternary loss can be used to construct the second model loss, ensuring that the compressed target content representation is suitable for content classification tasks.

[0080] In an exemplary embodiment, Figure 6 As shown, the above-mentioned construction of the second model loss based on the similarity between the target content representation sample of each content data sample and the target content representation sample of the content data sample of the same content classification may include:

[0081] Step S601 : determining a content representation cluster center for each content classification based on a target content representation sample of each content data sample.

[0082] In this step, target content representation samples for each content data sample of each content classification are obtained. For each content classification, the content representation cluster center is determined based on the target content representation samples for each content data sample of that content classification. The content representation cluster center represents the center of the target content representation samples for each content data sample of that content classification in the representation space. Thus, the content representation cluster center for each content classification can be determined.

[0083] Step S602 : constructing a second model loss based on the similarity between the target content representation sample of each content data sample and the content representation cluster center of the corresponding content classification.

[0084] In this step, a second model loss can be constructed based on the similarity between the target content representation sample of each content data sample and the content representation cluster center of the content classification to which it belongs. The similarity can be represented by the distance between the target content representation sample and the content representation cluster center, thereby constructing the second model loss.

[0085] The solution of this embodiment can construct a second model loss based on the similarity between the target content representation sample of each content data sample and the content representation cluster center of the content classification to which it belongs, so that the compressed target content representation can be applicable to the content classification task.

[0086] In an exemplary embodiment, determining the content representation cluster center of each content classification based on the target content representation sample of each content data sample in step S601 may include:

[0087] Determine target content representation samples corresponding to each content category; for each content category, obtain a content representation cluster center of the content category based on the mean of the target content representation samples corresponding to the content category.

[0088] In this embodiment, the target content representation samples corresponding to each content classification can be first determined. Then, for each content classification, the target content representation samples corresponding to the content classification are averaged to obtain a mean value, and the mean value is used as the content representation cluster center of the content classification. This simplifies the construction of the second model loss while ensuring that the compressed target content representation is applicable to the content classification task.

[0089] In one exemplary embodiment, applying the target content representation to the target task in step S304 may include:

[0090] According to the target content representation, a training sample of a task processing model to be trained for the target task is obtained; and the task processing model to be trained is trained using the training sample to obtain a trained task processing model for the target task.

[0091] In this embodiment, the terminal or server can use the target content representation as a training sample of the task processing model to be trained for the downstream target task, and use the training sample to train the task processing model to be trained to obtain a trained task processing model for the target task, wherein the target content representation can retain the main characteristics of the original content features and can be applicable to the current target task, thereby ensuring the training effect of the task processing model to be trained.

[0092] In another exemplary embodiment, applying the target content representation to the target task in step S304 includes:

[0093] The target content representation is input into a trained task processing model of the target task, and the task processing result output by the task processing model is obtained.

[0094] In this embodiment, the terminal or server can also use the target content representation for the processing of the trained task processing model of the target task, input the target content representation into the trained task processing model of the target task, and the trained task processing model obtains the task processing result based on the target content representation, so that the terminal or server can obtain the task processing result. As an example, the terminal or server can input the target content representation of the audio data into the trained task processing model of the audio classification task, and the task processing model can be a model for classifying the genres of music, thereby obtaining the genre corresponding to the audio data output by the task processing model. Among them, the target content representation can retain the main characteristics of the original content features and can be applied to the current target task. Using the target content representation for the processing of the trained task processing model of the target task can improve the efficiency of obtaining the target content representation while improving the accuracy of the processing of the task processing model.

[0095] In an exemplary embodiment, the method is applied to audio as an example for explanation. The content representation model can be an audio representation model. The audio representation model can convert the audio data into an abstract vector representation (content representation) by learning and understanding the audio data. The vector representation is used for audio downstream tasks, such as genre classification, etc., which will achieve better results than traditional audio features such as MFCC (Mel Frequency Cepstral Coefficients). The main architecture of the current audio representation model is Transformers, and its output content representation includes multiple layers. The multi-layer content representation will result in the need to try layer by layer when used in audio downstream tasks, which is cumbersome and limited. The method of this embodiment can perform multi-layer vector representation mapping for audio downstream tasks, and can obtain the vector representation (target content representation) that is most suitable for audio downstream tasks, avoiding cumbersome and limited layer-by-layer attempts, and can achieve better application effects. Figure 7 As shown, the method of this embodiment may include the following steps:

[0096] 1. After training, the large audio representation model (large content representation model) can output a multi-layer content representation sample [20, 1024] based on the input audio data sample (content data sample). This indicates that the multi-layer content representation sample has 20 layers, and the dimension of each content representation sample is 1024. The EncodecMAE is used as an example in this large audio representation model. Other large audio representation models can have different numbers of layers, and each layer of representation can have different dimensions.

[0097] 2. Use samples from downstream tasks to train a representation compression model. This representation compression model can employ a convolutional autoencoder network. Taking genre classification as an example, the aforementioned audio data samples can be audio data samples from different genres. The multi-layer content representation samples [20, 1024] output by the large audio representation model can be randomly masked, meaning that a certain percentage of points are randomly selected and set to zero.

[0098] 3. The randomly masked multi-layer content representation sample [20, 1024] can be input into the encoder of the representation compression model, and the encoder outputs the target content representation sample [1, 1024]. The target content representation sample [1, 1024] is input into the decoder, and the decoder outputs the multi-layer reconstructed content representation [20, 1024].

[0099] Thus, the first loss L1 can be constructed using multi-layer content representation samples [20, 1024] and multi-layer reconstruction content representation [20, 1024]:

[0100]

[0101] Where M is the number of audio data samples in the current training batch, x_rec_i represents the multi-layer reconstructed content representation of the current audio data sample, and x_raw_i represents the multi-layer content representation sample of the current audio data sample. The first loss L1 ensures that the compressed representation retains the key characteristics of the original representation.

[0102] 4. In order to ensure that the compressed representation is suitable for the current downstream genre task, the second loss L2 is introduced to perform genre clustering on the representation at the encoder output:

[0103]

[0104] Where x_i is the target content representation sample for the current audio data sample, and x_center is the mean of all target content representation samples in the current training batch that belong to the same genre as the current audio data sample. This mean is used as the cluster center for the content representation of the current genre. The second loss, L2, clusters all target content representation samples belonging to the same genre within the compressed representation space, making the compressed target content representation suitable for the current genre classification task.

[0105] 5. The total loss can be expressed as L = L1 + L2. By combining these two losses, the sample's inherent characteristics can be preserved during compression and adapted to downstream tasks. After the representation compression model is trained until the total loss L converges, the representation compression model is saved. The compressed representation (target content representation) can be obtained from the encoder output of the representation compression model for training or use in downstream tasks.

[0106] The solution of this embodiment can compress 20 layers of representation into 1 layer while ensuring that the compressed representation is suitable for the current downstream task, avoiding the difficulty of layer-by-layer selection. The optimal representation can be obtained by training the representation compression model once without having to repeat the training 20 times on the downstream task. Audio representation is an important part of audio downstream tasks. The solution of this embodiment can use the representation compression model to automatically compress the representation, avoiding repeated and tedious layer-by-layer traversal, so that the large audio representation model can be smoothly applied to downstream tasks, solving the problem of representation layer selection when the large audio representation model is applied to downstream tasks in related technologies, so that the representation of the large audio representation model can be efficiently applied to downstream tasks, bringing better results for downstream tasks.

[0107] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0108] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device can be used to communicate with an external device through a network connection. When the computer program is executed by the processor, a content representation application processing method is implemented.

[0109] Those skilled in the art will understand that Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0110] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0111] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0112] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0113] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0114] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0115] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0116] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A content representation application processing method, characterized in that: The method comprises: Obtaining a multi-layer content representation obtained by a content representation model based on content data; Inputting the multi-layer content representation into a trained representation compression model suitable for a target task; the representation compression model is used to compress the number of layers of the multi-layer content representation; Obtaining a target content representation obtained by compressing the multi-layer content representation with the trained representation compression model; the number of layers of the target content representation meets the layer number condition of the target task; The target content representation is applied to the target task.

2. The method according to claim 1, characterized in that Before inputting the multi-layer content representation into a trained representation compression model suitable for the target task, the method further includes: Acquire multiple content data samples of the target task; Obtaining a multi-layer content representation sample of each of the content data samples; the content representation sample is obtained by the content representation macro model according to the content data sample; The representation compression model to be trained is trained according to the multi-layer content representation samples of each of the content data samples to obtain the trained representation compression model.

3. The method according to claim 2, characterized in that The step of training the representation compression model to be trained based on the multi-layer content representation samples of each content data sample to obtain the trained representation compression model includes: Inputting the multi-layer content representation sample of each content data sample into the representation compression model to be trained; wherein the encoder of the representation compression model to be trained compresses the multi-layer content representation sample of each content data sample to obtain a target content representation sample that meets the layer number condition of the target task; and the decoder of the representation compression model to be trained reconstructs the target content representation sample of each content data sample into a multi-layer reconstructed content representation; the multi-layer reconstructed content representation corresponds to the number of layers of the multi-layer content representation sample; The representation compression model to be trained is trained according to the multi-layer content representation samples, the target content representation samples and the multi-layer reconstructed content representation of each of the content data samples to obtain the trained representation compression model.

4. The method according to claim 3, characterized in that The training of the representation compression model to be trained based on the multi-layer content representation samples, the target content representation samples, and the multi-layer reconstructed content representation of each content data sample to obtain the trained representation compression model includes: constructing a first model loss according to the similarity between the multi-layer content representation sample and the multi-layer reconstructed content representation of each of the content data samples; constructing a second model loss applicable to the target task according to a target content representation sample of each content data sample; The representation compression model to be trained is trained according to the first model loss and the second model loss to obtain the trained representation compression model.

5. The method according to claim 4, characterized in that The target task includes a content classification task, which includes multiple content classifications; and constructing a second model loss applicable to the target task based on a target content representation sample of each content data sample, including: constructing the second model loss according to the similarity between the target content representation sample of each content data sample and the target content representation sample of the content data sample of the same content classification; or, The second model loss is constructed based on the similarity between the target content representation sample of each content data sample and the target content representation sample of the content data sample of the same content classification, and the difference between the target content representation sample of each content data sample and the target content representation sample of the content data sample of a different content classification.

6. The method according to claim 5, characterized in that The constructing the second model loss according to the similarity between the target content representation sample of each content data sample and the target content representation sample of the content data sample of the same content classification includes: determining a content representation cluster center for each content classification based on a target content representation sample of each content data sample; The second model loss is constructed according to the similarity between the target content representation sample of each content data sample and the content representation cluster center of the corresponding content classification.

7. The method according to claim 6, characterized in that Determining the content representation cluster center of each content classification based on the target content representation sample of each content data sample includes: Determining each target content representation sample corresponding to each content classification; For each content category, a content representation cluster center of the content category is obtained according to the mean of each target content representation sample corresponding to the content category.

8. The method according to any one of claims 1 to 7, characterized in that The applying the target content representation to the target task includes: Obtaining, according to the target content representation, a training sample of a task processing model to be trained for the target task; Using the training samples to train the task processing model to be trained to obtain a trained task processing model for the target task; or, The target content representation is input into a trained task processing model of the target task to obtain a task processing result output by the task processing model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images