Data processing method and apparatus

Abstract

The present specification provides a data processing method and device, wherein the data processing method comprises: receiving multimedia content to be evaluated; splicing the multimedia content to be evaluated and a preset evaluation template to generate content evaluation prompt information, wherein the preset evaluation template comprises example information of at least one evaluation score word element; inputting the content evaluation prompt information into a content evaluation model to obtain at least one predicted evaluation score word element and evaluation explanation text output by the content evaluation model. Through the method, a single content evaluation model outputs a composite result of high-precision scoring and natural language explanation.

Description

Technical Field

[0001] This specification relates to the field of large language model technology, and in particular to data processing methods. This specification also relates to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the explosive growth of digital content creation and dissemination, the evaluation of multimedia content (such as videos and images) has become a core supporting technology in fields such as intelligent content review, personalized recommendations, and creative assistance tools. The core requirement for media content evaluation has evolved from the traditional binary judgment of "qualified or unqualified" to a composite requirement of "precise quantitative scoring + interpretable explanation." This not only necessitates outputting objective evaluation value scores but also simultaneously generating natural language explanations that align with public understanding to enhance the credibility and usability of the evaluation results.

[0003] However, the current scoring and text interpretation modules are set up and deployed relatively independently, requiring multiple interfaces for data exchange. A single model cannot accomplish the task of scoring and natural language interpretation. Furthermore, the scoring module lacks accuracy, and the natural language interpretation is not highly targeted. Therefore, providing a design solution for evaluating media content to address these issues has become a pressing technical problem for engineers. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a data processing method. This specification also relates to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the aforementioned problems existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a data processing method is provided, comprising: Receive multimedia content to be evaluated; The multimedia content to be evaluated and the preset evaluation template are combined to generate content evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring terminology; The content evaluation prompt information is input into the content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model.

[0006] According to a second aspect of the embodiments of this specification, a method for training a content evaluation model is provided, comprising: Acquire training sample data, wherein the training sample data includes sample multimedia content and sample evaluation scoring terms and sample explanation text corresponding to the sample multimedia content; The sample multimedia content and the preset evaluation template are combined to generate sample evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring terminology; The sample evaluation prompt information is input into the content evaluation model to obtain at least one predicted evaluation score term and predicted explanation text output by the content evaluation model; The model loss value is calculated based on the sample evaluation scoring terminology, the sample explanation text, the at least one predicted evaluation scoring terminology, and the predicted explanation text. Adjust the model parameters of the content evaluation model based on the model loss value, and continue training the content evaluation model until the model training stops.

[0007] According to a third aspect of the embodiments of this specification, a data processing method is provided, applied to a cloud-side device, comprising: The multimedia content to be evaluated sent by the receiving device; The multimedia content to be evaluated and the preset evaluation template are combined to generate content evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring terminology; The content evaluation prompt information is input into the content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model; The content score is calculated based on the at least one predicted evaluation scoring terminology, and the at least one predicted evaluation scoring terminology, the content score, and the evaluation explanation text are sent to the end device.

[0008] According to a fourth aspect of the embodiments of this specification, a data processing apparatus is provided, comprising: The receiving module is configured to receive multimedia content to be evaluated. The splicing module is configured to splice the multimedia content to be evaluated and the preset evaluation template to generate content evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring terminology; The evaluation module is configured to input the content evaluation prompt information into the content evaluation model and obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model.

[0009] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above method.

[0010] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0011] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0012] The data processing method provided in this specification includes receiving multimedia content to be evaluated; concatenating the multimedia content to be evaluated with a preset evaluation template to generate content evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring term; inputting the content evaluation prompt information into a content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model.

[0013] One embodiment of this specification implements a method for aesthetically evaluating multimedia content by incorporating example information of evaluation scoring terms into the input prompts of the content evaluation model. This guides the content evaluation model to output predicted evaluation scoring terms and evaluation explanation text. By customizing evaluation scoring terms that are tied to precision weights and designing a score encoding rule based on the number of evaluation scoring terms, the score encoding sequence and the explanation text sequence are concatenated into a consistent generated sequence. Based on the training paradigm of an autoregressive large language model, the content evaluation model is generated, achieving a composite result of high-precision scoring and natural language explanation from a single content evaluation model. Attached Figure Description

[0014] Figure 1 This is a flowchart of a data processing method provided in one embodiment of this specification; Figure 2 This is a flowchart illustrating a data processing method for an aesthetic evaluation scenario of an image, provided in one embodiment of this specification. Figure 3 This is a schematic flowchart of the model training method for a content evaluation model provided in one embodiment of this specification; Figure 4 This is a flowchart illustrating a data processing method provided in another embodiment of this specification; Figure 5 This is a schematic diagram of the structure of a data processing device provided in one embodiment of this specification; Figure 6 This is an architecture diagram of a data processing system provided in one embodiment of this specification; Figure 7 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0015] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0016] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0017] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0018] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0019] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0020] Token: The basic discrete unit for the model to process and measure text, which may correspond to a single character, subword, punctuation mark, or special symbol. The input text is segmented into a token sequence by a token segmenter, and the model encodes, predicts, and generates tokens at the token level. Context length, billing, and computational overhead are usually measured in terms of the number of tokens.

[0021] Autoregressive Large Language Models (ALMs): A type of language model that generates text sequentially, commonly used for tasks such as dialogue, writing, and question answering. Its basic objective is to predict the next token given the generated content, outputting a sequence progressively from left to right. It typically employs a Transformer structure with causal masking constraints to ensure that the current position can only utilize preceding information, thus achieving continuous and controllable generation. The model acquires general language capabilities through pre-training on large-scale corpora and can be adapted to specific business scenarios through fine-tuning with instructions.

[0022] Multimedia content aesthetic evaluation refers to the use of computer vision and artificial intelligence technologies to automatically judge the visual aesthetics of images in multimedia content. Its core goal is to simulate human perception of beauty and to quantify or rank dimensions such as composition, color, lighting, and subject expression of images.

[0023] With the explosive growth of digital content creation and dissemination, the evaluation of multimedia content (such as videos and images) has become a core supporting technology in fields such as intelligent content review, personalized recommendations, and creative assistance tools. Currently, the industry's core demand for aesthetic evaluation has evolved from the traditional binary judgment of "qualified / unqualified" to a composite requirement of "precise quantitative scoring + interpretable explanation"—not only must objective aesthetic value scores be output, but also natural language explanations that conform to human cognition must be generated simultaneously to improve the credibility and practicality of the evaluation results. From a technological evolution perspective, aesthetic evaluation of multimedia content is undergoing two key transformations: First, the evaluation accuracy has been upgraded from discrete classification to continuous high-precision quantization.

[0024] Second, the system architecture is evolving from deploying more models separately to generating a single, integrated model.

[0025] The rise of Large Language Models (LLMs) has provided a technological foundation for integrated generation solutions, but it also faces the challenge that "high-precision scores are difficult to adapt to the LLM training paradigm." Traditional classification models output aesthetic levels by pre-setting discrete labels such as "excellent," "good," and "poor," which achieves basic evaluation functions but is difficult to meet the requirements of high-precision quantification. At the same time, the scoring results of these models are independent of the text interpretation module, requiring additional interaction with the text generation model, resulting in weak system coupling and low inference efficiency.

[0026] In recent years, with the development of large language models, autoregressive large language models using the Next Token Prediction training paradigm have been gradually introduced. However, their conventional lexical units cannot efficiently carry the structured information of high-precision scores, making it difficult to achieve end-to-end training of "scoring + explanation". In addition, although current research combines text-image alignment and aesthetic preferences, it still relies on the concatenation of multiple models and has not broken through the technical bottleneck of integrated generation.

[0027] Current aesthetic rating models face the following problems: First, the system suffers from poor task coupling and high complexity. For example, the scoring module and text interpretation module are designed and deployed independently, requiring data interaction through multiple interfaces. A single model cannot accomplish the combined task of "high-precision quantitative scoring + natural language interpretation." This separate architecture not only increases the complexity of system deployment and maintenance but also significantly increases inference latency due to the substantial time consumed in data transmission and conversion between modules, failing to meet real-time evaluation requirements.

[0028] Second, it is difficult to balance accuracy and efficiency in score representation. Traditional classification-based scoring modules can only output discrete labels or roughly quantified scores, failing to meet the demands for high-precision quantification and struggling to match users' varying perceptions of aesthetic details. Conversely, directly outputting numerical text to represent high-precision scores cannot be deeply integrated with the LLM training process, resulting in poor coordination between score generation and text interpretation.

[0029] Third, the explanatory texts lack specificity and the task collaboration is insufficient. Because the scoring and explanation tasks are decoupled, the explanation module can only passively receive the scoring results and generate generic explanations, failing to distinguish the aesthetic differences between closely spaced scores. This significantly reduces the rationality and practicality of the explanations. Furthermore, the lack of a collaborative optimization mechanism between the two tasks further reduces the consistency of the overall evaluation results.

[0030] Based on this, a data processing method is provided in this specification. This specification also relates to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0031] Figure 1 A flowchart of a data processing method according to an embodiment of this specification is shown, which specifically includes the following steps: Step 102: Receive the multimedia content to be evaluated.

[0032] The data processing methods provided in the embodiments of this specification are applied to terminals, such as personal computers, servers, cloud servers, etc. These terminals have the capability to deploy and run the methods provided in the embodiments of this specification.

[0033] The multimedia content to be evaluated can be understood as various multimedia materials that require aesthetic attribute analysis and aesthetic evaluation in the aesthetic evaluation method provided in the embodiments of this specification. Its core scope focuses on multimedia content containing image elements, specifically including but not limited to single static images, dynamic video frames, short video clips, and image portions in combined text and image content. Any image-based multimedia material that requires aesthetic evaluation to determine its visual aesthetic level and optimize its visual presentation falls under the category of multimedia content to be evaluated in this solution. This multimedia content to be evaluated can originate from various scenarios such as content uploaded by content creators for publication, existing content published on the platform, and user-generated UGC content. Its quality and aesthetic appeal directly affect the user's browsing experience, the content's dissemination effect, and the overall quality of the platform's content ecosystem. Therefore, conducting scientific and accurate aesthetic evaluation of it has significant practical implications.

[0034] Aesthetic evaluation refers to the process of automatically analyzing, judging, and evaluating the visual aesthetics of image elements in multimedia content using computer vision technology, artificial intelligence algorithms, and image processing technology, through preset evaluation models and quantitative standards. Unlike the subjectivity and variability of human aesthetic judgment, this solution automates, standardizes, and refines aesthetic judgment, effectively avoiding the problems of low efficiency, high cost, and inconsistent evaluation standards associated with manual evaluation. Its core objective is to simulate the human visual system's perception logic and judgment habits regarding beauty through technological means, replicating the human cognitive process of image aesthetics. It comprehensively and multidimensionally deconstructs and quantifies the various visual attributes of images, ultimately outputting evaluation results in the form of scores, rankings, and grade classifications, providing data support for subsequent content optimization, filtering, and recommendation operations.

[0035] In the method provided in the embodiments of this specification, the core execution logic is to conduct a comprehensive and accurate aesthetic attribute analysis and aesthetic judgment of the multimedia content to be evaluated. To achieve this core goal, the primary prerequisite is to obtain the multimedia content to be evaluated that meets the evaluation requirements and needs to be aesthetically evaluated. This step is the foundation of the entire aesthetic evaluation process and is also the core prerequisite for the smooth implementation of all subsequent data processing, feature extraction, model calculation and other operations.

[0036] Specifically, acquiring the multimedia content to be evaluated is the starting step in the aesthetic evaluation process. Its core purpose is to provide real and effective data support for all subsequent data processing stages, ensuring that the entire evaluation process has a reliable data foundation.

[0037] In practical applications, the multimedia content to be evaluated can be video or images. There are also various ways to receive this multimedia content. For example, users can connect an external storage device (such as a USB flash drive, external hard drive, or memory card) to the current terminal, allowing the terminal to retrieve the multimedia content from the external storage device. Alternatively, users can connect the current terminal to the internet and download the multimedia content from the internet. Or, the multimedia content to be evaluated can be pre-saved to the terminal's local storage and retrieved from local storage based on its location.

[0038] This step obtains the multimedia content to be evaluated, providing a data foundation for subsequent data processing.

[0039] Step 104: Combine the multimedia content to be evaluated and the preset evaluation template to generate content evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring term.

[0040] Among them, the preset evaluation template can be understood as a special prompt word template adapted to the aesthetic evaluation scenario. Its core essence is a set of pre-set and standardized prompt words based on the operating logic of the large language model. It is specifically used to guide the large language model to accurately execute the aesthetic evaluation-related tasks of the multimedia content to be evaluated. It is the core bridge connecting the multimedia content to be evaluated and the evaluation capability of the large language model, providing the large language model with clear evaluation guidance and parameter specifications.

[0041] During the operation of a large language model, prompts serve as the interaction medium between the user and the model. Their core function is to clearly convey the contextual information of the input, task requirements, and various parameters of the input model, establishing a communication channel between the user's intent and the model's response. Unlike randomly input natural language commands, prompts are standardized and can accurately define task boundaries and clarify input requirements, preventing response deviations caused by ambiguous context or unclear intent, and ensuring that the model output meets expected requirements.

[0042] The prompts provided by the pre-set evaluation templates effectively help large language models better understand input intent and make accurate responses that meet aesthetic evaluation requirements. Through contextual guidance within the pre-set evaluation templates, this method enables large language models to quickly identify that they need to perform a multimedia content aesthetic evaluation task, avoiding evaluation bias caused by task confusion. Furthermore, the standardized prompt design of the pre-set evaluation templates significantly improves the interpretability and accessibility of large language models in aesthetic evaluation scenarios.

[0043] In the methods provided in the embodiments of this specification, to ensure that the large language model can accurately and efficiently complete the aesthetic scoring task of the multimedia content to be evaluated, and to avoid evaluation bias caused by inconsistent scoring standards and lack of scoring carriers, a pre-designed preset evaluation template is provided. The preset evaluation template is a dedicated template formed after being designed, tested and optimized in advance based on the core requirements of aesthetic evaluation, the operating logic of the large language model and the technical requirements of scoring coding. Its core function is to provide clear scoring guidance, standardized lexical references and standardized scoring coding carriers for the aesthetic evaluation process of the large language model, ensuring the standardization, consistency and accuracy of the entire aesthetic evaluation process.

[0044] Specifically, the preset evaluation template, designed for the aesthetic evaluation scenario of this solution, integrates various guiding information, parameter configurations, and reference examples related to aesthetic evaluation. One of its core components is example information for at least one evaluation scoring term. This example information provides a clear reference for the large language model, guiding it to accurately understand the usage of evaluation scoring terms, avoiding scoring encoding errors due to misunderstandings of terms, and ensuring the reliability of the scoring encoding results.

[0045] Furthermore, the evaluation scoring special token is a dedicated and customized token specifically proposed in the methods provided in the embodiments of this specification. It is used exclusively to represent the aesthetic scoring result of the multimedia content to be evaluated, possessing specificity and delimitation. It is applied only to the aesthetic evaluation scoring encoding process of this solution and is not used for other types of text processing, feature recognition, or model interaction tasks. The core design of this evaluation scoring token is to solve the technical problems in existing technologies, such as the lack of a dedicated carrier for aesthetic scoring results, non-standard scoring encoding, and inaccurate transmission of scoring information. Through customized token design, it achieves a precise match between aesthetic scoring results and the encoding process. The evaluation scoring special token provides a dedicated carrier for the final score encoding in the entire aesthetic evaluation process.

[0046] Specifically, to reflect the accuracy of subsequent scoring, the method provided in the embodiments of this specification provides at least two scoring term types, each representing a precision bit. For a detailed explanation of the evaluation scoring terms provided in the embodiments of this specification, "Score-E" is used to represent the evaluation scoring term. In this method, different evaluation scoring terms represent different scoring term types. For example, "Score-E-1" represents a score with 0.1 precision, "Score-E-2" represents a score with 0.01 precision, and "Score-E-3" represents a score with 0.001 precision. Therefore, "Score-E-1," "Score-E-2," and "Score-E-3" are three scoring term types, each representing a different scoring precision.

[0047] For example, if the model output is " <score-e-1> 、 <score-e-1> 、 <score-e-2> 、 <score-e-3>If '', it means the model output score is 0.211, where 2 "' <score-e-1>"Represents 0.2, 1" <score-e-2>"Represents 0.01, 1 unit" <score-e-3>" represents 0.001, and the sum of the three evaluation scoring terms is 0.211.

[0048] In practical applications, the number of rating term types represents the accuracy of the rating. For example, two rating term types correspond to a rating accuracy of 0.01; three rating term types correspond to a rating accuracy of 0.001; and four rating term types correspond to a rating accuracy of 0.0001.

[0049] The preset evaluation template includes example information for evaluation scoring terms. This information serves to inform the content evaluation model of the examples used for aesthetic evaluation output. For instance, the preset evaluation template might be: "Please give an aesthetic score to {×××} and provide an explanation, using the output..." <score-e-1> 、 <score-e-2> 、 <score-e-3>The quantity of the score is represented by, for example <score-e-1> 、 <score-e-2> 、 <score-e-3>"0.111 points". Among them, " <score-e-1> 、 <score-e-2> 、 <score-e-3>The example information is "0.111 points". Its purpose is to standardize the output of the content evaluation model so that the output of the content evaluation model is more in line with the requirements.

[0050] Taking the aforementioned preset evaluation template as an example, "{×××}" represents the multimedia content to be evaluated. In practical applications, the multimedia content to be evaluated needs to be combined with the preset evaluation template, that is, the multimedia content to be evaluated is filled into "{×××}" of the preset evaluation template to generate content evaluation prompt information.

[0051] Content evaluation prompts can be understood as prompts that need to be input into the content evaluation model. They provide the content evaluation model with context and parameter information about the input, helping the model to better understand the user's input intent and guiding it to respond accordingly.

[0052] In the method provided in the embodiments of this specification, evaluation scoring terms specifically designed for scoring are used. Different evaluation scoring terms of different types represent different scoring accuracies, enabling the content evaluation model to quantify the scoring accuracy within the 0-1 scoring range.

[0053] Step 106: Input the content evaluation prompt information into the content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model.

[0054] In the above steps, if content evaluation prompts are obtained, these prompts can be input into the content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model.

[0055] Among them, the predicted evaluation scoring terminology can be understood as the terminology output by the content evaluation model after evaluating the multimedia content to be evaluated based on the content evaluation prompts. It is used to represent the score of the multimedia content to be evaluated by the content evaluation model.

[0056] The evaluation explanatory text can be understood as an aesthetic natural language commentary generated by the content evaluation model to explain the scoring results while scoring the multimedia content to be evaluated.

[0057] The method provided in the embodiments of this specification uses an autoregressive large language model as its content evaluation model. The autoregressive large language model is one of the mainstream areas in the current field of large language models. Its core feature is that it uses an autoregressive mechanism as its core operating logic to achieve the gradual generation and accurate prediction of sequence data. Essentially, it is a generative language model built on causal relationships. Its core idea is to model the sequence generation task as an element-wise conditional probability prediction problem, that is, each time based only on the already generated historical sequence content, it predicts the probability distribution of the next sequence element, generating a complete sequence output through gradual iteration.

[0058] In the method provided in the embodiments of this specification, the content evaluation model does not directly output a score, but rather outputs predicted evaluation score terms. This avoids directly outputting a score that cannot be corrected when an incorrect score occurs. For example, if the content evaluation model initially predicts a score of 0.816, directly outputting the score would generate a score of 0.816, making it impossible to modify the score later. The method provided in the embodiments of this specification represents the score by evaluating score terms. When the predicted score is 0.816, eight terms are first output... <score-e-1>During this process, the model can modify the prediction score based on the prediction results. For example, it can adjust the score from 0.816 to 0.725, even if the model has only output the fourth score at this point. <score-e-1>In subsequent processing, only 7 will be output. <score-e-1>That's it. By using predicted evaluation terms instead of directly outputting content evaluation scores, the final evaluation results are more accurate and have a better margin for error.

[0059] The method provided in the embodiments of this specification integrates content scoring and content explanation text into a content evaluation model for processing, supporting a single model to simultaneously generate composite results of scoring and explanation based on the same input content.

[0060] In the methods provided in the embodiments of this specification, the content evaluation model includes an embedding layer, a decoder, and an output layer; The content evaluation prompt information is input into the content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model, including S1062-S1068: S1062. Parse the content evaluation prompt information and generate the image-text fusion information to be evaluated.

[0061] In the method provided in the embodiments of this specification, content evaluation prompts are input into the content evaluation model for processing, which is specifically divided into a data preprocessing stage and a model actual processing stage. This step provides a detailed explanation of the data preprocessing stage.

[0062] Specifically, in practical applications, the multimedia content to be evaluated can be multimodal data, including images and videos. To ensure the consistency of the input model data, preprocessing of the multimedia content to be evaluated is necessary. This involves parsing the content evaluation prompts to generate the corresponding image-text fusion information to be evaluated.

[0063] Specifically, the image-text fusion information to be evaluated refers to the fusion information including images and text. This step, through standardized preprocessing, transforms the multimedia content to be evaluated into a format recognizable by the content evaluation model, providing adaptable input for subsequent model processing.

[0064] In one specific embodiment provided in this specification, the content evaluation prompt information is parsed to generate image-text fusion information to be evaluated, including: Analyze the multimedia content in the content evaluation prompt information to obtain at least one multimedia image; The multimedia image and the preset evaluation template are used to generate image-text fusion information to be evaluated.

[0065] In the methods provided in the embodiments of this specification, the content evaluation model can process at least image and text data types. Based on this, multimedia content in the content evaluation prompt information can be parsed and preprocessed to generate at least one multimedia image. The multimedia image and a preset evaluation template are then combined to generate the image-text fusion information to be evaluated.

[0066] The processing methods for different modalities of data are further explained below. Specifically, the multimedia content in the content evaluation prompt information is parsed to obtain at least one multimedia image, including: When the multimedia content is video, at least one video frame image is obtained by sampling the video based on a preset sampling rule.

[0067] When multimedia content includes video, video can be sampled based on preset sampling rules. For example, video frames can be sampled at preset time intervals (such as sampling one video frame every 1 second or every 2 seconds). To ensure the processing efficiency of the model, the number of sampled video frame images should not exceed a preset threshold.

[0068] For example, if the video frame count threshold is set to 64, then the number of video frame images obtained during video sampling must not exceed 64. Furthermore, this video frame count threshold can be used as a standard to guide the sampling frequency for the video. For example, if the video duration is 60 seconds, a sampling rule of one video frame per second can be used. Or, for example, if the video duration is 120 seconds, a sampling rule of one video frame per two seconds can be used, and so on. In the methods provided in the embodiments of this specification, the video sampling rules and video sampling count threshold are not limited; the actual application shall prevail.

[0069] When the multimedia content is in the form of images, the images can be directly used as multimedia images. In practical applications, in order to improve the processing efficiency of subsequent models, when there are multiple input images, the input images can be pre-filtered to remove duplicate or highly similar images. This can reduce the processing pressure on subsequent models and improve data processing efficiency.

[0070] S1064. Input the image-text fusion information to be evaluated into the embedding layer to obtain the content evaluation prompt feature information output by the embedding layer.

[0071] After the above steps, the image-text fusion information to be evaluated can be obtained for aesthetic evaluation. This includes both the image information to be evaluated and the fixed task instructions in the prompt template (such as "Please give an aesthetic score to this video and image and explain it").

[0072] To input the image-text fusion information to be evaluated into the content evaluation model for processing, it is necessary to first convert it into computer language that can be processed by a computer, that is, to input the natural language content into the embedding layer for embedding processing.

[0073] Embedding layers are one of the most fundamental and crucial components in deep learning, especially in natural language processing and recommendation systems. Their core function is to transform discrete, high-dimensional symbols into continuous meaning vectors, enabling neural networks to process abstract concepts such as language and user behavior, and building upon this foundation to construct complex semantic understanding and generation capabilities. Embedding layers can map each word to a low-dimensional, dense, continuous real-valued vector.

[0074] The embedding layer is essentially a learnable embedding vocabulary, taking integer indices as input and outputting the embedding vector corresponding to each index. For example, if the ID of a token is 3, the vector corresponding to 3 can be found through this lookup table. In a specific embodiment provided in this specification, the method further includes: Assign evaluation term identifiers to each evaluation scoring term in the embedded vocabulary corresponding to the content evaluation model.

[0075] In the method provided in the embodiments of this specification, evaluation scoring lexical units are specifically designed. Referring to the design specifications of the embedded vocabulary of large language models, the evaluation scoring lexical units designed in the embodiments of this specification are added to the embedded vocabulary, and corresponding evaluation lexical unit identifiers (token IDs) are assigned to each type of evaluation scoring lexical unit. This processing can ensure that it is physically isolated from regular lexical units (such as Chinese characters, English words, etc.), and avoid semantic confusion during training and application.

[0076] Specifically, the embedding vocabulary of the content evaluation model can be queried to determine the reserved identifier intervals within the vocabulary. Then, an independent reserved identifier can be assigned to each type of evaluation scoring terminology from these intervals. To ensure the proper functioning of each evaluation scoring terminology in subsequent processing, the embedding vectors of newly added evaluation scoring terms need to be initialized with the same dimensions as the general vectors of the content evaluation model. These vectors are then integrated into the parameter set of the embedding layer of the content evaluation model. By training the embedding vectors, the content evaluation model can learn the mapping relationship between terms and precision weights.

[0077] The above describes the preparatory work for the content evaluation model before training or application. In the method provided in the embodiments of this specification, the image-text fusion information to be evaluated is input into the embedding layer to obtain the content evaluation prompt feature information output by the embedding layer. That is, the embedding layer performs embedding processing on the image-text fusion information to be evaluated, generating content evaluation prompt feature information that can be processed in a computer.

[0078] In one specific embodiment provided in this specification, the image-text fusion information to be evaluated includes image information and text information; The image-text fusion information to be evaluated is input into the embedding layer to obtain the content evaluation prompt feature information output by the embedding layer, including: The image information and the text information are input into the embedding layer to obtain image feature information and text feature information; The image feature information and the text feature information are combined to generate and output content evaluation prompt feature information.

[0079] As described in the steps above, the image-text fusion information to be evaluated specifically includes image information and text information. However, in practical applications, the embedding processing for images differs from that for text. Therefore, in the method provided in this embodiment, the image information and text are embedded separately in the embedding layer to obtain image feature information corresponding to the image information and text feature information corresponding to the text information. The image feature information and text feature information are then concatenated to generate the final content evaluation prompt feature information.

[0080] Specifically, the embedding layer includes an image embedding sublayer and a text embedding sublayer; The image information and the text information are input into the embedding layer to obtain image feature information and text feature information, including: The image information is input into the image embedding sublayer to obtain image feature information; The text information is input into the text embedding sub-layer to obtain text feature information.

[0081] Two types of embedding sub-layers are pre-defined in the embedding layer: an image embedding sub-layer and a text embedding sub-layer. The image embedding sub-layer is used to embed image information to obtain image feature information, while the text embedding sub-layer is used to embed text information to obtain text feature information. These two embedding sub-layers will be explained further below.

[0082] In practical applications, image features can be encoded using an image embedding sublayer to extract the corresponding image feature information for each image. In practice, image feature information can include global visual features, local visual features, or a combination of both. The specific content of the image feature information depends on the training of the image embedding sublayer and is not specifically limited here.

[0083] For text information, the text embedding sublayer can be used to perform word segmentation, token mapping, and other processing on the text to obtain the text feature information corresponding to the text information. In the text embedding layer, over-segmentation (tokenization) can be provided first to segment the text information, resulting in a token sequence composed of at least one token.

[0084] Tokenization, in natural language processing, is the process of segmenting raw text into tokens that the model can process. In large language models, tokenization is not simply segmented by character or word, but rather employs a sub-word strategy to achieve a balance between vocabulary size, generalization ability, and efficiency. A token can be understood as the word corresponding to each piece of text information determined after word segmentation. By obtaining the token ID (lexical identifier) ​​corresponding to each token, the word embedding information corresponding to each token ID can be retrieved from a pre-defined vocabulary. Then, based on the position of each token in the text information, the word embedding information corresponding to each token is concatenated to obtain the text feature information corresponding to the text information.

[0085] Finally, the image feature information and text feature information are concatenated to form content evaluation prompt feature information that the content evaluation model can directly process.

[0086] S1066. Input the content evaluation prompt feature information into the decoder to obtain the content evaluation decoding feature information output by the decoder.

[0087] In the steps described above, after a series of operations including data preprocessing, feature extraction, and prompt information fusion, content evaluation prompt feature information adapted to the operational requirements of the content evaluation model has been successfully obtained. This content evaluation prompt feature information is a comprehensive feature vector that integrates multi-dimensional information such as the aesthetic features of the multimedia content to be evaluated, the prompt word features of the preset evaluation template, and the example features of evaluation scoring terms. Essentially, it transforms various input information related to aesthetic evaluation into a feature form that the model can recognize and process. It is the core data foundation for the smooth operation of subsequent model decoding and a key prerequisite for ensuring that the content evaluation model can accurately output evaluation results.

[0088] It should be noted that the acquisition process of content evaluation prompt feature information strictly follows the model's input specifications, undergoing standardized feature normalization, dimension alignment, and other preprocessing operations to ensure that its feature format and dimensionality fully match the input requirements of the content evaluation model's decoder. This avoids decoding failures or deviations in decoding results caused by incompatible feature formats or inconsistent dimensions. Simultaneously, this content evaluation prompt feature information fully preserves the core aesthetic features of the multimedia content to be evaluated (global image features, local features, etc.) and the guiding information of the preset evaluation template (such as evaluation criteria, scoring rules, lexical examples, etc.). This provides comprehensive and accurate contextual support for the decoder's decoding operation, guiding the decoder to focus on the core task of aesthetic evaluation and generate decoding results that meet the requirements.

[0089] After obtaining qualified content evaluation prompt feature information, this information can be input into the decoder of the content evaluation model to perform decoding, thereby obtaining the content evaluation decoded feature information output by the decoder. The content evaluation model used in this solution is based on an autoregressive large language model as its core architecture. Its decoder, as the core functional module of the model, plays a crucial role in feature decoding, logical reasoning, and result generation. Essentially, it is a serialization processing module built on the Transformer decoder architecture, possessing powerful contextual understanding and feature parsing capabilities. It can accurately interpret the various semantics, features, and guiding information contained in the content evaluation prompt feature information and, through preset decoding logic, transform the input feature information into meaningful content evaluation decoded feature information.

[0090] The decoder's decoding process follows an autoregressive mechanism, and its core operating logic is adapted to the inference process of the autoregressive large language model. It is mainly divided into two core stages: The first stage is the feature reception and initialization stage. The decoder first receives the input content evaluation prompt feature information, performs preliminary feature parsing and dimensional transformation on the feature information, maps it to the model's preset feature space, and initializes the KV Cache (key-value cache) to store the key and value vectors generated during feature parsing, providing data support for subsequent stepwise decoding, avoiding redundant calculations, and improving decoding efficiency. The second stage is the stepwise decoding and feature generation stage. Based on the autoregressive mechanism, the decoder starts from the end of the content evaluation prompt feature information and performs feature decoding and inference element by element. Each decoding is based on the previously decoded historical feature information to predict the probability distribution of the next decoded feature element. The optimal feature element is selected through a preset sampling strategy, and the complete content evaluation decoding feature information is generated stepwise iteratively.

[0091] During the decoding process, the decoder fully utilizes the example information of evaluation scoring terms contained in the preset evaluation template to accurately associate them with the aesthetic features of the multimedia content to be evaluated. This ensures that the decoded content evaluation feature information accurately reflects the aesthetic score of the multimedia content. For example, the decoder uses aesthetic feature parameters such as color and lighting contained in the content evaluation prompt feature information, combined with the scoring criteria in the preset evaluation template, to decode and generate corresponding dimension scoring features. Simultaneously, it integrates the evaluation scoring term (Special Token) as a dedicated carrier into the decoding feature sequence, achieving precise binding between aesthetic scoring information and terms, providing clear feature support for subsequent scoring encoding and result analysis.

[0092] The content evaluation decoding feature information output by the decoder is a high-dimensional feature vector generated after deep model analysis and inference. Its core value lies in transforming the input content evaluation prompt feature information into core feature data that can be directly used for aesthetic scoring calculation and scoring encoding. This content evaluation decoding feature information not only includes quantitative features of various aesthetic dimensions of the multimedia content to be evaluated (such as compositional balance scoring features, color harmony scoring features, etc.), but also includes the associated features required for scoring encoding (such as the encoding features of evaluation scoring terms, scoring level features, etc.). It can directly serve as the core data foundation for subsequent scoring calculation and result output, avoiding additional feature transformation operations and significantly improving the efficiency of the entire aesthetic evaluation process.

[0093] During the decoding process, the decoder also utilizes its attention mechanism to focus on the core features of the content evaluation prompts, filtering out invalid and interfering features to ensure the accuracy and effectiveness of the output content evaluation decoding features. Simultaneously, the decoder possesses a certain degree of fault tolerance; when the input content evaluation prompts contain slight noise or bias, it can reduce the impact of these biases on the decoding results through contextual reasoning and feature correction, ensuring that the content evaluation decoding features truly and accurately reflect the aesthetic level of the multimedia content being evaluated.

[0094] S1068. Input the content evaluation decoding feature information into the output layer to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the output layer.

[0095] After obtaining the content evaluation decoding feature information, it is input into the output layer of the content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the output layer.

[0096] In the method provided in the embodiments of this specification, after decoding operations in the content evaluation model, content evaluation decoding feature information that reflects the aesthetic features and scoring correlation information of the multimedia content to be evaluated has been obtained. This content evaluation decoding feature information, as a high-dimensional core feature vector output by the decoder, retains the quantitative features of each aesthetic dimension (composition, color, lighting, subject expression, etc.) of the multimedia content to be evaluated, the encoded correlation features of evaluation scoring terms, and the guiding information in the preset evaluation template. It serves as the core data support for the subsequent output layer to generate the final evaluation result. Its feature format and dimensionality have been standardized to fully adapt to the input requirements of the output layer, ensuring efficient and accurate feature conversion and result output after input to the output layer.

[0097] The content evaluation decoding feature information is input into the output layer of the content evaluation model. Through feature mapping, probability calculation, and result generation in the output layer, at least one predicted evaluation rating terminology and evaluation explanation text are ultimately obtained. The output layer, as the terminal functional module of the content evaluation model, is essentially built upon a fully connected layer and a language generation module. Its core function is to transform the high-dimensional, abstract content evaluation decoding feature information output by the decoder into concrete output content with clear practical meaning that can be directly used for aesthetic evaluation result presentation and analysis—namely, predicted evaluation rating terms and evaluation explanation text. It serves as a crucial bridge connecting the model's internal feature processing with the application of external evaluation results.

[0098] The output layer's operation is mainly divided into two core stages, corresponding to the generation of predicted evaluation scoring terms and evaluation explanation text, respectively. The two stages work together and are interconnected to ensure the accuracy and consistency of the output results.

[0099] In one specific embodiment provided in this specification, the evaluation scoring terminology includes at least two scoring terminology types; Obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model, including: Obtain at least one predicted evaluation scoring term and evaluation explanation text corresponding to each scoring term type output by the content evaluation model.

[0100] In the method provided in the embodiments of this specification, the evaluation scoring terms include at least two scoring term types, each corresponding to a different scoring precision. Obtaining at least one predicted evaluation scoring term output by the content evaluation model specifically means obtaining the predicted evaluation scoring terms corresponding to each scoring term type output by the content evaluation model.

[0101] In practical applications, to ensure the efficiency and rationality of the model output, corresponding constraints are set for the predicted evaluation scoring terms. Specifically, the number of each scoring term type is limited to no more than 9. This constraint avoids exceeding the value range of a single precision bit, ensuring that the length of the output sequence is controllable and does not significantly increase the inference burden of the content evaluation model.

[0102] Each scoring term type represents the scoring accuracy, and the number of predicted evaluation scoring terms corresponding to each scoring term type represents the scoring value. In the method provided in the embodiments of this specification, high-precision score encoding is achieved based on each scoring term type and the number of predicted evaluation scoring terms corresponding to each scoring term type. In another specific embodiment provided in this specification, the method further includes: The content score corresponding to the multimedia content to be evaluated is calculated based on the predicted evaluation scoring terms corresponding to each scoring term type.

[0103] For example, the rating terminology type includes " <score-e-1> 、 <score-e-2> 、 <score-e-3>For example, <score-e-1>"Represents 0.1," <score-e-2>"Represents 0.01," <score-e-3>"Represents 0.001. If the model outputs a total of 7..." <score-e-1>", 8" <score-e-2>"and 3" <score-e-3>Then, based on the precision and quantity of each predicted evaluation scoring term, the content score corresponding to the multimedia content to be evaluated can be calculated as 0.783 (i.e., 7×0.1+8×0.01+3×0.001). This method can fully reflect the core encoding logic of the precision weight corresponding to the number of tokens provided in the embodiments of this specification.

[0104] The data processing method provided in this specification, during the aesthetic evaluation of multimedia content, incorporates example information of evaluation scoring terms into the prompts input to the content evaluation model, guiding the model to output predicted evaluation scoring terms and evaluation explanation text. By customizing evaluation scoring terms bound to precision weights and designing score encoding rules based on the number of evaluation scoring terms, the score encoding sequence and explanation text sequence are concatenated into a consistent generated sequence. Based on the training paradigm of an autoregressive large language model, the content evaluation model is generated, achieving a composite result of high-precision scoring and natural language explanation from a single content evaluation model.

[0105] By employing dedicated evaluation scoring terms and natural language interpretation sequences to characterize scoring accuracy, a single content evaluation model can simultaneously perform high-precision quantitative scoring and natural language interpretation, improving system coupling and reducing model deployment complexity and inference latency. Furthermore, this method allows the content evaluation model to adjust interpretation details based on subtle differences in evaluation scoring terms, strengthening the consistency between the final score and the natural language interpretation. Additionally, through seamless task coupling and regression task transformation, the capabilities of a large language model are fully utilized to improve the accuracy of aesthetic scoring, support multi-task merging deployment, and enhance computational efficiency and maintenance effectiveness.

[0106] Furthermore, the methods provided in the embodiments of this specification allow the content evaluation model to directly reuse the framework of native generative large language models without structural modifications, resulting in low adaptation costs. Adding new evaluation dimensions only requires supplementing the corresponding training data for training. This approach is suitable for scenarios that are not sensitive to timeliness but require detailed natural language interpretation.

[0107] The following is in conjunction with the appendix Figure 2 Taking the application of the data processing method provided in this specification in the aesthetic evaluation of images as an example, the data processing method will be further explained. Figure 2 The present specification provides a flowchart of a data processing method for an aesthetic evaluation scenario of an image, which includes the following steps: Step 202: Receive the image to be evaluated.

[0108] In this embodiment, an image 1 to be evaluated is received for aesthetic evaluation.

[0109] Step 204: Combine the image to be evaluated and the preset evaluation template to generate content evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring term.

[0110] In this embodiment, the preset evaluation template is "Please give an aesthetic score to {×××} and provide a corresponding explanation, using the output..." <score-e-1> 、 <score-e-2> 、 <score-e-3>The number represents the score, for example <score-e-1> 、 <score-e-2> 、 <score-e-3>"0.111 points". Replace "{×××}" in the preset evaluation template with the image to be evaluated (image 1), and generate the content evaluation prompt message: "Please give an aesthetic score to the image to be evaluated (image 1) and provide a corresponding explanation, using the output..." <score-e-1> 、 <score-e-2> 、 <score-e-3>The number represents the score, for example <score-e-1> <score-e-2> <score-e-3>0.111 points.

[0111] Step 206: Input the content evaluation prompt information into the embedding layer of the content evaluation model to obtain the content evaluation prompt feature information output by the embedding layer.

[0112] Step 208: Input the content evaluation prompt feature information into the decoder of the content evaluation model to obtain the content evaluation decoding feature information output by the decoder.

[0113] Step 210: Input the content evaluation decoding feature information into the output layer of the content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the output layer.

[0114] Step 212: Calculate the content score corresponding to the image to be evaluated based on at least one predicted evaluation scoring term.

[0115] In the specific implementation provided in this specification, the content evaluation model outputs the final data result after the above processing as " <score-e-1> <score-e-2> <score-e-3> <score-e-3>0.112 points. "The image is meaningless and extremely unattractive."

[0116] The data processing method provided in this specification, during the aesthetic evaluation of multimedia content, incorporates example information of evaluation scoring terms into the prompts input to the content evaluation model, guiding the model to output predicted evaluation scoring terms and evaluation explanation text. By customizing evaluation scoring terms bound to precision weights and designing score encoding rules based on the number of evaluation scoring terms, the score encoding sequence and explanation text sequence are concatenated into a consistent generated sequence. Based on the training paradigm of an autoregressive large language model, the content evaluation model is generated, achieving a composite result of high-precision scoring and natural language explanation from a single content evaluation model.

[0117] See Figure 3 , Figure 3 This specification illustrates a flowchart of a model training method for a content evaluation model provided in one embodiment. Figure 3 As shown, the method includes: Step 302: Obtain training sample data, wherein the training sample data includes sample multimedia content and sample evaluation scoring terms and sample explanation text corresponding to the sample multimedia content.

[0118] Step 304: Combine the sample multimedia content and the preset evaluation template to generate sample evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring term.

[0119] Step 306: Input the sample evaluation prompt information into the content evaluation model to obtain at least one predicted evaluation score term and predicted explanation text output by the content evaluation model.

[0120] Step 308: Calculate the model loss value based on the sample evaluation scoring terminology, the sample explanation text, the at least one predicted evaluation scoring terminology, and the predicted explanation text.

[0121] Step 310: Adjust the model parameters of the content evaluation model according to the model loss value, and continue training the content evaluation model until the model training stops.

[0122] The method provided in the embodiments of this specification also provides a model training method for the content evaluation model, which adopts an end-to-end training process.

[0123] First, training sample data for model training is obtained. This training sample data specifically includes sample multimedia content, as well as sample evaluation scoring terms and sample explanation text corresponding to the sample multimedia content.

[0124] The sample multimedia content is concatenated with a pre-defined evaluation template to generate sample evaluation prompts. These prompts, along with the sample evaluation scoring terms and sample explanation text, form a training sample pair in an "input sequence-generated sequence" format. Dedicated sample evaluation scoring terms, score text terms, and explanation text terms are used as a unified prediction target. This is then input into a content evaluation model built on a general large language model for supervised fine-tuning.

[0125] Specifically, the sample evaluation prompts are input into the content evaluation model to obtain at least one predicted evaluation rating term and a predicted explanatory text output by the content evaluation model. At this point, there is still a certain gap between the content output by the content evaluation model and the sample evaluation rating term and sample explanatory text. The difference needs to be calculated based on a preset loss function. For example, the cross-entropy loss function can be used to calculate the cross-entropy loss value between the sample evaluation rating term, the sample explanatory text, the at least one predicted evaluation rating term, and the predicted explanatory text. The model parameters of the content evaluation model are then adjusted based on this cross-entropy loss value.

[0126] The above describes the training process for one training iteration. In practical applications, multiple rounds of model training can be conducted to train the content evaluation model until the training stops, thus obtaining a well-trained content evaluation model.

[0127] The model training stopping condition can be adaptively adjusted according to the actual business situation. For example, the model training stopping condition can be that the model loss value is less than a preset threshold, or that the model training epochs have reached a preset number of training epochs, or both conditions can be met simultaneously, etc. In the methods provided in the embodiments of this specification, the model training stopping condition is not specifically limited.

[0128] The method provided in the embodiments of this specification constructs training sample data between sample evaluation scoring terms and sample explanation text and sample multimedia content. Through an end-to-end training process, the score encoding sequence and explanation text sequence are concatenated into a unified generation sequence, realizing a composite result of high-precision score and natural language explanation output by a single model.

[0129] See Figure 4 , Figure 4 This specification shows a flowchart illustrating a data processing method according to another embodiment. This embodiment applies a cloud-side device and specifically includes: Step 402: The multimedia content to be evaluated is sent by the receiving device.

[0130] Step 404: Combine the multimedia content to be evaluated and the preset evaluation template to generate content evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring term.

[0131] Step 406: Input the content evaluation prompt information into the content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model.

[0132] Step 408: Calculate the content score based on the at least one predicted evaluation scoring terminology, and send the at least one predicted evaluation scoring terminology, the content score, and the evaluation explanation text to the end device.

[0133] In practical applications, content evaluation models are built upon large language models. The application and deployment of large language models require substantial computing resources, but the computing capabilities of edge devices vary widely, and some edge devices may lack the necessary resources. To ensure a consistent user experience for all users, the data processing method can be implemented on a cloud-based device. The cloud-based device can connect to the edge device via the internet, receive the multimedia content to be evaluated from the edge device, and perform an aesthetic evaluation of the multimedia content using the data processing method provided in this specification. It then sends at least one predicted evaluation scoring term, a content score, and an evaluation explanation text generated during the evaluation to the edge device. Users with different edge devices can all enjoy the application experience of the content evaluation model.

[0134] Corresponding to the above method embodiments, this specification also provides data processing apparatus embodiments. Figure 5 A schematic diagram of the structure of a data processing apparatus provided in one embodiment of this specification is shown. For example... Figure 5 As shown, the device includes: The receiving module 502 is configured to receive multimedia content to be evaluated. The splicing module 504 is configured to splice the multimedia content to be evaluated and the preset evaluation template to generate content evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring terminology; The evaluation module 506 is configured to input the content evaluation prompt information into the content evaluation model and obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model.

[0135] In one specific embodiment provided in this specification, the content evaluation model includes an embedding layer, a decoder, and an output layer; The evaluation module 506 is further configured as follows: The content evaluation prompts are analyzed to generate the image-text fusion information to be evaluated; The image-text fusion information to be evaluated is input into the embedding layer to obtain the content evaluation prompt feature information output by the embedding layer; The content evaluation prompt feature information is input into the decoder to obtain the content evaluation decoding feature information output by the decoder; The content evaluation decoding feature information is input into the output layer to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the output layer.

[0136] In one specific embodiment provided in this specification, the evaluation module 506 is further configured to: Analyze the multimedia content in the content evaluation prompt information to obtain at least one multimedia image; The multimedia image and the preset evaluation template are used to generate image-text fusion information to be evaluated.

[0137] In one specific embodiment provided in this specification, the evaluation module 506 is further configured to: When the multimedia content is video, at least one video frame image is obtained by sampling the video based on a preset sampling rule.

[0138] In one specific embodiment provided in this specification, the image-text fusion information to be evaluated includes image information and text information; The evaluation module 506 is further configured as follows: The image information and the text information are input into the embedding layer to obtain image feature information and text feature information; The image feature information and the text feature information are combined to generate and output content evaluation prompt feature information.

[0139] In one specific embodiment provided in this specification, the embedding layer includes an image embedding sublayer and a text embedding sublayer; The evaluation module 506 is further configured as follows: The image information is input into the image embedding sublayer to obtain image feature information; The text information is input into the text embedding sub-layer to obtain text feature information.

[0140] In one specific embodiment provided in this specification, the evaluation scoring terminology includes at least two scoring terminology types; The evaluation module 506 is further configured as follows: Obtain at least one predicted evaluation scoring term and evaluation explanation text corresponding to each scoring term type output by the content evaluation model.

[0141] In one specific embodiment provided in this specification, the evaluation module 506 is further configured to: The content score corresponding to the multimedia content to be evaluated is calculated based on the predicted evaluation scoring terms corresponding to each scoring term type.

[0142] In one specific embodiment provided in this specification, the device further includes a lexical identifier adding module, configured to: Assign evaluation term identifiers to each evaluation scoring term in the embedded vocabulary corresponding to the content evaluation model.

[0143] The data processing method provided in this specification, during the aesthetic evaluation of multimedia content, incorporates example information of evaluation scoring terms into the prompts input to the content evaluation model, guiding the model to output predicted evaluation scoring terms and evaluation explanation text. By customizing evaluation scoring terms bound to precision weights and designing score encoding rules based on the number of evaluation scoring terms, the score encoding sequence and explanation text sequence are concatenated into a consistent generated sequence. Based on the training paradigm of an autoregressive large language model, the content evaluation model is generated, achieving a composite result of high-precision scoring and natural language explanation from a single content evaluation model.

[0144] By employing dedicated evaluation scoring terms and natural language interpretation sequences to characterize scoring accuracy, a single content evaluation model can simultaneously perform high-precision quantitative scoring and natural language interpretation, improving system coupling and reducing model deployment complexity and inference latency. Furthermore, this method allows the content evaluation model to adjust interpretation details based on subtle differences in evaluation scoring terms, strengthening the consistency between the final score and the natural language interpretation. Additionally, through seamless task coupling and regression task transformation, the capabilities of a large language model are fully utilized to improve the accuracy of aesthetic scoring, support multi-task merging deployment, and enhance computational efficiency and maintenance effectiveness.

[0145] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0146] See Figure 6 , Figure 6 This specification illustrates an architecture diagram of a data processing system according to one embodiment of the present specification. The data processing system may include a client 100 and a server 200. Client 100 is used to send multimedia content to be evaluated to server 200; Server 200 is used to concatenate the multimedia content to be evaluated and a preset evaluation template to generate content evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring term; input the content evaluation prompt information into a content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model; calculate the content score based on the at least one predicted evaluation scoring term; and send at least one predicted evaluation scoring term, the content score, and the evaluation explanation text to client 100. Client 100 is also configured to receive at least one predictive evaluation rating term, the content rating, and the evaluation explanation text sent by server 200.

[0147] The data processing system may include multiple clients 100 and a server 200. Clients 100 can be referred to as edge devices, and server 200 can be referred to as cloud devices. Multiple clients 100 can establish communication connections through server 200. In a content evaluation scenario, server 200 is used to provide content evaluation services between multiple clients 100. Each client 100 can act as a sender or receiver, communicating through server 200.

[0148] Users can interact with server 200 through client 100 to receive data sent by other clients 100, or send data to other clients 100, etc. In a content evaluation scenario, users can publish data streams to server 200 through client 100. Server 200 generates predicted evaluation scoring terms, content scores, and evaluation explanation text based on the data stream, and pushes the predicted evaluation scoring terms, content scores, and evaluation explanation text to other clients that have established communication.

[0149] In this system, client 100 and server 200 establish a connection via a network. The network provides the medium for communication between client 100 and server 200. The network can include various connection types, such as wired or wireless communication links or fiber optic cables. Data transmitted by client 100 may need to undergo encoding, transcoding, compression, or other processing before being published to server 200.

[0150] Client 100 can be a browser, an app (application), a web application such as an H5 (HyperText Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. Client 100 can be developed based on the software development kit (SDK) of the corresponding service provided by server 200, such as a real-time communication (RTC) SDK. Client 100 can be deployed on a computing device and depends on the device or certain apps on the device to run. The computing device may have a display screen and support information browsing, such as a personal mobile terminal like a mobile phone, tablet, or personal computer. Various other types of applications can also be configured on the computing device, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.

[0151] Server 200 may include servers providing various services, such as servers providing communication services to multiple clients, servers supporting backend training of models used on clients, and servers processing data sent by clients. It should be noted that server 200 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server integrated with blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0152] It is worth noting that the data processing methods provided in the embodiments of this specification are generally executed by the server. However, in other embodiments of this specification, the client may also have similar functions to the server, thereby executing the data processing methods provided in the embodiments of this specification. In other embodiments, the data processing methods provided in the embodiments of this specification may also be executed jointly by the client and the server.

[0153] Figure 7 A structural block diagram of a computing device 700 according to an embodiment of this specification is shown. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0154] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0155] In one embodiment of this specification, the above-described components of the computing device 700 and Figure 7 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 7 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0156] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 700 can also be a mobile or stationary server.

[0157] The processor 720 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0158] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data processing method described above.

[0159] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0160] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data processing method described above.

[0161] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0162] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data processing method described above.

[0163] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0164] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0165] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this specification is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this specification. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this specification.

[0166] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0167] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. These embodiments have been selected and specifically described in this specification to better explain the principles and practical applications of this specification, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents. < / score-e-3> < / score-e-2> < / score-e-1> < / score-e-2> < / score-e-1> < / score-e-2> < / score-e-1> < / score-e-2> < / score-e-1> < / score-e-2> < / score-e-1> < / score-e-2> < / score-e-1> < / score-e-2> < / score-e-1> < / score-e-2> < / score-e-1> < / score-e-2> < / score-e-1> < / score-e-2> < / score-e-1> < / score-e-1>

Claims

1. A data processing method, characterized in that, include: Receive multimedia content to be evaluated; The multimedia content to be evaluated and the preset evaluation template are combined to generate content evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring terminology; The content evaluation prompt information is input into the content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model.

2. The method as described in claim 1, characterized in that, The content evaluation model includes an embedding layer, a decoder, and an output layer; The content evaluation prompt information is input into the content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model, including: The content evaluation prompts are analyzed to generate the image-text fusion information to be evaluated; The image-text fusion information to be evaluated is input into the embedding layer to obtain the content evaluation prompt feature information output by the embedding layer; The content evaluation prompt feature information is input into the decoder to obtain the content evaluation decoding feature information output by the decoder; The content evaluation decoding feature information is input into the output layer to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the output layer.

3. The method as described in claim 2, characterized in that, The content evaluation prompt information is parsed to generate image-text fusion information to be evaluated, including: Analyze the multimedia content in the content evaluation prompt information to obtain at least one multimedia image; The multimedia image and the preset evaluation template are used to generate image-text fusion information to be evaluated.

4. The method as described in claim 3, characterized in that, Parsing the multimedia content in the content evaluation prompt information to obtain at least one multimedia image includes: When the multimedia content is video, at least one video frame image is obtained by sampling the video based on a preset sampling rule.

5. The method as described in claim 2, characterized in that, The image-text fusion information to be evaluated includes image information and text information; The image-text fusion information to be evaluated is input into the embedding layer to obtain the content evaluation prompt feature information output by the embedding layer, including: The image information and the text information are input into the embedding layer to obtain image feature information and text feature information; The image feature information and the text feature information are combined to generate and output content evaluation prompt feature information.

6. The method as described in claim 5, characterized in that, The embedding layer includes an image embedding sublayer and a text embedding sublayer; The image information and the text information are input into the embedding layer to obtain image feature information and text feature information, including: The image information is input into the image embedding sublayer to obtain image feature information; The text information is input into the text embedding sub-layer to obtain text feature information.

7. The method as described in claim 1, characterized in that, The evaluation scoring lemmas include at least two scoring lemma types; Obtain at least one predicted evaluation score term and evaluation explanation text output by the content evaluation model, including: Obtain at least one predicted evaluation scoring term and evaluation explanation text corresponding to each scoring term type output by the content evaluation model.

8. The method as described in claim 7, characterized in that, The method further includes: The content score corresponding to the multimedia content to be evaluated is calculated based on the predicted evaluation scoring terms corresponding to each scoring term type.

9. The method as described in claim 1, characterized in that, The method further includes: Assign evaluation term identifiers to each evaluation scoring term in the embedded vocabulary corresponding to the content evaluation model.

10. A method for training a content evaluation model, characterized in that, include: Acquire training sample data, wherein the training sample data includes sample multimedia content and sample evaluation scoring terms and sample explanation text corresponding to the sample multimedia content; The sample multimedia content and the preset evaluation template are combined to generate sample evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring terminology; The sample evaluation prompt information is input into the content evaluation model to obtain at least one predicted evaluation score term and predicted explanation text output by the content evaluation model; The model loss value is calculated based on the sample evaluation scoring terminology, the sample explanation text, the at least one predicted evaluation scoring terminology, and the predicted explanation text. Adjust the model parameters of the content evaluation model based on the model loss value, and continue training the content evaluation model until the model training stops.

11. A data processing method, characterized in that, Applications in cloud-side devices, including: The multimedia content to be evaluated sent by the receiving device; The multimedia content to be evaluated and the preset evaluation template are combined to generate content evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring terminology; The content evaluation prompt information is input into the content evaluation model to obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model; The content score is calculated based on the at least one predicted evaluation scoring terminology, and the at least one predicted evaluation scoring terminology, the content score, and the evaluation explanation text are sent to the end device.

12. A data processing apparatus, characterized in that, include: The receiving module is configured to receive multimedia content to be evaluated. The splicing module is configured to splice the multimedia content to be evaluated and the preset evaluation template to generate content evaluation prompt information, wherein the preset evaluation template includes example information of at least one evaluation scoring terminology; The evaluation module is configured to input the content evaluation prompt information into the content evaluation model and obtain at least one predicted evaluation scoring term and evaluation explanation text output by the content evaluation model.

13. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 11.

Images