Method, apparatus and medium for processing multimedia content

Abstract

The embodiment of the application discloses a multimedia content processing method, device, equipment and medium, which is applied to the technical field of data processing. The method comprises the following steps: acquiring multimedia information associated with multimedia content, extracting image modal features of first multimedia information, and extracting text modal features of second multimedia information; performing feature fusion processing on the image modal features and the text modal features to obtain multi-modal features of the multimedia content; performing feature processing on the multi-modal features to obtain initial parameter values of the multimedia content under recommendation parameters; determining parameter distribution probabilities of parameter values of the multimedia content under the recommendation parameters in N parameter intervals according to the initial parameter values; and determining target parameter values of the multimedia content under the recommendation parameters through the parameter distribution probabilities corresponding to the N parameter intervals. The embodiment of the application can improve the prediction accuracy of the recommendation parameters.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, device and medium for processing multimedia content. Background Technology

[0002] In multimedia recommendation tasks, the main objective is to predict multimedia content that users may be interested in and recommend it to their devices. For example, this involves predicting potentially interesting notes based on recommendation parameters (such as click-through rate) and then recommending those notes. Therefore, determining accurate recommendation parameters is a crucial task. Current methods typically involve building a binary classification model to predict the multimedia features of the content and determine which parameter classifies the content (e.g., whether the click-through rate is high or low). However, this approach cannot determine the specific values ​​of the recommendation parameters. Furthermore, the threshold for classifying the parameters can negatively impact model training, leading to low prediction accuracy for the recommendation parameters. Summary of the Invention

[0003] This application provides a method, apparatus, device, and medium for processing multimedia content, which can improve the accuracy of predictions for recommended parameters.

[0004] On one hand, embodiments of this application provide a method for processing multimedia content, the method comprising:

[0005] Obtain multimedia information associated with multimedia content; the media information includes first multimedia information in the image modality and second multimedia information in the text modality;

[0006] The image modal features of the first multimedia information and the text modal features of the second multimedia information are extracted. The image modal features and the text modal features are then fused to obtain the multimodal features of the multimedia content.

[0007] Multimodal features are processed to obtain the initial parameter values ​​of multimedia content under the recommendation parameters, and the parameter distribution probability of the multimedia content under the recommendation parameters in N parameter intervals is determined based on the initial parameter values; one parameter interval corresponds to one parameter distribution probability; N is a positive integer greater than 1.

[0008] The target parameter values ​​for multimedia content under the recommended parameters are determined by the parameter distribution probabilities corresponding to N parameter intervals.

[0009] On one hand, embodiments of this application provide a multimedia content processing apparatus, the apparatus comprising:

[0010] The acquisition module is used to acquire multimedia information associated with multimedia content; the media information includes first multimedia information of the multimedia content in the image modality and second multimedia information in the text modality;

[0011] The processing module is used to extract the image modal features of the first multimedia information and the text modal features of the second multimedia information, and to perform feature fusion processing on the image modal features and the text modal features to obtain the multimodal features of the multimedia content;

[0012] The processing module is also used to perform feature processing on multimodal features to obtain the initial parameter values ​​of multimedia content under the recommendation parameters, and to determine the parameter distribution probability of the parameter values ​​of multimedia content under the recommendation parameters falling within N parameter intervals based on the initial parameter values; one parameter interval corresponds to one parameter distribution probability; N is a positive integer greater than 1;

[0013] The processing module is also used to determine the target parameter value of multimedia content under the recommended parameters by using the parameter distribution probability corresponding to N parameter intervals.

[0014] On one hand, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute some or all of the steps in the above method.

[0015] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, are used to perform some or all of the steps in the above-described method.

[0016] Accordingly, according to one aspect of this application, a computer program product or computer program is provided, which includes computer instructions that, when executed by a processor, can implement some or all of the steps in the above-described method.

[0017] In this embodiment, multimedia information associated with multimedia content can be obtained; image modal features of the first multimedia information and text modal features of the second multimedia information can be extracted; feature fusion processing is performed on the image modal features and text modal features to obtain multimodal features of the multimedia content; feature processing is performed on the multimodal features to obtain the initial parameter values ​​of the multimedia content under the recommendation parameters; the initial parameter values ​​can be understood as an intermediate parameter value determined by regression; the parameter distribution probability of the multimedia content under the recommendation parameters in N parameter intervals is determined based on the initial parameter values; this can be understood as determining the probability distribution of the recommendation parameters by classification, that is, converting the initial parameter values, which are intermediate parameter values, into distribution probabilities in N parameter intervals; the target parameter values ​​of the multimedia content under the recommendation parameters are determined by the parameter distribution probabilities corresponding to the N parameter intervals; the target parameter values ​​are used to determine the recommendation strategy for the multimedia content; therefore, the specific recommendation parameters can be determined by first determining the initial parameter values, converting them into parameter probability distributions, and then determining the target parameter values. Combining regression and classification methods to determine the recommendation parameters can more accurately fit the correlation between the multimodal features of the multimedia content and the recommendation parameters, thereby improving the accuracy of the recommendation parameter determination. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A schematic diagram illustrating a data processing scenario provided in an embodiment of this application;

[0020] Figure 2 A flowchart illustrating a multimedia content processing method provided in this application embodiment. Figure 1 ;

[0021] Figure 3 A flowchart illustrating a multimedia content processing method provided in this application embodiment. Figure 2 ;

[0022] Figure 4 This application provides a schematic diagram illustrating a process for determining a target parameter value.

[0023] Figure 5 A schematic diagram of the structure of a multimedia content processing device provided in an embodiment of this application;

[0024] Figure 6This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0026] The multimedia content processing method proposed in this application is implemented in an electronic device, which can be a server or a terminal. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet computer, laptop computer, desktop computer, etc., but is not limited to these.

[0027] A schematic diagram of a data processing scenario proposed based on this multimedia content processing method can be shown as follows: Figure 1 As shown, Figure 1 A network architecture is proposed, which may include a service server and a user terminal cluster. The user terminal cluster may include one or more user terminals; the number of user terminals in the cluster is not limited. Communication connections can exist between the user terminals in the cluster. Simultaneously, any user terminal in the cluster can have a communication connection with the service server, enabling each user terminal in the cluster to interact with the service server via this connection. The communication connection method is not limited; it can be established directly or indirectly via wired communication, wireless communication, or other methods. This application does not impose any limitations on this method. Furthermore, it is understood that the electronic devices involved in the embodiments of this application may be… Figure 1 The business server shown can also be Figure 1 Any user terminal in the user terminal cluster shown.

[0028] For example, in this embodiment, the server can obtain multimedia information associated with any multimedia content and determine the recommended parameters for the multimedia content based on the multimedia information using the multimedia content processing method proposed in this application. For instance, when obtaining multimedia content 10, the server can obtain multimedia information 11 associated with that multimedia content 10. This multimedia information 11 includes first multimedia information 12 in the image modality and second multimedia information 13 in the text modality. Based on the first multimedia information 12 and the second multimedia information 13, the server determines the multimodal features 14 of the multimedia content 10. For example, it can extract the image modality information 15 of the first multimedia information 12 and the text modality features 16 of the second multimedia information 13, and perform feature fusion processing on the image modality features 15 and the text modality features 16 to obtain the multimodal features 14 of the multimedia content 10. The multimodal features 14 are processed to obtain the initial parameter values ​​17 of the multimedia content 10 under the recommendation parameters. Based on the initial parameter values ​​17, the parameter distribution probabilities 18 of the multimedia content 10 under the recommendation parameters in N parameter intervals are determined. Each parameter interval corresponds to a parameter distribution probability, such as parameter interval 1, parameter interval 2, and parameter interval 3. The target parameter value 19 of the multimedia content 10 under the recommendation parameters is determined through the parameter distribution probabilities 18 corresponding to the N parameter intervals. This target parameter value 19 is used to determine the recommendation strategy for the multimedia content.

[0029] Optionally, in some embodiments, the electronic device can execute the multimedia content processing method according to actual business needs to achieve efficient object segmentation. The technical solution of this application can be applied to any multimedia content recommendation parameter prediction scenario. For example, the multimedia content can be multimedia content in a social context, such as image content, video content, etc. (specifically, it could be user-posted notes). Or, the multimedia content can be multimedia content in an e-commerce context, such as product information content, etc. The type of multimedia content is not limited here. The electronic device can use the technical solution of this application to determine the target parameter value of the multimedia content under recommendation parameters, such as determining the click-through rate of the multimedia content, in order to make relevant recommendations for the multimedia content.

[0030] Optionally, the data involved in this application, such as multimedia information associated with multimedia content and target parameter values ​​of multimedia content under recommended parameters, can be stored in a database or in a blockchain. This application does not limit the storage of such data through a blockchain distributed system.

[0031] It should be noted that in specific embodiments of this application, when scenarios involving the acquisition of user information and related data, such as acquiring multimedia content uploaded by users, user permission or consent is required. That is, when the embodiments of this application are applied to specific products or technologies, the collection, use, and processing of relevant user data comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, prompts can be issued through an interactive interface to indicate what data will be collected or acquired. Specifically, the types and content of this data can be presented to the user through lists or other means. Further data collection and processing will only proceed after a confirmation or instruction to allow data collection is received on the interactive interface.

[0032] It is understood that the above scenarios are merely examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, as those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0033] Based on the foregoing description, this application proposes a method for processing multimedia content, which can be executed by the aforementioned electronic device. Please refer to... Figure 2 , Figure 2 This is a flowchart illustrating a multimedia content processing method provided in an embodiment of this application. Figure 2 As shown, the process flow of the multimedia content processing method in this application embodiment may include the following:

[0034] S101. Obtain multimedia information associated with multimedia content.

[0035] In some embodiments, the multimedia content can be of any type. For example, it can be image content, video content, etc., specifically, notes posted by users on social media platforms. No limitation is made here. This application does not limit the source of the multimedia content. For example, the multimedia content can be user-uploaded multimedia content; or online multimedia content scraped from a website; or content documents obtained from relevant databases, such as multimedia content obtained from a product database, etc. Furthermore, the multimedia content is the multimedia content to be recommended. And the multimedia information associated with the multimedia content can be information in multiple multimedia modalities. For example, the multimedia information associated with the multimedia content can include first multimedia information in the image modality and second multimedia information in the text modality. This application does not limit the multimedia modality of the multimedia information.

[0036] It can be understood that the first multimedia information is image information extracted from multimedia content. For example, if the multimedia content is notes, the image information could be pictures or video frames contained in the notes. The second multimedia information is text information extracted from multimedia content. This text information can be descriptive information about the image information. For example, if the multimedia content is an image, the text could be descriptive text about the image, such as the image title. In other words, the specific content of the first and second multimedia information is determined according to the specific type of multimedia content. It can be understood that the first and second multimedia information within the same multimedia content are mutually matching multimedia information.

[0037] S102. Extract the image modal features of the first multimedia information and the text modal features of the second multimedia information. Perform feature fusion processing on the image modal features and the text modal features to obtain the multimodal features of the multimedia content.

[0038] Specifically, image modal features can be obtained by extracting features from the first multimedia information using an image extraction model, and text modal features can be obtained by extracting features from the second multimedia information using a text extraction model. The specific extraction methods for the first and second multimedia information can be found in the relevant descriptions of the following embodiments.

[0039] One approach to feature fusion is to sum the image modal features and text modal features, and use the summed result as a multimodal feature. This multimodal feature represents both the feature information in the first multimedia information and the feature information in the second multimedia information.

[0040] In addition, another method of feature fusion processing is to concatenate image modal features and text modal features, and use the concatenated feature result as a multimodal feature.

[0041] It is understood that image extraction models and text extraction models can be sub-models within feature extraction models. Feature extraction models can also include a feature fusion layer, which performs feature fusion processing on image modal features and text modal features. No limitations are placed on the feature fusion processing described here.

[0042] Specifically, in the feature fusion layer, image modal features and text modal features are used as a feature set. Feature crosses (i.e., inner products of pairwise vectors) are performed on each pair of features in the feature set to obtain cross features. The image modal features and text modal features are then concatenated to obtain concatenated features. Multimodal features are obtained based on the cross features and concatenated features. Specifically, obtaining multimodal features based on the cross features and concatenated features can be achieved by sequentially tiling each element of the cross features and concatenated features to form a target vector, and then using this target vector as the multimodal feature.

[0043] Furthermore, the feature cross-referencing between pairs of features draws upon the idea of ​​Factorization Machine (FM), namely:

[0044]

[0045] Where <> denotes the vector dot product, V i V j Let N represent any two features in the feature set, and let N represent the number of features in the feature set.

[0046] It is understandable that this multimodal feature contains the correlation information between features within image modal features, between text modal features, and between image modal features and text modal features. It reflects the relationship between features obtained through explicit interaction between multiple features, thus enabling the combination of the relationship between two features in the multimodal feature, so that the multimodal feature can have rich feature information.

[0047] S103. Perform feature processing on the multimodal features to obtain the initial parameter values ​​of the multimedia content under the recommended parameters, and determine the parameter distribution probability of the multimedia content under the recommended parameters falling within N parameter intervals based on the initial parameter values.

[0048] The recommendation parameters can be any parameters applicable to any recommendation scenario, such as click-through rate, conversion rate, etc. The specific parameter values ​​(i.e., initial parameter values) of multimedia content under the recommendation parameters can be obtained through multimodal features. For example, the initial parameter values ​​can be obtained by performing feature processing on the multimodal features through a parameter prediction network (such as a fully connected layer) in a trained parameter processing model.

[0049] Specifically, the initial parameter values ​​can be processed by a parameter processing network in the parameter processing model to determine the parameter distribution probability of the multimedia content's parameter values ​​falling within N parameter intervals under the recommended parameters, which is the parameter distribution probability of the multimedia content's recommended parameters. The specific method for determining the parameter distribution probability through the parameter processing network can be found in the relevant description of the following embodiments.

[0050] Here, N parameter intervals are continuous intervals divided based on the preset parameter value range of the recommended parameters. For example, the parameter value range of the recommended parameters may be 0-1 or 0.4-1, etc., without limitation. The parameter value range can be divided using N-1 parameter dividing points, such as equal division, to obtain N parameter intervals. N is a positive integer greater than 1. N can be set according to the actual scenario. For example, if N is 10, the divided parameter intervals will be 0-0.1, 0.1-0.2, ..., 0.9-1, respectively. There is no limitation on the method of dividing the parameter intervals.

[0051] It is understood that this application's embodiments employ ordinal regression to determine the recommendation parameters for multimedia content. That is, initial parameter values ​​(intermediate values) are determined through regression, and then the parameter distribution probability of the multimedia content's recommendation parameters is determined based on these initial parameter values ​​through classification. This transforms the click-through rate (CTR) prediction problem into an ordinal regression problem of CTR, which in turn transforms the CTR regression problem into an interval classification problem. Different initial parameter values ​​for a given multimedia content result in different parameter distribution probabilities. Finally, the predicted value of the recommendation parameters is obtained through the parameter distribution probability, thus avoiding directly fitting a single parameter value but instead fitting the parameter probability distribution. Therefore, it retains the advantages of both classification and regression algorithms. Compared to using only classification or regression algorithms, this solution largely retains the advantages of both, exhibiting scale insensitivity and fast optimization, thereby improving the accuracy of recommendation parameter determination.

[0052] S104. Determine the target parameter value of multimedia content under the recommended parameters by using the parameter distribution probability corresponding to N parameter intervals.

[0053] Specifically, the parameter interval value for each parameter interval can be obtained. The target parameter value is then obtained by weighted summing of the reference interval values ​​for the N parameter intervals using the parameter distribution probabilities corresponding to those N intervals. This parameter interval value can be understood as the median value of each parameter interval. For example, the recommended parameter is click-through rate (CTR), and the N parameter ranges are: range 1: 0–0.25, range 2: 0.25–0.5, range 3: 0.5–0.75, and range 4: 0.75–1. The parameter range value for range 1 is 0.125, range 2 is 0.325, range 3 is 0.625, and range 4 is 0.825. If the probability distribution of the parameter range is 0.3 for range 1, 0.4 for range 2, 0.5 for range 3, and 0.7 for range 4, and the target parameter value is 0.125*0.3 + 0.325*0.4 + 0.625*0.5 + 0.825*0.7 = 0.847, then the CTR of the multimedia content is 0.847.

[0054] It's understandable that this target parameter value can be used to determine the recommendation strategy for multimedia content. For example, multimedia content with a target parameter value greater than a threshold value can be recommended to the target audience.

[0055] In this embodiment, multimedia information associated with multimedia content can be obtained; image modal features of the first multimedia information and text modal features of the second multimedia information can be extracted; feature fusion processing is performed on the image modal features and text modal features to obtain multimodal features of the multimedia content; feature processing is performed on the multimodal features to obtain the initial parameter values ​​of the multimedia content under the recommendation parameters; the initial parameter values ​​can be understood as an intermediate parameter value determined by regression; the parameter distribution probability of the multimedia content under the recommendation parameters in N parameter intervals is determined based on the initial parameter values; this can be understood as determining the probability distribution of the recommendation parameters by classification, that is, converting the initial parameter values, which are intermediate parameter values, into distribution probabilities in N parameter intervals; the target parameter values ​​of the multimedia content under the recommendation parameters are determined by the parameter distribution probabilities corresponding to the N parameter intervals; the target parameter values ​​are used to determine the recommendation strategy for the multimedia content; therefore, the specific recommendation parameters can be determined by first determining the initial parameter values, converting them into parameter probability distributions, and then determining the target parameter values. Combining regression and classification methods to determine the recommendation parameters can more accurately fit the correlation between the multimodal features of the multimedia content and the recommendation parameters, thereby improving the accuracy of the recommendation parameter determination.

[0056] Please see Figure 3 , Figure 3This is a flowchart illustrating a multimedia content processing method provided in an embodiment of this application. This method can be executed by the aforementioned electronic device. Figure 3 As shown, the process flow of the multimedia content processing method in this embodiment may include the following:

[0057] S201. Obtain multimedia information associated with the multimedia content. The specific implementation of step S201 can be found in the relevant descriptions of the above embodiments, and will not be repeated here.

[0058] S202. Extract the image modal features of the first multimedia information through the image extraction model, and extract the text modal features of the second multimedia information through the text extraction model. Perform feature fusion processing on the image modal features and text modal features to obtain the multimodal features of the multimedia content.

[0059] The extraction of image modal features using an image extraction model can be achieved by: segmenting the first multimedia information into multiple sub-images; obtaining image coding features (i.e., image coding vectors) for each sub-image; and sequentially inputting these features into an image extraction model. The model then performs feature extraction on these features to obtain image extraction features (i.e., image extraction vectors) for each sub-image. Based on these image extraction features, the image modal features of the first multimedia information are determined. Alternatively, the image extraction features of each sub-image can be used individually, or the sum of the image extraction features of all sub-images can be used as the final image extraction feature.

[0060] The image segmentation process for the first multimedia information can be performed by dividing the first multimedia information according to the image pixel granularity, or by dividing the first multimedia information into a target number of segmented images, and then using these target number of segmented images as multiple sub-images. Alternatively, the image segmentation process can be performed on the first multimedia information to obtain multiple segmented images of target sizes, and then using these multiple target size segmented images as multiple sub-images. No limitation is imposed here.

[0061] The text modal features extracted using a text extraction model can be achieved by segmenting the second multimedia information into multiple sub-texts, obtaining text encoding features (i.e., text encoding vectors) for each sub-text, and then sequentially inputting these features into a text extraction model. The model then performs feature extraction on these sub-texts to obtain text extraction features (i.e., text encoding vectors) for each sub-text. Based on these text extraction features, the text modal features of the second multimedia information are determined. Alternatively, the text extraction features of each sub-text can be used individually, or the sum of the text extraction features of all sub-texts can be used as the final text extraction feature.

[0062] The text segmentation process for the second multimedia information can be performed by segmenting the second multimedia information into multiple text segments, and then using these multiple text segments as multiple sub-texts.

[0063] Furthermore, the image extraction model and the text extraction model can be independent models trained separately, or they can be sub-models within a feature extraction model trained simultaneously. For example, the image extraction model can be the encoder in a Vision Transformer (a neural network model for extracting visual features), and the text extraction model can be the encoder in a Text Transformer (a neural network model for extracting text features).

[0064] One method for simultaneously training the image extraction model and the text extraction model is to acquire sample multimedia information associated with multiple sample multimedia content. The sample multimedia information associated with any sample multimedia content includes first sample multimedia information in the image modality and second sample multimedia information in the text modality. The initial image extraction model is called to extract the sample image modal features of the first sample multimedia information associated with each sample multimedia content, and the initial text extraction model is called to extract the sample image modal features of the second sample multimedia information associated with each sample multimedia content. Positive sample modal feature pairs and negative sample modal feature pairs are constructed using the sample image modal features and the corresponding sample text modal features for each sample multimedia content. A positive sample modal feature pair contains one sample image modal feature and one sample text modal feature, and the positive sample modal feature pair... The sample multimedia content associated with the included sample image modal features is the same as the sample multimedia content associated with the included sample text modal features of the positive sample modal feature pair; the negative sample modal feature pair includes one sample image modal feature and one sample text modal feature, and the sample multimedia content associated with the included sample image modal features of the negative sample modal feature pair is different from the sample multimedia content associated with the included sample text modal features of the positive sample modal feature pair; the similarity of positive sample features between the included sample image modal features and the included sample text modal features of the positive sample modal feature pair is determined, and the similarity of negative sample features between the included sample image modal features and the included sample text modal features of the negative sample modal feature pair is determined; the initial image extraction model and the initial text extraction model are trained using the positive sample feature similarity and the negative sample feature similarity to obtain the trained image extraction model and text extraction model.

[0065] It is understandable that the first and second multimedia information in a multimedia content are mutually matching media information. Therefore, during model training, text features and image features can be aligned based on contrastive learning, which makes the text features and visual features highly matched in the feature space. This allows the finally determined multimodal features to better represent the feature information in various modalities of multimedia information.

[0066] It can be understood that positive sample modal features are constructed from sample multimedia information associated with the same sample multimedia content, while negative sample modal features are constructed from sample multimedia information associated with different sample multimedia content. For example, if the sample multimedia content is content 1-3, the sample multimedia information 1 associated with multimedia content 1 includes first sample multimedia information 1 and second sample multimedia information 1; the sample multimedia information 2 associated with multimedia content 2 includes first sample multimedia information 2 and second sample multimedia information 2; and the sample multimedia information 3 associated with multimedia content 3 includes first sample multimedia information 3 and second sample multimedia information 3. Positive sample modal feature pairs can be as follows: Feature pair 1 includes: sample image modal features corresponding to first sample multimedia information 1 and sample text modal features corresponding to second sample multimedia information 1; sample image modal features corresponding to first sample multimedia information 2 and sample text modal features corresponding to second sample multimedia information 2; and sample image modal features corresponding to first sample multimedia information 3 and sample text modal features corresponding to second sample multimedia information 3. The negative sample modal feature pairs can be as follows: Feature pair 1 includes: sample image modal features corresponding to the first sample multimedia information 1 and sample text modal features corresponding to the second sample multimedia information 2, sample image modal features corresponding to the first sample multimedia information 1 and sample text modal features corresponding to the second sample multimedia information 3, sample image modal features corresponding to the first sample multimedia information 2 and sample text modal features corresponding to the second sample multimedia information 1, sample image modal features corresponding to the first sample multimedia information 2 and sample text modal features corresponding to the second sample multimedia information 3, sample image modal features corresponding to the first sample multimedia information 3 and sample text modal features corresponding to the second sample multimedia information 1, and sample image modal features corresponding to the first sample multimedia information 3 and sample text modal features corresponding to the second sample multimedia information 2.

[0067] It is understandable that the training of the initial image extraction model and the initial text extraction model can be carried out iteratively in the direction of increasing the similarity of positive sample features and decreasing the similarity of negative sample features to obtain the image extraction model and the text extraction model.

[0068] For details on the specific methods of feature fusion processing of image modal features and text modal features, please refer to the relevant descriptions in the above embodiments.

[0069] S203. The multimodal features are processed using a parameter processing model to obtain the initial parameter values ​​of the multimedia content under the recommended parameters. The specific implementation of step S203 can be found in the relevant descriptions of the above embodiments, and will not be repeated here.

[0070] S204. Determine the parameter distribution probability of the multimedia content under the recommended parameters being within the N parameter intervals based on the initial parameter values ​​and the distribution influence of the N parameter intervals.

[0071] The method for determining the parameter distribution probability can be as follows: obtain the start and end positions of the target parameter interval; the target parameter interval is any one of N parameter intervals; obtain the first distribution influence associated with the start position of the interval and the second distribution influence associated with the end position of the interval; determine the parameter distribution probability of the associated interval where the parameter value is at the start position of the interval under the recommended parameters based on the initial parameter value and the first distribution influence, and determine the parameter distribution probability of the associated region where the parameter value is at the end position of the interval under the recommended parameters based on the initial parameter value and the second distribution influence; determine the parameter distribution probability corresponding to the target parameter interval through the parameter distribution probability corresponding to the associated interval at the start position of the interval and the parameter distribution probability corresponding to the associated region at the end position of the interval.

[0072] It can be understood that the starting position of each parameter interval is associated with a first distribution influence, and the corresponding ending position is associated with a second distribution influence. These first and second distribution influences are model parameters in the parameter processing model, used to fit the parameter distribution probabilities of the recommended parameters under different initial parameter values.

[0073] The probability distribution of parameters, determined based on the initial parameter values, the influence of the first distribution, and the influence of the second distribution, can be found in the following formula:

[0074] P(Y i <x<Y j )=P(θ j -x)-P(θ i -x)=σ(θ j -x)-σ(θ i -x)

[0075]

[0076] Where Yi represents the starting position of the parameter interval, and Yj represents the ending position of the parameter interval; θ i The first distribution influence degree associated with the starting position of the interval, θ j The second distribution influence degree associated with the termination position of the interval is represented by σ(z); σ(z) represents the sigmoid function (a normalized activation function).

[0077] For example, if the starting position of the parameter interval is 0.1 and the ending position of the parameter interval is 0.3, the influence of the first distribution associated with the starting position of the interval is 3, and the influence of the second distribution associated with the ending position of the interval is 6, then the parameter distribution probability of the parameter interval 0.1 to 0.3 is: σ(0.3-6)-σ(0.1-3).

[0078] It can be understood that this application's embodiments transform the real parameter values ​​output by the parameter processing model into parameter distribution probabilities belonging to each parameter interval. The actual parameter values ​​of the recommended parameters within each parameter interval can be determined using the initial parameter values, thereby determining the final target parameter values. This can be understood as determining the parameter distribution probabilities associated with the initial parameter values ​​using the initial parameter values, and adjusting the initial parameter values ​​using these parameter distribution probabilities to obtain the final target parameter values. Wherein, the aforementioned σ(θ) j -x) represents the parameter distribution probability corresponding to the associated interval at the termination position of the target parameter interval. The associated interval at the termination position is the sum of the intervals formed by the parameter intervals preceding the termination position among the N parameter intervals. σ(θ) i -x) represents the parameter distribution probability corresponding to the associated interval of the starting position of the target parameter interval. The associated interval of the starting position is the sum of the intervals formed by the parameter intervals that are located before the starting position of the N parameter intervals.

[0079] For example, the N parameter intervals are 0-0.1, 0.1-0.2, ..., 0.8-0.9, and 0.9-1 respectively; the interval ends at 0.9 and begins at 0.8. Therefore, the associated interval at the end of the interval is the sum of the intervals formed by the parameter intervals before 0.9 among the N parameter intervals, which is 0-0.9. Thus, the parameter distribution probability corresponding to the associated interval at the end of the interval is the parameter distribution probability that the recommended parameter value falls within the interval 0-0.9. Similarly, the associated interval at the beginning of the interval is the sum of the intervals formed by the parameter intervals before 0.8 among the N parameter intervals, which is 0-0.8. Thus, the parameter distribution probability corresponding to the associated interval at the beginning of the interval is the parameter distribution probability that the recommended parameter value falls within the interval 0-0.8.

[0080] In other words, the probability that the recommended parameter value falls within the target parameter range can be determined by the parameter distribution probability of the recommended parameter value falling within the associated interval at the end of the interval and the parameter distribution probability of the recommended parameter value falling within the associated interval at the beginning and end of the interval. It can be understood that the difference between the associated interval at the end of the interval and the associated interval at the beginning and end of the interval is the target parameter range; therefore, the difference between the parameter distribution probability of the associated interval at the end of the interval and the parameter distribution probability of the associated interval at the beginning and end of the interval is the probability of falling within the target parameter range.

[0081] Therefore, the recommended parameter values ​​fall within a parameter range (Y). i <x<Y j The probability within a certain range can be expressed as P(Y). i <x<Y j Assuming P is the cumulative distribution function, and using the sigmoid function as the cumulative distribution function, the probability of each interval can be written as σ(θ). j -x)-σ(θ i -x), and thus the negative log-likelihood loss function can be used to optimize this parameter processing model. The advantage of this is that in normal classification tasks, different classes are encoded using hot-only methods, and the distance between any two classes is the same. This means that regardless of the deviation of the model's predicted class from the true value, the loss will be the same. However, in ordered regression, the distance between different classes is affected by their inter-class relationships. Therefore, the greater the deviation between the predicted and actual values, the larger the loss function, which is more conducive to training and optimization.

[0082] Therefore, the parameter processing model can be used to determine initial parameter values ​​based on multimodal features, and then determine the parameter distribution probability of the recommended parameters based on the initial parameter values, thereby determining the target parameter values. The training process can be as follows: Obtain the initial parameter processing model; the model parameters of the initial parameter processing model include the first initial distribution influence associated with the start position of each parameter interval and the second initial distribution influence associated with the corresponding end position of the interval; obtain sample multimodal features; the sample multimodal features carry label parameter values; input the sample multimodal features into the initial parameter processing model to obtain the sample initial parameter values ​​of the sample multimodal features under the recommended parameters, and determine the sample target parameter values ​​of the sample multimodal features under the recommended parameters based on the sample initial parameter values, the first initial distribution influence associated with each parameter interval, and the corresponding second initial distribution influence; train the initial parameter processing model using the label parameter values ​​and the sample target parameter values ​​to obtain the trained parameter processing model; the parameter processing model is used to determine the target parameter values ​​of multimedia content under the recommended parameters; the model parameters of the parameter processing model include the first distribution influence associated with the start position of each parameter interval and the second distribution influence associated with the corresponding end position of the interval.

[0083] Specifically, the initial parameter values ​​of the samples can be determined through the initial parameter prediction network in the initial parameter processing model, and the parameter distribution probability can be determined through the initial parameter processing network itself. That is, the initial parameter processing network includes the influence of each initial distribution, and the influence of each distribution can be obtained by training the initial parameter processing network. In other words, in the initial parameter processing network, the sample parameter distribution probability corresponding to each parameter interval is determined based on the initial parameter values, the first initial distribution influence corresponding to each parameter interval, and the corresponding second initial distribution influence. Then, the sample target parameter value under the recommended parameters is determined based on the sample parameter distribution probability corresponding to each parameter interval. The principle and process of determining the sample parameter distribution probability can be found in the relevant description above.

[0084] It is understandable that the parameter processing model and the feature extraction model can be trained independently or simultaneously. Furthermore, training the parameter processing model can involve iteratively training the initial parameter processing model in a direction that reduces the difference between the label parameter values ​​and the sample target parameter values, thus obtaining the final parameter processing model.

[0085] S205. Determine the target parameter value of multimedia content under the recommended parameters by using the parameter distribution probability corresponding to N parameter intervals.

[0086] The target parameter value is used to determine the recommendation strategy for multimedia content. This target parameter value can be determined within the trained parameter processing model. For example, the parameter processing model determines the parameter distribution probability based on the first distribution influence associated with the start position of each parameter interval and the second distribution influence associated with the end position of the corresponding interval, along with the initial parameter value. The target parameter value is then determined based on the parameter distribution probability. For a detailed description of how to determine the target parameter value, please refer to the relevant description in the above embodiments.

[0087] For example, such as Figure 4 As shown, Figure 4This is a schematic diagram illustrating a process for determining a target parameter value according to an embodiment of this application. The process involves acquiring multimedia information 41 associated with multimedia content 40, which includes first multimedia information 41a in an image modality and second multimedia information 41b in a text modality. The first multimedia information 41a and the second multimedia information 41b are input into a feature extraction model 42. An image extraction model 43a in the feature extraction model 42 extracts image modality features 44a of the first multimedia information 41a, and a text extraction model 43b in the feature extraction model 42 extracts text modality features 44b of the second multimedia information 41b. Finally, a feature fusion layer 45 fuses the image modality features 44a and the text modality features. 44b performs feature fusion processing to obtain multimodal features 46 of multimedia content 40. The multimodal features 46 are input into parameter processing model 47. The parameter prediction network 48a in parameter processing model 47 performs feature processing on multimodal features 46 to obtain initial parameter values ​​49. The parameter processing network 48b in parameter processing model 47 performs parameter processing on the initial parameter values ​​49 based on the relevant distribution influence degree 410, so as to determine the parameter distribution probability 411 of the parameter values ​​of multimedia content 40 under recommended parameters in N parameter intervals based on the initial parameter values ​​49. Then, the target parameter values ​​412 of multimedia content 40 under recommended parameters are determined through the parameter distribution probability 411 of the N parameter intervals.

[0088] In this embodiment, multimedia information associated with multimedia content can be obtained; image modal features of the first multimedia information and text modal features of the second multimedia information can be extracted; feature fusion processing is performed on the image modal features and text modal features to obtain multimodal features of the multimedia content; feature processing is performed on the multimodal features to obtain the initial parameter values ​​of the multimedia content under the recommendation parameters; the initial parameter values ​​can be understood as an intermediate parameter value determined by regression; the parameter distribution probability of the multimedia content under the recommendation parameters in N parameter intervals is determined based on the initial parameter values; this can be understood as determining the probability distribution of the recommendation parameters by classification, that is, converting the initial parameter values, which are intermediate parameter values, into distribution probabilities in N parameter intervals; the target parameter values ​​of the multimedia content under the recommendation parameters are determined by the parameter distribution probabilities corresponding to the N parameter intervals; the target parameter values ​​are used to determine the recommendation strategy for the multimedia content; therefore, the specific recommendation parameters can be determined by first determining the initial parameter values, converting them into parameter probability distributions, and then determining the target parameter values. Combining regression and classification methods to determine the recommendation parameters can more accurately fit the correlation between the multimodal features of the multimedia content and the recommendation parameters, thereby improving the accuracy of the recommendation parameter determination.

[0089] Please see Figure 5 , Figure 5This is a schematic diagram of a multimedia content processing device provided in this application. It should be noted that... Figure 5 The multimedia content processing apparatus shown is used to execute this application. Figure 2 and Figure 3 The methods in the illustrated embodiments are shown only in the parts relevant to the embodiments of this application for ease of explanation; specific technical details are not disclosed. Reference to this application is required. Figure 2 and Figure 3 The illustrated embodiment. The multimedia content processing apparatus 500 may include: an acquisition module 501 and a processing module 502. Wherein:

[0090] The acquisition module 501 is used to acquire multimedia information associated with multimedia content; the media information includes first multimedia information of the multimedia content in the image mode and second multimedia information in the text mode;

[0091] The processing module 502 is used to extract the image modal features of the first multimedia information and the text modal features of the second multimedia information, and to perform feature fusion processing on the image modal features and the text modal features to obtain the multimodal features of the multimedia content;

[0092] The processing module 502 is also used to perform feature processing on the multimodal features to obtain the initial parameter values ​​of the multimedia content under the recommendation parameters, and to determine the parameter distribution probability of the multimedia content under the recommendation parameters falling within N parameter intervals based on the initial parameter values; one parameter interval corresponds to one parameter distribution probability; N is a positive integer greater than 1;

[0093] The processing module 502 is also used to determine the target parameter value of the multimedia content under the recommended parameters by using the parameter distribution probability corresponding to the N parameter intervals.

[0094] Specifically, when the processing module 502 is used to extract the image modal features of the first multimedia information, it is used for:

[0095] The first multimedia information is subjected to image segmentation processing to obtain multiple sub-images corresponding to the first multimedia information;

[0096] The image coding features of multiple sub-images are obtained separately, and the image coding features of multiple sub-images are sequentially input into the image extraction model. The image extraction model performs feature extraction processing on the image coding features of multiple sub-images to obtain the image extraction features of each sub-image.

[0097] The image modality features of the first multimedia information are determined based on the image extraction features of each sub-image.

[0098] Specifically, when extracting the text modal features of the second multimedia information, the processing module 502 is used for:

[0099] The second multimedia information is processed by text segmentation to obtain multiple sub-texts corresponding to the second multimedia information;

[0100] The text encoding features of multiple sub-texts are obtained separately, and the text encoding features of multiple sub-texts are sequentially input into the text extraction model. The text extraction model performs feature extraction processing on the text encoding features of multiple sub-texts to obtain the text extraction features of each sub-text.

[0101] The text modal features of the second multimedia information are determined based on the text extraction features of each subtext.

[0102] The processing module 502 is also used for:

[0103] Obtain sample multimedia information associated with multiple sample multimedia contents; the sample multimedia information associated with any sample multimedia content includes the first sample multimedia information of any sample multimedia content in the image modality and the second sample multimedia information in the text modality;

[0104] The initial image extraction model is invoked to extract the sample image modal features of the first sample multimedia information associated with the multimedia content of each sample, and the initial text extraction model is invoked to extract the sample image modal features of the second sample multimedia information associated with the multimedia content of each sample.

[0105] Positive and negative sample modal feature pairs are constructed using the sample image modal features and the corresponding sample text modal features for each sample multimedia content. A positive sample modal feature pair contains one sample image modal feature and one sample text modal feature, and the sample multimedia content associated with the sample image modal feature in the positive sample modal feature pair is the same as the sample multimedia content associated with the sample text modal feature in the positive sample modal feature pair. A negative sample modal feature pair contains one sample image modal feature and one sample text modal feature, and the sample multimedia content associated with the sample image modal feature in the negative sample modal feature pair is different from the sample multimedia content associated with the sample text modal feature in the positive sample modal feature pair.

[0106] Determine the positive sample feature similarity between the positive sample modal feature pairs and the included sample image modal features and sample text modal features, and determine the negative sample feature similarity between the negative sample modal feature pairs and the included sample image modal features and sample text modal features;

[0107] The initial image extraction model and the initial text extraction model are trained by using positive sample feature similarity and negative sample feature similarity to obtain the trained image extraction model and text extraction model.

[0108] Specifically, when processing module 502 determines the probability distribution of the parameter values ​​of multimedia content within N parameter intervals under the recommended parameters based on the initial parameter values, it is used for:

[0109] Obtain the start and end positions of the target parameter interval; the target parameter interval can be any one of N parameter intervals.

[0110] Obtain the first distribution influence associated with the start position of the interval and the second distribution influence associated with the end position of the interval;

[0111] The parameter distribution probability of the associated interval where the parameter value is at the beginning of the interval is determined based on the initial parameter value and the first distribution influence, and the parameter distribution probability of the associated region where the parameter value is at the end of the interval is determined based on the initial parameter value and the second distribution influence.

[0112] The parameter distribution probability of the target parameter interval is determined by the parameter distribution probability of the associated interval at the start position of the interval and the parameter distribution probability of the associated region at the end position of the interval.

[0113] The processing module 502 is also used for:

[0114] Obtain the initial parameter processing model; the model parameters of the initial parameter processing model include the first initial distribution influence degree associated with the start position of each parameter interval and the second initial distribution influence degree associated with the end position of the corresponding interval.

[0115] Obtain multimodal features of the sample; the multimodal features of the sample carry label parameter values;

[0116] The sample multimodal features are input into the initial parameter processing model to obtain the sample initial parameter values ​​of the sample multimodal features under the recommended parameters. The sample target parameter values ​​of the sample multimodal features under the recommended parameters are determined based on the sample initial parameter values, the first initial distribution influence degree corresponding to each parameter interval, and the corresponding second initial distribution influence degree.

[0117] The initial parameter processing model is trained by using the label parameter values ​​and the sample target parameter values ​​to obtain the trained parameter processing model. The parameter processing model is used to determine the target parameter values ​​of multimedia content under the recommendation parameters. The model parameters of the parameter processing model include the first distribution influence degree associated with the start position of each parameter interval and the second distribution influence degree associated with the end position of the corresponding interval.

[0118] Specifically, when processing module 502 determines the target parameter value of multimedia content under the recommended parameters based on the parameter distribution probabilities corresponding to N parameter intervals, it is used for:

[0119] Obtain the reference interval value for each parameter interval;

[0120] The target parameter value is obtained by weighted summing of the reference interval values ​​of the N reference intervals using the parameter distribution probabilities corresponding to the N parameter intervals.

[0121] The specific implementation methods of the acquisition module and the processing module can be found in the description of the above embodiments, and will not be repeated here. It should be understood that the beneficial effects obtained by using the same method will also not be repeated here.

[0122] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 600 includes at least one processor 601 and a memory 602. Optionally, the electronic device may also include a network interface. The processor 601, memory 602, and network interface can exchange data. The network interface, controlled by the processor 601, is used to send and receive messages. The memory 602 stores computer programs, including program instructions. The processor 601 executes the program instructions stored in the memory 602. The processor 601 is configured to invoke the program instructions to execute the aforementioned method.

[0123] Memory 602 may include volatile memory, such as random-access memory (RAM); memory 602 may also include non-volatile memory, such as flash memory, solid-state drive (SSD), etc.; memory 602 may also include combinations of the above types of memory.

[0124] Processor 601 may be a central processing unit (CPU). In one embodiment, processor 601 may also be a graphics processing unit (GPU). Processor 601 may also be a combination of a CPU and a GPU.

[0125] In one possible implementation, memory 602 is used to store program instructions, which processor 601 can invoke to perform the following steps:

[0126] Obtain multimedia information associated with multimedia content; the media information includes first multimedia information in the image modality and second multimedia information in the text modality;

[0127] The image modal features of the first multimedia information and the text modal features of the second multimedia information are extracted. The image modal features and the text modal features are then fused to obtain the multimodal features of the multimedia content.

[0128] Multimodal features are processed to obtain the initial parameter values ​​of multimedia content under the recommendation parameters, and the parameter distribution probability of the multimedia content under the recommendation parameters in N parameter intervals is determined based on the initial parameter values; one parameter interval corresponds to one parameter distribution probability; N is a positive integer greater than 1.

[0129] The target parameter values ​​for multimedia content under the recommended parameters are determined by the parameter distribution probabilities corresponding to N parameter intervals.

[0130] Specifically, when extracting image modal features of the first multimedia information, the processor 601 is used for:

[0131] The first multimedia information is subjected to image segmentation processing to obtain multiple sub-images corresponding to the first multimedia information;

[0132] The image coding features of multiple sub-images are obtained separately, and the image coding features of multiple sub-images are sequentially input into the image extraction model. The image extraction model performs feature extraction processing on the image coding features of multiple sub-images to obtain the image extraction features of each sub-image.

[0133] The image modality features of the first multimedia information are determined based on the image extraction features of each sub-image.

[0134] Specifically, when extracting the text modal features of the second multimedia information, the processor 601 is used for:

[0135] The second multimedia information is processed by text segmentation to obtain multiple sub-texts corresponding to the second multimedia information;

[0136] The text encoding features of multiple sub-texts are obtained separately, and the text encoding features of multiple sub-texts are sequentially input into the text extraction model. The text extraction model performs feature extraction processing on the text encoding features of multiple sub-texts to obtain the text extraction features of each sub-text.

[0137] The text modal features of the second multimedia information are determined based on the text extraction features of each subtext.

[0138] The processor 601 is also used for:

[0139] Obtain sample multimedia information associated with multiple sample multimedia contents; the sample multimedia information associated with any sample multimedia content includes the first sample multimedia information of any sample multimedia content in the image modality and the second sample multimedia information in the text modality;

[0140] The initial image extraction model is invoked to extract the sample image modal features of the first sample multimedia information associated with the multimedia content of each sample, and the initial text extraction model is invoked to extract the sample image modal features of the second sample multimedia information associated with the multimedia content of each sample.

[0141] Positive and negative sample modal feature pairs are constructed using the sample image modal features and the corresponding sample text modal features for each sample multimedia content. A positive sample modal feature pair contains one sample image modal feature and one sample text modal feature, and the sample multimedia content associated with the sample image modal feature in the positive sample modal feature pair is the same as the sample multimedia content associated with the sample text modal feature in the positive sample modal feature pair. A negative sample modal feature pair contains one sample image modal feature and one sample text modal feature, and the sample multimedia content associated with the sample image modal feature in the negative sample modal feature pair is different from the sample multimedia content associated with the sample text modal feature in the positive sample modal feature pair.

[0142] Determine the positive sample feature similarity between the positive sample modal feature pairs and the included sample image modal features and sample text modal features, and determine the negative sample feature similarity between the negative sample modal feature pairs and the included sample image modal features and sample text modal features;

[0143] The initial image extraction model and the initial text extraction model are trained by using positive sample feature similarity and negative sample feature similarity to obtain the trained image extraction model and text extraction model.

[0144] Specifically, when determining the parameter distribution probability of multimedia content falling within N parameter intervals under recommended parameters based on initial parameter values, processor 601 is used for:

[0145] Obtain the start and end positions of the target parameter interval; the target parameter interval can be any one of N parameter intervals.

[0146] Obtain the first distribution influence associated with the start position of the interval and the second distribution influence associated with the end position of the interval;

[0147] The parameter distribution probability of the associated interval where the parameter value is at the beginning of the interval is determined based on the initial parameter value and the first distribution influence, and the parameter distribution probability of the associated region where the parameter value is at the end of the interval is determined based on the initial parameter value and the second distribution influence.

[0148] The parameter distribution probability of the target parameter interval is determined by the parameter distribution probability of the associated interval at the start position of the interval and the parameter distribution probability of the associated region at the end position of the interval.

[0149] The processor 601 is also used for:

[0150] Obtain the initial parameter processing model; the model parameters of the initial parameter processing model include the first initial distribution influence degree associated with the start position of each parameter interval and the second initial distribution influence degree associated with the end position of the corresponding interval.

[0151] Obtain multimodal features of the sample; the multimodal features of the sample carry label parameter values;

[0152] The sample multimodal features are input into the initial parameter processing model to obtain the sample initial parameter values ​​of the sample multimodal features under the recommended parameters. The sample target parameter values ​​of the sample multimodal features under the recommended parameters are determined based on the sample initial parameter values, the first initial distribution influence degree corresponding to each parameter interval, and the corresponding second initial distribution influence degree.

[0153] The initial parameter processing model is trained by using the label parameter values ​​and the sample target parameter values ​​to obtain the trained parameter processing model. The parameter processing model is used to determine the target parameter values ​​of multimedia content under the recommendation parameters. The model parameters of the parameter processing model include the first distribution influence degree associated with the start position of each parameter interval and the second distribution influence degree associated with the end position of the corresponding interval.

[0154] Specifically, when the processor 601 determines the target parameter value of multimedia content under the recommended parameters based on the parameter distribution probabilities corresponding to N parameter intervals, it is used for:

[0155] Obtain the reference interval value for each parameter interval;

[0156] The target parameter value is obtained by weighted summing of the reference interval values ​​of the N reference intervals using the parameter distribution probabilities corresponding to the N parameter intervals.

[0157] In specific implementations, the devices, processors, memory, etc., described in the embodiments of this application can execute the implementation methods described in the above method embodiments, or they can execute the implementation methods described in the embodiments of this application, which will not be repeated here.

[0158] This application also provides a computer-readable storage medium storing a computer program. The computer program includes program instructions, which, when executed by a processor, enable the processor to perform some or all of the steps described in the above method embodiments. Optionally, the computer storage medium can be volatile or non-volatile. The computer-readable storage medium may primarily include a program storage area and a data storage area. The program storage area may store an operating system, at least one application program required for a given function, etc.; the data storage area may store data created based on the use of blockchain nodes, etc.

[0159] This application provides a computer program product, which may include a computer program. When the computer program is executed by a processor, it can implement some or all of the steps in the above method, which will not be elaborated here.

[0160] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0161] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer storage medium, which can be a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0162] The above-disclosed embodiments are merely some of the embodiments of this application, and should not be construed as limiting the scope of this application. Those skilled in the art can understand that all or part of the processes for implementing the above embodiments, and equivalent changes made in accordance with the claims of this application, still fall within the scope of this application.

Claims

1. A method for processing multimedia content, characterized in that, The method includes: Obtain multimedia information associated with multimedia content; the multimedia information includes first multimedia information of the multimedia content in image mode and second multimedia information in text mode; Extract the image modal features of the first multimedia information and the text modal features of the second multimedia information, and perform feature fusion processing on the image modal features and the text modal features to obtain the multimodal features of the multimedia content; The multimodal features are processed to obtain the initial parameter values ​​of the multimedia content under the recommendation parameters, and the parameter distribution probability of the multimedia content under the recommendation parameters in N parameter intervals is determined based on the initial parameter values; one parameter interval corresponds to one parameter distribution probability; N is a positive integer greater than 1. The target parameter value of the multimedia content under the recommended parameters is determined by the parameter distribution probability corresponding to the N parameter intervals.

2. The method according to claim 1, characterized in that, The extraction of image modal features of the first multimedia information includes: The first multimedia information is subjected to image segmentation processing to obtain multiple sub-images corresponding to the first multimedia information; The image coding features of the multiple sub-images are obtained respectively, and the image coding features of the multiple sub-images are sequentially input into the image extraction model. The image extraction model performs feature extraction processing on the image coding features of the multiple sub-images to obtain the image extraction features of each sub-image. The image modal features of the first multimedia information are determined based on the image extraction features of each sub-image.

3. The method according to claim 2, characterized in that, The extraction of text modal features of the second multimedia information includes: The second multimedia information is subjected to text segmentation processing to obtain multiple sub-texts corresponding to the second multimedia information; The text encoding features of the multiple sub-texts are obtained respectively, and the text encoding features of the multiple sub-texts are sequentially input into the text extraction model. The text extraction model performs feature extraction processing on the text encoding features of the multiple sub-texts to obtain the text extraction features of each sub-text. The text modal features of the second multimedia information are determined based on the text extraction features of each sub-text.

4. The method according to claim 3, characterized in that, The method further includes: Obtain sample multimedia information associated with multiple sample multimedia contents; the sample multimedia information associated with any sample multimedia content includes the first sample multimedia information of the any sample multimedia content in the image modality and the second sample multimedia information in the text modality; The initial image extraction model is invoked to extract the sample image modal features of the first sample multimedia information associated with each sample multimedia content, and the initial text extraction model is invoked to extract the sample image modal features of the second sample multimedia information associated with each sample multimedia content. Positive and negative sample modal feature pairs are constructed using the sample image modal features and corresponding sample text modal features corresponding to each sample multimedia content. Each positive sample modal feature pair includes one sample image modal feature and one sample text modal feature, and the sample multimedia content associated with the sample image modal feature in the positive sample modal feature pair is the same as the sample multimedia content associated with the sample text modal feature in the positive sample modal feature pair. Each negative sample modal feature pair includes one sample image modal feature and one sample text modal feature, and the sample multimedia content associated with the sample image modal feature in the negative sample modal feature pair is different from the sample multimedia content associated with the sample text modal feature in the positive sample modal feature pair. Determine the positive sample feature similarity between the positive sample modal feature pairs and the sample image modal features and sample text modal features contained therein; and determine the negative sample feature similarity between the negative sample modal feature pairs and the sample image modal features and sample text modal features contained therein. The initial image extraction model and the initial text extraction model are trained using the positive sample feature similarity and the negative sample feature similarity to obtain the trained image extraction model and the text extraction model.

5. The method according to any one of claims 1-4, characterized in that, The step of determining the parameter distribution probability of the multimedia content falling within N parameter intervals under the recommended parameters based on the initial parameter values ​​includes: Obtain the start and end positions of the target parameter interval; the target parameter interval is any one of the N parameter intervals. Obtain the first distribution influence degree associated with the starting position of the interval and the second distribution influence degree associated with the ending position of the interval; The parameter distribution probability of the associated interval where the parameter value under the recommended parameters is at the beginning position of the interval is determined based on the initial parameter value and the first distribution influence degree, and the parameter distribution probability of the associated region where the parameter value under the recommended parameters is at the end position of the interval is determined based on the initial parameter value and the second distribution influence degree. The parameter distribution probability corresponding to the target parameter interval is determined by the parameter distribution probability corresponding to the associated interval at the starting position of the interval and the parameter distribution probability corresponding to the associated region at the ending position of the interval.

6. The method according to claim 5, characterized in that, The method further includes: Obtain the initial parameter processing model; the model parameters of the initial parameter processing model include the first initial distribution influence degree associated with the start position of the interval corresponding to each parameter interval and the second initial distribution influence degree associated with the end position of the interval. Obtain multimodal features of the sample; the multimodal features of the sample carry label parameter values; The sample multimodal features are input into the initial parameter processing model to obtain the sample initial parameter values ​​of the sample multimodal features under the recommended parameters. The sample target parameter values ​​of the sample multimodal features under the recommended parameters are determined based on the sample initial parameter values, the first initial distribution influence degree corresponding to each parameter interval, and the corresponding second initial distribution influence degree. The initial parameter processing model is trained using the label parameter values ​​and the sample target parameter values ​​to obtain the trained parameter processing model; the parameter processing model is used to determine the target parameter values ​​of the multimedia content under the recommendation parameters; the model parameters of the parameter processing model include the first distribution influence degree associated with the start position of each parameter interval and the second distribution influence degree associated with the end position of the corresponding interval.

7. The method according to any one of claims 1-4, characterized in that, The step of determining the target parameter value of the multimedia content under the recommended parameters by using the parameter distribution probability corresponding to the N parameter intervals includes: Obtain the reference interval value for each parameter interval; The target parameter value is obtained by weighted summing of the reference interval values ​​of the N parameter intervals using the parameter distribution probabilities corresponding to the N parameter intervals.

8. A multimedia content processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire multimedia information associated with multimedia content; the multimedia information includes first multimedia information of the multimedia content in image mode and second multimedia information in text mode; The processing module is used to extract the image modal features of the first multimedia information and the text modal features of the second multimedia information, and to perform feature fusion processing on the image modal features and the text modal features to obtain the multimodal features of the multimedia content; The processing module is further configured to perform feature processing on the multimodal features to obtain the initial parameter values ​​of the multimedia content under the recommendation parameters, and determine the parameter distribution probability of the parameter values ​​of the multimedia content under the recommendation parameters being in N parameter intervals based on the initial parameter values; one parameter interval corresponds to one parameter distribution probability; N is a positive integer greater than 1; The processing module is further configured to determine the target parameter value of the multimedia content under the recommended parameters by using the parameter distribution probability corresponding to the N parameter intervals.

9. An electronic device, characterized in that, The system includes a processor and a memory, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-7.

Images