Multimedia information recommendation model training method, recommendation method and device

Abstract

The application provides a multimedia information recommendation model training method and device and electronic equipment, the method comprises the following steps: extracting a pre-training sample set based on the basic historical data; training a basic recommendation model based on the pre-training sample set to obtain model parameters of the basic recommendation model; obtaining industry historical data in a multimedia information recommendation environment; and determining the model parameters of the multimedia information recommendation model according to the industry historical data, thereby enhancing the accuracy and relevance of multimedia information recommendation and improving the generalization of the multimedia information recommendation model. The embodiments of the application can also be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation and auxiliary driving.

Description

Technical Field

[0001] This invention relates to information processing technology, and more particularly to a training method for a multimedia information recommendation model, a multimedia information recommendation method, an apparatus, and an electronic device. Background Technology

[0002] Artificial Intelligence (AI) is a comprehensive technology within computer science that studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. AI technology is a multidisciplinary field, encompassing a wide range of areas, including natural language processing and machine learning / deep learning. It is believed that with technological advancements, AI will be applied in more fields and play an increasingly important role.

[0003] In traditional technologies, various multimedia information recommendation systems typically use general data recommendation models to ensure recommendation speed when recommending multimedia information to users. However, general data recommendation models are trained on general data from multiple domains, resulting in overly conventional recommendation results that lack industry-specific multimedia information recommendations. Conversely, building a model specifically for a particular industry can lead to overfitting due to a lack of training data, affecting the accuracy of recommendations and severely impacting the user experience. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a multimedia information recommendation model training method, apparatus, electronic device, and storage medium. The technical solution of the embodiments of the present invention is implemented as follows:

[0005] This invention provides a method for training a multimedia information recommendation model, including:

[0006] Acquire basic historical data in a multimedia information recommendation environment;

[0007] Based on the aforementioned historical data, a pre-training sample set is extracted;

[0008] Based on the pre-trained sample set, the basic recommendation model is trained to obtain the model parameters of the basic recommendation model;

[0009] Acquire historical industry data in a multimedia information recommendation environment;

[0010] The parameters of the embedding layer network of the basic recommendation model are extracted from the model parameters of the basic recommendation model, and the parameters of the embedding layer network are transferred to the multimedia information recommendation model, wherein the basic recommendation model and the multimedia information recommendation model have the same model structure.

[0011] Based on the historical industry data, the multimedia information recommendation model is trained to determine the model parameters, so as to adjust the recall strategy of multimedia information in the target industry through the multimedia information recommendation model and recommend multimedia information through the recall strategy.

[0012] This invention also provides a method for obtaining multimedia information to be recommended from a multimedia information data source;

[0013] The multimedia information recommendation model is used to process different multimedia information to be recommended and determine the priority of different multimedia information to be recommended.

[0014] The recall strategy for multimedia information is adjusted according to the priority of different multimedia information to be recommended, and multimedia information is recommended through the recall strategy.

[0015] This invention also provides a multimedia information recommendation model training device, comprising:

[0016] The information transmission module is used to acquire basic historical data in a multimedia information recommendation environment;

[0017] The information processing module is used to extract a pre-training sample set based on the aforementioned basic historical data;

[0018] The information processing module is used to train the basic recommendation model based on the pre-trained sample set to obtain the model parameters of the basic recommendation model.

[0019] The information processing module is used to acquire historical industry data in the multimedia information recommendation environment;

[0020] The information processing module is used to train the multimedia information recommendation model based on the industry historical data, determine the model parameters of the multimedia information recommendation model, so as to adjust the recall strategy of multimedia information in the target industry through the multimedia information recommendation model, and recommend multimedia information through the recall strategy.

[0021] In the above scheme,

[0022] The information processing module is used to adjust the embedding features of the embedding layer network of the basic recommendation model based on the industry historical data to obtain the embedding features of the multimedia information recommendation model.

[0023] The information processing module is used to configure training cycle parameters for the multimedia information recommendation model according to the type of the target industry.

[0024] The information processing module is used to adjust the network parameters of the multimedia information recommendation model by keeping the parameters of the embedding layer network unchanged when the number of training iterations of the multimedia information recommendation model is less than or equal to the training cycle number parameter;

[0025] When the loss function corresponding to the multimedia information recommendation model reaches the corresponding convergence condition, the first model parameters of the multimedia information recommendation model are determined, wherein the model parameters of the multimedia information recommendation model include the first model parameters.

[0026] In the above scheme,

[0027] The information processing module is used to train the basic recommendation model using the embedding features of the multimedia information recommendation model when the number of training iterations of the multimedia information recommendation model is greater than the training cycle number parameter, so as to adjust the first model parameter of the multimedia information recommendation model and obtain the model parameter of the multimedia information recommendation model.

[0028] In the above scheme,

[0029] The information processing module is used to determine the convergence speed of the loss function corresponding to the multimedia information recommendation model;

[0030] The information processing module is used to dynamically adjust the training cycle parameters according to the convergence speed, so as to match the training cycle parameters with the convergence speed.

[0031] In the above scheme,

[0032] The information processing module is used to determine invalid samples in the pre-training sample set based on the industry historical data.

[0033] The information processing module is used to determine industry feature samples that match the industry historical data based on the industry historical data.

[0034] The information processing module is used to adjust the pre-training sample set using the invalid samples and the industry feature samples to obtain an industry training sample set.

[0035] In the above scheme,

[0036] The information processing module is used to configure training cycle parameters for the multimedia information recommendation model according to the type of the target industry.

[0037] The information processing module is used to extract the embedding features of the embedding layer network of the basic recommendation model;

[0038] The information processing module is used to keep the parameters of the embedding layer network unchanged when the number of training iterations of the multimedia information recommendation model is less than or equal to the training cycle number parameter, and to adjust the embedding features through the industry training sample set to obtain the second model parameters of the multimedia information recommendation model, wherein the model parameters of the multimedia information recommendation model include the second model parameters;

[0039] The information processing module is used to train the basic recommendation model using the industry training sample set when the number of training iterations of the multimedia information recommendation model is greater than the training cycle parameter, so as to adjust the second model parameter of the multimedia information recommendation model and obtain the model parameter of the multimedia information recommendation model.

[0040] In the above scheme,

[0041] The information processing module is used when the multimedia information is a video advertisement.

[0042] The information processing module is used to send the exposure parameters of the video advertisement during playback to the detection server, so that the detection server can obtain the exposure parameters of the video advertisement.

[0043] The information processing module is used to use the exposure parameter as an evaluation parameter for the playback effect of the multimedia information, and to find the target exposure parameter based on the adjustment result of the recall strategy.

[0044] In the above scheme,

[0045] The information processing module is used to obtain the historical browsing information of the audience corresponding to the target industry;

[0046] The information processing module is used to determine the multimedia information exposure history corresponding to the historical browsing information based on the historical browsing information of the audience corresponding to the target industry.

[0047] The information processing module is used to dynamically adjust the recall strategy of the multimedia information based on the exposure history of the multimedia information corresponding to the historical browsing information.

[0048] In the above scheme,

[0049] The information processing module is used to determine the category of multimedia information to be recommended based on the multimedia information recommendation environment.

[0050] The information processing module is used to respond to the category of the multimedia information to be recommended, trigger a matching multimedia information data source, so as to adjust the multimedia information to be recommended by using a multimedia information data source that matches the category of the multimedia information to be recommended.

[0051] This invention also provides a multimedia information recommendation model training device, comprising:

[0052] The data transmission module is used to acquire multimedia information to be recommended from the multimedia information data source;

[0053] The data processing module is used to process different multimedia information to be recommended through a multimedia information recommendation model and determine the priority of different multimedia information to be recommended.

[0054] The data processing module is used to adjust the recall strategy for multimedia information according to the priority of different multimedia information to be recommended, and to recommend multimedia information through the recall strategy.

[0055] This invention also provides an electronic device, the electronic device comprising:

[0056] Memory, used to store executable instructions;

[0057] When the processor executes the executable instructions stored in the memory, it implements the aforementioned multimedia information recommendation model training method or the aforementioned multimedia information recommendation method.

[0058] This invention also provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement the aforementioned multimedia information recommendation model training method or the aforementioned multimedia information recommendation method.

[0059] The embodiments of the present invention have the following beneficial effects:

[0060] This invention acquires basic historical data from a multimedia information recommendation environment; extracts a pre-training sample set based on the basic historical data; trains a basic recommendation model based on the pre-training sample set to obtain the model parameters of the basic recommendation model; acquires industry historical data from the multimedia information recommendation environment; extracts the parameters of the embedding layer network of the basic recommendation model from the model parameters of the basic recommendation model, and transfers the parameters of the embedding layer network to the multimedia information recommendation model, wherein the basic recommendation model and the multimedia information recommendation model have the same model structure; trains the multimedia information recommendation model according to the industry historical data to determine the model parameters of the multimedia information recommendation model, so as to adjust the recall strategy of multimedia information in the target industry through the multimedia information recommendation model, and perform multimedia information recommendation through the recall strategy. Therefore, the multimedia information recommendation model can recommend multimedia information to users in different industries in the usage environment, while enhancing the accuracy and relevance of multimedia information recommendations, effectively improving the quality of multimedia information recommendations, and completing model training with fewer samples, reducing overfitting of the multimedia information recommendation model, improving the generalization of the multimedia information recommendation model, and enhancing the user experience. Attached Figure Description

[0061] Figure 1 This is a schematic diagram illustrating a usage scenario of the multimedia information recommendation model training method provided in an embodiment of the present invention.

[0062] Figure 2 A schematic diagram of the composition structure of the multimedia information recommendation model training device provided in an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram of the transfer training process in an embodiment of the present invention.

[0064] Figure 4 A schematic diagram of an optional process for training a multimedia information recommendation model provided in an embodiment of the present invention;

[0065] Figure 5 A schematic diagram of an optional process for training a multimedia information recommendation model provided in an embodiment of the present invention;

[0066] Figure 6 As described in the embodiments of the present invention Figure 5 The diagram shows the process of obtaining the peripheral and central features.

[0067] Figure 7 A schematic diagram of an optional process for training a multimedia information recommendation model provided in an embodiment of the present invention;

[0068] Figure 8This is a schematic diagram illustrating the application environment of the multimedia information recommendation method based on the multimedia information recommendation model in this embodiment of the invention;

[0069] Figure 9 This is a schematic diagram of the multimedia information recommendation method in an embodiment of the present invention;

[0070] Figure 10 A schematic diagram of an optional process for training a multimedia information recommendation model provided in an embodiment of the present invention;

[0071] Figure 11 This is a schematic diagram illustrating an optional multimedia information recommendation in an embodiment of the present invention;

[0072] Figure 12 This is a schematic diagram illustrating an optional multimedia information recommendation in an embodiment of the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0075] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0076] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention will be explained, and the nouns and terms involved in the embodiments of the present invention shall be interpreted as follows.

[0077] 1) In response to, used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0078] 2) Based on, used to indicate the conditions or states on which the operation is performed depends. When the conditions or states on which it depends are met, one or more operations can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0079] 3) Model training involves multi-class classification learning on the image dataset. This model can be built using deep learning frameworks such as TensorFlow and Torch, employing multiple layers of neural networks like CNNs to form a multi-class classification model. The model's input is a three-channel or original-channel matrix generated from images read using tools like OpenCV. The model output is the multi-class probability, ultimately outputting the multimedia information similarity judgment through algorithms such as softmax. During training, the model approximates the correct trend using objective functions such as cross-entropy.

[0080] 4) Neural Network (NN): Artificial Neural Network (ANN), also known as neural network or neural network-like network, is a mathematical or computational model in the fields of machine learning and cognitive science that imitates the structure and function of biological neural networks (the central nervous system of animals, especially the brain) and is used to estimate or approximate functions.

[0081] 5) Multi-objective recall: This involves considering multiple objectives within a single recall model. In recommendation systems, it's often necessary to optimize multiple business objectives simultaneously to generate greater business revenue. For example, in e-commerce, the goal is to simultaneously optimize click-through rate and conversion rate, giving the platform more objectives; in news feeds, the aim is to increase user engagement such as click-through rate, likes, and comments, creating a better community atmosphere and thus improving user retention.

[0082] 6) Recommendation Accuracy: The recommended multimedia content has a certain effect over a period of time, and this effect is measured by the user's interest in the multimedia content. Accuracy plays an important role in online user retention, clicks, and CTR on the client side.

[0083] 7) softmax: A very commonly used and important function in machine learning, especially in multi-class classification scenarios. It maps some inputs to real numbers between 0 and 1, and normalization ensures that the sum is 1.

[0084] 8) Multimedia information, including various forms of information available on the Internet, such as advertising information, video files, recommended multimedia information, news information, etc., presented on clients or smart devices.

[0085] In this invention, embodiments can be implemented using cloud technology. Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. It can also be understood as a general term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies based on cloud computing business models. The backend services of network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites; therefore, cloud technology needs cloud computing as its support.

[0086] It's important to note that cloud computing is a computing model that distributes computing tasks across a resource pool comprised of numerous computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" are infinitely scalable, readily available, and can be used on demand, expanded at any time, and paid for based on usage. As the foundational providers of cloud computing capabilities, they establish cloud resource pool platforms, often referred to as cloud platforms or Infrastructure as a Service (IaaS). These platforms deploy various types of virtual resources within the resource pool for external customers to choose from. The cloud resource pool primarily includes: computing devices (which can be virtualized machines containing operating systems), storage devices, and network devices.

[0087] Figure 1 This is a schematic diagram illustrating a usage scenario of the multimedia information recommendation model training method provided in this embodiment of the invention. (See attached diagram.) Figure 1 The terminals (including terminals 10-1 and 10-2) are equipped with corresponding clients capable of playing embedded multimedia information. The terminals connect to server 200 via network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both. Data transmission is achieved using a wireless link. The multimedia information includes, but is not limited to, videos, images, GIF animations, and advertising information. The types of multimedia information obtained by the terminals (including terminals 10-1 and 10-2) from the corresponding server 200 via network 300 can be the same or different. For example, the terminals (including terminals 10-1 and 10-2) can obtain video advertisements or image advertisements from the same industry from the corresponding server 200 via network 300. The specific types are not limited in this application. Server 200 can store different multimedia information, including advertising multimedia information in different dynamic formats, such as GIF, MP4, and MOV.

[0088] During the process of the terminal (terminal 10-1 and / or terminal 10-2) obtaining and displaying the corresponding service with embedded multimedia information from the server 200 via the network 300, the user can perform different operations on the multimedia information presented in the multimedia information playback window through the terminal (terminal 10-1 and / or terminal 10-2), generating different user usage process data record information. For example, when the multimedia information is a video advertisement, the user can share and / or like the exposed video advertisement while watching the information, or click on it. When the multimedia information is a dynamic GIF advertisement, during the exposure of the advertisement through the terminal (terminal 10-1 and / or terminal 10-2), the user can forward and / or comment on the advertisement, or jump to the corresponding product purchase link page through the GIF advertisement.

[0089] In some embodiments of the present invention, the multimedia information recommendation model can also recommend financial information to meet users' financial needs, such as recommending stock information or fund information to users in the financial industry, so that users in the financial industry can carry out financial activities through virtual or physical resources, or pay for the recommended multimedia information through virtual resources (such as digital RMB).

[0090] As an example, when server 200 determines which multimedia information to recommend to user terminal 10-1 or 10-2, it needs to adjust the recommended multimedia information in a timely manner. For example, it may replace any multimedia information in the set of recommended multimedia information to adapt to the viewing needs of audiences corresponding to different target industries. Taking video advertising multimedia information as an example, the multimedia information recommendation model provided by this invention can be applied to video advertising playback. In video advertising playback, different video advertising multimedia information from different data sources is usually processed, and finally, the corresponding different multimedia information and the corresponding recommended video to be presented on the user interface (UI) are presented. The accuracy and timeliness of the characteristics of different multimedia information directly affect the user experience. The background database of video playback receives a large amount of video data from different sources every day. The different multimedia information obtained for recommending multimedia information to audiences corresponding to the target industry can also be called by other applications (e.g., the recommendation results of the video advertising recommendation process are migrated to the long video recommendation process or the news recommendation process). Of course, the multimedia information recommendation model that matches the audiences corresponding to the target industry can also be migrated to different video recommendation processes (e.g., webpage video recommendation process, mini-program video recommendation process, or long video client video recommendation process).

[0091] As an example, server 200 is used to deploy a corresponding multimedia information recommendation model to implement the multimedia information recommendation model training method provided by this invention, or to deploy a multimedia information recommendation model training device to implement the multimedia information recommendation model training method. Specifically, it acquires basic historical data in the multimedia information recommendation environment; extracts a pre-training sample set based on the basic historical data; trains the basic recommendation model based on the pre-training sample set to obtain the model parameters of the basic recommendation model; acquires industry historical data in the multimedia information recommendation environment; trains the multimedia information recommendation model according to the industry historical data to determine the model parameters of the multimedia information recommendation model, so as to adjust the recall strategy of multimedia information in the target industry through the multimedia information recommendation model, and recommend multimedia information through the recall strategy, and display and output the recommended multimedia information matching the audience corresponding to the target industry through the terminal (terminal 10-1 and / or terminal 10-2). Taking multimedia information as an example, the multimedia information recommendation model provided by this invention can be applied to video ad playback. During video ad playback, different multimedia information from different data sources is typically processed, and finally, the corresponding multimedia information and the recommended multimedia information corresponding to the corresponding video ad recommendation process are presented on the user interface (UI). The accuracy and timeliness of the characteristics of different multimedia information directly affect the user experience. The background database of video playback receives a large amount of multimedia information data from different sources every day. The obtained multimedia information, which is used to recommend multimedia information to the target industry's corresponding audience, can also be called by other applications (e.g., the recommendation results of the video ad recommendation process can be migrated to the recommendation process in an instant messaging client or a news recommendation process). Of course, the multimedia information recommendation model matching the corresponding target industry's corresponding audience can also be migrated to different video recommendation processes (e.g., webpage video recommendation process, mini-program video recommendation process, or video recommendation process in an instant messaging client). The recommended video ads can meet the user's viewing needs.

[0092] The multimedia information recommendation model training method provided in this application is based on artificial intelligence (AI). AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making functions.

[0093] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0094] In the embodiments of this application, the main artificial intelligence software technologies involved include the aforementioned speech processing technologies and machine learning. For example, it may involve Automatic Speech Recognition (ASR) technology in speech technology, including speech signal preprocessing, speech signal frequency analyzing, speech signal feature extraction, speech signal feature matching / recognition, and speech training.

[0095] For example, this could involve machine learning (ML), a multidisciplinary field encompassing probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning typically includes techniques such as deep learning, which includes artificial neural networks, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and deep neural networks (DNNs).

[0096] It is understood that the multimedia information recommendation model training method and speech processing provided in this application can be applied to intelligent devices. Intelligent devices can be any device with information display function, such as smart terminals, smart home devices (such as smart speakers, smart washing machines, etc.), smart wearable devices (such as smartwatches), in-vehicle intelligent central control systems (which display multimedia information to users through mini-programs that perform different tasks), or AI intelligent medical devices (which display treatment cases by showing multimedia information), etc.

[0097] The structure of the multimedia information recommendation model training device according to an embodiment of the present invention will be described in detail below. The multimedia information recommendation model training device can be implemented in various forms, such as a dedicated terminal with multimedia information recommendation processing function, or a server equipped with multimedia information recommendation model training device processing function, for example, the preceding... Figure 1 Server 200. Figure 2 This is a schematic diagram of the composition structure of the multimedia information recommendation model training device provided in the embodiments of the present invention. It can be understood that... Figure 2 This is only an exemplary structure of the multimedia information recommendation model training device, not the entire structure; it can be implemented as needed. Figure 2 The structure shown may be part or all of the structure.

[0098] The multimedia information recommendation model training device provided in this embodiment of the invention includes: at least one processor 201, a memory 202, a user interface 203, and at least one network interface 204. The various components in the multimedia information recommendation model training device are coupled together through a bus system 205. It can be understood that the bus system 205 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 205 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 205.

[0099] The user interface 203 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0100] It is understood that memory 202 can be volatile memory or non-volatile memory, or both. In this embodiment of the invention, memory 202 is capable of storing data to support the operation of a terminal (such as 10-1). Examples of this data include any computer programs used to operate on the terminal (such as 10-1), such as operating systems and applications. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications.

[0101] In some embodiments, the multimedia information recommendation model training device provided in this invention can be implemented using a combination of hardware and software. For example, the multimedia information recommendation model training device provided in this invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the training method of the multimedia information recommendation model provided in this invention. For instance, the processor in the form of a hardware decoding processor can employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0102] As an example of the multimedia information recommendation model training device provided in this embodiment of the invention, which adopts a combination of hardware and software, the multimedia information recommendation model training device provided in this embodiment of the invention can be directly embodied as a combination of software modules executed by processor 201. The software modules can be located in a storage medium, which is located in memory 202. Processor 201 reads the executable instructions included in the software modules in memory 202 and combines them with necessary hardware (e.g., including processor 201 and other components connected to bus 205) to complete the training method of the multimedia information recommendation model provided in this embodiment of the invention.

[0103] As an example, processor 201 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0104] As an example of the hardware implementation of the multimedia information recommendation model training device provided in this embodiment of the invention, the device provided in this embodiment of the invention can be directly executed by a processor 201 in the form of a hardware decoding processor. For example, it can be executed by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components to implement the training method of the multimedia information recommendation model provided in this embodiment of the invention.

[0105] In this embodiment of the invention, the memory 202 is used to store various types of data to support the operation of the multimedia information recommendation model training device. Examples of such data include: any executable instructions for operation on the multimedia information recommendation model training device, such as executable instructions that can be included in the program implementing the training method of the multimedia information recommendation model according to this embodiment of the invention.

[0106] In other embodiments, the multimedia information recommendation model training device provided in this invention can be implemented in software. Figure 2 A multimedia information recommendation model training device stored in memory 202 is shown. This device can be software in the form of programs and plugins, and includes a series of modules. As an example of a program stored in memory 202, it may include the multimedia information recommendation model training device, which includes the following software modules:

[0107] Information transmission module 2081 and information processing module 2082. When the software modules in the multimedia information recommendation model training device are read into RAM and executed by processor 201, the training method of the multimedia information recommendation model provided in this embodiment of the invention will be implemented. The functions of each software module in the multimedia information recommendation model training device include:

[0108] The information transmission module 2081 is used to acquire basic historical data in a multimedia information recommendation environment.

[0109] Information processing module 2082 is used to extract a pre-training sample set based on the basic historical data.

[0110] The information processing module 2082 is used to train the basic recommendation model based on the pre-trained sample set to obtain the model parameters of the basic recommendation model.

[0111] The information processing module 2082 is used to acquire historical industry data in a multimedia information recommendation environment.

[0112] The information processing module 2082 is used to train the multimedia information recommendation model based on the industry historical data, determine the model parameters of the multimedia information recommendation model, so as to adjust the recall strategy of multimedia information in the target industry through the multimedia information recommendation model, and recommend multimedia information through the recall strategy.

[0113] Once the multimedia information recommendation model has been trained, it can be deployed in an electronic device to execute the multimedia information recommendation method provided in this application, which may specifically include:

[0114] The data transmission module is used to acquire multimedia information to be recommended from the multimedia information data source.

[0115] The data processing module is used to process different multimedia information to be recommended through a multimedia information recommendation model and determine the priority of different multimedia information to be recommended.

[0116] The data processing module is used to adjust the recall strategy for multimedia information according to the priority of different multimedia information to be recommended, and to recommend multimedia information through the recall strategy.

[0117] according to Figure 2 The electronic device shown, in one aspect of this application, also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform various embodiments and combinations of embodiments provided in the various optional implementations of the multimedia information recommendation model training method described above.

[0118] Before introducing the multimedia information recommendation method provided in this application, the shortcomings of multimedia information recommendation in related technologies will be briefly explained. When performing multimedia information recommendation in related technologies, the following methods can be used:

[0119] 1) A separate recommendation model is built for each industry, trained, and deployed to recommend multimedia information. The drawback is that a lack of training data can lead to overfitting, affecting recommendation accuracy and severely impacting the user experience.

[0120] 2) Using a general model to complete multimedia information recommendation. However, general data recommendation models are trained on general data from multiple domains, and their recommendation results are too conventional and lack multimedia information recommendations tailored to industry characteristics.

[0121] 3) Train the multimedia information recommendation model using data transfer, referring to... Figure 3 , Figure 3 This is a schematic diagram of the transfer learning process in an embodiment of the present invention. A typical method for performing transfer learning using a deep neural network is to fine-tune all parameters of a model pre-trained on the source domain using data from the target domain. However, it is unclear whether fine-tuning all parameters of all instances in the target domain is the optimal solution. Figure 3 The method shown may directly use transfer learning models, which may lead to negative transfer between model parameters due to different data distributions across industries, thus failing to achieve accurate multimedia information recommendation.

[0122] Combination Figure 2 The multimedia information recommendation model training device shown illustrates the multimedia information recommendation model training method provided in this embodiment of the invention. See also: Figure 4 , Figure 4 This is an optional flowchart illustrating the multimedia information recommendation model training method provided in this embodiment of the invention. It can be understood that... Figure 4 The steps shown can be performed by various electronic devices running a multimedia information recommendation model training device, such as a dedicated terminal, server, or server cluster equipped with a multimedia information recommendation model training device. The dedicated terminal equipped with the multimedia information recommendation model training device can be a preceding... Figure 2 The illustrated embodiment is an electronic device with a multimedia information recommendation model training device. The following section addresses... Figure 4 The steps shown are explained.

[0123] Step 401: The multimedia information recommendation model training device acquires basic historical data in the multimedia information recommendation environment.

[0124] The basic historical data comprises the total data from various industries' multimedia information recommendation processes. This could include the sum of all basic data across multiple multimedia information recommendation environments, such as product recommendations, advertising recommendations, e-commerce advertising recommendations, and financial advertising recommendations. When acquiring basic historical data, it's possible to effectively extract data from the raw logs of user usage data. For example, this includes extracting the user's device ID (user account), the type of multimedia information, the viewing duration of the multimedia information, and the recommendation environment of the multimedia information to obtain basic historical data from different dimensions.

[0125] It is understood that in the specific implementation of this application, user-related data such as basic historical data and industry historical data in the media information recommendation environment are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0126] Step 402: The multimedia information recommendation model training device extracts a pre-training sample set based on the aforementioned basic historical data.

[0127] In some embodiments of the present invention, when extracting the pre-training sample set using basic historical data, the user's historical click behavior when browsing video information over the network, as well as the browsing duration of the corresponding video advertisements or image advertisements, are recorded through a subscription service and stored in a remote dictionary service (Redis Remote Dictionary Server) as basic historical data. When extracting the pre-training sample set, the historical click behavior and browsing duration of the corresponding user's video advertisements or image advertisements can be retrieved, thereby determining the text tags, video tags, and channel tags in the pre-training sample set.

[0128] In some embodiments of the present invention, taking multimedia information as an example of video advertising, the pre-training sample set includes at least:

[0129] 1) Text Tags: These text tags are obtained from the title text. There are two ways to obtain text tags: using text information with "#" tags in the title information, which are provided by the video ad user; or calling existing keyword extraction services to extract keywords from the title text, such as video ad keywords like "air conditioner" and "mobile phone".

[0130] 2) Video Tags: These can be tags obtained by the ad server using a video classification model. For example, video tags can be obtained through a deep residual ResNet50 model. The pre-trained convolutional neural network of deep residual ResNet50 performs feature extraction, extracting the image information of the video into a 128-dimensional feature vector.

[0131] 3) Channel Tags: These can be obtained by the ad server using a text classification BERT model. The BERT model takes the video title text features as input. The Bidirectional Encoder Representation from Transformers (BERT) neural network is used to feed the video title sentence into the model to obtain a 64-dimensional (dimensionality can be customized) title feature vector. The BERT model further enhances the generalization ability of the word vector model, achieving sentence-level representation capabilities.

[0132] Step 403: The multimedia information recommendation model training device trains the basic recommendation model based on the pre-trained sample set to obtain the model parameters of the basic recommendation model.

[0133] The basic recommendation model can be used to recommend multimedia information to users in any industry. However, since the basic recommendation model is trained based on basic historical data, it cannot yet be used to recommend multimedia information to a specific industry. Therefore, it is necessary to continue training based on industry historical data.

[0134] Step 404: The multimedia information recommendation model training device acquires historical industry data in the multimedia information recommendation environment.

[0135] Step 405: Extract the parameters of the embedding layer network of the basic recommendation model from the model parameters of the basic recommendation model, and transfer the parameters of the embedding layer network to the multimedia information recommendation model, wherein the basic recommendation model and the multimedia information recommendation model have the same model structure.

[0136] Since the basic recommendation model and the multimedia information recommendation model have the same model structure, and the parameters of the embedding layer network are transferred to the multimedia information recommendation model, the multimedia recommendation model can inherit the fitting effect of the basic recommendation model, avoiding the overfitting defect that occurs when training the multimedia information recommendation model using only industry historical data. In addition, in order to enable the multimedia information recommendation model to adjust the recall strategy of multimedia information in the target industry, step 406 is also required to train the multimedia information recommendation model.

[0137] Step 406: The multimedia information recommendation model training device trains the multimedia information recommendation model based on the industry historical data, determines the model parameters of the multimedia information recommendation model, so as to adjust the recall strategy of multimedia information in the target industry through the multimedia information recommendation model, and recommend multimedia information through the recall strategy.

[0138] Combination Figure 2 The multimedia information recommendation model training device shown illustrates the multimedia information recommendation model training method provided in this embodiment of the invention. See also: Figure 5 , Figure 5 This is an optional flowchart illustrating the multimedia information recommendation model training method provided in this embodiment of the invention. It can be understood that... Figure 5The steps shown can be performed by various electronic devices running a multimedia information recommendation model training device, such as a dedicated terminal, server, or server cluster equipped with a multimedia information recommendation model training device. The dedicated terminal equipped with the multimedia information recommendation model training device can be a preceding... Figure 2 The illustrated embodiment is an electronic device with a multimedia information recommendation model training device. The following section addresses... Figure 5 The steps shown are explained.

[0139] Step 501: The multimedia information recommendation model training device adjusts the embedding features of the embedding layer network of the basic recommendation model based on the industry historical data to obtain the embedding features of the multimedia information recommendation model.

[0140] Specifically, due to the diverse types of target industries during recommendation, the model parameters of multimedia information recommendation models change according to the type of target industry to achieve recommendation accuracy. If each type of target industry is trained entirely based on historical industry data, the application cost of the multimedia information recommendation model will increase. Furthermore, for multimedia information recommendation models with limited historical industry data, overfitting of model parameters will occur, affecting the accuracy of multimedia information recommendations. To address these issues, the multimedia information recommendation model training method provided in this application uses the same model structure for both the base recommendation model and the multimedia information recommendation model. When training the multimedia information recommendation model, the parameters of the embedding layer network of the base recommendation model are extracted based on its model parameters, and these parameters are then transferred to the multimedia information recommendation model. Thus, the multimedia information recommendation model can inherit the generalization ability of the base recommendation model.

[0141] Meanwhile, since the multimedia information recommendation model requires embedded features during training, and there is some overlap between industry historical data and basic historical data, this application does not require feature processing of industry historical data to obtain the embedded features of the multimedia information recommendation model during training. Instead, it adjusts the embedded features of the embedding layer network of the basic recommendation model based on the industry historical data, deleting some embedded features of the embedding layer network of the basic recommendation model and adding features corresponding to the industry historical data. This further reduces the training time of the multimedia information recommendation model. At the same time, the obtained embedded features of the multimedia information recommendation model retain the fitting effect of the basic historical data and make the embedded features of the multimedia information recommendation model suitable for the target industry, thus improving the accuracy of the recommendation.

[0142] Step 502: The multimedia information recommendation model training device configures the training cycle parameters for the multimedia information recommendation model according to the type of the target industry.

[0143] The accuracy of multimedia information recommendations varies depending on the target industry. For example, in multimedia information recommendation environments where the recommendation accuracy (recommendation accuracy = number of user triggers / number of information recommendations) needs to be greater than or equal to 0.8, the loop iteration parameter needs to be a positive integer greater than or equal to 980, as the multimedia information recommendation model needs to improve recommendation accuracy and guide users to trigger consumer goods or financial product advertisements. Similarly, in multimedia information recommendation environments where the recommendation accuracy is less than or equal to 0.8, the loop iteration parameter needs to be a positive integer less than or equal to 840, as the multimedia information recommendation model needs to inherit the fitting properties of the base recommendation model.

[0144] Step 503: Multimedia Information Recommendation Model Training Device. When the number of training iterations of the multimedia information recommendation model is less than or equal to the training cycle parameter, the parameters of the embedding layer network are kept unchanged. The network parameters of the multimedia information recommendation model are adjusted using the embedding features of the multimedia information recommendation model and the multi-task loss function of the multimedia information recommendation model until the loss function corresponding to the multimedia information recommendation model reaches the corresponding convergence condition. The first model parameters of the multimedia information recommendation model are then determined, wherein the model parameters of the multimedia information recommendation model include the first model parameters.

[0145] It should be noted that since the first model parameters are obtained by training the multimedia information recommendation model while keeping the parameters of the embedding layer network unchanged, the multimedia information recommendation model using the first model parameters can inherit the fitting effect of the base recommendation model and improve the generalization of the multimedia information recommendation model. However, since the embedding layer network of the multimedia information recommendation model has not participated in the training, the multimedia information recommendation model using the first model parameters has not yet reached the optimal state for targeted recommendations of the target industry (for example, the accuracy of ad recommendations has not yet reached the accuracy threshold of 0.8). Therefore, the embedding layer network of the multimedia information recommendation model needs to continue to participate in the training to adjust the first model parameters.

[0146] Step 504: Multimedia Information Recommendation Model Training Device When the number of training iterations of the multimedia information recommendation model is greater than the training cycle parameter, the basic recommendation model is trained using the embedding features of the multimedia information recommendation model to adjust the first model parameter of the multimedia information recommendation model and obtain the model parameter of the multimedia information recommendation model.

[0147] After the model parameters of the multimedia information recommendation model are determined through steps 503 and 504, the trained multimedia information recommendation model inherits the fitting effect of the base recommendation model, avoids the overfitting defect that occurs when training the multimedia information recommendation model using only historical industry data, improves the generalization of the multimedia information recommendation model, and makes the multimedia information recommendation model more in line with the user's needs.

[0148] It should be noted that, since this application does not impose specific restrictions on the model structure of the multimedia information recommendation model, the training cycle parameters need to be dynamically adjusted during configuration. This can be achieved by: determining the convergence speed of the loss function corresponding to the multimedia information recommendation model; and dynamically adjusting the training cycle parameters according to the convergence speed to match the training cycle parameters with the convergence speed.

[0149] Specifically, when superlinear convergence is determined by the convergence speed of the loss function, the training cycle parameters are reduced; when linear convergence is determined, the training cycle parameters are kept unchanged; when sublinear convergence is determined, the training cycle parameters are increased. In this way, the training cycle parameters are dynamically adjusted to ensure that the convergence speed of the loss function changes smoothly and to ensure the training accuracy of the multimedia information recommendation model.

[0150] Therefore, training with a pre-trained sample set ensures sufficient data, yields useful model parameters, and prevents the recommendation model from overfitting. Training with historical industry data ensures the accuracy of multimedia information recommendations within a specific industry, avoids overly conventional recommendations, and better attracts user attention.

[0151] refer to Figure 6 , Figure 6 As described in the embodiments of the present invention Figure 5 The diagram illustrates the process of obtaining peripheral and central features. Through steps 501 and 502 in the preceding embodiments, the original features are mapped using the central sub-model, and the domain features are mapped using the peripheral sub-model, reducing the dimensionality of the 128-dimensional feature vector to a lower-dimensional space (64 or 32 dimensions). Under the same constraints, the central features of the original features in the target semantic space and the peripheral features of the domain features in the target semantic space are obtained. For example, if x represents the input vector of one side of the network and y represents the output vector… W represents the hidden layer in the middle. i Let b represent the weight matrix of the i-th layer, and b i Let represent the bias term of the i-th layer. Then, the output of the hidden layer and the output of the central submodel can be expressed as Equation 1:

[0152] Formula 1

[0153] Referring to Formula 2, the following can be used: The function serves as the activation function for both the output layer and the hidden layer li:

[0154] Formula 2

[0155] When ranking content, referring to Formula 3, the cosine similarity between peripheral and central features can be used as the priority ranking criterion for recommended videos, based on the peripheral sub-model (e.g., the outer tower in a twin-tower structure) and the central sub-model (e.g., the central tower in a twin-tower structure).

[0156] Formula 3

[0157] Combination Figure 2 The multimedia information recommendation model training device shown illustrates the multimedia information recommendation model training method provided in this embodiment of the invention. See also: Figure 7 , Figure 7 This is an optional flowchart illustrating the multimedia information recommendation model training method provided in this embodiment of the invention. It can be understood that... Figure 7 The steps shown can be performed by various electronic devices running a multimedia information recommendation model training device, such as a dedicated terminal, server, or server cluster equipped with a multimedia information recommendation model training device. The dedicated terminal equipped with the multimedia information recommendation model training device can be a preceding... Figure 2 The illustrated embodiment is an electronic device with a multimedia information recommendation model training device. The following section addresses... Figure 7 The steps shown are explained.

[0158] Step 701: The multimedia information recommendation model training device determines invalid samples in the pre-training sample set based on the industry historical data.

[0159] Because the target industries are diverse, the multimedia information recommendation model needs to be constantly adjusted to meet the usage requirements of multimedia information recommendation. However, there is some overlap between industry historical data and basic historical data. Therefore, in this application, the multimedia information recommendation model does not need to perform feature processing on industry historical data to obtain the embedding features of the multimedia information recommendation model during training. Instead, based on the industry historical data, the embedding features of the embedding layer network of the basic recommendation model are adjusted, deleting some embedding features of the basic recommendation model's embedding layer network and adding features corresponding to the industry historical data. For example, the pre-training sample set obtained based on the basic historical data includes: information samples from short videos and information samples corresponding to industry historical data of consumer goods advertisements or financial product advertisements. For the multimedia information recommendation model that recommends short videos, the information samples corresponding to industry historical data of consumer goods advertisements or financial product advertisements are invalid samples. Similarly, for the multimedia information recommendation model that recommends advertisements, the information samples from short videos are invalid samples during training.

[0160] Step 702: The multimedia information recommendation model training device determines industry feature samples that match the industry historical data based on the industry historical data.

[0161] Step 703: The multimedia information recommendation model training device uses the invalid samples and the industry feature samples to adjust the pre-training sample set to obtain the industry training sample set.

[0162] In this process, invalid samples can be removed from the pre-training samples, and industry feature samples can be added to obtain an industry training sample set. The industry training sample set can be stored in a cloud server to ensure that when the multimedia information recommendation model of the corresponding industry is triggered again, the training samples can be directly called to save model training time.

[0163] Step 704: The multimedia information recommendation model training device configures the training cycle parameters for the multimedia information recommendation model according to the type of the target industry.

[0164] Step 705: The multimedia information recommendation model training device extracts the embedding features of the embedding layer network of the basic recommendation model.

[0165] Step 706: Multimedia Information Recommendation Model Training Device When the number of training iterations of the multimedia information recommendation model is less than or equal to the training cycle parameter, the parameters of the embedding layer network are kept unchanged, and the embedding features are adjusted using the industry training sample set to obtain the second model parameters of the multimedia information recommendation model.

[0166] It should be noted that since the second model parameters are obtained by training the multimedia information recommendation model while keeping the parameters of the embedding layer network unchanged, the multimedia information recommendation model using the second model parameters can inherit the fitting effect of the basic recommendation model and improve the generalization of the multimedia information recommendation model. Although the multimedia information recommendation model using the second model parameters can achieve the recommendation of multimedia information, since the embedding layer network of the multimedia information recommendation model has not participated in the training, the targeted recommendation effect of the multimedia information recommendation model using the second model parameters for the target industry has not yet reached the optimal state (for example, the recommendation accuracy of advertisements has not yet reached the accuracy threshold of 0.8). Therefore, the embedding layer network of the multimedia information recommendation model needs to continue to participate in the training to adjust the second model parameters. At the same time, since the pre-training sample set is adjusted using invalid samples and the industry feature samples in steps 701-703 to obtain the industry training sample set, when the multimedia information model is used in different target industries (or reused in the same target industry), the industry training sample set can be directly obtained from the cloud server, further compressing the training time of the multimedia information recommendation model, so that the multimedia information recommendation model can obtain the highest recommendation accuracy in the shortest training time.

[0167] Step 707: Multimedia Information Recommendation Model Training Device When the number of training iterations of the multimedia information recommendation model is greater than the training cycle parameter, the basic recommendation model is trained using the industry training sample set to adjust the second model parameters of the multimedia information recommendation model and obtain the model parameters of the multimedia information recommendation model.

[0168] In some embodiments of the present invention, when the multimedia information recommendation model is embedded in a corresponding hardware device (e.g., a news reading terminal, an e-book terminal, or a financial news terminal), and the usage environment involves pushing different news multimedia information to users through a news reading terminal or e-book terminal, fixing the corresponding fixed noise threshold of the multimedia information recommendation model can effectively improve the training speed of the multimedia information recommendation model and reduce the user's waiting time. In this usage environment with fixed noise, the training sample set can come from historical data of the target industry's corresponding audience. Historical recommended multimedia information browsing data can be the recommended multimedia information viewing behavior data generated when recommending multimedia information to the target industry's corresponding audience, which can be extracted from historical browsing logs. Here, historical recommended multimedia information browsing data can be all historical recommended multimedia information browsing data; alternatively, considering the timeliness of the behavioral data, it can only include historical recommended multimedia information browsing data within a preset time period, such as historical recommended multimedia information browsing data within a week, etc.

[0169] The following uses a video recommendation scenario in a video ad playback interface as an example to illustrate the multimedia information recommendation method provided in this embodiment of the invention. Figure 8 This is a schematic diagram illustrating the application environment of the multimedia information recommendation method based on a multimedia information recommendation model in an embodiment of the present invention, wherein, as shown... Figure 8 As shown, the video ad playback interface can be displayed within the corresponding app or triggered through an instant messaging client mini-program (the multimedia information recommendation model can be trained and encapsulated in the corresponding app or stored as a plugin in the instant messaging client mini-program). With the continuous development and increase of video ad applications, the amount of video information carried far exceeds that of text information. Video ads can be continuously recommended to users through corresponding applications. Therefore, recommending fresh video ads to users and avoiding repetitive recommendations can maintain user engagement. Effective subsequent recommendations of related videos can significantly improve the user experience. Figure 9 This is a schematic diagram of the multimedia information recommendation method in an embodiment of the present invention, including the following steps:

[0170] Step 901: Obtain the multimedia information to be recommended from the multimedia information data source.

[0171] Step 902: Process different multimedia information to be recommended using a multimedia information recommendation model to determine the priority of different multimedia information to be recommended.

[0172] For example, consider two multimedia information products: video advertisement A and advertisement B for consumer goods. When the multimedia information recommendation model provided in this application determines that the target multimedia information score of A is 1 and the target multimedia information score of B is 2, it can be determined that the priority of advertisement B is higher than that of advertisement A. This indicates that the current user may be more interested in video advertisement B. Therefore, based on the scores of the target multimedia information, advertisement B is recommended to the user first, and more playback traffic is allocated to advertisement B to increase its exposure rate, thereby enabling the user to obtain a better viewing experience and increasing the trigger rate of advertisement B.

[0173] Step 903: Adjust the recall strategy for multimedia information according to the priority of different multimedia information to be recommended, and recommend multimedia information through the recall strategy.

[0174] Figure 10This is a schematic diagram of the multimedia information recommendation method in an embodiment of the present invention. The basic recommendation model and the multimedia information recommendation model adopt a dual-tower structure. After obtaining the basic recommendation model, the basic recommendation model can be further trained and fine-tuned using historical industry data of e-commerce advertisements to recommend different e-commerce advertisements to users. The initial value of the training cycle parameter is 4. When the convergence speed of the loss function corresponding to the multimedia information recommendation model exceeds the convergence speed threshold, the training cycle parameter is adjusted.

[0175] Specifically, the following steps are included:

[0176] Step 1001: Extract a pre-training sample set based on basic historical data.

[0177] Step 1002: Based on the pre-trained sample set, train the basic recommendation model to obtain the model parameters of the basic recommendation model.

[0178] Step 1003: Obtain historical data of the e-commerce advertising industry in the multimedia information recommendation environment.

[0179] Step 1004: Adjust the embedding features of the embedding layer network of the basic recommendation model based on historical data from the e-commerce advertising industry.

[0180] Step 1005: Use the multi-task loss function of the e-commerce advertising multimedia information recommendation model to adjust the network parameters and determine the network parameters of the e-commerce advertising multimedia information recommendation model.

[0181] In some embodiments of the present invention, see Figure 11 , Figure 11This is a schematic diagram of an optional multimedia information recommendation in an embodiment of the present invention. The category of the multimedia information to be recommended can be determined. In response to the category of the multimedia information to be recommended, a matching multimedia information data source is triggered. For example, when the category of the multimedia information to be recommended is determined to be advertising information, the target resources include different advertisements from the same industry. Different video advertisements included in different resource groups can be played sequentially in different ad slot video ad playback windows (e.g., ad slot 1, ad slot 2, and ad slot 3 each play three different advertisements from the same industry). Alternatively, when all different ad slot video ad playback areas in the display interface are occupied by the same industry, the advertising information is displayed in a loop. The advertising information from the same industry can be presented in a loop in different ad slot video ad playback windows in the advertising information display interface. Simultaneously, when different ad slot video ads from the same industry are video ads, the video ad information from the same industry can be presented in a loop, and the audio volume carried by the video can be adjusted to the maximum to prompt the user to watch the played video ad. Replacing ad A with ad B allocates more playback bandwidth to ad B, resulting in a better viewing experience for the user. Specifically, based on the traffic parameters and iterative experimental parameters matched by the advertising information recall strategy, the advertising information recall strategy can be dynamically adjusted. This can increase the advertising exposure rate. In some embodiments of the present invention, the exposure channel of advertisement A can be adjusted from the current multimedia information playback client to the contact status information of the instant messaging client. Of course, when adjusting the exposure position of advertisement A, it can be adjusted from the status sharing interface advertisement of the instant messaging client to the splash screen advertisement, so as to conform to different dynamically adjusted recall strategies. This allows video advertisements in different ad positions to be recommended to different users in a short period of time, thereby achieving better video recommendation results. Figure 11 For example, when adjusting the recall strategy for advertisements in the consumer goods industry on an instant short video playback interface using the multimedia information recommendation model provided in this application, through... Figure 6 The multimedia information recommendation model shown can recommend different ads to users who have viewed ads in ad slots 1, 2, and 3. By dynamically adjusting the recall strategy, it can deliver consumer goods industry ads in the order of ad A, B, and C. Specifically, ad slot 1 displays ad A, ad slot 2 displays ad B, and ad slot 3 displays ad C. This ensures that users receive more novel advertising information (recommending previously unviewed ads to different types of users), resulting in a better user experience and increasing ad click-through rates for better advertising performance.

[0182] like Figure 11As shown, when playing video ads, the exposure parameters of the video ads can be sent to the detection server so that the detection server can obtain the exposure parameters of the video ads. The exposure parameters are used as evaluation parameters for the playback effect of the multimedia information, and the target exposure parameters are found based on the adjustment results of the recall strategy. For example, if the exposure parameters of ad slot 1, ad slot 2, and ad slot 3 are 100 times, 85 times, and 70 times, respectively, the recommendation effect of the ads can be determined by the exposure parameters of the video ads playing in the ad slots. Based on the adjustment results of the recall strategy, if the target exposure parameters of ads A, B, and C are found to be 65 times, 75 times, and 102 times, respectively, then ad A can be moved to ad slot 3, ad B to ad slot 2, and ad C to ad slot 1, so as to flexibly meet the needs of video ad placement.

[0183] At the same time, such as Figure 11 As shown, when dynamically adjusting the playback strategy of the time-sensitive short videos, historical browsing information of the target industry's corresponding audience can be obtained; based on the historical browsing information of the target industry's corresponding audience, the exposure history of the time-sensitive short videos corresponding to the historical browsing information is determined; based on the exposure history of the time-sensitive short videos corresponding to the historical browsing information, the playback strategy of the time-sensitive short videos is dynamically adjusted to achieve the desired effect. Figure 11 For example, due to different user preferences, ads in any target industry can be blocked. Therefore, if it is determined that audience 1 in the target industry has previously blocked ad B in their browsing history, the playback strategy can be dynamically adjusted to replace ad A with other ad information (such as ad C). Similarly, if it is determined that audience 2 in the target industry has previously blocked ad C in their browsing history, the playback strategy can be dynamically adjusted to replace ad A with other ad information (such as ad D) to conform to the usage habits of the target industry audience and provide users with a better user experience.

[0184] See Figure 12 , Figure 12 This is a schematic diagram of an optional multimedia information recommendation in an embodiment of the present invention. When the multimedia information recommendation model recommends financial industry funds to users watching short videos, it trains the basic recommendation model based on historical data from the financial industry. After determining the model parameters, it can recommend different fund products to users watching short videos in ad slots 1, 2, and 3. By dynamically adjusting the recall strategy, it can recommend financial industry fund products in the order of fund "Hongyuan XX", fund "Wansheng XX", and fund "XX Fund". Specifically, ad slot 1 displays "XX Fund", ad slot 2 displays "Wansheng XX", and ad slot 3 displays "Hongyuan XX" (ad C). Therefore, compared to... Figure 11The advertising recommendations shown can deliver different industry advertisements or product information in the same ad slots through a multimedia information recommendation model, while ensuring that users have the same viewing experience, making it convenient for users to purchase and search.

[0185] Beneficial technical effects:

[0186] This invention acquires basic historical data from a multimedia information recommendation environment; extracts a pre-training sample set based on the basic historical data; trains a basic recommendation model based on the pre-training sample set to obtain the model parameters of the basic recommendation model; acquires industry historical data from the multimedia information recommendation environment; extracts the parameters of the embedding layer network of the basic recommendation model from the model parameters of the basic recommendation model, and transfers the parameters of the embedding layer network to the multimedia information recommendation model, wherein the basic recommendation model and the multimedia information recommendation model have the same model structure; trains the multimedia information recommendation model according to the industry historical data to determine the model parameters of the multimedia information recommendation model, so as to adjust the recall strategy of multimedia information in the target industry through the multimedia information recommendation model, and perform multimedia information recommendation through the recall strategy. Therefore, the multimedia information recommendation model can recommend multimedia information to users in different industries in the usage environment, while enhancing the accuracy and relevance of multimedia information recommendations, effectively improving the quality of multimedia information recommendations, and completing model training with fewer samples, reducing overfitting of the multimedia information recommendation model, improving the generalization of the multimedia information recommendation model, and enhancing the user experience.

[0187] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for training a multimedia information recommendation model, characterized in that, The method includes: Acquire basic historical data in a multimedia information recommendation environment; extract a pre-training sample set based on the basic historical data; train a basic recommendation model based on the pre-training sample set to obtain the model parameters of the basic recommendation model; Based on the industry historical data in the multimedia information recommendation environment, invalid samples in the pre-training sample set are removed, and industry feature samples that match the industry historical data are added to obtain the industry training sample set. The parameters of the embedding layer network of the basic recommendation model are extracted from the model parameters of the basic recommendation model, and the parameters of the embedding layer network are transferred to the multimedia information recommendation model, wherein the basic recommendation model and the multimedia information recommendation model have the same model structure. Based on the type of the target industry, the training cycle parameters are configured for the multimedia information recommendation model; based on the convergence speed of the multi-task loss function of the multimedia information recommendation model, the training cycle parameters are dynamically adjusted to match the convergence speed. When the number of training iterations of the multimedia information recommendation model is less than or equal to the number of training iterations, the parameters of the embedding layer network remain unchanged. The embedding features of the embedding layer network of the basic recommendation model are adjusted using the industry training sample set to obtain the second model parameters of the multimedia information recommendation model. The model parameters of the multimedia information recommendation model include the second model parameters. When the number of training iterations of the multimedia information recommendation model is greater than the number of training iterations parameter, the basic recommendation model is trained using the industry training sample set to adjust the second model parameters of the multimedia information recommendation model and obtain the model parameters of the multimedia information recommendation model. The multimedia information recommendation model is used to adjust the recall strategy for multimedia information in the target industry, and the recall strategy is used to recommend multimedia information.

2. The method according to claim 1, characterized in that, The method further includes: Based on the historical industry data, the embedding features of the embedding layer network of the basic recommendation model are adjusted to obtain the embedding features of the multimedia information recommendation model. Based on the type of the target industry, configure the training cycle parameters for the multimedia information recommendation model; When the number of training iterations of the multimedia information recommendation model is less than or equal to the training cycle number parameter, the parameters of the embedding layer network remain unchanged, and the network parameters of the multimedia information recommendation model are adjusted using the embedding features of the multimedia information recommendation model and the multi-task loss function of the multimedia information recommendation model. When the loss function corresponding to the multimedia information recommendation model reaches the corresponding convergence condition, the first model parameters of the multimedia information recommendation model are determined, wherein the model parameters of the multimedia information recommendation model include the first model parameters.

3. The method according to claim 2, characterized in that, The method further includes: When the number of training iterations of the multimedia information recommendation model exceeds the training cycle parameter, the basic recommendation model is trained using the embedding features and the multi-task loss function of the multimedia information recommendation model to adjust the first model parameter of the multimedia information recommendation model and obtain the model parameter of the multimedia information recommendation model.

4. The method according to claim 1, characterized in that, The method further includes: When the multimedia information is a video advertisement The exposure parameters of the video advertisement during playback are sent to the detection server so that the detection server can obtain the exposure parameters of the video advertisement. The exposure parameter is used as an evaluation parameter for the playback effect of the multimedia information, and the target exposure parameter is found based on the adjustment result of the recall strategy.

5. The method according to claim 1, characterized in that, The method further includes: Obtain historical browsing information of viewers corresponding to the target industry; Based on the historical browsing information of the audience corresponding to the target industry, determine the exposure history of multimedia information corresponding to the historical browsing information; Based on the exposure history of the multimedia information corresponding to the historical browsing information, the recall strategy for the multimedia information is dynamically adjusted.

6. The method according to claim 1, characterized in that, The method further includes: Based on the multimedia information recommendation environment, determine the category of multimedia information to be recommended; In response to the category of the multimedia information to be recommended, a matching multimedia information data source is triggered to adjust the multimedia information to be recommended by using a multimedia information data source that matches the category of the multimedia information to be recommended.

7. A multimedia information recommendation method, characterized in that, The method includes: Retrieve multimedia information to be recommended from multimedia information data sources; The multimedia information recommendation model is used to process different multimedia information to be recommended and determine the priority of different multimedia information to be recommended. The recall strategy for multimedia information is adjusted according to the priority of different multimedia information to be recommended, and multimedia information is recommended through the recall strategy, wherein the multimedia information recommendation model is trained based on any one of claims 1 to 6.

8. A multimedia information recommendation model training device, characterized in that, The device includes: The information transmission module is used to acquire basic historical data in a multimedia information recommendation environment; The information processing module is used to extract a pre-training sample set based on the aforementioned basic historical data; The information processing module is used to train the basic recommendation model based on the pre-trained sample set to obtain the model parameters of the basic recommendation model. The information processing module is used to remove invalid samples from the pre-training sample set and add industry feature samples that match the industry historical data based on the industry historical data in the multimedia information recommendation environment, so as to obtain an industry training sample set. The information processing module is used to extract the parameters of the embedding layer network of the basic recommendation model from the model parameters of the basic recommendation model, and transfer the parameters of the embedding layer network to the multimedia information recommendation model, wherein the basic recommendation model and the multimedia information recommendation model have the same model structure. The information processing module is used to configure training cycle parameters for the multimedia information recommendation model according to the type of the target industry; dynamically adjust the training cycle parameters according to the convergence speed of the multi-task loss function of the multimedia information recommendation model to match the training cycle parameters with the convergence speed; when the number of training iterations of the multimedia information recommendation model is less than or equal to the training cycle parameters, keep the parameters of the embedding layer network unchanged, and adjust the embedding features of the embedding layer network of the basic recommendation model through the industry training sample set to obtain the second model parameters of the multimedia information recommendation model, wherein the model parameters of the multimedia information recommendation model include the second model parameters; When the number of training iterations of the multimedia information recommendation model exceeds the training cycle parameter, the basic recommendation model is trained using the industry training sample set to adjust the second model parameter of the multimedia information recommendation model, thereby obtaining the model parameter of the multimedia information recommendation model. The multimedia information recommendation model is then used to adjust the recall strategy for multimedia information in the target industry, and the recall strategy is used to recommend multimedia information.

9. The apparatus as claimed in claim 8, characterized in that, The information processing module is also used for: Based on the historical industry data, the embedding features of the embedding layer network of the basic recommendation model are adjusted to obtain the embedding features of the multimedia information recommendation model. Based on the type of the target industry, configure the training cycle parameters for the multimedia information recommendation model; When the number of training iterations of the multimedia information recommendation model is less than or equal to the training cycle number parameter, the parameters of the embedding layer network remain unchanged, and the network parameters of the multimedia information recommendation model are adjusted using the embedding features of the multimedia information recommendation model and the multi-task loss function of the multimedia information recommendation model. When the loss function corresponding to the multimedia information recommendation model reaches the corresponding convergence condition, the first model parameters of the multimedia information recommendation model are determined, wherein the model parameters of the multimedia information recommendation model include the first model parameters.

10. The apparatus as claimed in claim 9, characterized in that, The information processing module is also used for: When the number of training iterations of the multimedia information recommendation model exceeds the training cycle parameter, the basic recommendation model is trained using the embedding features and the multi-task loss function of the multimedia information recommendation model to adjust the first model parameter of the multimedia information recommendation model and obtain the model parameter of the multimedia information recommendation model.

11. The apparatus as claimed in claim 8, characterized in that, The information processing module is also used for: When the multimedia information is a video advertisement, the exposure parameters of the video advertisement during playback are sent to the detection server so that the detection server can obtain the exposure parameters of the video advertisement. The exposure parameter is used as an evaluation parameter for the playback effect of the multimedia information, and the target exposure parameter is found based on the adjustment result of the recall strategy.

12. The apparatus as claimed in claim 8, characterized in that, The information processing module is also used for: Obtain historical browsing information of viewers corresponding to the target industry; Based on the historical browsing information of the audience corresponding to the target industry, determine the exposure history of multimedia information corresponding to the historical browsing information; Based on the exposure history of the multimedia information corresponding to the historical browsing information, the recall strategy for the multimedia information is dynamically adjusted.

13. The apparatus as claimed in claim 8, characterized in that, The information processing module is also used for: Based on the multimedia information recommendation environment, determine the category of multimedia information to be recommended; In response to the category of the multimedia information to be recommended, a matching multimedia information data source is triggered to adjust the multimedia information to be recommended by using a multimedia information data source that matches the category of the multimedia information to be recommended.

14. A multimedia information recommendation device, characterized in that, The device includes: The data transmission module is used to acquire multimedia information to be recommended from the multimedia information data source; The data processing module is used to process different multimedia information to be recommended through a multimedia information recommendation model and determine the priority of different multimedia information to be recommended. The data processing module is used to adjust the recall strategy of multimedia information according to the priority of different multimedia information to be recommended, and to recommend multimedia information through the recall strategy, wherein the multimedia information recommendation model is trained based on any one of claims 1 to 6.

15. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the multimedia information recommendation model training method according to any one of claims 1 to 6, or the multimedia information recommendation method according to claim 7.

16. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; When the processor executes the executable instructions stored in the memory, it implements the multimedia information recommendation model training method according to any one of claims 1 to 6, or the multimedia information recommendation method according to claim 7.

17. A computer-readable storage medium storing executable instructions, characterized in that, When the executable instructions are executed by the processor, they implement the multimedia information recommendation model training method according to any one of claims 1 to 6, or the multimedia information recommendation method according to claim 7.

Images