A model training and information recommendation method, device, storage medium and equipment

Abstract

This specification discloses a model training and information recommendation method, device, storage medium and equipment. The model training method includes: obtaining multimedia data, and determining each user's browsing record for the multimedia data in history; according to the browsing record, determining each element point information contained in the multimedia data as label element point information, the element point information is used to characterize the information content corresponding to the key time node in the multimedia data; inputting the multimedia data into the information extraction model to be trained, so as to extract each element point information contained in the multimedia data through the information extraction model as predicted element point information; training the information extraction model according to the predicted element point information and the label element point information. The model trained based on the element point information can fully meet the user's expectations when recommending information, ensure the accuracy of information recommendation, and further improve the effect of information recommendation.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, storage medium and equipment for model training and information recommendation. Background Art

[0002] Information recommendation is an effective marketing method. On the one hand, it can fully meet user needs, and on the other hand, it can attract users to use products or enjoy services. Therefore, it is widely used in the promotion and publicity of industrial products, food, games, film and television works, and service projects.

[0003] However, in the current process of recommending information to users, some key points in the recommended information are often difficult to meet the expectations of users, resulting in low accuracy of information recommendation, which seriously affects the effect of information recommendation.

[0004] Therefore, how to improve the accuracy of information recommendation is an urgent problem to be solved. Summary of the invention

[0005] This specification provides a model training and information recommendation method, device, storage medium and equipment to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a model training method, including:

[0008] Acquire multimedia data, and determine historical browsing records of each user for the multimedia data, wherein the browsing records are used to reflect the interactive operations performed by each user when browsing the information content at each time node in the multimedia data;

[0009] Determine, according to the browsing record, each element point information contained in the multimedia data as label element point information, wherein the element point information is used to represent information content corresponding to a key time node in the multimedia data;

[0010] Inputting the multimedia data into the information extraction model to be trained, so as to extract each element point information contained in the multimedia data as the predicted element point information through the information extraction model;

[0011] The information extraction model is trained according to the predicted feature point information and the label feature point information.

[0012] Optionally, before the step of determining information of each element point contained in the multimedia data according to the browsing record, the method further includes:

[0013] Acquire feedback data for the multimedia data, where the feedback data is used to represent data content corresponding to key time nodes in the multimedia data fed back by each user when browsing the multimedia data in the past;

[0014] The step of determining the information of each element point contained in the multimedia data according to the browsing record specifically includes:

[0015] The information of each element point is determined according to the browsing record and the feedback data.

[0016] Optionally, before the step of inputting the multimedia data into the information extraction model to be trained, the method further comprises:

[0017] Dividing the multimedia data to obtain at least one type of sub-data, the type including: text, image, and audio;

[0018] The step of inputting the multimedia data into the information extraction model to be trained so as to extract information of each element point contained in the multimedia data through the information extraction model specifically includes:

[0019] The at least one type of sub-data is input into the information extraction model, so as to extract information of each element point contained in the multimedia data through the information extraction model.

[0020] Optionally, the step of extracting information of each element point contained in the multimedia data by using the information extraction model specifically includes:

[0021] For each time node in the multimedia data, determining a richness value corresponding to each type of sub-data at the time node;

[0022] The information of each element point included in the multimedia data is determined according to the richness value corresponding to each type of sub-data at each time node.

[0023] Optionally, the step of determining the information of each element point included in the multimedia data according to the richness value corresponding to each type of sub-data at each time node specifically includes:

[0024] For each time node in the multimedia data, determine a criticality value corresponding to the time node according to a richness value corresponding to each type of sub-data at the time node and a preset weight corresponding to each type of sub-data;

[0025] If the criticality value corresponding to the time node is greater than the preset criticality value, the feature point information is determined according to the information content corresponding to the time node.

[0026] Optionally, the extracting, by the information extraction model, information of each element point contained in the multimedia data as a step of predicting the element point information specifically includes:

[0027] Extracting each element point information contained in the multimedia data as predicted element point information through the information extraction model, and determining an evaluation value corresponding to each predicted element point information, wherein for each element point information, the evaluation value corresponding to the element point information is used to characterize the contribution of the element point information to the expected browsing situation of the multimedia data;

[0028] The step of training the information extraction model according to the predicted feature point information and the label feature point information specifically includes:

[0029] The information extraction model is trained according to the predicted feature point information, the label feature point information and the evaluation value.

[0030] Optionally, training the information extraction model according to the predicted element point information, the label element point information and the evaluation value specifically includes:

[0031] The information extraction model is trained with minimizing the deviation between the predicted feature point information and the label feature point information and maximizing the evaluation value as optimization goals.

[0032] This specification provides an information recommendation method, including:

[0033] Acquire multimedia data to be recommended;

[0034] Inputting the multimedia data to be recommended into a pre-trained information extraction model, so as to extract information of each element point contained in the multimedia data to be recommended through the information extraction model, wherein the information extraction model is trained by the above-mentioned model training method;

[0035] Information recommendation is performed according to information of each element point contained in the multimedia data to be recommended.

[0036] This specification provides a model training device, including:

[0037] An acquisition module, used to acquire multimedia data and determine each user's browsing record for the multimedia data in history, wherein the browsing record is used to reflect the interactive operations performed by each user when browsing the information content at each time node in the multimedia data;

[0038] A determination module, configured to determine, based on the browsing record, each element point information contained in the multimedia data as label element point information, wherein the element point information is used to characterize information content corresponding to a key time node in the multimedia data;

[0039] An extraction module, used for inputting the multimedia data into the information extraction model to be trained, so as to extract each element point information contained in the multimedia data as the predicted element point information through the information extraction model;

[0040] A training module is used to train the information extraction model according to the predicted feature point information and the label feature point information.

[0041] This specification provides an information recommendation device, including:

[0042] An acquisition module, used for acquiring multimedia data to be recommended;

[0043] An input module, used for inputting the multimedia data to be recommended into a pre-trained information extraction model, so as to extract information of each element point contained in the multimedia data to be recommended through the information extraction model, wherein the information extraction model is trained by the above-mentioned model training method;

[0044] The recommendation module is used to recommend information according to the information of each element point contained in the multimedia data to be recommended.

[0045] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above method is implemented.

[0046] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the above method is implemented when the processor executes the program.

[0047] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0048] In the information recommendation method provided in the present specification, multimedia data is obtained, and the browsing records of each user in history for the multimedia data are determined; based on the browsing records, each element point information contained in the multimedia data is determined as label element point information, and the element point information is used to characterize the information content corresponding to the key time nodes in the multimedia data; the multimedia data is input into the information extraction model to be trained, so as to extract each element point information contained in the multimedia data through the information extraction model as predicted element point information; the information extraction model is trained based on the predicted element point information and the label element point information.

[0049] It can be seen from the above method that during the model training process, this scheme can determine the feature point information in the multimedia data and use it as a label based on the user's actual browsing of the multimedia data. In this way, the model trained by the label can accurately extract the feature point information contained in the multimedia data. Using the feature point information for information recommendation can fully meet user expectations, ensure the accuracy of information recommendation, and further improve the information recommendation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0051] Figure 1 A flowchart of a model training method provided in this specification;

[0052] Figure 2 A schematic diagram of a process for training an information extraction model provided in this specification;

[0053] Figure 3 A flowchart of an information recommendation method provided in this specification;

[0054] Figure 4 A schematic diagram of a model training device provided in this specification;

[0055] Figure 5 A schematic diagram of an information recommendation device provided in this specification;

[0056] Figure 6 A method corresponding to the Figure 1 or Figure 3 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0058] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0059] Figure 1 A flowchart of an information recommendation method provided in this specification includes the following steps:

[0060] S101: Acquire multimedia data, and determine each user's historical browsing record for the multimedia data, wherein the browsing record is used to reflect the interactive operations performed by each user when browsing the information content of each time node in the multimedia data.

[0061] Advertising is crucial to business promotion, and the key points in advertising materials are important factors in determining advertising effectiveness. Traditional methods for extracting key points in advertising materials mainly rely on manual methods, which are inefficient and prone to errors. With the development of deep learning technology, it has become possible to extract key points in advertising materials using deep learning models.

[0062] Based on this, this specification provides a model training method, which extracts key point information in multimedia data (advertising materials) through a deep learning model, determines sample labels based on the user's browsing history of the multimedia data, and then trains the model, so that the model can accurately extract key points in the multimedia data and use the key point information for subsequent information recommendations.

[0063] In this specification, the execution entity used to implement a model training and information recommendation method can be a designated device such as a server, and of course, it can also be a client. For the sake of ease of description, the following will only take the server as the execution entity as an example to illustrate a model training and information recommendation method provided in this specification.

[0064] The server may obtain the multimedia data and the user's historical browsing records for the multimedia data.

[0065] The multimedia data may include at least one of video, text and audio. In this specification, the multimedia data may be advertising material in a target application scenario.

[0066] The browsing record is used to indicate the interactive operations performed by each user when browsing the information content of each time node in the multimedia data, such as the number of likes, the number of favorites, the click rate, the browsing rate, the playback rate, etc.

[0067] In other words, the historical recommendation information may be a time series of audio and video data, each frame of the time series corresponds to a browsing record, and the time series of the browsing records and the time series of the audio and video data are aligned in time scale.

[0068] After acquiring the multimedia data, the server can pre-process the multimedia data, wherein the server can process the multimedia data to obtain a data sequence of a specified number of frames. For example, for a 30-frame video, the server can extract an image every 3 frames to obtain a data sequence containing 10 frames of images, thereby achieving data compression and improving overall efficiency.

[0069] In addition, the server may further divide the multimedia data to obtain at least one type of sub-data, and the sub-data types may include text, image, and audio. Taking multimedia data composed of video, text, and audio as an example, the server may divide it into pure image data, pure audio data, and pure text data, and the data sequences of different types of data are aligned with each other in time.

[0070] It should be pointed out that the server can first determine a data sequence of a specified number of frames, and then divide the data sequence of the specified number of frames into different types of sub-data based on the data sequence of the specified number of frames as processed data. The server can also first divide the multimedia data into different types of sub-data based on the data sequence of the specified number of frames, and then determine the data sequence of the specified number of frames corresponding to the different types of sub-data. Of course, the server can also use any of these methods to pre-process the multimedia data to obtain processed data.

[0071] S102: Determine, according to the browsing record, each element point information contained in the multimedia data as label element point information, wherein the element point information is used to represent information content corresponding to a key time node in the multimedia data.

[0072] The server may determine, according to the browsing history, each element point information included in the multimedia data as the tag element point information.

[0073] In this specification, the element point information is used to characterize the information content corresponding to the key time node in the multimedia data. The key time node can be understood as the time point or time period where the interactive operation situation may change significantly, such as the time point where the user's click rate or browsing rate changes significantly, or the time point where the user stays for a long time.

[0074] Among them, the time node can refer to a frame or a moment in the time series of multimedia data. Of course, it can also refer to a specific time period. The length of each time period can be determined according to actual conditions, and this specification does not make specific restrictions on this.

[0075] Taking the browsing history containing the user's click rate and browsing rate as an example, the key time node can be the time node when the change in the browsing rate and / or click rate in the browsing history exceeds the preset change amount, and the label element point information can be the information content corresponding to the time node.

[0076] Of course, the key time node may also be a time node when the browsing rate and / or click rate exceeds the first preset threshold (i.e., a time node when the browsing rate and / or click rate is relatively high), and a time node when the browsing rate and / or click rate is lower than the second preset threshold (i.e., a time node when the browsing rate and / or click rate is relatively low). The above-mentioned preset change amount, the first preset threshold, and the second preset threshold may be set according to actual conditions, and this specification does not specifically limit this.

[0077] In addition, the server can also determine to obtain feedback data for the multimedia data, which is used to characterize the data content corresponding to the key time nodes in the multimedia data that each user has fed back when historically browsing the multimedia data (such as the user's targeted comments on the content at certain time nodes in the multimedia data).

[0078] In addition, the above information content includes not only images, audio and text content, but also special effects added to the video.

[0079] The server can then determine the information of each tag feature point based on the browsing history and feedback data.

[0080] S103: Inputting the multimedia data into the information extraction model to be trained, so as to extract each element point information contained in the multimedia data as predicted element point information through the information extraction model.

[0081] The server can input the multimedia data into a pre-trained information extraction model, and extract the information of each element point contained in the processed data through the information extraction model.

[0082] Specifically, when the multimedia data is preprocessed sub-data of different types, the server can input each sub-data into the information extraction model to be trained, so as to extract each element point information contained in the multimedia data as the predicted element point information through the information extraction model.

[0083] For each time node in the multimedia data, the information extraction model can determine the richness value corresponding to each type of sub-data at the time node. For each sub-data, the richer the information contained in the sub-data, the higher the richness value corresponding to the sub-data.

[0084] For text data, its richness can be determined based on the number of words in the text. The more words there are, the greater the richness value, and vice versa. For image data, its richness can be used to express the diversity of elements in the image (such as people, buildings, objects, etc.) and the vividness of colors. For audio data, its richness can be used to express the diversity of melody and pitch changes.

[0085] Afterwards, the information of each element point included in the multimedia data can be determined according to the richness value corresponding to each type of sub-data at each time node.

[0086] Among them, for each time node in the multimedia data, a weighted sum can be performed according to the richness value corresponding to each type of sub-data at the time node and the preset weight corresponding to each type of sub-data, so as to determine the criticality value corresponding to the time node.

[0087] If the criticality value corresponding to the time node is greater than the preset criticality value, it means that the time node is a critical time node in the multimedia data, and the server can determine the element point information according to the information content corresponding to the time node.

[0088] For example, the server may use the feature vector of the information content corresponding to the time node as the feature point information. Of course, the server may also directly use the information content corresponding to the time node as the feature point information.

[0089] Of course, the server may not pre-process the multimedia data, but directly input the multimedia data into the information extraction model, so as to extract the information of each element point contained in the multimedia data through the information extraction model.

[0090] S104: Training the information extraction model according to the predicted feature point information and the label feature point information.

[0091] After determining the predicted feature point information and the label feature point information, the server can train the information extraction model with minimizing the deviation between the predicted feature point information and the label feature point information as the optimization goal.

[0092] Furthermore, in the process of extracting predicted element point information through the information extraction model, the server can determine the evaluation value corresponding to each predicted element point information through the evaluation network integrated in the information extraction model. The evaluation value is used to characterize the contribution of the element point information to the expected browsing situation (such as click-through rate and browsing rate) of the multimedia data.

[0093] Among them, the greater the expected contribution of feature point information to the browsing situation, the greater the corresponding evaluation value, and vice versa.

[0094] The server may train the information extraction model according to the predicted feature point information, the label feature point information and the evaluation value.

[0095] The server can train the information extraction model by minimizing the deviation between the predicted feature point information and the label feature point information and maximizing the evaluation value corresponding to each predicted feature point information as the optimization goal. For ease of understanding, this specification provides a schematic diagram of the process of training the information extraction model, such as Figure 2 shown.

[0096] Figure 2 A schematic diagram of a process for training an information extraction model provided in this specification.

[0097] The server may collect multimedia data, which may be various game advertising materials, including images, videos, etc. The advertising materials are then preprocessed, such as grayscale and normalization of the images.

[0098] In addition, the server can construct a deep learning model for extracting feature point information in the advertising material as an information extraction model, and use the trained deep learning model to extract feature point information in the advertising material as predicted feature point information.

[0099] At the same time, the server can collect users' browsing records and feedback data on advertisements, such as click-through rate, viewing time, comments, etc. Then, the collected data is analyzed to find out the element point information related to the key points of the advertisement material as the label element point information. Then, the information extraction model is trained based on the predicted element point information and the label element point information.

[0100] After the training of the information extraction model reaches the training target (such as the model converges to a preset range, or reaches a preset number of training times), the server can deploy the information extraction model for subsequent information recommendation.

[0101] For ease of understanding, this specification provides a schematic diagram of an information recommendation method applied to an information extraction model trained by the above-mentioned model training method, such as Figure 3 shown.

[0102] Figure 3 A flowchart of an information recommendation method provided in this specification includes the following steps:

[0103] S301: Acquire multimedia data to be recommended.

[0104] S302: Inputting the multimedia data to be recommended into a pre-trained information extraction model, so as to extract information of each element point contained in the multimedia data to be recommended through the information extraction model, wherein the information extraction model is trained by the above-mentioned model training method.

[0105] S303: Recommend information according to the information of each element point included in the multimedia data to be recommended.

[0106] The server may obtain the multimedia data to be recommended, and then pre-process the multimedia data (please see step S101 for details), and then input the pre-processed multimedia data into the trained information extraction model.

[0107] The information extraction model can extract information of each element point contained in the multimedia data to be recommended, and make information recommendations based on the information of each element point.

[0108] In this specification, there can be multiple ways to recommend information based on element point information. Among them, the server can obtain the query words entered by the user and extract the query elements therein. At the same time, it determines the element point information corresponding to each piece of information to be recommended. Then, based on the matching degree between the query elements and the element point information corresponding to each piece of information to be recommended, the target information to be recommended is determined and recommended to the user.

[0109] In addition, the server may also optimize the multimedia information to be recommended based on the determined element point information, and recommend information to the user based on the optimized multimedia information.

[0110] Specifically, after determining each element point information included in the multimedia data, the server may evaluate each element point information through the evaluation network of the information extraction model to determine the evaluation value corresponding to each element point information.

[0111] In addition, for different business scenarios, different information content contributes differently to browsing situations. For example, in game scenarios, for action games, users may prefer tense and exciting fighting scenes and dynamic music, while for development games, what attracts users may be warm and comfortable game scenes and soothing music.

[0112] Therefore, in the process of evaluating the element point information through the evaluation model, the server can input each element point information and the scene information of the corresponding application scene into the evaluation model, so as to determine the evaluation value of each element point information in its corresponding application scene through the evaluation model, thereby ensuring that the determined evaluation value matches the scene information of the application scene.

[0113] In this specification, the above-mentioned application scenarios may include: product recommendations, game recommendations, film and television work recommendations, and service item recommendations, etc. The scenario information of the application scenarios may include classification information of the application scenarios. Taking the game recommendation scenario as an example, its scenario information may include the style type of the game, such as action, strategy, development, etc., and taking film and television works as an example, its scenario information may include the type of film and television works, such as martial arts, romance, reasoning, science fiction, etc.

[0114] After determining the evaluation value corresponding to each element point information, the server may optimize the multimedia data according to the evaluation value corresponding to each element point information.

[0115] Specifically, for each element point information, if the evaluation value corresponding to the element point information is lower than a preset threshold, the server may adjust the information content at the time node corresponding to the element point information to optimize the multimedia data.

[0116] Among them, for each element point information, if the evaluation value corresponding to the element point information is lower than the preset threshold, several element point information located before the time node corresponding to the element point information and several element point information located after the time node corresponding to the element point information will be used as the context information corresponding to the element point information.

[0117] Then, the information content at the time node corresponding to the element point information can be adjusted according to the context information, so as to obtain optimized multimedia data.

[0118] Of course, the server may also push the information of the feature points whose evaluation values ​​are lower than the preset threshold to the production staff in the form of information prompts, so that the production staff can adjust the multimedia data based on the information prompts.

[0119] After obtaining the optimized multimedia data, the server can recommend the optimized multimedia data to the user.

[0120] From the above method, it can be seen that this solution can more accurately extract key points in multimedia data and improve information recommendation effect. At the same time, using browsing records and feedback data to optimize the training of deep learning models can improve the accuracy and efficiency of key point extraction and save labor costs.

[0121] For example, the video of the advertising material contains elements such as the actions of the game characters and the environmental background. Through the method provided by this solution, the key points in the video can be extracted using the deep learning model first, and then the browsing history and feedback data of the advertisement can be collected, such as the user's click-through rate and comments. Then, based on this data, the training of the deep learning model is optimized and the key points in the advertising material are re-extracted. In this way, the key points in the advertising material can be extracted more accurately and the advertising effect can be improved.

[0122] The above are one or more implementation model training or information recommendation methods of this specification. Based on the same idea, this specification also provides corresponding model training or information recommendation devices, such as Figure 4 or Figure 5 shown.

[0123] Figure 4 A schematic diagram of a model training device provided in this specification includes:

[0124] The acquisition module 401 is used to acquire multimedia data and determine the browsing records of each user for the multimedia data in history, wherein the browsing records are used to reflect the interactive operations performed by each user when browsing the information content of each time node in the multimedia data;

[0125] A determination module 402 is used to determine, according to the browsing record, each element point information contained in the multimedia data as label element point information, wherein the element point information is used to represent the information content corresponding to the key time node in the multimedia data;

[0126] An extraction module 403, used for inputting the multimedia data into the information extraction model to be trained, so as to extract each element point information contained in the multimedia data as the predicted element point information through the information extraction model;

[0127] The training module 404 is used to train the information extraction model according to the predicted feature point information and the label feature point information.

[0128] Optionally, before the step of determining the information of each element point contained in the multimedia data according to the browsing record, the acquisition module 401 is further used to acquire feedback data for the multimedia data, the feedback data being used to characterize the data content corresponding to the key time nodes in the multimedia data fed back by each user when browsing the multimedia data in the past;

[0129] The determining module 402 is specifically configured to determine the information of each element point according to the browsing history and the feedback data.

[0130] Optionally, before the step of inputting the multimedia data into the information extraction model to be trained, the extraction module 403 is further used to divide the multimedia data to obtain at least one type of sub-data, the type including: text, image, audio;

[0131] The extraction module 403 is specifically configured to input the at least one type of sub-data into the information extraction model, so as to extract information of each element point contained in the multimedia data through the information extraction model.

[0132] Optionally, the extraction module 403 is specifically used to determine, for each time node in the multimedia data, a richness value corresponding to each type of sub-data at the time node; and determine information of each element point contained in the multimedia data based on the richness value corresponding to each type of sub-data at each time node.

[0133] Optionally, the extraction module 403 is specifically used to determine, for each time node in the multimedia data, a criticality value corresponding to the time node based on the richness value corresponding to each type of sub-data at the time node and a preset weight corresponding to each type of sub-data; if the criticality value corresponding to the time node is greater than the preset criticality value, determining the element point information based on the information content corresponding to the time node.

[0134] Optionally, the determination module 402 is specifically configured to extract, through the information extraction model, each element point information contained in the multimedia data as predicted element point information, and determine an evaluation value corresponding to each predicted element point information, wherein for each element point information, the evaluation value corresponding to the element point information is used to characterize the contribution of the element point information to the expected browsing situation of the multimedia data;

[0135] The training module 404 is specifically used to train the information extraction model according to the predicted feature point information, the label feature point information and the evaluation value.

[0136] Optionally, the training module 404 is specifically used to train the information extraction model by minimizing the deviation between the predicted feature point information and the label feature point information and maximizing the evaluation value as an optimization goal.

[0137] Figure 5 A schematic diagram of an information recommendation device provided in this specification includes:

[0138] An acquisition module 501 is used to acquire multimedia data to be recommended;

[0139] An input module 502 is used to input the multimedia data to be recommended into a pre-trained information extraction model, so as to extract information of each element point contained in the multimedia data to be recommended through the information extraction model, wherein the information extraction model is trained by the above-mentioned model training method;

[0140] The recommendation module 503 is used to recommend information according to the information of each element point contained in the multimedia data to be recommended.

[0141] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 and Figure 3 A model training and information recommendation method is provided.

[0142] This manual also provides Figure 6 The one shown corresponds to Figure 1 or Figure 2 A schematic diagram of the electronic device. Figure 6 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 or Figure 2 The model training or information recommendation method. Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0143] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0144] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0145] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0146] For the convenience of description, the above device is described by dividing it into various units according to its functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0147] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0148] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0149] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0150] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0151] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0152] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0153] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0154] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0155] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0157] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0158] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A model training method, characterized in that: include: Acquire multimedia data, and determine historical browsing records of each user for the multimedia data, wherein the browsing records are used to reflect the interactive operations performed by each user when browsing the information content at each time node in the multimedia data; Determine, according to the browsing record, each element point information contained in the multimedia data as label element point information, wherein the element point information is used to represent information content corresponding to a key time node in the multimedia data; The multimedia data is input into the information extraction model to be trained, so as to extract the information of each element point contained in the multimedia data as the predicted element point information through the information extraction model, wherein, for each time node in the multimedia data, the richness value corresponding to each type of sub-data contained in the multimedia data at the time node is determined, and the predicted element point information is determined according to the richness value corresponding to each type of sub-data at each time node; The information extraction model is trained according to the predicted feature point information and the label feature point information.

2. The method according to claim 1, characterized in that Before the step of determining information of each element point included in the multimedia data according to the browsing record, the method further includes: Acquire feedback data for the multimedia data, where the feedback data is used to represent data content corresponding to key time nodes in the multimedia data fed back by each user when browsing the multimedia data in the past; The step of determining the information of each element point contained in the multimedia data according to the browsing record specifically includes: The information of each element point is determined according to the browsing record and the feedback data.

3. The method according to claim 1, characterized in that Before the step of inputting the multimedia data into the information extraction model to be trained, the method further includes: Dividing the multimedia data to obtain at least one type of sub-data, the type including: text, image, and audio; The step of inputting the multimedia data into the information extraction model to be trained so as to extract information of each element point contained in the multimedia data through the information extraction model specifically includes: The at least one type of sub-data is input into the information extraction model, so as to extract information of each element point contained in the multimedia data through the information extraction model.

4. The method according to claim 1, characterized in that The step of determining the information of each element point included in the multimedia data according to the richness value corresponding to each type of sub-data at each time node specifically includes: For each time node in the multimedia data, determine a criticality value corresponding to the time node according to a richness value corresponding to each type of sub-data at the time node and a preset weight corresponding to each type of sub-data; If the criticality value corresponding to the time node is greater than the preset criticality value, the feature point information is determined according to the information content corresponding to the time node.

5. The method according to claim 1, characterized in that The step of extracting information of each element point contained in the multimedia data by using the information extraction model as a step of predicting the element point information specifically includes: Extracting each element point information contained in the multimedia data as predicted element point information through the information extraction model, and determining an evaluation value corresponding to each predicted element point information, wherein for each element point information, the evaluation value corresponding to the element point information is used to characterize the contribution of the element point information to the expected browsing situation of the multimedia data; The step of training the information extraction model according to the predicted feature point information and the label feature point information specifically includes: The information extraction model is trained according to the predicted feature point information, the label feature point information and the evaluation value.

6. The method according to claim 5, characterized in that Training the information extraction model according to the predicted element point information, the label element point information and the evaluation value specifically includes: The information extraction model is trained with minimizing the deviation between the predicted feature point information and the label feature point information and maximizing the evaluation value as optimization goals.

7. An information recommendation method, characterized in that: include: Acquire multimedia data to be recommended; Inputting the multimedia data to be recommended into a pre-trained information extraction model, so as to extract information of each element point contained in the multimedia data to be recommended through the information extraction model, wherein the information extraction model is trained by the model training method according to any one of claims 1 to 6 above; Information recommendation is performed according to information of each element point contained in the multimedia data to be recommended.

8. A model training device, characterized in that: include: An acquisition module, used to acquire multimedia data and determine each user's browsing record for the multimedia data in history, wherein the browsing record is used to reflect the interactive operations performed by each user when browsing the information content at each time node in the multimedia data; A determination module, configured to determine, based on the browsing record, each element point information contained in the multimedia data as label element point information, wherein the element point information is used to characterize information content corresponding to a key time node in the multimedia data; an extraction module, configured to input the multimedia data into an information extraction model to be trained, so as to extract information of each element point contained in the multimedia data as predicted element point information through the information extraction model, wherein, for each time node in the multimedia data, a richness value corresponding to each type of sub-data contained in the multimedia data at the time node is determined, and the predicted element point information is determined according to the richness value corresponding to each type of sub-data at each time node; A training module is used to train the information extraction model according to the predicted feature point information and the label feature point information.

9. An information recommendation device, characterized in that: include: An acquisition module, used for acquiring multimedia data to be recommended; An input module, used for inputting the multimedia data to be recommended into a pre-trained information extraction model, so as to extract information of each element point contained in the multimedia data to be recommended through the information extraction model, wherein the information extraction model is trained by the model training method according to any one of claims 1 to 6 above; The recommendation module is used to recommend information according to the information of each element point contained in the multimedia data to be recommended.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images