Model training method, device, equipment and storage medium

Abstract

The present application provides a model training method, apparatus, device and storage medium, the method comprising: obtaining at least one initial label and corresponding labeling personnel information of a first content to be labeled; inputting at least one initial label and corresponding labeling personnel information into a labeling page generation model to obtain at least one target label and page display parameter; sending at least one target label and page display parameter to a corresponding user device, so that the user device displays the labeling page corresponding to the first content to be labeled based on at least one target label and page display parameter; using at least one initial label and corresponding labeling personnel information as a state in a reinforcement learning algorithm, and at least one target label and page display parameter as an action in a reinforcement learning algorithm, to train the labeling page generation model. Thus, the diversity of the labeling pages can be guaranteed, and the user experience can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a model training method, apparatus, device and storage medium. Background Art

[0002] The security and health of multimedia content are of vital importance, so it is usually necessary to label multimedia content. In order to ensure the efficiency and accuracy of labeling, a combination of machine labeling and manual labeling is generally adopted. The process is as follows: the multimedia server can obtain the content to be labeled, and input the content to be labeled into the neural network model to obtain the label of the content to be labeled. Further, the content to be labeled and the corresponding label are sent to the user device. The user device renders the page to display the content to be labeled and the corresponding label on the labeled page, so that the labeler can manually label on the labeled page. However, the current labeling page is relatively simple. Summary of the invention

[0003] The present application provides a model training method, apparatus, device and storage medium, thereby ensuring the diversity of annotated pages and thereby improving user experience.

[0004] In a first aspect, a model training method is provided, comprising: obtaining at least one initial label and corresponding labeling personnel information of a first content to be labeled; inputting at least one initial label and corresponding labeling personnel information of the first content to be labeled into a labeling page generation model to obtain at least one target label and page display parameter of the first content to be labeled, wherein the labeling page generation model is a model based on a reinforcement learning algorithm; sending at least one target label and page display parameter of the first content to be labeled to a corresponding user device, so that the user device displays the labeling page corresponding to the first content to be labeled based on at least one target label and page display parameter of the first content to be labeled; using at least one initial label and corresponding labeling personnel information of the first content to be labeled as a state in the reinforcement learning algorithm, and using at least one target label and page display parameter of the first content to be labeled as an action in the reinforcement learning algorithm, so as to train the labeling page generation model.

[0005] In a second aspect, a page display method is provided, including: obtaining at least one initial label and corresponding labeling personnel information of the second content to be annotated; inputting at least one initial label and corresponding labeling personnel information of the second content to be annotated into the annotation page generation model trained by the above-mentioned model training method to obtain at least one target label and page display parameters of the second content to be annotated; sending at least one target label and page display parameters of the second content to be annotated to a corresponding user device, so that the user device displays the annotation page corresponding to the second content to be annotated based on at least one target label and page display parameters of the second content to be annotated.

[0006] In a third aspect, a model training device is provided, including: an acquisition module, an input module, a sending module and a training module, wherein the acquisition module is used to acquire at least one initial label of a first content to be annotated and corresponding annotation personnel information; the input module is used to input at least one initial label of the first content to be annotated and corresponding annotation personnel information into an annotation page generation model to obtain at least one target label and page display parameter of the first content to be annotated, and the annotation page generation model is a model based on a reinforcement learning algorithm; the sending module is used to send at least one target label and page display parameter of the first content to be annotated to a corresponding user device, so that the user device displays the annotation page corresponding to the first content to be annotated based on at least one target label and page display parameter of the first content to be annotated; the training module is used to use at least one initial label of the first content to be annotated and corresponding annotation personnel information as a state in the reinforcement learning algorithm, and at least one target label and page display parameter of the first content to be annotated as an action in the reinforcement learning algorithm, so as to train the annotation page generation model.

[0007] In a fourth aspect, a page display device is provided, including: an acquisition module, an input module and a sending module, the acquisition module is used to acquire at least one initial label of the second content to be annotated and the corresponding annotation personnel information; the input module is used to input at least one initial label of the second content to be annotated and the corresponding annotation personnel information into the annotation page generation model trained by the above-mentioned model training method, so as to obtain at least one target label and page display parameters of the second content to be annotated; the sending module is used to send at least one target label and page display parameters of the second content to be annotated to a corresponding user device, so that the user device displays the annotation page corresponding to the second content to be annotated based on the at least one target label and page display parameters of the second content to be annotated.

[0008] In a fifth aspect, an electronic device is provided, comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the method of the first aspect or the second aspect.

[0009] In a sixth aspect, a computer-readable storage medium is provided for storing a computer program, wherein the computer program enables a computer to execute the method of the first aspect or the second aspect.

[0010] Through the technical solution provided by the present application, since the information of the annotator is taken into consideration, and even the feature information of the content to be annotated and / or the preference information of the annotator can be combined, the trained model can output a variety of target labels and page display parameters, so that a variety of annotation pages can be presented to the annotators, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A schematic diagram of a system architecture involved in an embodiment of the present application;

[0013] Figure 2 A flowchart of a model training method provided in one embodiment of the present application;

[0014] Figure 3 A flowchart of a model training method provided in another embodiment of the present application;

[0015] Figure 4 A schematic diagram of the training process of the annotation page generation model provided in one embodiment of the present application;

[0016] Figure 5 A schematic diagram of the training process of a labeling page generation model provided in another embodiment of the present application;

[0017] Figure 6 A flow chart of a page display method provided in an embodiment of the present application;

[0018] Fig. 7A and Figure 7B Two schematic diagrams for the same marked page provided in one embodiment of the present application;

[0019] Fig. 8A and Figure 8B Two schematic diagrams for the same marked page provided for another embodiment of the present application;

[0020] Fig. 9A and Fig. 9B Two schematic diagrams for the same marked page provided in yet another embodiment of the present application;

[0021] Fig. 10A and Fig. 10B Two schematic diagrams for the same marked page provided for another embodiment of the present application;

[0022] Fig.11A and Fig. 11B Two schematic diagrams for the same marked page provided in yet another embodiment of the present application;

[0023] Fig. 12A and Fig. 12BTwo schematic diagrams for different annotated pages provided in yet another embodiment of the present application;

[0024] Fig.13 A schematic diagram of a model training device 1300 provided in an embodiment of the present application;

[0025] Fig.14 A schematic diagram of a page display device 1400 provided in an embodiment of the present application;

[0026] Fig.15 It is a schematic block diagram of an electronic device 1500 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0029] Before introducing the technical solution of this application, the following is an explanation of the relevant knowledge of this application:

[0030] Reinforcement Learning (RL) algorithm: is a field of machine learning that focuses on how a software agent should act in an environment to maximize the idea of ​​cumulative rewards. The basic elements of reinforcement learning: environment, agent, state, action, and reward.

[0031] The environment is an external system. The subject is in this system and can perceive this system and take certain actions based on the perceived state.

[0032] An agent is a system embedded in an environment that is able to change the state of the environment by taking actions.

[0033] A state is a time slice of the current environment.

[0034] Action refers to the behavior performed by the subject.

[0035] Reward is a scalar that refers to the reward given by the environment to the current action or state.

[0036] Q Learning: It is a value-based algorithm in reinforcement learning algorithms. Q is Q(s,a), which refers to the expected benefit of taking action a under state s at a certain moment. The environment will feedback corresponding rewards based on the subject's actions, so the main idea of ​​the algorithm is to construct the state (State) and action (Action) into a Q-table to store the Q value, and then select the action that can obtain the maximum reward based on the Q value.

[0037] Deep Q-Network (DQN) algorithm: It is an algorithm that combines deep learning and reinforcement learning. The motivation for this algorithm is that the storage space of the Q_table in the traditional reinforcement learning algorithm Q-learning is limited, while the states in the real world or even the virtual world are close to infinite. Therefore, it is impossible to build a Q_table that can store a very large state space. However, in machine learning, neural networks can be used to treat states and actions as inputs to the neural network, and then the Q value of the action is obtained after analysis by the neural network. In this way, there is no need to record the Q value in a table, but the neural network can be used to directly predict the Q value.

[0038] As mentioned above, the current annotation page is presented in a relatively simple manner. In order to solve this technical problem, the present application can present a diversified annotation page in combination with the annotation personnel information.

[0039] In some embodiments, the system architecture of the embodiments of the present application is as follows: Figure 1 shown.

[0040] Figure 1 This is a schematic diagram of a system architecture involved in an embodiment of the present application, including a user device 101, a data collection device 102, a training device 103, an execution device 104, a database 105 and a content library 106.

[0041] The data acquisition device 102 is used to read the training data from the content library 106 and store the read training data in the database 105. The training data involved in the embodiment of the present application includes the initial label of the content to be annotated and the corresponding annotator information, and optionally, may also include feature information of the content to be annotated and / or annotator preference information.

[0042] The training device 103 trains the annotation page generation model based on the training data maintained in the database 105 , so that the trained annotation page generation model can obtain the target label and page display parameters of the content to be annotated.

[0043] In addition, refer to Figure 1 , the execution device 104 is configured with an I / O interface 107 to exchange data with external devices. For example, the content to be annotated sent by the user device 101 is received through the I / O interface. The calculation module 108 in the execution device 104 processes the content to be annotated, obtains the initial label of the content to be annotated and the corresponding annotator information, or can also obtain the feature information of the content to be annotated and / or the annotator preference information. The trained annotation page generation model is used to process the obtained information, output the target label and page display parameters of the content to be annotated, and send the corresponding results to the user device 101 through the I / O interface.

[0044] The user device 101 may include a mobile phone, a tablet computer, a laptop computer, a PDA, a mobile internet device (MID), a desktop computer, or other terminal devices with a browser function installed.

[0045] The execution device 104 may be a server.

[0046] Exemplarily, the server may be a computing device such as a rack server, a blade server, a tower server or a cabinet server, etc. The server may be an independent test server or a test server cluster composed of multiple test servers.

[0047] In this embodiment, the execution device 104 is connected to the user device 101 through a network. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobilecommunication (GSM), Wideband Code Division Multiple Access (WCDMA), the 4th Generation (4G) network, the 5th Generation (5G) network, Bluetooth, wireless fidelity (Wi-Fi), a call network, etc.

[0048] It should be noted that Figure 1 It is only a schematic diagram of a system architecture provided by an embodiment of the present application. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. In some embodiments, the data acquisition device 102 and the user device 101, the training device 103 and the execution device 104 can be the same device. The database 105 can be distributed on one server or on multiple servers, and the content library 106 can be distributed on one server or on multiple servers.

[0049] The technical solutions of the embodiments of the present application are described in detail below through some embodiments. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0050] Figure 2 A flowchart of a model training method provided in an embodiment of the present application, which can be executed by any electronic device such as a mobile phone, tablet computer, laptop computer, PDA, MID, desktop computer, etc., for example, can be executed by Figure 1 The training device in the application is not limited to this. Figure 2 As shown, the method may include:

[0051] S210: Obtain at least one initial label of the first content to be annotated and corresponding annotator information;

[0052] S220: Inputting at least one initial tag of the first content to be annotated and corresponding annotator information into an annotation page generation model to obtain at least one target tag and page display parameters of the first content to be annotated;

[0053] S230: Sending at least one target tag and a page display parameter of the first content to be annotated to a corresponding user device, so that the user device displays an annotated page corresponding to the first content to be annotated based on the at least one target tag and the page display parameter of the first content to be annotated;

[0054] S240: Using at least one initial label of the first content to be annotated and the corresponding annotator information as the state in the reinforcement learning algorithm, and using at least one target label of the first content to be annotated and the page display parameter as the action in the reinforcement learning algorithm to train the annotated page generation model.

[0055] Optionally, in the embodiment of the present application, the content to be annotated may be a video to be annotated, an image, etc., but is not limited thereto. The video to be annotated here may be a short video to be annotated or a normal video to be annotated.

[0056] It should be understood that the first content to be annotated is used to train the content to be annotated of the annotation page generation model, wherein the first content to be annotated may be multiple. For example, for a short video platform, the electronic device can train the annotation page generation model once through the short video to be annotated of at least one annotator for one day.

[0057] Optionally, the electronic device may obtain at least one initial label of the first content to be annotated in the following manner, but is not limited thereto: the electronic device may input the first content to be annotated into at least one neural network model to obtain at least one initial label of the first content to be annotated. For example, for a short video platform, the electronic device may input a short video to be annotated into at least one video understanding model to obtain at least one initial label of the first content to be annotated.

[0058] Optionally, the at least one neural network model and the at least one initial label may be in a one-to-one correspondence. For example, there are two neural network models, one neural network model is used to identify whether the content of the short video to be labeled involves unhealthy content, so as to label the video to be labeled as involving unhealthy content; the other neural network model is used to identify the character behavior in the content of the short video to be labeled, so as to label the video to be labeled with labels about the character behavior, such as dance.

[0059] It should be understood that the annotator information corresponding to the first content to be annotated refers to the information of the annotator who manually annotates the first content to be annotated.

[0060] Optionally, the annotator information corresponding to the first content to be annotated includes at least one of the following, but is not limited to: age, gender, and on-the-job time of the annotator.

[0061] Optionally, the electronic device may also obtain feature information of the first content to be annotated and / or annotator preference information.

[0062] Optionally, the first content to be annotated may be a video to be annotated, and accordingly, the feature information of the first content to be annotated includes at least one of the following: an image, text, and audio of the video to be annotated.

[0063] Optionally, the electronic device may input the first content to be annotated into at least one neural network model to obtain feature information of the first content to be annotated. For example, there are three neural network models, which respectively extract image, text, and audio features of the video to be annotated.

[0064] Optionally, the neural network model used here to extract feature information of the first content to be labeled may be the same as or different from the neural network model used to obtain the initial label.

[0065] Optionally, the labeler's preference information may be determined based on the labeling behavior of the labeler. The electronic device may count the clicks of each labeler on each label position within a preset time period to determine which labels each labeler is more inclined to click. For example, labeler A is more inclined to click on labels at the front position, and labeler B is more inclined to click on labels at the middle position.

[0066] Optionally, the electronic device can input the click situation of each annotator on each label position within a preset time period into the neural network model to obtain the labels of which positions each annotator is more inclined to click.

[0067] It should be understood that the annotated page generation model is a model based on a reinforcement learning algorithm. For example, the annotated page generation model may be a DQN model or a Q Learning model, but is not limited thereto.

[0068] Optionally, the input of the annotation page generation model may include at least one initial tag of the content to be annotated and the corresponding annotator information, and may also include feature information of the content to be annotated and / or annotator preference information. The output of the annotation page generation model is at least one target tag of the content to be annotated and page display parameters.

[0069] In one possible implementation, at least one initial tag of the first content to be annotated has the same content and order as at least one target tag. For example, the first content to be annotated has two initial tags, namely, tag 1 and tag 2, where tag 1 is arranged before tag 2. After being processed by the annotation page generation model, two target tags are also obtained, namely, tag 1 and tag 2, where tag 1 is arranged before tag 2.

[0070] In another possible implementation, at least one initial tag of the first content to be annotated is the same as the content of at least one target tag but in a different order. For example, the first content to be annotated has two initial tags, namely, tag 1 and tag 2, where tag 1 is arranged before tag 2. After being processed by the annotation page generation model, two target tags are also obtained, namely, tag 1 and tag 2, where tag 1 is arranged after tag 2.

[0071] In another possible implementation, at least one initial tag of the first content to be annotated is not completely the same as the content of at least one target tag. For example, the first content to be annotated has two initial tags, namely, tag 1 and tag 2. After being processed by the annotation page generation model, three target tags are obtained, namely, tag 1, tag 2, and tag 3.

[0072] It should be understood that the corresponding user equipment is a user equipment corresponding to a labeler who labels the first content to be labeled.

[0073] Optionally, the page display parameters include: HTML parameters and / or CSS parameters of the annotated page, wherein the HTML parameters define the page structure of the annotated page, and the CSS parameters determine the page performance and style of the annotated page.

[0074] Optionally, if the input of the annotation page generation model also includes feature information of the content to be annotated and / or annotator preference information, the electronic device can use at least one of the feature information of the first content to be annotated and the annotator preference information, at least one initial label of the first content to be annotated and the corresponding annotator information as states in the reinforcement learning algorithm.

[0075] Optionally, in addition to determining the state and action in the reinforcement learning algorithm, the electronic device may also determine the reward of the reinforcement learning algorithm. Based on this, S240 may be implemented in the following achievable manner:

[0076] Alternatively, if Figure 3 As shown, before S240, it may include:

[0077] S310: Determine the time from display to completion of annotation of the annotated page corresponding to the first content to be annotated;

[0078] Accordingly, S240 may include:

[0079] S320: Use at least one initial label of the first content to be annotated and the corresponding annotator information as the state in the reinforcement learning algorithm, and use at least one target label of the first content to be annotated and the page display parameter as the action in the reinforcement learning algorithm, and determine the reward value in the reinforcement learning algorithm according to the time from display to completion of the annotated page corresponding to the first content to be annotated, so as to train the annotated page generation model.

[0080] Optionally, the user device can determine the time from the display to the completion of the annotation of the annotation page corresponding to the first content to be annotated, and push the time to the electronic device. For example, the display time of a short video to be annotated is t1, and the time when the annotator completes the annotation of the short video to be annotated is t2, then the time from the display to the completion of the annotation of the annotation page corresponding to the short video to be annotated is t2-t1, and the user device can push t2-t1 to the electronic device. Alternatively, the user device can send the display time and the annotation completion time corresponding to the annotation page to the electronic device, and the electronic device determines the time from the display to the completion of the annotation of the annotation page corresponding to the first content to be annotated based on these two times.

[0081] As described above, the first content to be annotated may be in multiple situations. For example, for a short video platform, the electronic device may train an annotation page generation model once through at least one annotator's short video to be annotated for one day.

[0082] In one achievable manner, the electronic device may sum the time from display to completion of annotation of the annotation pages corresponding to the plurality of first contents to be annotated to obtain a sum result; and determine the inverse of the sum result as a reward value in the reinforcement learning algorithm.

[0083] For example, for a short video platform, the electronic device can train the annotation page generation model once through at least one annotator's short videos to be annotated in one day. Assume that annotator A annotates N short videos to be annotated in one day, and the time from display to completion of the corresponding annotation pages is T(1), T(2)...T(N), respectively. Assuming that the annotation page generation model is trained through these N short videos to be annotated, the reward value can be calculated as the inverse of the sum of T(1), T(2)...T(N).

[0084] Figure 4 A schematic diagram of the training process of the annotation page generation model provided in an embodiment of the present application is shown in FIG. Figure 4As shown, for the short video to be annotated, the electronic device can input the short video to be annotated into the video understanding model to obtain at least one initial label of the short video to be annotated, and can also input the short video to be annotated into the feature extraction model to obtain the feature information of the short video to be annotated. In addition, the annotator information and the annotator preference information can also be obtained. The input of the annotation page generation model can include at least one initial label of the short video to be annotated, the corresponding annotator information, the feature information of the short video to be annotated, and the annotator preference information. The output of the annotation page generation model is at least one target label and page display parameters of the short video to be annotated. During the model training process, the feature information of the short video to be annotated, the annotator preference information, at least one initial label of the short video to be annotated, and the corresponding annotator information are used as the state in the reinforcement learning algorithm, and at least one target label and page display parameters of the short video to be annotated are used as the action in the reinforcement learning algorithm, and the reward value in the reinforcement learning algorithm is determined according to the time from the display to the completion of the annotation of the annotated page corresponding to the short video to be annotated, so as to train the annotation page generation model.

[0085] In another possible implementation, the electronic device may determine the annotation accuracy of the first content to be annotated; determine the reward value in the reinforcement learning algorithm according to the annotation accuracy of the first content to be annotated and the time from display to completion of the annotation page corresponding to the first content to be annotated. For example, when the first content to be annotated is a plurality of first contents to be annotated, the electronic device may sum the time from display to completion of the annotation page corresponding to the plurality of first contents to be annotated to obtain a first summation result; sum the annotation accuracy of the plurality of first contents to be annotated to obtain a second summation result; sum the inverse of the first summation result and the second summation result to obtain a third summation result; and determine the third summation result as the reward value in the reinforcement learning algorithm.

[0086] For example, for a short video platform, an electronic device can train a labeling page generation model once through at least one labeler's short videos to be labeled in one day. Assume that labeler A labels N short videos to be labeled in one day, and the time from display to labeling completion of their corresponding labeling pages is T(1), T(2)...T(N), and their corresponding labeling accuracy is M(1), M(2)...M(N). Assuming that the labeling page generation model is trained through these N short videos to be labeled, the reward value can be calculated as the reciprocal of the sum of T(1), T(2)...T(N), that is, P1, and the sum of M(1), M(2)...M(N) is P2. Finally, P1+P2 can be determined as the reward value in the reinforcement learning algorithm.

[0087] Figure 5 A schematic diagram of the training process of the annotation page generation model provided in another embodiment of the present application is shown in FIG. Figure 5As shown, for the short video to be annotated, the electronic device can input the short video to be annotated into the video understanding model to obtain at least one initial label of the short video to be annotated, and can also input the short video to be annotated into the feature extraction model to obtain the feature information of the short video to be annotated. In addition, the annotator information and the annotator preference information can also be obtained. The input of the annotation page generation model can include at least one initial label of the short video to be annotated, the corresponding annotator information, the feature information of the short video to be annotated, and the annotator preference information. The output of the annotation page generation model is at least one target label and page display parameters of the short video to be annotated. During the model training process, the feature information of the short video to be annotated, the annotator preference information, at least one initial label of the short video to be annotated, and the corresponding annotator information are used as the state in the reinforcement learning algorithm, and at least one target label and page display parameters of the short video to be annotated are used as the action in the reinforcement learning algorithm, and the reward value in the reinforcement learning algorithm is determined according to the annotation accuracy of the short video to be annotated and the time from display to completion of the annotated page corresponding to the short video to be annotated, so as to train the annotation page generation model.

[0088] In an embodiment of the present application, during the above-mentioned model training process, since the information of the annotation personnel is taken into consideration, it can even be combined with the feature information of the content to be annotated and / or the preference information of the annotation personnel, so that the trained model can output a variety of target labels and page display parameters, thereby presenting a variety of annotation pages to the annotators, thereby improving the user experience.

[0089] Furthermore, the electronic device can determine the reward value in the reinforcement learning algorithm according to the time from display to completion of the annotation page, so that a more suitable annotation page can be presented to each annotator, thereby improving the annotation efficiency.

[0090] Figure 6 A flowchart of a page display method provided in an embodiment of the present application, such as Figure 6 As shown, the method can be executed by any electronic device such as a mobile phone, a tablet computer, a laptop computer, a PDA, a MID, a desktop computer, etc. Figure 1 The execution device in the embodiment is executed, and this application does not limit this. Figure 6 As shown, the method may include:

[0091] S610: Obtain at least one initial label of the second content to be annotated and corresponding annotator information;

[0092] S620: Inputting at least one initial label of the second content to be annotated and the corresponding annotator information into the annotated page generation model trained by the above model training method to obtain at least one target label and page display parameters of the second content to be annotated;

[0093] S630: Send at least one target tag and page display parameters of the second content to be annotated to a corresponding user device, so that the user device displays an annotated page corresponding to the second content to be annotated based on the at least one target tag and page display parameters of the second content to be annotated.

[0094] As described above, the first content to be annotated is used to train the content to be annotated of the annotation page generation model, and the second content to be annotated in this embodiment is the content to be annotated involved in the model testing, that is, the execution stage.

[0095] Optionally, the second content to be annotated is a video to be annotated, and accordingly, the feature information of the second content to be annotated includes at least one of the following: an image, text, and audio of the video to be annotated.

[0096] Optionally, the annotator information corresponding to the second content to be annotated includes at least one of the following: age, gender, and on-the-job time of the annotator.

[0097] Optionally, the electronic device can also obtain at least one of the characteristic information of the second content to be annotated and the annotator's preference information; based on this, the electronic device can input at least one of the characteristic information of the second content to be annotated and the annotator's preference information, at least one initial label of the second content to be annotated and the corresponding annotator information into the annotation page generation model trained by the above-mentioned model training method to obtain at least one target label and page display parameters of the second content to be annotated.

[0098] It should be understood that the explanation of S610 and S620 may refer to the explanation of S210 and S220, and this application will not elaborate on this.

[0099] It should be understood that since the input of the annotation page generation model may include at least one initial label of the content to be annotated and the corresponding annotation personnel information, or may also include feature information of the content to be annotated and / or annotation personnel preference information. Therefore, for the same content to be annotated, such as the second content to be annotated mentioned above, for different annotators, at least one target label obtained may be different, and / or the page display parameters obtained may also be different. Accordingly, the annotation pages presented to different annotators may be different. Among them, the difference in at least one target label includes: differences in content, sorting, quantity, etc.

[0100] Example 1, Fig. 7A and Figure 7B Two schematic diagrams for the same annotation page are provided in one embodiment of the present application, such as Fig. 7AAs shown, for annotator A, since she is a female, 24 years old, and has been on the job for half a year, the annotator is more inclined to click on label 1 about whether it is beneficial to children's health. Then the annotation page presented by the electronic device to the annotator may include label 1 about whether it is beneficial to children's health. Figure 7B As shown, for the annotator B, since he is a male, 40 years old, and has been on the job for 10 years, the annotator is more inclined to click on the label 2 of the character behavior. Then the label in the middle of the annotation page presented by the electronic device to the annotator may include the label 2 of the character behavior.

[0101] Example 2, Fig. 8A and Figure 8B Two schematic diagrams for the same marked page are provided for another embodiment of the present application, such as Fig. 8A As shown in FIG. 1 , for annotator A, the order of the tags of the annotation page presented by the electronic device to the annotator is tag 1, tag 2, tag 3, and tag 4. Figure 8B As shown, for annotator B, the label order of the annotation page presented by the electronic device to the annotator is label 2, label 1, label 3, label 4.

[0102] Example 3, Fig. 9A and Fig. 9B Two schematic diagrams for the same marked page are provided for another embodiment of the present application, such as Fig. 9A As shown in FIG. 1 , for annotator A, the order of the tags of the annotated page presented by the electronic device to the annotator is tag 1, tag 2, and tag 3. Fig. 9B As shown, for annotator B, the label order of the annotation page presented by the electronic device to the annotator is label 1, label 2, label 3, label 4.

[0103] Example 4, Fig. 10A and Fig. 10B Two schematic diagrams for the same marked page are provided for another embodiment of the present application, such as Fig. 10A As shown in FIG. 1 , for annotator A, since she is a female, 24 years old, and has been on the job for half a year, she is more inclined to click on the tags at the front. Therefore, the tags at the front of the annotation page presented to the annotator by the electronic device are more prominent. Fig. 10B As shown, for annotator B, since he is a male, 40 years old, and has been on the job for 10 years, he is more inclined to click on the label in the middle position, so the label in the middle position of the annotation page presented to the annotator by the electronic device is more prominent.

[0104] Example 5, Fig.11A and Fig. 11B Two schematic diagrams for the same marked page are provided for another embodiment of the present application, such as Fig.11AAs shown in FIG. 1 , for annotator A, since she is a female, 24 years old, and has been on the job for half a year, the style of the annotation page presented by the electronic device to the annotator is more obvious, for example, different labels have different background patterns. Fig. 11B As shown, for the annotator B, since he is a male, 40 years old, and has been on the job for 10 years, the page style of the annotation page presented by the electronic device to the annotator is simpler, for example: different labels have no background.

[0105] It should be understood that for different contents to be annotated, for the same annotator, at least one target tag obtained may be different, and / or the page display parameters obtained may also be different. Accordingly, the annotation pages presented to different annotators may be different. The difference in at least one target tag includes: differences in content, order, quantity, etc.

[0106] Example 6, Fig. 12A and Fig. 12B Two schematic diagrams for different annotated pages are provided for another embodiment of the present application, such as Fig. 12A As shown, corresponding to the content 1 to be annotated, since the content 1 to be annotated contains sensitive content, the annotation page presented by the electronic device to the annotator can highlight these sensitive contents, such as enlarging / circling sensitive words or images. Fig. 12B As shown, corresponding to the content 2 to be annotated, since the content 2 to be annotated does not contain sensitive content, the annotation page presented by the electronic device to the annotator can display the annotation page normally.

[0107] It should be understood that for the same annotator, if the annotator information changes, such as the change in the on-the-job time, then even for the same content to be annotated, at least one target tag obtained may be different, and / or the page display parameters obtained may be different. Accordingly, the annotation pages presented to different annotators may be different. Among them, the difference in at least one target tag includes: differences in content, order, quantity, etc.

[0108] In the embodiment of the present application, since the information of the annotator is taken into consideration, it can even be combined with the feature information of the content to be annotated and / or the preference information of the annotator, so that a variety of annotation pages can be presented to the annotator, thereby improving the user experience.

[0109] Fig.13 A schematic diagram of a model training device 1300 provided in an embodiment of the present application is shown as follows: Fig.13As shown, the device 1300 includes: an acquisition module 1310, an input module 1320, a sending module 1330 and a training module 1340, wherein the acquisition module 1310 is used to acquire at least one initial label of the first content to be annotated and the corresponding annotation personnel information; the input module 1320 is used to input at least one initial label of the first content to be annotated and the corresponding annotation personnel information into the annotation page generation model to obtain at least one target label and page display parameters of the first content to be annotated, and the annotation page generation model is a model based on a reinforcement learning algorithm; the sending module 1330 is used to send at least one target label and page display parameters of the first content to be annotated to a corresponding user device, so that the user device displays the annotation page corresponding to the first content to be annotated based on the at least one target label and page display parameters of the first content to be annotated; the training module 1340 is used to use at least one initial label of the first content to be annotated and the corresponding annotation personnel information as the state in the reinforcement learning algorithm, and at least one target label and page display parameters of the first content to be annotated as the action in the reinforcement learning algorithm to train the annotation page generation model.

[0110] Optionally, the acquisition module 1310 is also used to: obtain at least one item of the characteristic information of the first content to be annotated and the annotation personnel's preference information; accordingly, the input module 1320 is specifically used to: input at least one item of the characteristic information of the first content to be annotated and the annotation personnel's preference information, at least one initial label of the first content to be annotated and the corresponding annotation personnel information into the annotation page generation model to obtain at least one target label and page display parameters of the first content to be annotated; the training module 1340 is specifically used to: use at least one item of the characteristic information of the first content to be annotated and the annotation personnel's preference information, at least one initial label of the first content to be annotated and the corresponding annotation personnel information as the state in the reinforcement learning algorithm.

[0111] Optionally, the device 1300 also includes: a determination module 1350, used to determine the time from the display to the completion of the annotation of the annotated page corresponding to the first content to be annotated; accordingly, the training module 1340 is specifically used to: use at least one initial label of the first content to be annotated and the corresponding annotation personnel information as the state in the reinforcement learning algorithm, at least one target label of the first content to be annotated and the page display parameter as the action in the reinforcement learning algorithm, and determine the reward value in the reinforcement learning algorithm according to the time from the display to the completion of the annotation of the annotated page corresponding to the first content to be annotated, so as to train the annotation page generation model.

[0112] Optionally, the first content to be annotated is a plurality of first contents to be annotated. Accordingly, the training module 1340 is specifically used to: sum the time from display to completion of annotation of the annotation pages corresponding to the plurality of first contents to be annotated to obtain a sum result; and determine the inverse of the sum result as the reward value in the reinforcement learning algorithm.

[0113] Optionally, the determination module 1350 is also used to determine the annotation accuracy of the first content to be annotated; accordingly, the training module 1340 is specifically used to determine the reward value in the reinforcement learning algorithm based on the annotation accuracy of the first content to be annotated and the time from the display to the completion of the annotation of the annotation page corresponding to the first content to be annotated.

[0114] Optionally, the first content to be annotated is multiple first contents to be annotated. Accordingly, the training module 1340 is specifically used to: sum the time from display to completion of annotation of the annotation pages corresponding to the multiple first contents to be annotated to obtain a first summation result; sum the annotation accuracy of the multiple first contents to be annotated to obtain a second summation result; sum the inverse of the first summation result and the second summation result to obtain a third summation result; and determine the third summation result as the reward value in the reinforcement learning algorithm.

[0115] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Fig.13 The device 1300 shown may perform Figure 2 The corresponding method embodiments, and the aforementioned and other operations and / or functions of each module in the device 1300 are respectively to implement Figure 2 For the sake of brevity, the corresponding processes in each method are not repeated here.

[0116] The above describes the device 1300 of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0117] Fig.14 A schematic diagram of a page display device 1400 provided in an embodiment of the present application is shown as follows: Fig.14As shown, the device 1400 includes: an acquisition module 1410, an input module 1420 and a sending module 1430, wherein the acquisition module 1410 is used to acquire at least one initial label of the second content to be annotated and the corresponding annotation personnel information; the input module 1420 is used to input at least one initial label of the second content to be annotated and the corresponding annotation personnel information into the annotation page generation model trained by the above-mentioned model training method to obtain at least one target label and page display parameters of the second content to be annotated; the sending module 1430 is used to send at least one target label and page display parameters of the second content to be annotated to the corresponding user device, so that the user device displays the annotation page corresponding to the second content to be annotated based on the at least one target label and page display parameters of the second content to be annotated.

[0118] Optionally, the acquisition module 1410 is also used to obtain at least one item of the characteristic information of the second content to be annotated and the annotator's preference information; accordingly, the input module 1420 is specifically used to: input at least one item of the characteristic information of the second content to be annotated and the annotator's preference information, at least one initial label of the second content to be annotated and the corresponding annotator information into the annotation page generation model trained by the above-mentioned model training method, so as to obtain at least one target label and page display parameters of the second content to be annotated.

[0119] Optionally, the second content to be annotated is a video to be annotated, and accordingly, the feature information of the second content to be annotated includes at least one of the following: an image, text, and audio of the video to be annotated.

[0120] Optionally, the annotator information corresponding to the second content to be annotated includes at least one of the following: age, gender, and on-the-job time of the annotator.

[0121] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Fig.14 The device 1400 shown may perform Figure 6 The corresponding method embodiments, and the aforementioned and other operations and / or functions of each module in the device 1400 are respectively to implement Figure 6 For the sake of brevity, the corresponding processes in each method are not repeated here.

[0122] The above describes the device 1400 of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0123] Fig.15 It is a schematic block diagram of an electronic device 1500 provided in an embodiment of the present application.

[0124] like Fig.15 As shown, the electronic device 1500 may include:

[0125] The memory 1510 and the processor 1520, the memory 1510 is used to store the computer program and transmit the program code to the processor 1520. In other words, the processor 1520 can call and run the computer program from the memory 1510 to implement the method in the embodiment of the present application.

[0126] For example, the processor 1520 may be configured to execute the above method embodiments according to instructions in the computer program.

[0127] In some embodiments of the present application, the processor 1520 may include but is not limited to:

[0128] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0129] In some embodiments of the present application, the memory 1510 includes but is not limited to:

[0130] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0131] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 1510 and executed by the processor 1520 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0132] like Fig.15 As shown, the electronic device may also include:

[0133] The transceiver 1530 may be connected to the processor 1520 or the memory 1510 .

[0134] The processor 1520 may control the transceiver 1530 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 1530 may include a transmitter and a receiver. The transceiver 1530 may further include an antenna, and the number of antennas may be one or more.

[0135] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0136] The present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0137] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0138] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0139] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0140] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0141] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model training method, characterized in that: include: Obtaining at least one initial label of the first content to be annotated and corresponding annotator information; Inputting at least one initial label of the first content to be annotated and corresponding annotator information into an annotation page generation model to obtain at least one target label and page display parameters of the first content to be annotated, wherein the annotation page generation model is a model based on a reinforcement learning algorithm; Sending at least one target tag and a page display parameter of the first content to be annotated to a corresponding user device, so that the user device displays an annotated page corresponding to the first content to be annotated based on the at least one target tag and the page display parameter of the first content to be annotated; Determine the time from when the annotated page corresponding to the first content to be annotated is displayed to when the annotator completes the annotating of the annotated page; At least one initial label of the first content to be annotated and the corresponding annotator information are used as states in a reinforcement learning algorithm, and at least one target label of the first content to be annotated and a page display parameter are used as actions in the reinforcement learning algorithm. The reward value in the reinforcement learning algorithm is determined according to the time from the display of the annotated page corresponding to the first content to be annotated to the completion of the annotating of the annotated page by the annotator, so as to train the annotated page generation model.

2. The method according to claim 1, characterized in that Also includes: Acquire at least one of the feature information of the first content to be annotated and the annotator preference information; The step of inputting at least one initial tag of the first content to be annotated and corresponding annotator information into an annotation page generation model to obtain at least one target tag of the first content to be annotated and page display parameters includes: Inputting at least one of the feature information of the first content to be annotated and the annotator preference information, at least one initial label of the first content to be annotated and the corresponding annotator information into the annotated page generation model to obtain at least one target label and page display parameters of the first content to be annotated; The step of using at least one initial label of the first content to be annotated and the corresponding annotator information as a state in a reinforcement learning algorithm includes: At least one of the feature information of the first content to be annotated and the annotator preference information, at least one initial label of the first content to be annotated and the corresponding annotator information is used as states in a reinforcement learning algorithm.

3. The method according to claim 1 or 2, characterized in that: The first content to be annotated is a plurality of first content to be annotated. Accordingly, determining the reward value in the reinforcement learning algorithm according to the time from when the annotated page corresponding to the first content to be annotated is displayed to when the annotator completes the annotating of the annotated page includes: Sum the time from when the annotated pages corresponding to the plurality of first to-be-annotated contents are displayed to when the annotators complete annotating the annotated pages, and obtain a summation result; The inverse of the summation result is determined as a reward value in the reinforcement learning algorithm.

4. The method according to claim 1 or 2, characterized in that: Also includes: Determining the annotation accuracy of the first content to be annotated; The step of determining the reward value in the reinforcement learning algorithm according to the time from when the annotated page corresponding to the first content to be annotated is displayed to when the annotator completes the annotating of the annotated page includes: The reward value in the reinforcement learning algorithm is determined according to the annotation accuracy rate of the first content to be annotated and the time from when the annotated page corresponding to the first content to be annotated is displayed to when the annotator completes the annotation of the annotated page.

5. The method according to claim 4, characterized in that The first content to be annotated is a plurality of first content to be annotated. Accordingly, determining the reward value in the reinforcement learning algorithm according to the annotation accuracy of the first content to be annotated and the time from when the annotated page corresponding to the first content to be annotated is displayed to when the annotator completes the annotation of the annotated page includes: Sum the time from when the annotated pages corresponding to the plurality of first to-be-annotated contents are displayed to when the annotators complete annotating the annotated pages, to obtain a first summation result; Summing the annotation accuracy rates of the plurality of first contents to be annotated to obtain a second summation result; Summing the reciprocal of the first summation result and the second summation result to obtain a third summation result; The third summation result is determined as a reward value in a reinforcement learning algorithm.

6. A page display method, characterized in that: include: Obtaining at least one initial label of the second content to be annotated and corresponding annotator information; Inputting at least one initial label of the second content to be annotated and corresponding annotator information into the annotation page generation model trained by the method according to any one of claims 1 to 5 to obtain at least one target label and page display parameters of the second content to be annotated; Sending at least one target tag and page display parameters of the second content to be annotated to a corresponding user device, so that the user device displays an annotated page corresponding to the second content to be annotated based on the at least one target tag and page display parameters of the second content to be annotated.

7. The method according to claim 6, characterized in that Also includes: Acquire at least one of the feature information of the second content to be annotated and the annotator preference information; The step of inputting at least one initial label of the second content to be annotated and corresponding annotator information into the annotated page generation model trained by the method according to any one of claims 1 to 5 to obtain at least one target label and page display parameters of the second content to be annotated includes: At least one of the feature information of the second content to be annotated and the annotator preference information, at least one initial label of the second content to be annotated and the corresponding annotator information are input into the annotation page generation model trained by the method described in any one of claims 1 to 5 to obtain at least one target label and page display parameters of the second content to be annotated.

8. The method according to claim 7, characterized in that The second content to be annotated is a video to be annotated. Accordingly, the feature information of the second content to be annotated includes at least one of the following: an image, text, and audio of the video to be annotated.

9. A model training device, characterized in that: include: An acquisition module, used to acquire at least one initial label of the first content to be annotated and corresponding annotator information; an input module, configured to input at least one initial label of the first content to be annotated and corresponding annotator information into an annotation page generation model, to obtain at least one target label and page display parameters of the first content to be annotated, wherein the annotation page generation model is a model based on a reinforcement learning algorithm; a sending module, configured to send at least one target tag and a page display parameter of the first content to be annotated to a corresponding user device, so that the user device displays an annotated page corresponding to the first content to be annotated based on the at least one target tag and the page display parameter of the first content to be annotated; A determination module, used to determine the time from when the annotated page corresponding to the first content to be annotated is displayed to when the annotator completes the annotating of the annotated page; A training module is used to use at least one initial label of the first content to be annotated and the corresponding annotator information as the state in the reinforcement learning algorithm, at least one target label of the first content to be annotated and the page display parameter as the action in the reinforcement learning algorithm, and determine the reward value in the reinforcement learning algorithm according to the time from the display of the annotated page corresponding to the first content to be annotated to the completion of the annotating of the annotated page by the annotator, so as to train the annotated page generation model.

10. A page display device, characterized in that: include: An acquisition module, used to acquire at least one initial label of the second content to be annotated and corresponding annotator information; An input module, used to input at least one initial label of the second content to be annotated and corresponding annotator information into the annotation page generation model trained by the method according to any one of claims 1 to 5, so as to obtain at least one target label and page display parameters of the second content to be annotated; The sending module is used to send at least one target tag and page display parameters of the second content to be annotated to the corresponding user equipment, so that the user equipment displays the annotated page corresponding to the second content to be annotated based on the at least one target tag and page display parameters of the second content to be annotated.

11. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program causes a computer to execute the method according to any one of claims 1 to 8.

Images