Information recommendation method, apparatus and device

Abstract

The application relates to the technical field of recommendation, and discloses an information recommendation method, device and equipment, the method comprising: obtaining a document to be read; performing text recognition on the document to be read to obtain a text to be read; performing named entity recognition on the text to be read to determine knowledge points in the text to be read; determining a knowledge point list to be recommended in the document to be read according to long-term preferences and short-term preferences of a user, and marking and displaying the knowledge point list in the document to be read; and in response to an event triggered by a user at a position of a target knowledge point in the document to be read, recommending learning content matched with the target knowledge point list to the user. The application combines long-term preferences and short-term preferences of a user to recommend a knowledge point list to the user, and marks knowledge points recommended in the document, so that the user can intuitively understand the knowledge points in the document.

Description

Technical Field

[0001] This application relates to the field of recommendation technology, and in particular, to an information recommendation method, apparatus, and device. Background Technology

[0002] In the insurance industry, both agents and clients have a need to learn about insurance. Current methods of learning insurance knowledge largely rely on users (agents or clients) actively learning by searching for product information on apps or search engines. However, users unfamiliar with insurance may not know what to search for. Therefore, it's necessary to extract insurance knowledge and present it to users. Furthermore, different users have different needs for knowledge points; randomly recommending knowledge points to users is not engaging. Therefore, a method is needed to recommend knowledge points based on user preferences. Summary of the Invention

[0003] The purpose of this application is to provide an information recommendation method, apparatus, and device that combines users' long-term and short-term preferences to recommend a list of knowledge points and marks the knowledge points in the document to make it easy for users to understand the knowledge points in the document at a glance.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to a first aspect of the embodiments of this application, a data annotation method is provided, the method comprising:

[0006] Get the document to be read;

[0007] The document to be read is subjected to text recognition to obtain the text to be read;

[0008] Named entity recognition is performed on the text to be read to identify the knowledge points in the text.

[0009] Based on the user's long-term and short-term preferences, a list of knowledge points to be recommended in the document to be read is determined, and the list of knowledge points is marked and displayed in the document to be read;

[0010] In response to an event triggered by the location of the target knowledge point in the document to be read by the user, learning content matching the list of target knowledge points is recommended to the user.

[0011] In some embodiments of this application, based on the foregoing scheme, the step of performing text recognition on the document to be read to obtain the text to be read includes:

[0012] Based on the FPN algorithm, the document to be read is processed into a feature map to obtain a probability map;

[0013] The probability map is made differentiable and binarized to obtain a binary map;

[0014] Find connected regions on the binary graph to obtain a set of text box data, which includes the position information of multiple text boxes;

[0015] The text information within the multiple text boxes is identified to obtain the text to be read.

[0016] In some embodiments of this application, based on the foregoing scheme, the step of performing named entity recognition on the text to be read to determine the knowledge points in the text to be read includes:

[0017] The text to be read is converted into a token sequence that meets preset conditions;

[0018] Based on the SPAN-BERT model, the token sequence is processed hierarchically to obtain the encoded context representation;

[0019] Based on a pointer network, the context representation is processed to predict the location of the knowledge points and determine the knowledge points in the text to be read.

[0020] In some embodiments of this application, based on the foregoing scheme, the pointer network includes a first classifier and a second classifier; the step of predicting the location of the knowledge point based on the pointer network and the context representation includes:

[0021] The first classifier obtains the candidate set of the starting positions of the knowledge point and the corresponding probabilities based on the context representation.

[0022] The second classifier obtains the candidate set of the end positions of the knowledge point and the corresponding probabilities based on the context representation.

[0023] Based on the maximum likelihood function, the position of the knowledge point is determined according to the probability of the starting position and the probability of the ending position.

[0024] In some embodiments of this application, based on the foregoing scheme, the information recommendation method further includes:

[0025] Construct an entity binary classification model based on the span model and the binary classification model;

[0026] The data is augmented using preset positive entity samples to obtain negative entity samples;

[0027] The entity binary classification model is trained based on the entity positive samples and the entity negative samples to obtain the trained entity binary classification model;

[0028] The confidence level of the knowledge point is determined based on the location of the knowledge point using the trained entity binary classification model.

[0029] Based on the confidence level of the knowledge points, the knowledge points in the document to be read are determined.

[0030] In some embodiments of this application, based on the foregoing scheme, the step of determining the list of knowledge points to be recommended in response to an event triggered by a user at the location of the knowledge point, according to the user's long-term and short-term preferences, includes:

[0031] A recommendation model is constructed, which includes an embedding layer, an attention layer, an LSTM network, and a fusion layer;

[0032] The sequence of knowledge points triggered by the user in the past is taken as the long-term sequence, and the sequence of knowledge points triggered by the user recently is taken as the short-term sequence.

[0033] Through the embedding layer, the feature information of the long-term sequence and the corresponding knowledge points, as well as the feature information of the short-term sequence and the corresponding knowledge points, are encoded into low-dimensional dense vectors.

[0034] Through the attention layer, the weights of each knowledge point to the user are obtained from the low-dimensional dense vector corresponding to the long-term sequence. The user's long-term preference is obtained by multiplying the weights with the vector formed by the feature information of the long-term sequence and the corresponding knowledge points.

[0035] The user's short-term preferences are obtained through the LSTM network based on the low-dimensional dense vector corresponding to the short-term sequence.

[0036] The fusion layer weights and fuses the user's long-term preferences and short-term preferences to obtain the user's final preferences. Based on the final preferences, the probability of the user reading each knowledge point again is determined, and a list of knowledge points is generated from high to low based on the reading probabilities.

[0037] In some embodiments of this application, based on the foregoing scheme, the feature information of the knowledge point includes the text recognition confidence level, entity recognition confidence level, font size, location of the knowledge point, number of documents containing the knowledge point, and type of the knowledge point.

[0038] In some embodiments of this application, based on the foregoing scheme, the information recommendation method further includes:

[0039] Using the SimBERT model, each knowledge point is embedded and vectorized to obtain the semantic vector of each knowledge point.

[0040] Based on the cosine similarity algorithm, the similarity between each knowledge point is determined according to the semantic vector of each knowledge point;

[0041] Based on the similarity between the knowledge points, duplicate knowledge points are removed.

[0042] According to a second aspect of the embodiments of this application, an information recommendation device is provided, the device comprising:

[0043] The document retrieval unit is used to retrieve documents to be read.

[0044] A text recognition unit is used to perform text recognition on the document to be read to obtain the text to be read.

[0045] The entity recognition unit is used to perform named entity recognition on the text to be read, determine the knowledge points in the text to be read, and annotate and display the knowledge points.

[0046] The recommendation unit is used to respond to an event triggered by the user at the location of the knowledge point, and to determine a list of knowledge points to be recommended based on the user's long-term and short-term preferences.

[0047] The recommendation unit is also used to recommend learning content that matches the knowledge point list to the user.

[0048] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising:

[0049] One or more processors;

[0050] A storage device for storing one or more programs that, when executed by one or more processors, cause the electronic device to perform the method described in the first aspect above.

[0051] This application's embodiments combine users' long-term and short-term preferences to recommend a list of knowledge points, and mark the knowledge points in the document for the recommended knowledge points, so that users can easily understand the knowledge points in the document at a glance. In addition, for the target knowledge point that the user is reading, explanatory videos are recommended to help the user quickly understand and learn the target knowledge point.

[0052] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0053] The above and other features and advantages of this application will become more apparent from the detailed description of exemplary embodiments thereof with reference to the accompanying drawings.

[0054] Figure 1A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown.

[0055] Figure 2 A flowchart of an information recommendation method according to an embodiment of this application is shown.

[0056] Figure 3 A flowchart of a text recognition method according to an embodiment of this application is shown.

[0057] Figure 4 A flowchart of a method for named entity recognition according to an embodiment of this application is shown.

[0058] Figure 5 A schematic diagram illustrating information recommendations according to one embodiment of this application is shown.

[0059] Figure 6 A schematic diagram of an information recommendation device according to an embodiment of this application is shown.

[0060] Figure 7 A schematic diagram of a program product for implementing the above method according to an embodiment of this application is shown.

[0061] Figure 8 A schematic diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0062] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0063] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0064] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0065] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily need to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0066] It should also be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such uses of these terms can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described.

[0067] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown.

[0068] like Figure 1 As shown, the system architecture may include terminal devices (such as...) Figure 1 The device shown includes one or more of a smartphone 101, a tablet 102, and a portable computer 103, a network 104, and a server 105. The network 104 serves as a medium to provide a communication link between the terminal device and the server 105. The terminal device and the server 105 are connected via the network 104, and the network can include various connection types, such as wired communication links, wireless communication links, etc.

[0069] The information recommendation method provided in this application embodiment can be executed by server 105, and the knowledge recommendation result of the document to be read can be sent to the terminal device through the network, so that the user can view the document to be read through the terminal device.

[0070] It should also be noted that, Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (content distribution network), and big data and artificial intelligence platforms. Terminals can be smartphones, tablets, laptops, desktop computers, smart TVs, etc., but are not limited to these, and this application does not impose any restrictions.

[0071] It's important to explain that cloud computing, as described above, is a computing model that distributes computing tasks across a resource pool comprised of numerous computers. This allows various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, the resources in the "cloud" appear infinitely scalable and can be accessed, used on demand, and expanded at any time. By establishing a cloud computing resource pool (or simply a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform), various types of virtual resources are deployed within the resource pool for external customers to choose from. The cloud computing resource pool primarily includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0072] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0073] Figure 2 A flowchart of a data annotation method according to an embodiment of this application is shown. Figure 2 As shown, the method includes at least the following steps.

[0074] Step 210: Obtain the document to be read.

[0075] The document to be read may be a PPT or PDF document containing images, such as an insurance introduction PPT or PDF document that the user is looking at. The document contains a lot of professional insurance knowledge points, which the user may not understand while reading and may not know where to ask questions.

[0076] Step 220: Perform text recognition on the document to be read to obtain the text to be read.

[0077] Documents in formats such as PPT or PDF contain not only text but also images. Therefore, text recognition technology is needed to identify the text in the text and images of the document to prepare for subsequent identification of named entities in the text.

[0078] Step 230: Perform named entity recognition on the text to be read to identify the knowledge points in the text.

[0079] This application extracts knowledge points from the text to be read through named entity recognition. The entities to be identified include insurance terms, insurance product names, and disease names in insurance-related documents.

[0080] Step 240: Based on the user's long-term and short-term preferences, determine the list of knowledge points to be recommended in the document to be read, and mark and display the list of knowledge points in the document to be read.

[0081] Based on the user's long-term preferences (i.e., the knowledge points the user has been interested in historically, including knowledge points triggered in all documents the user has read in the past) and short-term preferences (i.e., the knowledge points the user has been interested in recently, including a preset number of knowledge points triggered in documents the user has recently read), a targeted list of knowledge points to be recommended to the user is determined. The knowledge points in the recommended list are marked and displayed in the document, for example, by making the knowledge points bold, underlined, changing the font color, highlighting, etc., to facilitate user reading.

[0082] Step 250: In response to an event triggered by the user's location of the target knowledge point in the document to be read, recommend learning content that matches the target knowledge point to the user.

[0083] By combining the position information of the text box corresponding to the knowledge point on the screen in step 220 text recognition, the position of the knowledge point on the screen can be determined, and the events triggered by the user at the location of the knowledge point can be monitored, such as when the user moves the mouse to the location of the target knowledge point or clicks on the location of the target knowledge point. For example, based on the principle of fuzzy matching, the explanatory video corresponding to the target knowledge point can be recommended to the user from the video database, and the target knowledge point can be explained to the user in the form of short videos, so as to facilitate the user's quick learning.

[0084] This application's embodiments combine users' long-term and short-term preferences to recommend a list of knowledge points, and mark the knowledge points in the document for the recommended knowledge points, so that users can easily understand the knowledge points in the document at a glance. In addition, for the target knowledge point that the user is reading, explanatory videos are recommended to help the user quickly understand and learn the target knowledge point.

[0085] Figure 3 A flowchart illustrating a text recognition method according to an embodiment of this application is shown. Figure 3 As shown, the text recognition method includes at least the following steps.

[0086] Step 310: Based on the FPN algorithm, perform feature map processing on the document to be read to obtain a probability map.

[0087] Step 320: Differentiable binarization of the probability map to obtain a binary map.

[0088] Step 330: Find connected regions on the binary graph to obtain a set of text box data, which includes the position information of multiple text boxes.

[0089] Text recognition comprises two main parts: text detection and text recognition. In practice, a DB-based text detection model can be used to implement the aforementioned text detection method. This model includes a feature extraction module, a probabilistic map prediction module, and a binarization module. The feature extraction module can employ a ResNet-18 or ResNet-50 deep convolutional neural network, using a feature pyramid (FPN) to fuse feature maps of different sizes, thereby extracting features from the segmented image of the input document. The extracted features are then input into the probabilistic map prediction module to obtain a probabilistic map. This module can consist of a 3x3 convolutional layer and two deconvolutional layers with a stride of 2. The probabilistic map is then input into the binarization module to obtain a binary map. Finally, connected components are found in the binary map to obtain the text boxes.

[0090] Step 340: Identify the text information within multiple text boxes to obtain the text to be read.

[0091] In practical implementation, a CRNN model can be used for text recognition. The document and its corresponding text boxes are input into the CRNN text recognition network model to determine the text information within each text box. The CRNN text recognition network includes a CNN module, a BiLSTM (Bi-directional Long Short-Term Memory) module, and a CTC network structure. The CNN module adopts the MobileNetV3-small structure to extract features from the text image; the BiLSTM module uses the extracted feature images for feature vector fusion to extract contextual features of the character sequence, obtaining the probability distribution of each feature column; the CTC network structure inputs the latent vector probability distribution to predict the text sequence.

[0092] Figure 4 A flowchart illustrating a named entity recognition method according to an embodiment of this application is shown. Figure 4 As shown, the named entity recognition method includes at least the following steps.

[0093] Step 410: Convert the text to be read into a token sequence that meets preset conditions.

[0094] The text to be read contains multiple sentences from the document. For example, a sentence might be, "Party B should strictly administer medication according to the principle of a 3-day supply for acute illnesses, a 7-day supply for chronic illnesses, and a maximum of 24 days." In this sentence, "acute illness" and "chronic illness" are named entities and need to be identified. Before using the model for named entity recognition, each sentence needs to be converted into a sequence of tokens in the form of individual characters. In specific implementations, converting the text to be read into a token sequence that meets preset conditions can include:

[0095] Convert characters in the text into numeric tokens; characters that cannot be matched are... <unk>To replace, completing the initial conversion;

[0096] Add start and end markers [CLS] and [SEP] before and after the sentence token;

[0097] The sentence token is truncated and padded according to the set length, with the padded token being 0.

[0098] The above sentence can be translated as "[CLS] Party B shall strictly adhere to the principle of administering medication according to a 3-day supply for acute illnesses, a 7-day supply for chronic illnesses, and a maximum supply of 24 days. [SEP]"

[0099] Step 420: Based on the SPAN-BERT model, perform hierarchical computation on the token sequence to obtain the encoded context representation.

[0100] In practical implementation, based on the SPAN-BERT model, hierarchical computation is performed on the token sequence to obtain the encoded context representation, which may include:

[0101] The sentence token sequence is taken as input and connected by bidirectional Transformers in SPAN-BERT to compute the context representation of the sentence in a hierarchical manner; the output of the last Transformer is used as the final context representation.

[0102] The Transformer itself is a Seq2Seq structure, using Attention instead of the traditional LSTM as the encoder to achieve parallel computation. The encoder consists of N identical layers, each containing two sub-layers: a multi-head self-attention mechanism and a fully connected feed-forward network. Each sub-layer also contains residual and normalization layers.

[0103] Step 430: Based on the pointer network, process the context representation, predict the location of knowledge points, and determine the knowledge points in the text to be read.

[0104] For example, in the sentence "Party B shall strictly administer medication according to the principle of 3 days' supply for acute illnesses, 7 days' supply for chronic illnesses, and a maximum of 24 days' supply," the pointer network predicts that an entity in the sentence starts at position 8 and ends at position 10. Therefore, it can be determined that "acute illness" is an entity.

[0105] It should be noted that before performing hierarchical calculations on the token sequence to obtain the encoded context representation, the process also includes training the SPAN-Bert model. The model training process may include the following steps:

[0106] (1) A dataset is obtained based on the preset entity types and the manual annotation results of some documents;

[0107] For insurance-related documents, named entity types include insurance terms, product names, and disease names. BERT is based on a supervised training model; therefore, before training, a portion of the document's text content was manually annotated, highlighting key knowledge points to obtain named entities and their corresponding locations, generating a dataset in <text, entity> format. For example, a sentence from a document, "Party B should strictly administer medication according to the principle of 3 days' dosage for acute illnesses, 7 days' dosage for chronic illnesses, and a maximum of 24 days' dosage," is used to generate data in the format <Party B should strictly administer medication according to the principle of 3 days' dosage for acute illnesses, 7 days' dosage for chronic illnesses, and a maximum of 24 days' dosage, acute illness, chronic illness>.

[0108] By using this method of training with the document to be identified, the BERT model can maintain a certain level of generalization ability while having a strong feature extraction capability from the document to be identified, thereby improving the accuracy of the BERT model in extracting features.

[0109] (2) Clean and structure the obtained dataset.

[0110] Specifically, this includes data cleaning, removing illegal characters, spaces, and line breaks from the text; segmenting the text at the word level and constructing a dictionary; and constructing an entity type dictionary.

[0111] Before generating training samples from the dataset, the dataset is cleaned to avoid introducing unnecessary noise during subsequent model training. A dictionary is then constructed to facilitate the mapping of each character in the token sequence to its index value between steps 410 and 420, thus converting the characters into numeric input parameters that SPAN-Bert can recognize. As mentioned earlier, for insurance-related documents, one entity type dictionary is {insurance terms, product names, disease names}.

[0112] (3) Based on the cleaned and structured dataset, construct a positive sample set and a negative sample set.

[0113] The samples are represented using the Span format, and the positive sample set consists of a labeled entity dataset. For example, "acute disease" in the sentence above is an entity of the disease name type. Its corresponding positive sample in the Span format is a triple (8, 10, 3). The first two digits represent a span, specifically the start and end positions of the entity in the sentence, and the third digit represents the type of the entity, specifically the position of the entity type in the entity type dictionary.

[0114] Ten percent of the characters in a sentence pair are randomly masked to obtain a negative sample set. This employs the masking mechanism used in BERT model training. The masked characters can be obtained from the predicted characters of a pre-trained prediction model. This random character masking method allows the model to make judgments during training, resulting in a model with stronger generalization and feature extraction capabilities. The sample set is then input into the initial BERT model for training, and the gradient descent algorithm is used to obtain the final BERT model.

[0115] In some embodiments of this application, based on the foregoing scheme, the pointer network includes a first classifier and a second classifier; based on the pointer network, the location of the knowledge point is predicted according to the context representation, including:

[0116] The first classifier is used to obtain the candidate set of the starting positions of knowledge points and their corresponding probabilities based on the context representation.

[0117] The second classifier is used to obtain the candidate set of the end positions of knowledge points and their corresponding probabilities based on the context representation.

[0118] Based on the maximum likelihood function, the location of knowledge points is determined according to the probability of the start position and the probability of the end position.

[0119] Traditional Seq2Seq models cannot address the issue that the output sequence vocabulary changes with the length of the input sequence; for such problems, the output is often a subset of the input set. Pointer networks, on the other hand, directly manipulate the input sequence instead of setting an output vocabulary, thus mapping pointers to elements of the input sequence. This characteristic of output elements originating from input elements makes pointer networks ideal for directly copying certain elements from the input sequence to the output sequence. This is a very effective approach for extraction tasks, and predicting only the start and end positions easily solves the classification consistency problem that previous sequence labeling tasks had tried to address.

[0120] For example, binary start and end labels can be used to mark the beginning and end positions of a sentence. In the start layer label, a token with a "1" indicates that the token is the starting position of a knowledge point, and in the end layer label, a token with a "0" indicates that the token is the ending position of a knowledge point. A classifier can then be used to obtain the probability that each token in the sentence is "1" or "0", thus obtaining a candidate set of start positions and the probability of each element in the candidate set.

[0121] In some embodiments of this application, based on the foregoing scheme, the information recommendation method further includes:

[0122] Construct an entity binary classification model based on the span model and the binary classification model;

[0123] The data is augmented using preset positive entity samples to obtain negative entity samples;

[0124] The entity binary classification model is trained based on entity positive samples and entity negative samples to obtain the trained entity binary classification model;

[0125] The confidence level of a knowledge point is determined based on its location using the trained entity binary classification model.

[0126] Based on the confidence level of the knowledge points, identify the knowledge points in the document to be read.

[0127] This application implements an entity binary classification model based on the BERT model and a binary classification model. This model can determine whether an input entity is a real entity or a fake entity, i.e., the confidence level of the input entity. Data from an existing entity database is used as positive entity samples for training. Based on the existing entity database, a large number of negative samples are generated using data augmentation techniques as negative entity samples for training. The entity binary classification model is trained using both positive and negative entity samples to obtain the trained entity binary classification model.

[0128] In this embodiment, instead of directly using the output of the pointer network as the final entity recognition result, the trained entity binary classification model is connected to the output of the pointer network. The knowledge points are obtained by their positions in the sentence sequence. The entity binary classification model can then derive the confidence level of these knowledge points. If the confidence level is greater than or equal to a preset threshold, the knowledge point is considered a real entity; if the confidence level is less than the preset threshold, the knowledge point is considered a fake entity. For example, in the sentence above, if the entity binary classification model determines that the entity "acute illness" output by the pointer network has a confidence level of 95%, then the entity is ultimately determined to be a real entity. If the entity binary classification model determines that the entity "Party B" output by the pointer network has a confidence level of 30%, then the entity is ultimately determined to be a fake entity.

[0129] It should be noted that during the training of the SPAN-BERT model and pointer network, multi-task joint training can be performed in conjunction with the trained entity binary classification model. The entity confidence output by the entity binary classification model can be used to promote the iterative training of the SPAN-BERT model and pointer network, thereby improving the entity recognition accuracy of the SPAN-BERT model and pointer network.

[0130] In some embodiments of this application, based on the foregoing scheme, in response to an event triggered by a user at the location of a knowledge point, a list of knowledge points to be recommended is determined according to the user's long-term and short-term preferences, including:

[0131] Construct a recommendation model, which includes an embedding layer, an attention layer, an LSTM network, and a fusion layer;

[0132] The sequence of knowledge points triggered by the user in the past is regarded as the long-term sequence, and the sequence of knowledge points triggered by the user recently is regarded as the short-term sequence.

[0133] Through the embedding layer, the feature information of long-term sequences and corresponding knowledge points and the feature information of short-term sequences and corresponding knowledge points are encoded into low-dimensional dense vectors.

[0134] Through the attention layer, the weights of each knowledge point to the user are obtained from the low-dimensional dense vector corresponding to the long-term sequence. The weights are multiplied by the vector formed by the feature information of the long-term sequence and the corresponding knowledge points, and then accumulated to obtain the user's long-term preference.

[0135] By using an LSTM network, the user's short-term preferences can be obtained from the low-dimensional dense vectors corresponding to short-term sequences.

[0136] The fusion layer weights and merges users' long-term and short-term preferences to obtain their final preferences. Based on these final preferences, the probability of users reading each knowledge point again is determined, and a list of knowledge points is generated from high to low based on the reading probability.

[0137] A user's viewing of a document is called a Session. Each viewing action is based on a specific need, and there can be significant differences between different Sessions. However, if only a single Session is considered, it is impossible to effectively utilize the user's long-term preferences for recommendation modeling. Figure 5 A schematic diagram of another information recommendation method according to an embodiment of this application is shown. Figure 5 As shown in the embodiment of this application, a recommendation model that integrates users' long-term memory and short-term preferences is constructed. First, feature engineering is performed to obtain the feature information corresponding to the long-term sequence and the short-term sequence. Then, knowledge point recommendations are made to users through the embedding layer, attention layer, LSTM network and fusion layer of the model.

[0138] In some embodiments of this application, based on the foregoing scheme, the feature information of a knowledge point may include the text recognition confidence level, entity recognition confidence level, font size, location of the knowledge point, number of documents containing the knowledge point, and type of the knowledge point.

[0139] Knowledge points contain multidimensional feature information. By using the multidimensional feature information of knowledge points, the above recommendation model can better learn the features of knowledge points learned by users in the long and short term, and output a recommendation list that is closer to the user's true preferences.

[0140] Among them, the text recognition confidence score, which is the confidence score of the output of the above text recognition model, is one of the factors that affect the final user's click on knowledge points. Although the influence weight is not high, this application still uses it as feature information to participate in the recommendation.

[0141] Entity recognition confidence score is the confidence score of the entity output by the above-mentioned entity binary classification model. It has a significant impact on subsequent user clicks because words with higher confidence scores are more likely to be professional terms in the insurance field. Therefore, this application uses it as feature information for recommendation.

[0142] Font size can be determined by the size of the text box obtained in the above text detection. The parts of the document with larger fonts are usually the content that the author emphasizes, while the notes with smaller fonts are likely to be ignored by users and are not attractive even if they are identified. Therefore, this application uses them as feature information for recommendation.

[0143] The location of knowledge points can be determined by the position of the text box obtained in the above text detection. Based on viewing experience, the center position of the screen is more obvious and easier to pay attention to, while the edge position is easy to ignore. Therefore, this application uses it as feature information for recommendation.

[0144] The number of documents that mention a knowledge point can be determined based on the knowledge point statistics of each document. If a knowledge point is mentioned in multiple documents, it means that it is a more frequently mentioned knowledge point. If the knowledge point is mentioned less often, it means that it is a less popular knowledge point. Therefore, this application uses it as a feature information for recommendation.

[0145] It should be noted that, in addition to the feature information of the above dimensions, other feature information such as the type of knowledge point and the page number where the knowledge point is located can also be used as feature information for recommendation.

[0146] In some embodiments of this application, based on the foregoing scheme, the method further includes:

[0147] By using the SimBERT model, each knowledge point is embedded and vectorized to obtain the semantic vector of each knowledge point.

[0148] Based on the cosine similarity algorithm, the similarity between knowledge points is determined according to the semantic vector of each knowledge point;

[0149] Based on the similarity between different knowledge points, duplicate knowledge points are removed.

[0150] The knowledge points to be compared are imported into a pre-trained BERT model to obtain semantic vectors for each knowledge point. These semantic vectors, leveraging the feature extraction capabilities of the BERT model, dynamically express the relationships between words, word positions, and sentences within the knowledge points. This reflects the characteristics of the knowledge points from multiple perspectives, including words and sentences, thus improving the accuracy of subsequent text similarity calculations. Employing cosine similarity, which focuses on direction as a key measure of similarity, highlights text similarity and further enhances the accuracy of text similarity assessments.

[0151] This application embodiment combines users' long-term and short-term preferences to recommend a list of knowledge points, and marks the knowledge points in the document with the recommended knowledge points. The recommended knowledge points are not repeated in the document, which makes it easy for users to understand the knowledge points in the document at a glance. In addition, for the target knowledge point that the user is reading, the application recommends explanatory videos to help the user quickly understand and learn the target knowledge point.

[0152] The following describes embodiments of the information recommendation device of this application, which can be used to execute the information recommendation method described above in this application. For details not disclosed in the embodiments of the information recommendation device of this application, please refer to the embodiments of the information recommendation method described above in this application.

[0153] Figure 6 A schematic diagram of an information recommendation device according to an embodiment of this application is shown. Figure 6 As shown, the information recommendation device includes at least a document acquisition unit 610, a text recognition unit 620, an entity recognition unit 630, and a recommendation unit 640.

[0154] Document acquisition unit 610 is used to acquire documents to be read;

[0155] The text recognition unit 620 is used to perform text recognition on the document to be read to obtain the text to be read;

[0156] The entity recognition unit 630 is used to perform named entity recognition on the text to be read, determine the knowledge points in the text to be read, and mark and display the knowledge points.

[0157] The recommendation unit 640 is used to respond to an event triggered by the user at the location of the knowledge point, and to determine a list of knowledge points to be recommended based on the user's long-term and short-term preferences.

[0158] The recommendation unit 640 is also used to recommend learning content that matches the knowledge point list to the user.

[0159] It should be noted that although several units of the information recommendation method and information recommendation device have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, two or more units and functions described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the solution in this application according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0160] In another aspect, this application also provides a computer-readable storage medium having a program product stored thereon capable of implementing the methods described above in this specification. In some possible implementations, various aspects of this application may also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to the various exemplary embodiments of this application.

[0161] refer to Figure 7 As shown, a program product 700 for implementing the above-described method according to an embodiment of this application is described. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product of this application is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0162] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0163] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0164] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0165] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0166] In another respect, this application also provides an electronic device capable of implementing the above-described method.

[0167] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented as: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0168] The following reference Figure 8 To describe an electronic device 800 according to this embodiment of the present application. Figure 8 The electronic device 800 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0169] like Figure 8 As shown, the electronic device 800 is manifested in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including storage unit 820 and processing unit 810).

[0170] The storage unit stores program code that can be executed by the processing unit 810, causing the processing unit 810 to perform the steps described in the "Embodiment Methods" section above according to various exemplary embodiments of this application.

[0171] Storage unit 820 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 821 and / or cache memory 822, and may further include a read-only memory (ROM) 823.

[0172] The storage unit 820 may also include a program / utility 824 having a set (at least one) of program modules 825, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0173] Bus 830 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0174] Electronic device 800 can also communicate with one or more external devices 900 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 800, and / or with any device that enables electronic device 800 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 850. Furthermore, electronic device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 860. As shown, network adapter 860 communicates with other modules of electronic device 800 via bus 830. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0175] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the method according to the embodiments of this application.

[0176] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this application, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0177] It should be understood that this application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.< / unk>

Claims

1. An information recommendation method, characterized in that, The method includes: Get the document to be read; The document to be read is subjected to text recognition to obtain the text to be read. The feature extraction module uses a Feature Pyramid Network (FPN) to fuse feature maps of different sizes to extract features from the segmented image of the document. The feature extraction module uses a ResNet-18 or ResNet-50 deep convolutional neural network. The extracted features are input into a probability map prediction module to obtain a probability map. The probability map prediction module consists of a 3×3 convolutional layer and two deconvolutional layers with a stride of 2. The probability map is input into a binarization module to obtain a binary map. Connected regions are found in the binary map to obtain text boxes. The text information within multiple text boxes is recognized to obtain the text to be read. Named entity recognition is performed on the text to be read to determine the knowledge points in the text. Specifically, the text is converted into a token sequence that meets the model's input requirements. The hierarchical computational capability of the SPAN-BERT model is used to generate an accurate contextual representation. Based on a pointer network, the location of the knowledge points is predicted according to the contextual representation. The first classifier of the pointer network predicts the start position of the knowledge point, the second classifier predicts the end position, and the maximum likelihood function is used to determine the location of the knowledge point. An entity binary classification model is constructed based on the BERT model and a binary classification model. The confidence level of the input knowledge point is determined by the entity binary classification model. If the confidence level of the knowledge point is less than a preset threshold, the knowledge point is a false entity; if the confidence level of the knowledge point is greater than or equal to the preset threshold, the knowledge point is a real entity and is considered a knowledge point in the text to be read. Based on the user's long-term and short-term preferences, a list of knowledge points to be recommended in the document to be read is determined, and the list of knowledge points is marked and displayed in the document to be read; In response to an event triggered by the location of the target knowledge point in the document to be read by the user, learning content matching the list of target knowledge points is recommended to the user.

2. The information recommendation method according to claim 1, characterized in that, The step of performing text recognition on the document to be read to obtain the text to be read includes: Based on the FPN algorithm, the document to be read is processed into a feature map to obtain a probability map; The probability map is made differentiable and binarized to obtain a binary map; Find connected regions on the binary graph to obtain a set of text box data, which includes the position information of multiple text boxes; The text information within the multiple text boxes is identified to obtain the text to be read.

3. The information recommendation method according to claim 1, characterized in that, The step of performing named entity recognition on the text to be read to determine the knowledge points in the text includes: The text to be read is converted into a token sequence that meets preset conditions; Based on the SPAN-BERT model, the token sequence is processed hierarchically to obtain the encoded context representation; Based on a pointer network, the context representation is processed to predict the location of the knowledge points and determine the knowledge points in the text to be read.

4. The information recommendation method according to claim 3, characterized in that, The pointer network includes a first classifier and a second classifier; the step of predicting the location of the knowledge point based on the pointer network and the context representation includes: The first classifier obtains the candidate set of the starting positions of the knowledge point and the corresponding probabilities based on the context representation. The second classifier obtains the candidate set of the end positions of the knowledge point and the corresponding probabilities based on the context representation. Based on the maximum likelihood function, the position of the knowledge point is determined according to the probability of the starting position and the probability of the ending position.

5. The information recommendation method according to claim 3, characterized in that, The information recommendation method also includes: Construct an entity binary classification model based on the span model and the binary classification model; The data is augmented using preset positive entity samples to obtain negative entity samples; The entity binary classification model is trained based on the entity positive samples and the entity negative samples to obtain the trained entity binary classification model; The confidence level of the knowledge point is determined based on the location of the knowledge point using the trained entity binary classification model. Based on the confidence level of the knowledge points, the knowledge points in the document to be read are determined.

6. The information recommendation method according to claim 1, characterized in that, The event triggered by the user at the location of the knowledge point determines a list of knowledge points to be recommended based on the user's long-term and short-term preferences, including: A recommendation model is constructed, which includes an embedding layer, an attention layer, an LSTM network, and a fusion layer; The sequence of knowledge points triggered by the user in the past is taken as the long-term sequence, and the sequence of knowledge points triggered by the user recently is taken as the short-term sequence. Through the embedding layer, the feature information of the long-term sequence and the corresponding knowledge points, as well as the feature information of the short-term sequence and the corresponding knowledge points, are encoded into low-dimensional dense vectors. Through the attention layer, the weights of each knowledge point to the user are obtained based on the low-dimensional dense vector corresponding to the long-term sequence. The user's long-term preference is obtained by multiplying the weights with the vector formed by the feature information of the long-term sequence and the corresponding knowledge points. The user's short-term preferences are obtained through the LSTM network based on the low-dimensional dense vector corresponding to the short-term sequence. The fusion layer weights and fuses the user's long-term and short-term preferences to obtain the user's final preferences. Based on the final preferences, the probability of the user reading each knowledge point again is determined, and a list of knowledge points is generated from high to low based on the reading probabilities.

7. The information recommendation method according to claim 6, characterized in that, The feature information of the knowledge point includes the text recognition confidence level, entity recognition confidence level, font size, location of the knowledge point, number of documents containing the knowledge point, and type of the knowledge point.

8. The information recommendation method according to claim 1, characterized in that, The information recommendation method also includes: Using the SimBERT model, each knowledge point is embedded and vectorized to obtain the semantic vector of each knowledge point. Based on the cosine similarity algorithm, the similarity between each knowledge point is determined according to the semantic vector of each knowledge point; Based on the similarity between the knowledge points, duplicate knowledge points are removed.

9. An information recommendation device, characterized in that, The device includes: The document retrieval unit is used to retrieve documents to be read. The text recognition unit is used to recognize the text of the document to be read, thereby obtaining the text to be read. The feature extraction module employs a Feature Pyramid Network (FPN) to fuse feature maps of different sizes to extract features from the segmented image of the document to be read. The feature extraction module uses a ResNet-18 or ResNet-50 deep convolutional neural network. The extracted features are input into a probability map prediction module to obtain a probability map. The probability map prediction module consists of a 3×3 convolutional layer and two deconvolutional layers with a stride of 2. The probability map is then input into a binarization module to obtain a binary map. Connected regions are found in the binary map to obtain text boxes. The text information within multiple text boxes is recognized to obtain the text to be read. An entity recognition unit is used to perform named entity recognition on the text to be read, determine the knowledge points in the text, and label and display the knowledge points. Specifically, the text is converted into a token sequence that meets the model input requirements. The hierarchical computation capability of the SPAN-BERT model is used to generate an accurate contextual representation. Based on a pointer network, the position of the knowledge point is predicted according to the contextual representation. The first classifier of the pointer network predicts the start position of the knowledge point, the second classifier predicts the end position, and the position of the knowledge point is determined by combining the maximum likelihood function. An entity binary classification model is constructed based on the BERT model and a binary classification model. The confidence level of the input knowledge point is judged by the entity binary classification model. If the confidence level of the knowledge point is less than a preset threshold, the knowledge point is a false entity; if the confidence level of the knowledge point is greater than or equal to the preset threshold, the knowledge point is a real entity and is considered a knowledge point in the text to be read. The recommendation unit is used to respond to an event triggered by the user at the location of the knowledge point, and to determine a list of knowledge points to be recommended based on the user's long-term and short-term preferences. The recommendation unit is also used to recommend learning content that matches the knowledge point list to the user.

10. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to perform the method as described in any one of claims 1-8.

Images