Doctor recommendation method for mining doctor network by using large-scale language model

Abstract

The invention discloses a doctor recommendation method for mining a doctor network by using a large-scale language model, which is used for recommending matched doctors for patients based on interactive behaviors of the patients in an online medical platform, the large-scale language model and doctor information. The technical problem to be solved is to alleviate the sparsity of user behaviors in a doctor recommendation system so as to improve the performance of the recommendation system. The forward propagation part is mainly composed of four parts, wherein the first part is based on doctor homepage information, and doctor vectors are obtained through a large-scale language model; in the second part, dimensionality reduction is carried out on a doctor vector, and a doctor network is mined; 3, according to the doctor network and the current interaction sequence of the patient, obtaining a patient vector fused with doctor network information; and the fourth part is to recommend doctors to the patients according to the patient vector representation.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet services, and in particular, relates to a doctor recommendation method for mining a doctor network using a large-scale language model. Background Art

[0002] Doctor recommendation refers to the use of doctor information, user information, and interaction information between users and doctors in online medical platforms to recommend matching doctors to users. Existing recommendation systems are mainly for e-commerce, news, and short video platforms. The items recommended by these platforms are commodities, news, or short videos. The number of these items is very large, and it is difficult to establish a network of related items. Therefore, the existing recommendation methods are to establish a network between platform users, that is, a social network, to alleviate the problem of data sparsity. For example, the recommendation system in the Kuaishou platform usually recommends short videos that users like.

[0003] The application scenario of this method is an online medical platform. The recommended items of the online medical platform are doctors, and the doctor resources are very limited. As of December 31, 2020, the number of registered doctors on WeDoctor is about 270,000, and the number of registered users on WeDoctor is about 222 million. It can be seen that in online medical platforms, the number of projects is far less than the number of users. The semantic information of doctors is mainly their department information and the diseases they are good at treating. For example, the introduction of a doctor A on the homepage of the online platform shows that his department is the thyroid diagnosis and treatment center, and his specialties are: neuroendocrine tumors, bone metastases, neuroendocrine cancer, hyperthyroidism, thyroid tumors, bone metastases and thyroid cancer. The recommendation system can connect doctors according to the department to which the doctor belongs and the disease information he is good at, thereby building a doctor network. Existing methods usually connect doctors by constructing a doctor-disease heterogeneous graph. This type of method ignores the connection between diseases and relies on manpower to remap the diseases that doctors are good at. For example, in the platform, the diseases that a doctor B is good at are: thyroid disease, dwarfism, precocious puberty, and diabetes. Then this doctor should be similar to Doctor A who is good at treating hyperthyroidism, thyroid tumors and thyroid cancer, but constructing a doctor-disease heterogeneous graph cannot connect the two doctors. Summary of the invention

[0004] The technical challenge that this method aims to solve is: in the doctor recommendation system, how to alleviate the sparsity of user behavior based on the doctor's homepage information, thereby improving the performance of the recommendation system. Users in online medical platforms refer to patients. This method alleviates the sparsity of user behavior from two technical perspectives: first, using a large-scale language model to obtain the doctor's vector, so that the traditional ID-based recommendation task becomes a content-based recommendation; based on the doctor vector, a doctor network is built offline to enrich user behavior with the doctor network.

[0005] Specifically, the present invention proposes a doctor recommendation method using a large-scale language model to mine a doctor network, and the process is as follows:

[0006] Based on the doctor's homepage information, a large-scale language model is used to obtain the doctor vector. The doctor's homepage on the online platform contains an introduction to his department and the diseases he is good at. This method converts the description text of doctor v into v Input into the large-scale language model to obtain the high-dimensional vector representation of the doctor. This process is represented as emb v =LLM(text v ).text v Includes the doctor's department information text depart v and good at treating diseases information text disease v .

[0007] Reduce the dimension of the doctor vector and mine the doctor network. This method uses a dimensionality reduction algorithm to reduce the dimension of the high-dimensional doctor vector. The reduced-dimensional doctor vector is represented by x v To express, that is, x v =Dim_Reduct(emb v ), Dim_Reducx represents the dimensionality reduction algorithm. Based on the low-dimensional doctor vector representation, the doctor network G is mined v . G v = {V, A}, where V is the set of all doctors in the online platform and A is the adjacency matrix. The first step in learning the adjacency matrix is ​​to use the vector similarity calculation formula sim to calculate the similarity between any two doctors and obtain a fully connected doctor network. in, and Doctor i and doctor v j The second step is to prune the fully connected doctor network to obtain a sparse doctor network. First, the A that is less than or equal to 0 i,j The value is pruned, and then the hyperparameter k is set to retain the k most similar doctors for each doctor. At this time, the doctor network is a directed graph. The specific calculation process is as follows:

[0008]

[0009] Among them, A i Top-k is to convert the i-th row of matrix A into i All values ​​in are sorted from large to small, and the set consisting of the top k values ​​is retained. The low-dimensional doctor vectors and doctor networks are stored offline in the system.

[0010] According to the doctor network and the patient's current interaction sequence, the patient vector that integrates the doctor network information is obtained. The current interaction sequence of patient u is expressed as |S u |It is S u Length. Doctor v i The vector is Initialize to Doctor vector Updated during training. Based on the vector representation of the doctor, the patient u’s current interaction sequence S u The vector representation is The patient vector is obtained through the sequence encoder Seq_Encoder. The process is as follows: Further use of doctor network G v To supplement the patient's current interaction sequence information, first use the doctor network G v For S u The doctor vector in the sequence is updated, and the process is expressed as GNN is a graph neural network. is node v i In Figure G v The set of neighbor nodes in the. Then the sequence encoder is used to obtain the patient vector that integrates the doctor network information. The calculation formula is The two vector representations of the patient are fused to obtain the final vector representation of the patient as p u , Merge is a method that merges two vectors.

[0011] Recommend doctors to patients based on their vector representation. Multiply by the patient vector p u , and then use the softmax function to calculate the score of doctor v2:

[0012]

[0013] Among them, p u represents the patient vector, is the vector representation of doctor v2, Indicates the possibility of doctor v2 being interacted. For this patient sample data, the loss function is:

[0014]

[0015] Among them, y2 represents the one-hot code of doctor v2. The function is optimized using the Adam optimizer.

[0016] The beneficial technical effects of the present invention are as follows:

[0017] (1) This method introduces the world knowledge learned from the large-scale language model into the recommendation system, inputs the doctor's homepage information into the large-scale language model to obtain the doctor's vector, and uses the reduced-dimensional vector to initialize the doctor's vector in the recommendation system to alleviate the sparsity of user behavior.

[0018] (2) This method mines the doctor network based on the doctor vectors output by the large-scale language model, and uses the doctor network to update the doctor vectors in the user behavior sequence, further alleviating the sparsity of user behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a flow chart of a doctor recommendation method for mining a doctor network using a large-scale language model according to the present invention;

[0020] Figure 2 This is a model framework diagram of a doctor recommendation method of the present invention that utilizes a large-scale language model to mine a doctor network. DETAILED DESCRIPTION

[0021] In order to further understand the present invention, the doctor recommendation method for mining the doctor network using a large-scale language model provided by the present invention is described in detail below in combination with a specific implementation method, but the present invention is not limited to this. Non-essential improvements and adjustments made by technical personnel in this field under the core guiding ideology of the present invention still fall within the scope of protection of the present invention.

[0022] The problem definition of this method is to recommend matching doctors to patients based on the patient's interactive behavior, large-scale language model and doctor information in the online medical platform. To this end, this method designs a doctor recommendation method that uses a large-scale language model to mine the doctor network. The forward propagation part of the algorithm mainly consists of four parts. The first part is based on the doctor's homepage information, using a large-scale language model to obtain the doctor vector; the second part is to reduce the dimension of the doctor vector and mine the doctor network; the third part is to obtain the patient vector that integrates the doctor network information based on the doctor network and the patient's current interaction sequence; the fourth part is to recommend doctors to patients based on the patient vector representation. The model framework diagram is shown in the figure below. Figure 2 As shown. Note that this method is not an end-to-end method. The first and second parts do not need to be trained, and the outputs of these two parts are stored offline in the system. Because these two parts do not participate in the training, they will not increase the complexity of the recommendation system.

[0023] like Figure 1 As shown, according to one embodiment of the present invention, the method includes the following steps:

[0024] S100, based on the doctor's homepage information, use a large-scale language model to obtain the doctor's vector. The doctor's homepage in the online platform contains an introduction to his department and the diseases he specializes in. For example, Doctor A's department is the Thyroid Diagnosis and Treatment Center, and he specializes in treating the following diseases: hypothyroidism, hyperthyroidism, thyroid cancer, parathyroid tumors, thyroid nodules, and parathyroid hyperplasia. Large-scale language models have learned universal knowledge through a lot of training, and large-scale language models provide semantically meaningful vector representations for each text. It has been proven that the text vectors it outputs are very effective for clustering tasks. The clustering task depends on the spatial distance between entities of different types, and the construction of the doctor network is based on the spatial distance of the doctor's vector. Therefore, this method obtains the doctor's vector representation based on the large-scale language model, and then mines the doctor network based on the doctor vector. This method designs the text text input to the large-scale language model. v , the text v Input into a large-scale language model to obtain a vector representation. This process is represented as emb v =LLM(text v ).text v It is a description of the doctor's information, including the doctor's department information. v and good at treating disease informationdisea v , specifically "The doctor's department is depart v , whose specialty is treating diseases v ". As mentioned above, the text text of Doctor A input to the large-scale language model v The doctor's department is the Thyroid Diagnosis and Treatment Center, and he is good at treating hypothyroidism, hyperthyroidism, thyroid cancer, parathyroid tumors, thyroid nodules, and parathyroid hyperplasia. The design of input text templates belongs to prompt engineering. Technical personnel can design input text templates based on recommended scenarios and platform information. v . v Input into the large-scale language model LLM to obtain the high-dimensional vector representation of the text emb v The large-scale language model LLM can use an open source large-scale language model or a closed-source commercial large-scale language model. By performing an average pooling operation on the last layer of latent vectors of the open source large-scale language model, we can obtain the high-dimensional vector emb of the doctor. v, you can choose the Vicuna model. The Vicuna model has different parameter versions. If Vicuna-7B is used, the doctor vector dimension is 4096, and if Vicuna-13B is used, the doctor vector dimension is 5120. Closed-source commercial LLMs include OpenAI's text-embedding-ada-v2 and text-embedding-3-large. If a closed-source commercial LLM is used, directly call the LLM's public API interface to obtain the doctor vector emb v The vector dimension obtained by text-embedding-ada-v2 is 1536, and the vector dimension output by text-embedding-3-large is 3072. The advantage of open source models is that they protect data and are free, but the effect is worse than that of commercial closed source LLM. Because the data input used in this method is all from the publicly available doctor homepage data, it does not involve doctor privacy, so this method uses a closed source large-scale language model.

[0025] S200, reduce the dimensionality of the doctor vector and mine the doctor network. The dimensionality of the doctor vector obtained by the large-scale language model is too high. This method uses a dimensionality reduction algorithm to reduce the dimensionality of the doctor vector. The doctor vector after dimensionality reduction is used as x v To express, that is, x v =Dim_Reduct(emb v ). Dim_Reduct represents the dimensionality reduction algorithm. Any existing dimensionality reduction algorithm can be used. This method uses principal component analysis (PCA), that is, x v = PCA(emb v ). Considering that the doctor vector after dimensionality reduction needs to be initialized for the doctor vector in the subsequent recommendation system, and in order to reduce the computational complexity of the doctor network, this method reduces the doctor vector to 64 dimensions. Based on the low-dimensional doctor vector representation, this method designs a graph structure mining algorithm to mine the doctor network G v . G v Represented as G v = {V, V}, where V is the set of all doctors in the online platform and V is the adjacency matrix. The first step in learning the adjacency matrix is ​​to use the vector similarity calculation formula to calculate the similarity between any two doctors and obtain a fully connected doctor network. The vector similarity calculation formula sim is the cosine similarity calculation formula. For doctor v i and Doctor V j , and its similarity is The second step is to prune the fully connected doctor network to obtain a sparse doctor network. i,jIt is obtained by the cosine similarity calculation formula, and the value range is [-1,1]. The number of doctors in different fields is different. For example, in an online consultation platform, there are only 36 doctors who can treat pediatric asthma, while there are 206 doctors who can treat depression. If a threshold is set, such as retaining A in the doctor network i,j ≥0.5 will cause some doctor nodes in the doctor network to have no adjacent nodes. Therefore, this method first i,j The value is pruned, and then the hyperparameter k is set to retain the k most similar doctors for each doctor. At this time, the doctor network is a directed graph. The specific calculation process is as follows:

[0026]

[0027] Among them, A i Top-k is to convert the i-th row of matrix A into i All values ​​in are sorted in descending order, and the set consisting of the top k values ​​is retained. After obtaining the low-dimensional doctor vector and doctor network, they are stored offline in the system. The doctor's department information and the diseases he is good at treating are relatively stable, and there is no need to update the doctor vector and network information frequently.

[0028] S300, based on the doctor network and the patient's current interaction sequence, obtain a patient vector that incorporates the doctor network information. The current interaction sequence of patient u is expressed as |S u |It is S u Length. Existing recommendation algorithms usually design complex sequence encoders to extract complex preferences from patient behavior sequences, but in medical platforms, patients have very few interactive behaviors. Therefore, this method proposes to build a doctor network to alleviate data sparsity, and use a large-scale language model to obtain a vector representation of the doctor. The doctor vector representation obtained by the large-scale language model is in the semantic space, and it needs to be aligned with the vector in the recommendation space. In the previous step, this method has obtained a high-dimensional doctor vector through a large-scale language model, and further obtained a low-dimensional doctor vector through a dimensionality reduction algorithm. This method uses the low-dimensional doctor vector to initialize the doctor vector in the recommendation system, and then updates the doctor vector through the recommendation task to achieve alignment between the semantic space and the recommendation space. The sequence encoder is represented as Seq_Encoder, which can be any existing sequence encoder, such as a recurrent neural network or an average pooling operation. Doctor v i The vector is Initialize to Doctor vector Updated during training. Based on the vector representation of the doctor, the current interaction sequence S of patient u u The vector representation is The process of obtaining the patient vector through the sequence encoder is: Further use of doctor network G v To supplement the patient's current interaction sequence information, first use the doctor network G v For S u The doctor vector in the sequence is updated, and the process is expressed as GNN can be any graph neural network. The function of graph neural network is to update the vector of a node with its neighbor nodes. i ,G v ) is the node v i In Figure G v The set of neighbor nodes in formula Represents the fusion node v i Neighboring nodes N(v i ,G v ) and v i Original vector representation Get node v i New vector representation Here, GNN can be implemented using any existing graph neural network, including LightGCN, GCN, and GraphSAGE. In order to reduce the complexity of the model, this method uses a layer of LightGCN to implement GNN. The formula is as follows:

[0029]

[0030] Then use the sequence encoder to get the patient vector that incorporates the doctor network information The calculation formula is Finally, the two vector representations of the patient are fused to obtain the final vector representation of the patient as p u , Merge is a method to merge two vectors. This method uses vector addition operation. It is also possible to use a layer of MLP network or an adaptive attention mechanism for fusion.

[0031] S400, recommend a doctor to the patient based on the patient vector representation. Multiply by the patient vector p u , and then use the softmax function to calculate the score of doctor v2:

[0032]

[0033] Among them, p u represents the patient vector, is the vector representation of doctor v2, Indicates the possibility of doctor v2 being interacted. For this patient sample data, the loss function is:

[0034]

[0035] Among them, y2 represents the one-hot encoding of doctor v2. The function is optimized using the Adam optimizer.

[0036] The above description of the embodiments is to facilitate the understanding and application of the present invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made to the present invention by those skilled in the art based on the disclosure of the present invention should be within the scope of protection of the present invention.

Claims

1. A doctor recommendation method using a large-scale language model to mine a doctor network, characterized by: Based on the doctor's homepage information, a large-scale language model is used to obtain the doctor vector; the doctor's homepage on the online platform contains an introduction to his department and the diseases he is good at. This method converts the description text of doctor v into v Input into the large-scale language model to obtain the high-dimensional vector representation of the doctor. This process is represented as emb v =LLM(text v ); text v Includes the doctor's department information text depart v and good at treating diseases information text disease v ; Reduce the dimension of the doctor vector and mine the doctor network; this method uses a dimensionality reduction algorithm to reduce the dimension of the high-dimensional doctor vector, and the reduced-dimensional doctor vector is used as x v To express, that is, x v =Dim_Reduct(emb v ), Dim_Reduct represents the dimensionality reduction algorithm; based on the low-dimensional doctor vector representation, the doctor network G is mined v ; G v ={V,A}, where V is the set of all doctors in the online platform and A is the adjacency matrix; The first step in learning the adjacency matrix is ​​to use the vector similarity calculation formula sim to calculate the similarity between any two doctors and obtain a fully connected doctor network. in, and It's Doctor V" and Doctor V $ The second step is to prune the fully connected doctor network to obtain a sparse doctor network. First, A", $ The value is pruned, and then the hyperparameter k is set to retain the k most similar doctors for each doctor. At this time, the doctor network is a directed graph. The specific calculation process is as follows: in, It is to sort all the values ​​in the i-th row A" of the matrix A in descending order and retain the top k values; the low-dimensional doctor vectors and doctor networks are stored offline in the system; According to the doctor network and the patient's current interaction sequence, the patient vector that integrates the doctor network information is obtained; the current interaction sequence of patient u is expressed as |S + |It is S + The length of the doctor v" is Initialize to Doctor vector Updated during training; based on the vector representation of the doctor, the patient u’s current interaction sequence S + The vector representation is The patient vector is obtained through the sequence encoder Seq_Encoder. The process is as follows: Further use of doctor network G v To supplement the patient's current interaction sequence information, first use the doctor network G v For S + The doctor vector in the sequence is updated, and the process is expressed as GNN is a graph neural network. is node v i In Figure G v The set of neighbor nodes in the network; then the sequence encoder is used to obtain the patient vector that integrates the doctor network information The calculation formula is The two vector representations of the patient are fused to obtain the final vector representation of the patient as p + , Merge is a method to fuse two vectors; Recommend doctors to patients based on their vector representation; represent the vector of doctor v2 in the doctor set Multiply by the patient vector p + , and then use the softmax function to calculate the score of doctor v2: Among them, p + represents the patient vector, is the vector representation of doctor v2, represents the possibility of doctor v2 being interacted; for the patient sample data, the loss function is: Among them, y2 represents the one-hot encoding of doctor v2; The function is optimized using the Adam optimizer.

2. The doctor recommendation method using a large-scale language model to mine a doctor network according to claim 1, characterized in that: The text v Specifically: The doctor's department is depart v , whose specialty is treating diseases v .

3. The doctor recommendation method using a large-scale language model to mine a doctor network according to claim 1, characterized in that: The large-scale language model LLM is an open source large-scale language model. The last layer of latent vector of the open source large-scale language model is averaged pooled to obtain a high-dimensional doctor vector.

4. The doctor recommendation method using a large-scale language model to mine a doctor network according to claim 1, characterized in that: The large-scale language model LLM is a closed-source commercial large-scale language model. The public API interface of LLM is called to obtain a high-dimensional doctor vector.

5. The doctor recommendation method using a large-scale language model to mine a doctor network according to claim 1, characterized in that: The dimension reduction algorithm Dim_Reduct is a principal component analysis method.

6. The doctor recommendation method using a large-scale language model to mine a doctor network according to claim 1, characterized in that: The vector similarity calculation formula sim adopts the cosine similarity calculation formula. i and doctor v $ , and its similarity is 7. The doctor recommendation method using a large-scale language model to mine a doctor network according to claim 1, characterized in that: The sequence encoder Seq_Encoder is a recurrent neural network or an average pool.

8. The doctor recommendation method using a large-scale language model to mine a doctor network according to claim 1, characterized in that: The GNN is implemented using a layer of LightGCN, and the formula is:

9. The doctor recommendation method using a large-scale language model to mine a doctor network according to claim 1, characterized in that: The fusion method Merge uses a vector addition operation, and the formula is:

Images