A training method and device for a clinical data processing model based on contrastive learning

Abstract

The present disclosure relates to a training method and device for a clinical data processing model based on contrastive learning, including: obtaining a current training set, including a first original entity word and a first standard entity word; obtaining a data enhancement training set; based on the current training set and the data enhancement training set, performing the following contrastive learning training process on a deep self-attention neural network model: performing initial training on the deep self-attention neural network model based on the data enhancement training set and a preset loss function to obtain a first network model; based on the first network model, calculating a target semantic relationship weight score; selecting the first multiple words with the highest semantic similarity from the current training set according to the target semantic relationship weight score, and obtaining a first training set based on the selected words and the first original entity word; using the first training set as a new current training set, and performing the above contrastive learning training process again. The present disclosure can improve the accuracy and generalization ability of the algorithm model.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a training method and device for a clinical data processing model based on contrastive learning. Background Art

[0002] In the process of clinical research, doctors often need to use deep neural network learning methods to perform statistical analysis on clinical case information. The general deep neural network learning method is built on the basis of general natural language corpus and general NLP (Natural Language Processing) tasks; the traditional machine learning method model itself has insufficient expressive power, and its performance is prone to bottlenecks. The trained models and parameters have low accuracy in actual applications, poor generalization ability, and cannot be actually applied in clinical research. Summary of the invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a training method and device for a clinical data processing model based on contrastive learning.

[0004] The present disclosure provides a training method for a clinical data processing model based on contrastive learning, the method comprising:

[0005] Acquire a current training set; wherein the training data in the current training set includes: a first original entity word and a plurality of first standard entity words annotated therewith, wherein the first original entity word is a non-internal standard clinical entity word, and the first standard entity word is an internal standard clinical entity word with similar semantics to the first original entity word;

[0006] Acquire a data augmentation training set; wherein the training data in the data augmentation training set includes: a triple consisting of an internal standard positive sample, a first type of negative sample, and a second type of negative sample, wherein the positive sample is a sample pre-labeled for the first original entity word, the first type of negative sample is a sample selected from the first standard entity word that is similar to the first original entity word, and the second type of negative sample is an arbitrarily selected sample that is not similar to the first original entity word;

[0007] Based on the current training set and the data augmentation training set, the following contrastive learning training process is performed on the deep self-attention neural network model:

[0008] Performing initial training on the deep self-attention neural network model based on the data augmentation training set and a preset loss function to obtain a first network model after initial training;

[0009] Based on the first network model, calculating a target semantic relationship weight score between the first standard entity word and the first original entity word in the current training set;

[0010] Selecting the first multiple words with the highest semantic similarity from the current training set according to the target semantic relationship weight score, and obtaining a first training set based on the selected words and the first original entity words;

[0011] Using the first training set as a new current training set, and combining it with the data enhancement training set to perform the above-mentioned contrastive learning training process again;

[0012] The comparative learning and training process is repeated multiple times until an optimized second network model and parameters are obtained, and the second network model is used as a clinical data processing model.

[0013] The present disclosure provides a training device for a clinical data processing model based on contrastive learning, the device comprising:

[0014] A first training set acquisition module is used to acquire a current training set; wherein the training data in the current training set includes: a first original entity word and a plurality of first standard entity words annotated therewith, wherein the first original entity word is a non-internal standard clinical entity word, and the first standard entity word is an internal standard clinical entity word with similar semantics to the first original entity word;

[0015] A second training set acquisition module is used to acquire a data enhancement training set; wherein the training data in the data enhancement training set includes: a triple consisting of an internal standard positive sample, a first type of negative sample, and a second type of negative sample, wherein the positive sample is a sample pre-labeled for the first original entity word, the first type of negative sample is a sample selected from the first standard entity word that is similar to the first original entity word, and the second type of negative sample is an arbitrarily selected sample that is not similar to the first original entity word;

[0016] A learning and training module is used to perform the following comparative learning and training process on the deep self-attention neural network model based on the current training set and the data enhancement training set:

[0017] The deep self-attention neural network model is initially trained based on the data augmented training set and a preset loss function to obtain a first network model after the initial training; based on the first network model, the target semantic relationship weight score between the first standard entity word and the first original entity word in the current training set is calculated; according to the target semantic relationship weight score, the first multiple words with the highest semantic similarity are selected from the current training set, and a first training set is obtained based on the selected words and the first original entity word; the first training set is used as a new current training set, and the above-mentioned comparative learning training process is performed again in combination with the data augmented training set; the comparative learning training process is repeated multiple times until an optimized second network model and parameters are obtained, and the second network model is used as a clinical data processing model.

[0018] The present disclosure provides an electronic device, which includes: a processor; a memory for storing instructions executable by the processor; and the processor is used to read the executable instructions from the memory and execute the instructions to implement the above method.

[0019] The present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the above method.

[0020] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0021] The disclosed embodiments provide a training method and device for a clinical data processing model based on contrastive learning. Based on a deep self-attention neural network, contrastive learning is performed through multiple rounds and cyclic enhancement, so that the current model and parameters can reasonably distinguish positive samples from the first type of negative samples and the second type of negative samples, and finally train an intelligent algorithm model and parameters that can have a powerful clinical entity semantic representation, maximize the mining and utilization of the implicit semantic information of the internal standard entity terms themselves, and ultimately improve the accuracy and generalization ability of the algorithm model, so as to efficiently and high-quality improve the problem of automated clinical entity mapping in doctors' scientific research. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] Figure 1 A flow chart of a training method for a clinical data processing model based on contrastive learning according to an embodiment of the present disclosure;

[0025] Figure 2 A schematic diagram of the architecture of the deep self-attention neural network model described in the embodiment of the present disclosure;

[0026] Figure 3 A structural block diagram of a training device for a clinical data processing model based on contrastive learning according to an embodiment of the present disclosure;

[0027] Figure 4 It is a schematic diagram of the structure of the electronic device described in the embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0030] Considering that the existing model training methods have poor generalization ability and cannot be actually applied in clinical research, based on this, the embodiments of the present disclosure provide a training method, device and equipment for a clinical data processing model based on contrastive learning. The technical solution can maximize the mining and utilization of the implicit semantic knowledge of the entity standard set itself based on contrastive learning training of a deep self-attention neural network, and ultimately improve the accuracy and generalization ability of the algorithm model; the technology can be well applied to clinical research and other fields. For ease of understanding, the embodiments of the present disclosure are described below.

[0031] Figure 1 A flow chart of a training method for a clinical data processing model based on contrastive learning provided in an embodiment of the present disclosure, the method comprising the following steps:

[0032] Step A1, obtaining a current training set; wherein the training data in the current training set includes: a first original entity word and a plurality of first standard entity words annotated therewith. The first original entity word is a non-internal standard clinical entity word, and the first standard entity word is an internal standard clinical entity word with similar semantics to the first original entity word.

[0033] In this embodiment, when the deep self-attention neural network model is trained for the first time, the current training set can be obtained, for example, by semantically integrating multiple specific standard databases of clinical entities (such as surgery); from historical data of clinical entity terms annotated by doctors, experts and other relevant staff in actual clinical research; from high-frequency clinical entity terms counted in a preset medical information database. The internal standard is a unified standard determined based on multiple specific standards (such as the national clinical version, the national medical insurance version, the Beijing clinical version, and the disease standard ICD-10), which can be used as a reference for accurate mapping of other specific standards.

[0034] Step A2, obtaining a data augmentation training set; wherein the training data in the data augmentation training set includes: a triple consisting of an internal standard positive sample, a first type of negative sample, and a second type of negative sample. The positive sample is a sample pre-labeled for the first original entity word, the first type of negative sample is a sample selected from the first standard entity word that is similar to the first original entity word, and the second type of negative sample is an arbitrarily selected sample that is not similar to the first original entity word.

[0035] In this embodiment, the positive samples in the data enhancement training set are words under the internal standard that are accurately mapped to the first original entity words. Since the number of positive samples is not required to be high, in order to ensure the accuracy of the positive samples, the positive samples can be obtained by manual annotation. The first type of negative samples are a certain number (such as 50) of word samples randomly selected from the first standard entity words in the current training set; the first type of negative samples and the first original entity words have a high semantic similarity and are relatively difficult to distinguish. The second type of negative samples are arbitrarily selected samples that are not similar to the first original entity words, such as can be randomly selected from an existing clinical entity word set. The positive samples, the first type of negative samples and the second type of negative samples are randomly combined to form a triple, that is, each triple includes a positive sample, a first type of negative sample and a second type of negative sample. It can be understood that using a small number of positive and negative samples, a large number of triples can be obtained by any combination, thereby forming a data enhancement training set with a large amount of data and high quality, which greatly expands the data set.

[0036] Next, based on the current training set and the data augmentation training set, the deep self-attention neural network model is subjected to the contrastive learning training process shown in the following steps A3 to A6.

[0037] Step A3, performing initial training on the deep self-attention neural network model based on the data enhancement training set and the preset loss function to obtain a first network model after the initial training.

[0038] In practical applications, the Transformer deep self-attention neural network model is used. This model uses a large amount of unlabeled Internet corpus for unsupervised learning to obtain a general model that can represent the deep semantic relationship in the current corpus, and then uses the corresponding manually labeled data set for training and parameter optimization in specific tasks. However, the general model is very ineffective in solving specific NLP prediction tasks. The model parameters of the neural network need to be optimized and adjusted according to specific fields and specific tasks, and it relies more on a large amount of high-quality manually labeled data.

[0039] Based on this, this embodiment uses a large amount of high-quality data to enhance the training set, and uses a preset loss function to train the deep self-attention neural network model. The most important feature of contrastive learning training is that it makes maximum use of the three types of samples mined in the data enhancement process by relying on a special loss function, a series of encoding processes (Encoding) and pooling processes (Pooling) and the deep self-attention neural network model representation ability of Transformer, and can distinguish positive samples from the first type of negative samples and the second type of negative samples, and then calculate and generate confidence scores that meet the actual situation.

[0040] Reference Figure 2 The deep self-attention neural network model includes: input layer, encoding layer, hidden layer and pooling layer. The deep self-attention neural network model is initially trained based on the data augmentation training set and the preset loss function, including:

[0041] The first original entity word (wWord) and its corresponding triples (vWord, sWord and tWord) are used as input data of the input layer; the input data is vectorized through the encoding layer; the output of the encoding layer is sent to the hidden layer for multiple nonlinear transformations and semantic information processing; the multiple original output results of the hidden layer are standardized vector aligned through the pooling layer to obtain a 768-dimensional unified layer deep semantic information vector representation. For the unified layer deep semantic information vector representation, the deep semantic relationship score is calculated; the deep semantic relationship score includes: the deep semantic relationship scores between the first original entity word and its corresponding positive samples, the first type of negative samples, and the second type of negative samples, respectively expressed as: D(w,v), D(w,s) and D(w,t).

[0042] Referring to the following formula (1), the corresponding total loss function in the training process is calculated according to the above deep semantic relationship score and the preset loss function;

[0043]

[0044] Among them, L is the total loss function corresponding to the training process, N is the number of triplets in the data augmentation training set, and D(W sj ,V nj )、D(W sj ,S pj )、D(W sj ,T qj ) are the first primitive entity words W sj and the corresponding positive sample V nj , the first type of negative sample S pj , the second type of negative samples T qj The deep semantic relationship score between them.

[0045] The deep self-attention neural network model is trained according to the total loss function until the total loss function meets the preset loss threshold, and the training is stopped to obtain the first network model.

[0046] Step A4, based on the first network model, calculate the target semantic relationship weight score between the first standard entity word and the first original entity word in the current training set. The implementation process includes the following steps:

[0047] Step A4.1, calculating the first semantic relationship weight score between the first standard entity word and the first original entity word in the current training set according to the preset character embedding model and word embedding model.

[0048] Specifically, first, based on the character embedding model, the first semantic weight between the first standard entity word and the first original entity word is calculated; then, based on the word embedding model, the second semantic weight between the first standard entity word and the first original entity word is calculated; and then, based on the first semantic weight, the second semantic weight and the preset semantic influence factor, the first semantic relationship weight score between the first standard entity word and the first original entity word is calculated.

[0049] Step A4.2, calculating the second semantic relationship weight score between the first standard entity word and the first original entity word in the current training set through the first network model.

[0050] This embodiment uses the first network model after initial training to re-score the words in the current training set to obtain a new weight Y of deep and complex semantic relationships that can reasonably utilize the Transformer deep self-attention neural network model and huge network parameters. The specific calculation method is:

[0051]

[0052] Among them, Y(A,B) is the new weight calculation result of the deep and complex semantic relationship between the first original entity word A and the first standard entity word B, and VA and V B They are the 768-dimensional unified layer deep semantic information vector representations generated after a series of processing such as the encoding layer, hidden layer, and pooling layer, and v ai and v bi They are respectively the probability values ​​of the deep semantic information vector representations corresponding to the first original entity word A and the first standard entity word B in the 768-dimensional semantic space.

[0053] Step A4.3, obtaining a target semantic relationship weight score between the first standard entity word and the first original entity word in the current training set according to the first semantic relationship weight score and the second semantic relationship weight score.

[0054] This embodiment combines and calculates a comprehensive scoring result, namely the target semantic relationship weight score, based on the first semantic relationship weight score (which can be expressed as T(A, B)) and the second semantic relationship weight score Y(A, B); the target semantic relationship weight score satisfies the five semantic relationships of the character index model, word index model, character embedding model, word embedding model and the tuned deep self-attention neural network model. The specific calculation method is:

[0055] Z(A,B)=q1·Y(A,B)+q2·T(A,B)+e (3)

[0056] Among them, Z(A,B) is the comprehensive scoring result of the first original entity word A and the first standard entity word B that satisfy the five semantic relationships, that is, the target semantic relationship weight score, q1 and q2 are the weight influence factors of Y(A,B) and T(A,B) corresponding to different semantic relationships, respectively, and e is a probability bias, so that the final calculated score is controlled within the standard result between [0,1].

[0057] Step A5, selecting the first multiple words with the highest semantic similarity from the current training set according to the target semantic relationship weight score, and obtaining a first training set based on the selected words and the first original entity words.

[0058] Step A6: Use the first training set as a new current training set and perform the above-mentioned contrastive learning training process again in combination with the data enhancement training set.

[0059] In this embodiment, the words in the current training set are sorted from high to low according to the target semantic relationship weight score, and the first multiple words with the highest semantic similarity are selected according to the sorting result. The selected words and the first original entity words form a first training set with higher data quality. The first training set is used as the new current training set, and the comparative learning training process shown in steps A3 to A6 is performed again to perform the next round of model training.

[0060] Step A7, repeat the comparative learning and training process multiple times until the optimized second network model and parameters are obtained, and the second network model is used as the clinical data processing model.

[0061] The training method of the clinical data processing model based on contrastive learning provided in this embodiment uses a deep self-attention neural network model and a preset loss function for the current training set and the data enhancement training set, performs multiple rounds of contrastive learning and parameter optimization and performs cyclic enhancement training until a neural network model and parameters that can well predict the current development set are found. In each round of training process performed in this embodiment, a semantic similarity relationship score is added to the model and parameter results of the previous round of data enhancement and contrastive learning to generate a new round of enhanced training set. By using the continuously enhanced training set, better model results can be trained out, and the overfitting problem that is easily caused by multiple neural network training can be effectively avoided, while retaining multiple semantic similarity relationships to the maximum extent. In the data enhancement training set, a triple, i.e., a set of a first-class negative sample, a second-class negative sample and a positive sample, is used as the smallest training unit, and the loss function in the model training is calculated, which can maximize the use of the parameters generated by the current training to reasonably distinguish the three types of samples in the triple. In contrastive learning training, the corresponding random sampling methods and weight influencing factors of different models can be dynamically adjusted according to the changes in the loss function value of the current training results, so that the representation ability of the model and parameters is closer to the actual data distribution of the data set.

[0062] Based on the above training results, this embodiment can also perform verification tests on the test set and return to the previous steps to continue training. According to the results of different test sets, various weight influencing factors and neural network models and parameters can be fine-tuned to ensure that the currently obtained optimal model has good generalization capabilities in new data sets in the future, and can solve the problem of automated mapping of clinical entities in doctors' scientific research with high quality and efficiency.

[0063] In this case, the method provided in this embodiment may also include the following content.

[0064] Step A8, obtaining a target test set; wherein the test data in the target test set includes: a second original entity word and multiple second standard entity words annotated therewith, the second original entity word is a non-internal standard clinical entity word, and the second standard entity word is an internal standard clinical entity word that is semantically similar to the second original entity word.

[0065] In one method of obtaining a target test set, the following may be included:

[0066] (1) Sampling an initial test set from a preset database; wherein the test data in the initial test set includes: a second original entity word that is not an internal standard and its annotated second standard entity word that is semantically similar and is an internal standard.

[0067] (2) From the second standard entity words in the initial test set, words that meet the preset similarity conditions with the second original entity words are selected, and the selected words and the second original entity words form a target test set; wherein the preset similarity condition is: the comprehensive semantic similarity of the character semantics and word semantics between the second original entity words and the second original entity words is higher than the preset similarity threshold.

[0068] In this embodiment, the multiple clinical entity words selected according to the above-mentioned similarity conditions are words that satisfy four semantic relationships including character index model, word index model, character embedding model and word embedding model. The selected words and the second original entity words constitute a target test set.

[0069] Step A9: Apply the second network model to the target test set to obtain a third semantic relationship weight score between the second standard entity word and the second original entity word in the target test set.

[0070] In this embodiment, the process of obtaining the third semantic relationship weight score for the target test set is basically the same as the implementation process of calculating the target semantic relationship weight score using the first network model, and will not be described in detail here.

[0071] Step A10: judging whether the accuracy of the prediction result of the second network model meets the requirement according to the third semantic relationship weight score.

[0072] If the prediction accuracy of the current target test set meets the requirements and is not much different from the prediction accuracy of the development set, it means that the currently trained deep self-attention neural network model is the optimal model and has sufficient generalization ability to be used to automatically predict new clinical entity terms. If the accuracy of the target test set is significantly different from the pre-set accuracy and does not meet the requirements, execute the following step A11.

[0073] Step A11, adjust the positive samples, the first category of negative samples, and the second category of negative samples in the data enhancement training set to obtain a new data enhancement training set, and re-cycle the comparative learning training process multiple times based on the new data enhancement training set and the current training set until the prediction result accuracy generated by the obtained deep self-attention neural network model and parameters meets the requirements; at this time, it is determined that the trained deep self-attention neural network model can produce ideal prediction results.

[0074] It should be noted that in the series of processes of data augmentation and contrastive learning, many hyperparameters can be continuously adjusted in these cyclic training to obtain the optimal results, such as the spatial dimension of the latent semantic vector representation of the character embedding model, the spatial dimension of the latent semantic vector representation of the word embedding model, and the number of random sampling for positive and negative sample mining, etc.

[0075] In order to apply the above-trained deep self-attention neural network model to actual production, or to provide reference value in other model training processes, the method provided in this embodiment may further include the following steps:

[0076] Step A12, save the currently obtained optimal deep self-attention neural network model and parameters, as well as the various models obtained previously and the corresponding combination weight influence factor results, and the training task is completed.

[0077] After multiple rounds of data enhancement and contrastive learning in the aforementioned embodiments and cyclic enhancement training verified by the target test set, an optimal character index model, word index model, character embedding model and word embedding model and corresponding combined weight influencing factors, as well as a tuned deep self-attention neural network model and parameter results have been obtained. The results of these optimal models and parameters are saved separately to solve the problem of automatic mapping of new clinical entities in future real scenarios, and then the training task is completed.

[0078] In summary, the training method of the clinical data processing model based on contrastive learning provided by the embodiment of the present disclosure is based on a deep self-attention neural network, and contrastive learning is performed through multiple rounds and cyclic enhancement, so that the model and parameters can reasonably distinguish positive samples from the first type of negative samples and the second type of negative samples, and finally train an intelligent algorithm model and parameters that can have a powerful clinical entity semantic representation, maximize the mining and utilization of the implicit semantic information of the internal standard entity words themselves, and ultimately improve the accuracy and generalization ability of the algorithm model, so as to efficiently and high-quality improve the problem of automated clinical entity mapping in doctors' scientific research.

[0079] Compared with the traditional method, the training method of the clinical data processing model based on contrastive learning in this technical solution has at least the following advantages:

[0080] This scheme can generate a large and high-quality data enhancement training set with a small amount of positive and negative samples. In the contrastive learning training process, the positive and negative samples mined in the data enhancement training set can be effectively utilized. Through clever loss function training, the positive and negative samples can be reasonably distinguished, and then the model results that satisfy multiple semantic relationships are used for comprehensive scoring calculation. Finally, an intelligent algorithm model and parameters with powerful clinical entity semantic representation are trained, which maximizes the mining and utilization of the implicit semantic information of the internal standard entity words themselves, and can derive a confidence score that is more in line with the actual situation, which significantly improves the accuracy and generalization ability of the algorithm model. Especially in clinical research, data processing processes such as data automatic mapping can be performed efficiently and with high quality.

[0081] Reference Figure 3 The present disclosure provides a training device for a clinical data processing model based on contrastive learning, the device comprising:

[0082] The first training set acquisition module 302 is used to acquire a current training set; wherein the training data in the current training set includes: a first original entity word and a plurality of first standard entity words annotated therewith, wherein the first original entity word is a non-internal standard clinical entity word, and the first standard entity word is an internal standard clinical entity word with similar semantics to the first original entity word;

[0083] The second training set acquisition module 304 is used to acquire a data enhancement training set; wherein the training data in the data enhancement training set includes: a triple consisting of an internal standard positive sample, a first type of negative sample, and a second type of negative sample, wherein the positive sample is a sample pre-labeled for the first original entity word, the first type of negative sample is a sample selected from the first standard entity word that is similar to the first original entity word, and the second type of negative sample is an arbitrarily selected sample that is not similar to the first original entity word;

[0084] The learning and training module 306 is used to perform the following comparative learning and training process on the deep self-attention neural network model based on the current training set and the data enhancement training set:

[0085] Based on the data augmented training set and a preset loss function, the deep self-attention neural network model is initially trained to obtain a first network model after the initial training; based on the first network model, the target semantic relationship weight score between the first standard entity word and the first original entity word in the current training set is calculated; according to the target semantic relationship weight score, the first multiple words with the highest semantic similarity are selected from the current training set, and a first training set is obtained based on the selected words and the first original entity word; the first training set is used as a new current training set, and the above-mentioned comparative learning training process is performed again in combination with the data augmented training set; the comparative learning training process is repeated multiple times until an optimized second network model and parameters are obtained.

[0086] The implementation principle and technical effects of the device provided in this embodiment are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0087] Figure 4 The structure diagram of an electronic device provided by the embodiment of the present disclosure is shown in FIG. Figure 4 As shown, the electronic device 400 includes one or more processors 401 and a memory 402 .

[0088] The processor 401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0089] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may run the program instructions to implement the method of the embodiment of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0090] In one example, the electronic device 400 may further include: an input device 403 and an output device 404, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0091] In addition, the input device 403 may also include, for example, a keyboard, a mouse, and the like.

[0092] The output device 404 can output various information to the outside, including the determined distance information, direction information, etc. The output device 404 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0093] Of course, to simplify, Figure 4 Only some of the components related to the present disclosure in the electronic device 400 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 400 may also include any other appropriate components.

[0094] Furthermore, this embodiment also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the above-mentioned training method of the clinical data processing model based on contrastive learning.

[0095] The computer program product of a training method, device, electronic device and medium for a clinical data processing model based on contrastive learning provided in the embodiments of the present disclosure includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0096] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0097] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A training method for a clinical data processing model based on contrastive learning, characterized in that: The method comprises: Acquire a current training set; wherein the training data in the current training set includes: a first original entity word and a plurality of first standard entity words annotated therewith, wherein the first original entity word is a non-internal standard clinical entity word, and the first standard entity word is an internal standard clinical entity word with similar semantics to the first original entity word; Acquire a data augmentation training set; wherein the training data in the data augmentation training set includes: a triple consisting of an internal standard positive sample, a first type of negative sample, and a second type of negative sample, wherein the positive sample is a sample pre-labeled for the first original entity word, the first type of negative sample is a sample selected from the first standard entity word that is similar to the first original entity word, and the second type of negative sample is an arbitrarily selected sample that is not similar to the first original entity word; Based on the current training set and the data augmentation training set, the following contrastive learning training process is performed on the deep self-attention neural network model: The contrastive learning training process includes: performing initial training on the deep self-attention neural network model based on the data augmentation training set and a preset loss function to obtain a first network model after initial training; and calculating the target semantic relationship weight score between the first standard entity word and the first original entity word in the current training set based on the first network model; and selecting the first multiple words with the highest semantic similarity from the current training set according to the target semantic relationship weight score, and obtaining a first training set based on the selected words and the first original entity word; Using the first training set as a new current training set, and combining it with the data enhancement training set to perform the above-mentioned contrastive learning training process again; The comparative learning and training process is repeated multiple times until an optimized second network model and parameters are obtained, and the second network model is used as a clinical data processing model.

2. The method according to claim 1, characterized in that The calculating, based on the first network model, a target semantic relationship weight score between the first standard entity word and the first original entity word in the current training set comprises: Calculate a first semantic relationship weight score between the first standard entity word and the first original entity word in the current training set according to a preset character embedding model and a word embedding model; Calculating, by using the first network model, a second semantic relationship weight score between the first standard entity word and the first original entity word in the current training set; A target semantic relationship weight score between the first standard entity word and the first original entity word in the current training set is obtained according to the first semantic relationship weight score and the second semantic relationship weight score.

3. The method according to claim 1, characterized in that The method further comprises: Acquire a target test set; wherein the test data in the target test set includes: a second original entity word and a plurality of second standard entity words annotated therewith, wherein the second original entity word is a non-internal standard clinical entity word, and the second standard entity word is an internal standard clinical entity word that is semantically similar to the second original entity word; Applying the second network model to the target test set to obtain a third semantic relationship weight score between the second standard entity word and the second original entity word in the target test set; According to the third semantic relationship weight score, determining whether the accuracy of the prediction result of the second network model meets the requirement; If not, the positive samples, the first category of negative samples, and the second category of negative samples in the data augmented training set are adjusted to obtain a new data augmented training set, and the comparative learning training process is re-circulated multiple times based on the new data augmented training set and the current training set until the prediction result accuracy generated by the obtained deep self-attention neural network model and parameters meets the requirements.

4. The method according to claim 3, characterized in that The obtaining of the target test set comprises: An initial test set is obtained by sampling from a preset database; wherein the test data in the initial test set includes: a second original entity word that is not an internal standard and a second standard entity word that is annotated with similar semantics and is an internal standard; Selecting words that meet a preset similarity condition with the second original entity words from the second standard entity words in the initial test set, and forming a target test set with the selected words and the second original entity words; Among them, the preset similarity condition is: the comprehensive semantic similarity of character semantics and word semantics between the second original entity word and the second original entity word is higher than a preset similarity threshold.

5. The method according to claim 1, characterized in that The deep self-attention neural network model includes: an input layer, an encoding layer, a hidden layer and a pooling layer; The initial training of the deep self-attention neural network model based on the data enhancement training set and the preset loss function includes: Using the first original entity word and the triplet corresponding thereto as input data of the input layer; Performing vector conversion on the input data through the encoding layer; Sending the output result of the encoding layer to the hidden layer for multiple nonlinear transformations and semantic information processing; Performing standardized vector alignment processing on multiple original output results of the hidden layer through the pooling layer to obtain a unified layer deep semantic information vector representation; Calculate the deep semantic relationship score for the unified layer deep semantic information vector representation; wherein the deep semantic relationship score includes: the deep semantic relationship scores between the first original entity word and the corresponding positive samples, the first type of negative samples, and the second type of negative samples; Calculate the corresponding total loss function in the training process according to the deep semantic relationship score and the preset loss function; The deep self-attention neural network model is initially trained according to the total loss function.

6. The method according to claim 1 or 5, characterized in that: The loss function includes: Where L is the total loss function corresponding to the training process, N is the number of triplets in the data augmentation training set, , , They are the first original entity words and the corresponding positive sample , the first type of negative samples , the second type of negative samples The deep semantic relationship score between them.

7. A training device for a clinical data processing model based on contrastive learning, characterized in that: The device comprises: A first training set acquisition module is used to acquire a current training set; wherein the training data in the current training set includes: a first original entity word and a plurality of first standard entity words annotated therewith, wherein the first original entity word is a non-internal standard clinical entity word, and the first standard entity word is an internal standard clinical entity word with similar semantics to the first original entity word; A second training set acquisition module is used to acquire a data enhancement training set; wherein the training data in the data enhancement training set includes: a triple consisting of an internal standard positive sample, a first type of negative sample, and a second type of negative sample, wherein the positive sample is a sample pre-labeled for the first original entity word, the first type of negative sample is a sample selected from the first standard entity word that is similar to the first original entity word, and the second type of negative sample is an arbitrarily selected sample that is not similar to the first original entity word; A learning and training module is used to perform the following comparative learning and training process on the deep self-attention neural network model based on the current training set and the data enhancement training set: The contrastive learning training process includes: performing initial training on the deep self-attention neural network model based on the data augmentation training set and a preset loss function to obtain a first network model after initial training; and calculating the target semantic relationship weight score between the first standard entity word and the first original entity word in the current training set based on the first network model; and selecting the first multiple words with the highest semantic similarity from the current training set according to the target semantic relationship weight score, and obtaining a first training set based on the selected words and the first original entity word; The first training set is used as a new current training set, and the above-mentioned comparative learning training process is performed again in combination with the data enhancement training set; the comparative learning training process is cycled multiple times until the optimized second network model and parameters are obtained, and the second network model is used as the clinical data processing model.

8. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 6.

Images