Case symptom extraction method, device, electronic device and storage medium based on pseudo-label

Abstract

The present invention relates to the fields of digital medicine and artificial intelligence technology, applies deep learning technology, and relates to a training method, device, electronic device, and storage medium for a symptom extraction model based on pseudo labels, including: performing adversarial training on an initial teacher model, generating an adversarial sample set, calculating a first loss value, updating the parameters of the initial teacher model to obtain a preliminary teacher model, making the model more robust to noise disturbances, using the preliminary teacher model to analyze an unlabeled data set to generate pseudo labels, using the labeled data and the unlabeled data with the pseudo labels to train the initial student model twice to obtain a second loss value, adding the loss values ​​to obtain a total loss value, comparing it with a preset threshold until the total loss value is less than the preset threshold, and outputting a target teacher model as a symptom extraction model. In the medical field, the present invention can improve the accuracy of extracting case symptoms.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, applies deep learning technology, and relates to a method, device, electronic device and storage medium for extracting case symptoms based on pseudo labels. Background Art

[0002] In the healthcare field, extracting patient symptoms from medical records is a critical task. By extracting patient symptom information from medical records, doctors can quickly identify the patient's condition and better diagnose and treat the disease. This not only improves medical efficiency but also enhances diagnostic accuracy, thereby increasing cure rates. In addition to traditional rule-based and machine learning algorithms, natural language processing-based methods, such as BiLSTM and BERT, are becoming increasingly popular. However, these methods have several drawbacks: 1. Due to the inherent complexity and variability of medical language, models are overly sensitive to noisy data in medical records, resulting in poor generalization. 2. High-quality annotated medical data is difficult and costly to obtain. Summary of the Invention

[0003] The present invention provides a method, device and computer-readable storage medium for extracting case symptoms based on pseudo labels, the main purpose of which is to improve the accuracy of case symptoms extracted by the model.

[0004] To achieve the above object, the present invention provides a method for extracting case symptoms based on pseudo labels, characterized in that the method comprises:

[0005] Obtaining a first sample set and a second sample set, wherein the data in the first sample set is labeled data, and the data in the second sample set is unlabeled data;

[0006] Performing adversarial training on the initial teacher model based on the first sample set to generate an adversarial sample set, calculating the gradient of the first sample set and the adversarial sample set according to a gradient function, adding the gradient of the first sample set and the gradient of the adversarial sample set to obtain a first loss value, and updating the parameters of the initial teacher model according to the first loss value to obtain a preliminary teacher model;

[0007] Analyzing the second sample set using the preparatory teacher model to generate corresponding pseudo labels, and performing a first training on the initial student model based on the pseudo labels to generate a preparatory student model;

[0008] The preparatory student model is trained for a second time using the first sample set to obtain a second loss value corresponding to the preparatory student model, the first loss value and the second loss value are added to obtain a total loss value, the total loss value is compared with a preset threshold value, and if the total loss value is less than the preset threshold value, the preparatory teacher model is used as the symptom extraction model; if the total loss value is greater than or equal to the preset threshold value, the parameters of the preparatory teacher model are updated until the total loss value is less than the preset threshold value, and a target teacher model is obtained, and the target teacher model is used as the symptom extraction model;

[0009] A medical record set of symptoms to be extracted is obtained, the medical record set of symptoms to be extracted is preprocessed, and the preprocessed medical record set of symptoms to be extracted is input into the symptom extraction model to extract the medical record symptoms.

[0010] Optionally, performing adversarial training on the initial teacher model based on the first sample set to generate an adversarial sample set, calculating the gradient of the first sample set and the adversarial sample set according to a gradient function, and adding the gradient of the first sample set and the gradient of the adversarial sample set to obtain a first loss value includes:

[0011] Calculating the gradient of the first sample set using a first gradient function of the initial teacher model;

[0012] Calculating the gradient of the first sample set according to the perturbation value function to obtain a perturbation value, and obtaining an adversarial sample set of the first sample set according to the perturbation value;

[0013] Calculate the gradient of the adversarial sample set using the second gradient function;

[0014] The gradient of the first sample set and the gradient of the adversarial sample set are added to obtain a first loss value.

[0015] Optionally, the formula of the first gradient function is:

[0016]

[0017] Among them, g is the gradient of the first sample set, p is the probability distribution of the first sample set, y is the label vector of the first sample set, and x is the text vector of the first sample set. To find the derivative with respect to x, θ is the preset parameter.

[0018] Optionally, the formula of the disturbance value function is:

[0019] r=∈g / ||g||2

[0020] Where r is the perturbation value, g is the gradient of the first sample set, and ∈ is a constant.

[0021] Optionally, the formula of the second gradient function is:

[0022]

[0023] Among them, h is the gradient of the adversarial sample set, p is the probability distribution of the adversarial sample set, b is the label vector of the adversarial sample set, and a is the text vector of the adversarial sample set. To find the derivative of a, r is the disturbance value and θ is the preset parameter.

[0024] Optionally, the performing a second training on the preparatory student model using the first sample set to obtain a second loss value corresponding to the preparatory student model includes:

[0025] Training the preparatory student model using the training set of the first sample set;

[0026] Using a sample set of the first sample set to verify the preparatory student model, and obtain a predicted label of the preparatory student model for the first sample set;

[0027] A loss value between the predicted label and the true label of the first sample is calculated to obtain a second loss value.

[0028] Optionally, the step of obtaining a set of medical records of symptoms to be extracted, preprocessing the set of medical records of symptoms to be extracted, and inputting the preprocessed set of medical records of symptoms to be extracted into the symptom extraction model to extract medical record symptoms includes:

[0029] Obtain a set of symptom medical records to be extracted, and perform word segmentation on the set of symptom medical records to be extracted using the symptom extraction model;

[0030] Extracting target features from each of the segmented words based on a preset symptom feature template to obtain target features for each of the segmented words;

[0031] Marking each of the segmented words using a preset marking method according to the target feature to obtain corresponding marking information, wherein the marking information includes information marking various symptoms;

[0032] Symptoms are extracted from the set of symptom medical records to be extracted based on the annotation information.

[0033] In order to solve the above problems, the present invention further provides a case symptom extraction device based on pseudo labels, characterized in that the device comprises:

[0034] A data extraction module, configured to obtain a first sample set and a second sample set, wherein the data in the first sample set is labeled data, and the data in the second sample set is unlabeled data;

[0035] an adversarial training module, configured to perform adversarial training on the initial teacher model based on the first sample set to generate an adversarial sample set, calculate the gradient of the first sample set and the adversarial sample set according to a gradient function, add the gradient of the first sample set and the gradient of the adversarial sample set to obtain a first loss value, and update the parameters of the initial teacher model according to the first loss value to obtain a preliminary teacher model;

[0036] a pseudo-label generation module, configured to analyze the second sample set using the preparatory teacher model to generate corresponding pseudo-labels, and perform a first training on the initial student model based on the pseudo-labels to generate a preparatory student model;

[0037] A loss value module is used to use the first sample set to perform a second training on the preparatory student model to obtain a second loss value corresponding to the preparatory student model, add the first loss value and the second loss value to obtain a total loss value, compare the total loss value with a preset threshold, and if the total loss value is less than the preset threshold, use the preparatory teacher model as the symptom extraction model; if the total loss value is greater than or equal to the preset threshold, update the parameters of the preparatory teacher model until the total loss value is less than the preset threshold, obtain a target teacher model, and use the target teacher model as the symptom extraction model.

[0038] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0039] a memory storing at least one instruction; and

[0040] The processor executes the instructions stored in the memory to implement the above-mentioned pseudo-label-based case symptom extraction method.

[0041] In order to solve the above problems, the present invention also provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the above-mentioned pseudo-label-based case symptom extraction method.

[0042] The present invention uses a labeled data set to perform adversarial training on the initial teacher model, generates an adversarial sample set, calculates a first loss value, updates the parameters of the initial teacher model to obtain a preparatory teacher model, makes the model more robust to noise disturbances, uses the preparatory teacher model to analyze the unlabeled data set, generates pseudo labels, uses the labeled data and the unlabeled data for generating pseudo labels to train the initial student model twice, obtains a second loss value, adds the loss values ​​to obtain a total loss value, compares it with a preset threshold value, and outputs the target teacher model as a symptom extraction model until the total loss value is less than the preset threshold value. The present invention can not only utilize a large amount of existing unlabeled medical data, but also fully consider the influence of noise data, making the model more stable and more generalizable, thereby improving the accuracy of the model in extracting case symptoms. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of a pseudo-label-based case symptom extraction method provided by one embodiment of the present invention;

[0044] Figure 2 A functional module diagram of a training device for a pseudo-label-based case symptom extraction model provided by one embodiment of the present invention;

[0045] Figure 3 A schematic structural diagram of an electronic device for implementing the pseudo-label-based case symptom extraction method provided in one embodiment of the present invention.

[0046] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0048] The embodiments of the present application provide a method for extracting case symptoms based on pseudo-labels. The execution subject of the method includes, but is not limited to, at least one of electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiments of the present application. In other words, the method for extracting case symptoms based on pseudo-labels can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to: a single server, a server cluster, a cloud server, or a cloud server cluster.

[0049] Reference Figure 1 FIG. 1 is a flow chart of a pseudo-label-based case symptom extraction method according to an embodiment of the present invention. In this embodiment, the pseudo-label-based case symptom extraction method includes:

[0050] S1. Obtain a first sample set and a second sample set, wherein the data in the first sample set is labeled data, and the data in the second sample set is unlabeled data.

[0051] In this embodiment of the present invention, the first sample set is a small amount of labeled medical data, and the second sample set is a large amount of unlabeled medical data. Specifically, the medical data set may be an electronic medical record, for example. The electronic medical record is a medical record recorded by a doctor for a patient, specifically including information such as the patient's name, address, contact information, illness, symptoms, treatment history, and medical diagnosis.

[0052] S2. Perform adversarial training on the initial teacher model based on the first sample set to generate an adversarial sample set, calculate the gradient of the first sample set and the adversarial sample set according to the gradient function, add the gradient of the first sample set and the gradient of the adversarial sample set to obtain a first loss value, and update the parameters of the initial teacher model according to the first loss value to obtain a preliminary teacher model.

[0053] In an embodiment of the present invention, performing adversarial training on the initial teacher model based on the first sample set to generate an adversarial sample set, calculating the gradient of the first sample set and the adversarial sample set according to a gradient function, and adding the gradient of the first sample set and the gradient of the adversarial sample set to obtain a first loss value includes:

[0054] Calculating the gradient of the first sample set using a first gradient function of the initial teacher model;

[0055] Calculating the gradient of the first sample set according to the perturbation value function to obtain a perturbation value, and obtaining an adversarial sample set of the first sample set according to the perturbation value;

[0056] Calculate the gradient of the adversarial sample set using the second gradient function;

[0057] The gradient of the first sample set and the gradient of the adversarial sample set are added to obtain a first loss value.

[0058] In an optional embodiment of the present application, the gradient of the first sample set is calculated using the first gradient function of the initial teacher model, and the formula is:

[0059]

[0060] Among them, g is the gradient of the first sample set, p is the probability distribution of the first sample set, y is the label vector of the first sample set, and x is the text vector of the first sample set. To find the derivative with respect to x, θ is the preset parameter, and (y|x; θ) is the predicted y value under the condition that the input is x and the model parameter is θ.

[0061] In an optional embodiment of the present application, the gradient of the first sample set is calculated according to the perturbation value function to obtain a perturbation value, and an adversarial sample set of the first sample set is obtained according to the perturbation value, and the formula is:

[0062] r=∈g / ||g||2

[0063] Where r is the perturbation value, g is the gradient of the first sample set, and ∈ is a constant.

[0064] In an optional embodiment of the present application, the gradient of the adversarial sample set is calculated using the second gradient function, and the formula is:

[0065]

[0066] Among them, h is the gradient of the adversarial sample set, p is the probability distribution of the adversarial sample set, b is the label vector of the adversarial sample set, and a is the text vector of the adversarial sample set. To find the derivative of a, r is the disturbance value, θ is the preset parameter, and (b|a+r; θ) represents the predicted value of b under the condition that the input is a+r and the model parameter is θ.

[0067] In detail, the present invention reduces the influence of noise data by performing adversarial training on the initial teacher model, making the model more stable and robust.

[0068] S3. Analyze the second sample set using the preparatory teacher model to generate corresponding pseudo labels, and perform a first training on the initial student model based on the pseudo labels to generate a preparatory student model.

[0069] In an embodiment of the present invention, the initial student model is trained on the pseudo labels generated by the second sample set using the second sample set and the preliminary teacher model to generate a preliminary student model. The teacher model and student model mentioned in the present invention are tools for obtaining the final symptom extraction model and are independent of the model structure. They can be a BERT structure or a GPT structure, and are not limited here.

[0070] S4. Use the first sample set to train the preparatory student model for the second time to obtain a second loss value corresponding to the preparatory student model, add the first loss value and the second loss value to obtain a total loss value, compare the total loss value with a preset threshold, if the total loss value is less than the preset threshold, use the preparatory teacher model as the symptom extraction model; if the total loss value is greater than or equal to the preset threshold, update the parameters of the preparatory teacher model until the total loss value is less than the preset threshold, obtain a target teacher model, and use the target teacher model as the symptom extraction model.

[0071] In the embodiment of the present invention, the second training of the preparatory student model using the first sample set to obtain a second loss value corresponding to the preparatory student model includes:

[0072] Training the preparatory student model using the training set of the first sample set;

[0073] Using a sample set of the first sample set to verify the preparatory student model, and obtain a predicted label of the preparatory student model for the first sample set;

[0074] A loss value between the predicted label and the true label of the first sample is calculated to obtain a second loss value.

[0075] In the embodiment of the present invention, the predicted labels are labels generated by the preliminary student model through analysis of medical data in the sample set.

[0076] In an embodiment of the present invention, a threshold is preset. When the loss value is greater than or equal to the preset threshold, it means that the teacher model and the student model do not meet the requirements. Therefore, the parameters of the teacher model and the student model are changed and training is continued until the loss value is less than the loss threshold. The target teacher model is obtained as the symptom extraction model.

[0077] S5. Obtain a set of medical records of symptoms to be extracted, pre-process the set of medical records of symptoms to be extracted, and input the pre-processed set of medical records of symptoms to be extracted into the symptom extraction model to extract medical record symptoms.

[0078] In an embodiment of the present invention, obtaining a set of medical records of symptoms to be extracted, preprocessing the set of medical records of symptoms to be extracted, and inputting the preprocessed set of medical records of symptoms to be extracted into the symptom extraction model to extract medical record symptoms includes:

[0079] Obtain a set of symptom medical records to be extracted, and perform word segmentation on the set of symptom medical records to be extracted using the symptom extraction model;

[0080] Extracting target features from each of the segmented words based on a preset symptom feature template to obtain target features for each of the segmented words;

[0081] Marking each of the segmented words using a preset marking method according to the target feature to obtain corresponding marking information, wherein the marking information includes information marking various symptoms;

[0082] Symptoms are extracted from the set of symptom medical records to be extracted based on the annotation information.

[0083] In this embodiment, the disease symptom text in the symptom medical record set to be extracted is first segmented, for example, the jieba segmentation tool is used to segment the disease symptom text; in another embodiment, since different clinicians have different writing styles, their description words and punctuation separation methods may be different. For example, for the same negative description, there may be "no headache or fever", "no headache or fever", "no fever or headache", "denied hypertension", etc. At this time, the writing style of each clinician is analyzed, and the writing style includes logical expression and customary words for symptoms. The disease symptom text is segmented according to each writing style, and the text is divided into a series of short sentences. Then, the forward maximum matching algorithm is used to segment the short sentences to obtain the words in the short sentences, their parts of speech, and relative position relationships.

[0084] After the disease symptom text is segmented, the target features of each segmentation are extracted according to the preset feature template to obtain the target features of each segmentation. The target features of the segmentation are associated with the part of speech of the associated segmentation before and / or after the segmentation. The above feature template can be a template provided by the CRF model. The CRF model can be built in advance based on the CRF++ open source tool, using the symptom description words of various doctors as training corpus. After the initial template is trained, the trained initial template can be tested with a preset test set. The training is considered completed only when the test results meet the test indicators, and the corresponding feature template is obtained. The target features of the segmentation can be obtained through the feature template. The target features include the part of speech of the segmentation, the part of speech of the previous segmentation, the part of speech of the next segmentation, and the association relationship between these segmentations.

[0085] Then, each word segment is labeled using the BIO labeling method to obtain corresponding labeling information. In this embodiment, a joint labeling (Joint segmentation and labeling) method can be used to label the descriptions of the same disease symptoms with the same label to obtain corresponding labeling information. That is, the above-mentioned labeling information includes information for labeling various disease symptoms, and then the corresponding symptoms are extracted from the target electronic medical record based on the labeling information.

[0086] The present invention uses a labeled data set to perform adversarial training on the initial teacher model, generates an adversarial sample set, calculates a first loss value, updates the parameters of the initial teacher model to obtain a preparatory teacher model, makes the model more robust to noise disturbances, uses the preparatory teacher model to analyze the unlabeled data set, generates pseudo labels, uses the labeled data and the unlabeled data for generating pseudo labels to train the initial student model twice, obtains a second loss value, adds the loss values ​​to obtain a total loss value, compares it with a preset threshold value, and outputs the target teacher model as a symptom extraction model until the total loss value is less than the preset threshold value. The present invention can not only utilize a large amount of existing unlabeled medical data, but also fully consider the influence of noise data, making the model more stable and more generalizable, thereby improving the accuracy of the model in extracting case symptoms.

[0087] like Figure 2 , which is a functional module diagram of a pseudo-label-based case symptom extraction device provided by one embodiment of the present invention.

[0088] The pseudo-label-based case symptom extraction device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the pseudo-label-based case symptom extraction device 100 can include a data extraction module 101, an adversarial training module 102, a pseudo-label generation module 103, and a loss value module 104. The module of the present invention, also known as a unit, refers to a series of computer program segments that can be executed by an electronic device processor and can perform fixed functions, which are stored in the memory of the electronic device.

[0089] In this embodiment, the functions of each module / unit are as follows:

[0090] A data extraction module 101 is configured to obtain a first sample set and a second sample set, wherein the data in the first sample set is labeled data and the data in the second sample set is unlabeled data;

[0091] An adversarial training module 102 is configured to perform adversarial training on the initial teacher model based on the first sample set to generate an adversarial sample set, calculate the gradient of the first sample set and the adversarial sample set according to a gradient function, add the gradient of the first sample set and the gradient of the adversarial sample set to obtain a first loss value, and update the parameters of the initial teacher model according to the first loss value to obtain a preliminary teacher model;

[0092] a pseudo-label generating module 103 for analyzing the second sample set using the preparatory teacher model to generate corresponding pseudo-labels, and performing a first training on the initial student model based on the pseudo-labels to generate a preparatory student model;

[0093] The loss value module 104 is used to use the first sample set to perform a second training on the preparatory student model to obtain a second loss value corresponding to the preparatory student model, add the first loss value and the second loss value to obtain a total loss value, compare the total loss value with a preset threshold, and if the total loss value is less than the preset threshold, use the preparatory teacher model as the symptom extraction model; if the total loss value is greater than or equal to the preset threshold, update the parameters of the preparatory teacher model until the total loss value is less than the preset threshold, obtain a target teacher model, and use the target teacher model as the symptom extraction model.

[0094] In detail, each module in the pseudo-label-based case symptom extraction device 100 according to the embodiment of the present invention adopts the same Figure 1 The training method of the pseudo-label-based symptom extraction model described in

[15] is the same technical means and can produce the same technical effects, so I will not go into details here.

[0095] like Figure 3 , which is a structural diagram of an electronic device for implementing a method for extracting case symptoms based on pseudo labels provided by an embodiment of the present invention.

[0096] The electronic device 1 may include a processor 10, a memory 11, a communication interface 12 and a bus 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a training program for a pseudo-label-based symptom extraction model.

[0097] Among them, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the electronic device 1. Furthermore, the memory 11 can also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 can not only be used to store application software and various types of data installed on the electronic device 1, such as the code of the training program of the pseudo-label-based symptom extraction model, but can also be used to temporarily store data that has been output or is to be output.

[0098] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing the programs or modules stored in the memory 11 (such as the training program of the pseudo-label-based symptom extraction model, etc.), as well as calling the data stored in the memory 11, to perform various functions of the electronic device 1 and process data.

[0099] The communication interface 12 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, for displaying information processed in the electronic device and for displaying a visual user interface.

[0100] The bus 13 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus 13 may be divided into an address bus, a data bus, a control bus, etc. The bus 13 is configured to enable communication between the memory 11 and at least one processor 10.

[0101] Figure 3 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0102] For example, although not shown, the electronic device 1 may further include a power source (such as a battery) for powering the various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charging management, discharging management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device 1 may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0103] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0104] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0105] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0106] The training program of the pseudo-label-based symptom extraction model stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve the following:

[0107] Obtaining a first sample set and a second sample set, wherein the data in the first sample set is labeled data, and the data in the second sample set is unlabeled data;

[0108] Performing adversarial training on the initial teacher model based on the first sample set to generate an adversarial sample set, calculating the gradient of the first sample set and the adversarial sample set according to a gradient function, adding the gradient of the first sample set and the gradient of the adversarial sample set to obtain a first loss value, and updating the parameters of the initial teacher model according to the first loss value to obtain a preliminary teacher model;

[0109] Analyzing the second sample set using the preparatory teacher model to generate corresponding pseudo labels, and performing a first training on the initial student model based on the pseudo labels to generate a preparatory student model;

[0110] The preparatory student model is trained for a second time using the first sample set to obtain a second loss value corresponding to the preparatory student model, the first loss value and the second loss value are added to obtain a total loss value, the total loss value is compared with a preset threshold value, and if the total loss value is less than the preset threshold value, the preparatory teacher model is used as the symptom extraction model; if the total loss value is greater than or equal to the preset threshold value, the parameters of the preparatory teacher model are updated until the total loss value is less than the preset threshold value, and a target teacher model is obtained, and the target teacher model is used as the symptom extraction model;

[0111] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0112] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0113] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0114] Obtaining a first sample set and a second sample set, wherein the data in the first sample set is labeled data, and the data in the second sample set is unlabeled data;

[0115] Performing adversarial training on the initial teacher model based on the first sample set to generate an adversarial sample set, calculating the gradient of the first sample set and the adversarial sample set according to a gradient function, adding the gradient of the first sample set and the gradient of the adversarial sample set to obtain a first loss value, and updating the parameters of the initial teacher model according to the first loss value to obtain a preliminary teacher model;

[0116] Analyzing the second sample set using the preparatory teacher model to generate corresponding pseudo labels, and performing a first training on the initial student model based on the pseudo labels to generate a preparatory student model;

[0117] The preparatory student model is trained for a second time using the first sample set to obtain a second loss value corresponding to the preparatory student model, the first loss value and the second loss value are added to obtain a total loss value, the total loss value is compared with a preset threshold value, and if the total loss value is less than the preset threshold value, the preparatory teacher model is used as the symptom extraction model; if the total loss value is greater than or equal to the preset threshold value, the parameters of the preparatory teacher model are updated until the total loss value is less than the preset threshold value, and a target teacher model is obtained, and the target teacher model is used as the symptom extraction model;

[0118] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0119] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0120] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0121] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0122] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0123] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0124] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Second-order terms are used to indicate names and do not imply any particular order.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for extracting case symptoms based on pseudo labels, characterized in that: The method comprises: Obtaining a first sample set and a second sample set, wherein the data in the first sample set is labeled data, and the data in the second sample set is unlabeled data; Calculating the gradient of the first sample set using the first gradient function of the initial teacher model, calculating the gradient of the first sample set according to the perturbation value function to obtain a perturbation value, obtaining an adversarial sample set for the first sample set according to the perturbation value, calculating the gradient of the adversarial sample set using the second gradient function, adding the gradient of the first sample set and the gradient of the adversarial sample set to obtain a first loss value, and updating the parameters of the initial teacher model according to the first loss value to obtain a preliminary teacher model; Analyzing the second sample set using the preparatory teacher model to generate corresponding pseudo labels, and performing a first training on the initial student model based on the pseudo labels to generate a preparatory student model; The preparatory student model is trained for a second time using the training set of the first sample set, the preparatory student model is verified using the sample set of the first sample set to obtain a predicted label of the preparatory student model for the first sample set, the loss value between the predicted label and the true label of the first sample is calculated to obtain a second loss value, the first loss value and the second loss value are added to obtain a total loss value, the total loss value is compared with a preset threshold, if the total loss value is less than the preset threshold, the preparatory teacher model is used as the symptom extraction model; if the total loss value is greater than or equal to the preset threshold, the parameters of the preparatory teacher model are updated until the total loss value is less than the preset threshold, to obtain a target teacher model, and the target teacher model is used as the symptom extraction model; Obtain a set of medical records of symptoms to be extracted, preprocess the set of medical records of symptoms to be extracted, input the preprocessed set of medical records of symptoms to be extracted into the symptom extraction model to extract medical record symptoms, including: obtaining a set of medical records of symptoms to be extracted, segmenting the set of medical records of symptoms to be extracted using the symptom extraction model, extracting target features of each segmentation based on a preset symptom feature template, labeling each segmentation using a preset labeling method based on the target feature to obtain corresponding labeling information, extracting symptoms from the set of medical records of symptoms to be extracted based on the labeling information, and the labeling information includes information for labeling various symptoms.

2. The pseudo-label-based case symptom extraction method according to claim 1, wherein: The formula of the first gradient function is: Among them, g is the gradient of the first sample set, p is the probability distribution of the first sample set, y is the label vector of the first sample set, and x is the text vector of the first sample set. To find the derivative with respect to x, θ is the preset parameter.

3. The pseudo-label-based case symptom extraction method according to claim 1, wherein: The formula of the disturbance value function is: Where r is the perturbation value, g is the gradient of the first sample set, and ϵ is a constant.

4. The method for extracting case symptoms based on pseudo labels according to claim 1, wherein: The formula of the second gradient function is: Among them, h is the gradient of the adversarial sample set, p is the probability distribution of the adversarial sample set, b is the label vector of the adversarial sample set, and a is the text vector of the adversarial sample set. To find the derivative of a, r is the disturbance value and θ is the preset parameter.

5. A pseudo-label-based case symptom extraction device, characterized in that: The device comprises: A data extraction module, configured to obtain a first sample set and a second sample set, wherein the data in the first sample set is labeled data, and the data in the second sample set is unlabeled data; An adversarial training module is configured to calculate the gradient of the first sample set using a first gradient function of the initial teacher model, calculate the gradient of the first sample set according to a perturbation value function to obtain a perturbation value, obtain an adversarial sample set for the first sample set according to the perturbation value, calculate the gradient of the adversarial sample set using a second gradient function, add the gradient of the first sample set and the gradient of the adversarial sample set to obtain a first loss value, and update the parameters of the initial teacher model according to the first loss value to obtain a preliminary teacher model; a pseudo-label generation module, configured to analyze the second sample set using the preparatory teacher model to generate corresponding pseudo-labels, and perform a first training on the initial student model based on the pseudo-labels to generate a preparatory student model; A loss value module is used to perform a second training on the preparatory student model using the training set of the first sample set, verify the preparatory student model using the sample set of the first sample set to obtain the predicted label of the preparatory student model for the first sample set, calculate the loss value between the predicted label and the true label of the first sample to obtain a second loss value, add the first loss value and the second loss value to obtain a total loss value, compare the total loss value with a preset threshold, and if the total loss value is less than the preset threshold, use the preparatory teacher model as the symptom extraction model; if the total loss value is greater than or equal to the preset threshold, update the parameters of the preparatory teacher model until the total loss value is less than the preset threshold to obtain a target teacher model, and use the target teacher model as the symptom extraction model, including: obtaining a set of symptom medical records to be extracted, using the symptom extraction model to segment the set of symptom medical records to be extracted, extracting target features of each of the segmentations according to a preset symptom feature template, labeling each of the segmentations using a preset labeling method according to the target feature to obtain corresponding labeling information, extracting symptoms from the set of symptom medical records to be extracted according to the labeling information, and the labeling information includes information for labeling various symptoms.

6. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the pseudo-label-based case symptom extraction method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the pseudo-label-based case symptom extraction method according to any one of claims 1 to 4 is implemented.

Images