Method, device and apparatus for data processing based on new word ratio

Abstract

The present disclosure provides a method for data processing based on the new word ratio, comprising: S1: processing all unlabeled data by constructing a queue system to obtain queue data; S2: performing new word ratio analysis on the obtained queue data; S3: performing quality assessment on model annotated data according to the new word ratio of each queue data; S4: selecting a group of queue data with the highest quality in the queue data to add to the training set for data expansion. The present disclosure also provides a data processing device, an electronic device, and a readable storage medium.

Description

Technical Field

[0001] The present disclosure relates to the technical field of entity recognition, and particularly to a method, apparatus, and device for data processing based on the proportion of new words. Background Art

[0002] Although named entity recognition research has been widely applied in the general field, there are still many problems when applied to the medical field, such as difficulty in recognizing professional terms and scarcity of labeled data. Different from the text in the general field, medical field texts such as electronic medical records have unique text structures and contain rich professional knowledge.

[0003] First, Chinese electronic medical records have the characteristic of numerous professional terms, such as "radical distal gastrectomy", "gastroscopy pathology", etc., and it is difficult for people without a medical background to understand the meanings of these terms; second, for the convenience of medical staff to write and read, there are many templatized expressions in electronic medical records, such as "perform...show...", "perform...surgery", and there are many English abbreviations in medical entities, such as "Billroth II anastomosis", which easily lead to errors in model recognition; finally, there are a large number of entity nesting problems in electronic medical records. For example, the entire entity of "radical distal gastrectomy" represents a kind of surgery, but among them, "radical" and "distal" are modifiers of the stomach and are parts of the approach in the surgical procedure, while "gastric大部" is the part, and "resection" is the surgical procedure, which together form a surgical entity and are difficult to accurately define the boundary in the recognition task. Summary of the Invention

[0004] To solve at least one of the above technical problems, the present disclosure provides a method, apparatus, and device for data processing based on the proportion of new words.

[0005] According to one aspect of the present disclosure, a method for data processing based on the proportion of new words is provided, including: S1: Processing all unlabeled data through constructing a queue system to obtain queue data; S2: Conducting new word proportion analysis on the obtained queue data; S3: Evaluating the quality of the model labeled data according to the new word proportion of each queue data; S4: Selecting a group of queue data with the highest quality in the queue data and adding it to the training set for data augmentation.

[0006] According to the method for data processing based on the proportion of new words according to at least one embodiment of the present disclosure, the unlabeled data is unlabeled data in Chinese electronic medical records.

[0007] According to the method for processing data based on the proportion of new words of at least one embodiment of the present disclosure, the queue data is obtained in step S1 through the following steps: S11: Model training is performed through training set and development set training, and the model with the best performance on the development set is selected as the label model, and all unlabeled data are labeled, and it is assumed that the labeling is the standard answer; S12: According to the growth trend of the model value during the model training process, n models obtained from single-round training are selected, and all unlabeled data are repeatedly labeled using the models obtained from single-round training; S13: Each single-round model is compared with the standard answer of the label model, and data samples with the same labeling and data samples with different labeling are placed at intervals. For the data with the current queue number n, the former is promoted to queue n+1, and the latter is reduced to queue n-1, and finally n+1 data queues are obtained.

[0008] According to the method for data processing based on the new word ratio in at least one embodiment of the present disclosure, the new word ratio in step S2 is calculated according to the following formula:

[0009] The proportion of new words = 1-[(the number of entities annotated by the model in the entity dictionary of the training set) / (the number of all entities annotated in the pseudo data)]×100%.

[0010] According to the method for processing data based on the proportion of new words of at least one embodiment of the present disclosure, in step S3, the lower the proportion of new words in the model annotation data, the higher the quality of the queue data.

[0011] According to the method for performing data processing based on the proportion of new words of at least one embodiment of the present disclosure, the training set obtained in step S4 is used for the next round of model training.

[0012] According to the method for data processing based on the new word ratio of at least one embodiment of the present disclosure, the model training uses a neural network model.

[0013] According to another aspect of the present disclosure, there is provided a data processing device, including: a queue data processing module, which processes all unlabeled data by constructing a queue system to obtain queue data; a new word ratio analysis module, which performs new word ratio analysis on the obtained queue data; a quality assessment module, which performs quality assessment on model annotated data according to the new word ratio of each queue data; and a data expansion module, which selects a group of queue data with the highest quality in the queue data and adds them to a training set for data expansion.

[0014] According to the data processing device of at least one embodiment of the present disclosure, the queue data processing module obtains the queue data through the following steps:

[0015] S11: Train the model through training set and development set training, select the model with the best performance on the development set as the label model, label all unlabeled data, and assume that the label is the standard answer;

[0016] S12: Select n models obtained through single-round training according to the growth trend of the model value during the model training process, and use the models obtained through single-round training to repeatedly label all unlabeled data;

[0017] S13: Compare the labeling results of each single-round model with those of the optimal model, and place data samples with the same labeling and data samples with different labeling at intervals. For the data with the current queue number n, the former is promoted to queue n+1 and the latter is demoted to queue n-1, and finally n+1 data queues are obtained.

[0018] According to another aspect of the present disclosure, a readable storage medium is provided, wherein the readable storage medium stores a computer program, wherein the computer program is used for a processor to execute one of the above-mentioned methods for performing data processing based on the new word ratio.

[0019] According to another aspect of the present disclosure, an electronic device is provided, which includes a processor and a readable storage medium, wherein the readable storage medium stores execution instructions, and the processor executes the execution instructions in the readable storage medium, so that the processor executes one of the above-mentioned methods for data processing based on the proportion of new words. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0021] Figure 1 It is a flowchart of a method for performing data processing based on the proportion of new words according to an embodiment of the present disclosure.

[0022] Figure 2 It is a flow chart of a queue construction system according to an embodiment of the present disclosure for processing all unlabeled data to obtain queue data.

[0023] Figure 3 It is a schematic diagram of the structure of a data processing device according to an embodiment of the present disclosure.

[0024] Figure 4 It is a schematic structural diagram of an electronic device having a data processing device according to an embodiment of the present disclosure.

[0025] Description of Reference Numerals

[0026] 1000 electronic devices

[0027] 1001 queue data processing module

[0028] 1002 New word ratio analysis module

[0029] 1003 Quality Assessment Module

[0030] 1004 Data Expansion Module

[0031] 1100 bus

[0032] 1200 processors

[0033] 1300 Memory

[0034] 1400 other circuits. DETAILED DESCRIPTION

[0035] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0036] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0037] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0038] The use of cross-hatching and / or shading in the accompanying drawings is generally used to make the boundaries between adjacent components clear. As such, unless otherwise specified, the presence or absence of cross-hatching or shading does not convey or indicate any preference or requirement for the specific materials, material properties, dimensions, proportions, commonalities between the components shown, and / or any other characteristics, attributes, properties, etc. of the components. In addition, in the accompanying drawings, the sizes and relative sizes of the components may be exaggerated for clarity and / or descriptive purposes. When the exemplary embodiments can be implemented differently, the specific process sequence can be performed in a different order than described. For example, two successively described processes can be performed substantially simultaneously or in an order opposite to the described order. In addition, the same figure numbers represent the same components.

[0039] When a component is referred to as being "on" or "over," "connected to," or "coupled to" another component, the component may be directly on, directly connected to, or directly coupled to the other component, or intervening components may be present. However, when a component is referred to as being "directly on," "directly connected to," or "directly coupled to" another component, there are no intervening components. For this purpose, the term "connected" may refer to a physical connection, an electrical connection, etc., with or without intervening components.

[0040] The terms used herein are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0041] Figure 1 It is a flowchart of a method for performing data processing based on the proportion of new words according to an embodiment of the present disclosure.

[0042] like Figure 1 As shown, the method for data processing based on the new word ratio of this embodiment includes: S1: obtaining queue data by processing all unlabeled data through constructing a queue system; S2: performing new word ratio analysis on the obtained queue data to obtain the new word ratio of each queue data; S3: performing quality assessment on the model annotation data based on the new word ratio of each queue data; S4: selecting a group of queue data with the highest quality from all queue data and adding it to the training set for data expansion.

[0043] In step S1, queue data is obtained by constructing a queue system to process all unlabeled data. Figure 2 It is a flow chart of a queue construction system according to an embodiment of the present disclosure for processing all unlabeled data to obtain queue data.

[0044] like Figure 2 As shown, the queue data is obtained by constructing a queue system to process all unlabeled data, including the following steps:

[0045] S11: Perform model training through training on the training set and the development set, select the model with the best performance on the development set as the labeling model, use the labeling model to label all unlabeled data, and use the labeled results as the standard answer; the model can use an existing neural network model;

[0046] S12: According to the growth trend of the model F value during the model training process in step S11, select n models obtained from single-round training, and use the models obtained from single-round training to repeatedly annotate all unlabeled data; wherein, the model F value is the harmonic mean of the model accuracy P and the model recall rate, F = (2×P×R)×100% / (P+R); P is the accuracy rate: P = the number of correctly identified entities × 100% / the number of identified entities; R is the recall rate: R = the number of correctly identified entities × 100% / the number of correct entities.

[0047] S13: Use each single-round model to compare with the standard answer of the label model, and place the data samples with the same label and the data samples with different label at intervals. For the data with the current queue number n, the former is promoted to queue n+1 and the latter is demoted to queue n-1, and finally n+1 data queues are obtained.

[0048] In step S2, the obtained queue data is analyzed for the proportion of new words, where new words refer to entities marked by the model that have not appeared in the original training set. The proportion of new words is calculated according to the following formula: new word proportion = 1-[(the number of entities marked by the model in the entity dictionary of the training set) / (the number of all entities marked in the pseudo data)]×100%. Wherein, pseudo data refers to the queue data obtained in step S1.

[0049] Step S3: perform quality assessment on the model annotation data according to the new word ratio of each queue data. For different queue data, the quality of the model annotation data of the queue is assessed according to its new word ratio. Since a low new word ratio indicates that most entities recognized by the model exist in the original entity dictionary, the accuracy of the queue data can be fully guaranteed. In S3, the lower the new word ratio in the model annotation data, the higher the quality of the queue data.

[0050] New word ratio New word ratio

[0051] S4. Select a group of queue data with the highest quality from the queue data and add them to the training set for data expansion. Since the low proportion of new words means that most of the entities recognized by the model exist in the original entity dictionary, the accuracy of the queue data can be fully guaranteed. Adding this data to the training set for data expansion can minimize the possibility of introducing noise samples while retaining all data samples. Then, delete the queue data added to the training set from the unlabeled data to obtain new unlabeled data. The new training set and new unlabeled data generated in S4 can be used for the next round of model training.

[0052] The lower the proportion of new words, the more entities recognized by the model will match the entities in the original training set. Adding such data to the training set can greatly reduce the risk of introducing noise. On the contrary, for queue data with a high proportion of new words, due to the lack of standard answers for such data, the accuracy of the data samples in the new words cannot be determined. Adding them to the training set will most likely introduce noise, resulting in a slight improvement in model performance.

[0053] Figure 3 A data processing device of an embodiment of the present disclosure is shown, including: a queue data processing module 1001, which processes all unlabeled data by constructing a queue system to obtain queue data; a new word ratio analysis module 1002, which performs new word ratio analysis on the obtained queue data; a quality assessment module 1003, which performs quality assessment on model annotated data according to the new word ratio of each queue data; and a data expansion module 1004, which selects a group of queue data with the highest quality in the queue data and adds them to the training set for data expansion.

[0054] According to a preferred embodiment of the present disclosure, the queue data processing module 1001 obtains the queue data through the following steps:

[0055] S11: Train the model through training set and development set training, select the model with the best performance on the development set as the label model, label all unlabeled data, and assume that the label is the standard answer;

[0056] S12: Select n models obtained through single-round training according to the growth trend of the model value during the model training process, and use the models obtained through single-round training to repeatedly label all unlabeled data;

[0057] S13: Compare the labeling results of each single-round model with those of the optimal model, and place data samples with the same labeling and data samples with different labeling at intervals. For the data with the current queue number n, the former is promoted to queue n+1 and the latter is demoted to queue n-1, and finally n+1 data queues are obtained.

[0058] Figure 4 It is a schematic structural diagram of an electronic device having a data processing device according to an embodiment of the present disclosure.

[0059] like Figure 4 As shown, the electronic device 1000 may include a corresponding module for executing each or several steps in the above method. Therefore, each step or several steps in the above method may be executed by a corresponding module, and the electronic device 1000 may include one or more modules in these modules. The module may be one or more hardware modules specially configured to execute the corresponding steps, or implemented by a processor configured to execute the corresponding steps, or stored in a computer-readable medium for implementation by a processor, or implemented by some combination.

[0060] The hardware structure of the device can be implemented using a bus architecture. The bus architecture can include any number of interconnected buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus 1100 connects various circuits including one or more processors 1200, memory 1300 and / or hardware modules together. The bus 1100 can also connect various other circuits 1400 such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0061] The bus 1100 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the figure only uses one connecting line, but does not mean that there is only one bus or one type of bus.

[0062] Any process or method description in the flowchart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes other implementations, in which the functions may not be performed in the order shown or discussed, including performing the functions in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by a person skilled in the art of the present disclosure. The processor performs the various methods and processes described above. For example, the method implementation in the present disclosure can be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via a memory and / or a communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods in any other appropriate manner (e.g., by means of firmware).

[0063] The logic and / or steps represented in the flowchart or otherwise described herein may be embodied in any readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions on an instruction execution system, apparatus, or device), or in combination with such instruction execution systems, apparatuses, or devices.

[0064] For the purposes of this specification, a "readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use with or in conjunction with an instruction execution system, device or apparatus. More specific examples of readable storage media (a non-exhaustive list) include the following: an electrical connection with one or more wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a memory.

[0065] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0066] A person skilled in the art may understand that all or part of the steps of implementing the above-mentioned implementation method may be completed by instructing related hardware through a program, and the program may be stored in a readable storage medium, which, when executed, includes one or a combination of the steps of the implementation method.

[0067] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a readable storage medium. The storage medium may be a read-only memory, a disk or an optical disk, etc.

[0068] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment / method or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments / methods or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments / methods or examples described in this specification and the features of the different embodiments / methods or examples, unless they are contradictory.

[0069] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0070] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. A method for data processing based on the proportion of new words, characterized in that: include: S1: Process all unlabeled data to obtain queue data by building a queue system; S2: New word proportion analysis was performed on the obtained cohort data; S3: Evaluate the quality of the model annotation data according to the proportion of new words in each cohort data; and S4: Select a group of queue data with the highest quality from the queue data and add them to the training set for data expansion; In step S1, the queue data is obtained by the following steps: S11: Train the model using the training set and the development set, select the model with the best performance on the development set as the labeling model, label all unlabeled data, obtain labeled data, and use the labeled data as the standard answer; S12: Select n models obtained through single-round training according to the growth trend of the model value during the model training process, and use the models obtained through single-round training to repeatedly label all unlabeled data; as well as S13: Use each single-round model to compare with the standard answer of the label model, place the data samples with the same label and the data samples with different label alternately, and for the data with the current queue number n, promote the former to queue n+1 and reduce the latter to queue n-1, and finally obtain n+1 data queues; The new word ratio in step S2 is calculated according to the following formula: The proportion of new words = 1-[(the number of entities annotated by the model in the entity dictionary of the training set) / (the number of all entities annotated in the pseudo data)]×100%.

2. The method for data processing based on the new word ratio according to claim 1, characterized in that: The unlabeled data are unlabeled data in Chinese electronic medical records.

3. The method for data processing based on new word ratio according to claim 1, characterized in that: In step S3, the lower the proportion of new words in the model annotation data, the higher the quality of the queue data.

4. The method for data processing based on new word ratio according to claim 1, characterized in that: The training set obtained in step S4 is used for the next round of model training.

5. A data processing device, characterized in that: include: A queue data processing module, wherein the queue data processing module processes all unlabeled data to obtain queue data by constructing a queue system; A new word ratio analysis module, wherein the new word ratio analysis module performs new word ratio analysis on the obtained queue data; A quality assessment module, wherein the quality assessment module performs quality assessment on the model annotation data according to the new word ratio of each queue data; as well as A data expansion module, wherein the data expansion module selects a group of queue data with the highest quality from the queue data and adds it to the training set for data expansion; The queue data processing module obtains the queue data through the following steps: S11: Train the model through training set and development set training, select the model with the best performance on the development set as the label model, label all unlabeled data, and assume that the label is the standard answer; S12: Select n models obtained through single-round training according to the growth trend of the model value during the model training process, and use the models obtained through single-round training to repeatedly label all unlabeled data; as well as S13: Compare the labeling results of each single-round model with those of the optimal model, place data samples with the same labeling and data samples with different labeling at intervals, and for data with the current queue number n, promote the former to queue n+1 and reduce the latter to queue n-1, and finally obtain n+1 data queues; The new word ratio is calculated according to the following formula: The proportion of new words = 1-[(the number of entities annotated by the model in the entity dictionary of the training set) / (the number of all entities annotated in the pseudo data)]×100%.

6. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and the computer program is used for a processor to execute the method for performing data processing based on the new word ratio according to any one of claims 1 to 4.

7. An electronic device, characterized in that: The method comprises a processor and a readable storage medium, wherein the readable storage medium stores execution instructions, and the processor executes the execution instructions in the readable storage medium, so that the processor executes the method for data processing based on the new word ratio according to any one of claims 1 to 4.

Images