Data segmentation method, system and equipment based on pre-training language model and medium

Abstract

The invention discloses a pre-training language model-based data segmentation method, system and device and a medium, and relates to the technical field of text segmentation, the method comprises the following steps: obtaining original medical record text data, and segmenting the original medical record text data to obtain N data; processing the N data through a pre-training language model to obtain a medical record word vector set; processing the obtained medical field standard words through a pre-training language model to obtain a standard word vector set; and based on the medical record word vector set and the standard word vector set, marking entity information in the original medical record text data through a similarity calculation result. According to the method, the key medical information can be accurately extracted and labeled from the original medical record text by utilizing the strong semantic understanding ability of the pre-trained language model and an efficient text segmentation and similarity calculation method, and the efficiency and accuracy of medical record data processing are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of text segmentation technology, and in particular to data segmentation methods, systems, devices and media based on pre-trained language models. Background Technology

[0002] In the medical field, medical record data is a crucial foundation for clinical diagnosis, treatment, and research. Medical record texts typically contain key information such as a patient's detailed medical history, symptoms, diagnostic results, and treatment process. However, medical record texts are usually unstructured data in natural language format, making the direct extraction and analysis of key medical information challenging.

[0003] Currently, common methods for processing medical record data include: Rule-based text segmentation methods identify and extract key information from medical records using predefined medical terms, keywords, or regular expressions. However, this method relies on manually defined rules, resulting in poor flexibility and difficulty adapting to complex and ever-changing medical record texts.

[0004] Traditional machine learning methods extract text features using feature engineering, and then extract information through classifiers or sequence labeling models. These methods require large amounts of labeled data to train the model, and the feature extraction process is complex, making it difficult to capture the deep semantic information of the text.

[0005] Deep learning-based text processing methods: In recent years, with the development of deep learning technology, some studies have begun to try to use convolutional neural networks (CN) or recurrent neural networks (RN) to process medical record texts. However, these methods suffer from gradient vanishing or exploding problems when processing long texts, and their ability to model contextual information is limited.

[0006] Although existing medical record data processing methods can extract key information to some extent, the following problems still exist: Insufficient flexibility: Rule-based methods rely on manually defined rules, making it difficult to adapt to medical record texts with different formats and content.

[0007] High demand for labeled data: Traditional machine learning methods require a large amount of labeled data to train models, and the labeling process is time-consuming and costly.

[0008] Limited context modeling capabilities: Traditional deep learning methods (such as CN and RN) suffer from gradient vanishing or gradient exploding problems when processing long texts, making it difficult to effectively capture long-distance contextual information.

[0009] Insufficient semantic understanding: Although existing medical record data processing methods based on pre-trained language models can capture semantic information, they still fall short in terms of refined segmentation and key information extraction, and cannot effectively identify key information such as disease entities and symptoms in medical records. Summary of the Invention

[0010] The technical problem to be solved by this invention is to address the shortcomings of existing technologies. Specifically, it provides a data segmentation method, system, device, and medium based on a pre-trained language model, as detailed below: 1) In a first aspect, the present invention provides a data segmentation method based on a pre-trained language model, the specific technical solution of which is as follows: Obtain the original medical record text data and segment it to obtain N data points; A set of medical record word vectors is obtained by processing N data points using a pre-trained language model; A set of standard word vectors is obtained by processing the standard words in the medical field through a pre-trained language model. Based on the medical record word vector set and the standard word vector set, entity information in the original medical record text data is labeled using the similarity calculation results.

[0011] The beneficial effects of the data segmentation method based on a pre-trained language model provided by this invention are as follows: Leveraging the powerful semantic understanding capabilities of pre-trained language models, this method accurately captures the deep semantic features of medical record texts and standard terms, thereby improving the accuracy and reliability of entity information annotation. Through N-gram segmentation and similarity calculation, it achieves refined segmentation and key information extraction of medical record texts, effectively identifying key information such as disease entities and symptoms, thus improving the efficiency and quality of medical record data processing. Furthermore, this method does not rely on manually defined rules, exhibiting high flexibility and scalability. It can adapt to medical record texts of different formats and contents, and the segmentation effect can be optimized by adjusting parameters such as the similarity threshold. This provides an efficient, flexible, and accurate solution for the processing and analysis of medical record data, contributing to the development of medical informatization and the effective utilization of medical data.

[0012] Based on the above solution, the present invention can be further improved as follows.

[0013] Furthermore, the process of segmenting the original medical record text data to obtain N data points is as follows: The original medical record text data is divided into N-grams using a sliding window to obtain N data points.

[0014] Furthermore, pre-trained language models include: Embedding layer, multi-head attention mechanism layer, and forward computation layer; The Embedding layer is used to perform matrix multiplication using the weight matrix and the ID mapped from the input data to obtain the embedding word vectors; The multi-head attention mechanism layer is used to determine the vector representation of each input data that incorporates attention information by matrix multiplication based on the embedding word vectors; The forward computation layer is used to generate output data after activation by two linear layers and an activation layer.

[0015] The beneficial effects of the above-mentioned further solutions are as follows: By using the sliding window method to perform N-gram partitioning on the original medical record text data, N data fragments can be systematically generated, ensuring that all parts of the medical record text are fully considered. This provides a more comprehensive input for subsequent vectorization processing and avoids the loss of key information due to missing text fragments. Simultaneously, a pre-trained language model including an embedding layer, a multi-head attention mechanism layer, and a forward computation layer can more accurately capture the semantic information of the text. The embedding layer generates embedding word vectors by multiplying the weight matrix and the IDs mapped to the input data, providing a basic semantic representation of the text. The multi-head attention mechanism layer further determines the vector representation of each input data point incorporating attention information through matrix multiplication, effectively capturing long-distance dependencies and contextual information in the text. The forward computation layer generates output data after activation by two linear layers and an activation layer, further optimizing the quality of the vector representation. This hierarchical model structure design enables the model to extract and fuse text features from different levels, thereby more accurately representing the semantic information of medical record text and standard words. This provides a more reliable basis for similarity calculation and entity information annotation, further improving the accuracy and efficiency of medical record data processing.

[0016] Furthermore, the process of annotating entity information in the original medical record text data based on similarity calculation results is as follows: For each vector in the medical record word vector set, calculate the cosine similarity with all vectors in the standard word vector set; The cosine similarity calculation result is compared with the preset similarity threshold one by one. When the cosine similarity calculation result is greater than the preset similarity threshold, the vector in the medical record word vector set corresponding to the cosine similarity calculation result is labeled to obtain the entity information in the original medical record text data.

[0017] The beneficial effects of the above-mentioned further solutions are as follows: By calculating the cosine similarity between each vector in the medical record word vector set and all vectors in the standard word vector set, the semantic similarity between words in the medical record text and standard words can be systematically quantified. The relationship between the cosine similarity calculation result and a preset similarity threshold is evaluated one by one to ensure that only medical record words that are highly semantically related to the standard words are labeled as entity information, thus effectively avoiding mislabeling and overlabeling. This method not only improves the accuracy and reliability of entity information labeling but also allows for flexible adjustment of the similarity threshold to adapt to different application scenarios and accuracy requirements. Furthermore, the cosine similarity-based calculation method is simple and efficient, enabling rapid completion of labeling tasks in large-scale medical record data processing, significantly improving the efficiency and practicality of medical record text processing, and providing higher-quality data support for medical data analysis and clinical decision-making.

[0018] 2) In a second aspect, the present invention also provides a data segmentation system based on a pre-trained language model, the specific technical solution of which is as follows: The acquisition module is used to: acquire the original medical record text data and segment the original medical record text data to obtain N data points; The first processing module is used to: process N data points using a pre-trained language model to obtain a set of medical record word vectors; The second processing module is used to: process the acquired standard words in the medical field through a pre-trained language model to obtain a set of standard word vectors; The segmentation module is used to: based on the medical record word vector set and the standard word vector set, and through similarity calculation results, to annotate entity information in the original medical record text data.

[0019] Based on the above solution, the present invention can be further improved as follows.

[0020] Furthermore, the process of segmenting the original medical record text data to obtain N data points is as follows: The original medical record text data is divided into N-grams using a sliding window to obtain N data points.

[0021] Furthermore, pre-trained language models include: Embedding layer, multi-head attention mechanism layer, and forward computation layer; The Embedding layer is used to perform matrix multiplication using the weight matrix and the ID mapped from the input data to obtain the embedding word vectors; The multi-head attention mechanism layer is used to determine the vector representation of each input data that incorporates attention information by matrix multiplication based on the embedding word vectors; The forward computation layer is used to generate output data after activation by two linear layers and an activation layer.

[0022] Furthermore, the process of annotating entity information in the original medical record text data based on similarity calculation results is as follows: For each vector in the medical record word vector set, calculate the cosine similarity with all vectors in the standard word vector set; The cosine similarity calculation result is compared with the preset similarity threshold one by one. When the cosine similarity calculation result is greater than the preset similarity threshold, the vector in the medical record word vector set corresponding to the cosine similarity calculation result is labeled to obtain the entity information in the original medical record text data.

[0023] 3) In a third aspect, the present invention also provides an electronic device, the electronic device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the electronic device to perform any of the methods described above.

[0024] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to enable a computer to implement any of the above methods.

[0025] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0026] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating a data segmentation method based on a pre-trained language model according to an embodiment of the present invention. Figure 2 This is a structural framework diagram of an electronic device according to the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0028] like Figure 1 As shown in the figure, a data segmentation method based on a pre-trained language model according to an embodiment of the present invention includes the following steps: S1, obtain the original medical record text data, and segment the original medical record text data to obtain N data; S2, the medical record word vector set is obtained by processing N data through a pre-trained language model; S3, the standard word vector set is obtained by processing the standard words in the medical field through a pre-trained language model; S4, based on the medical record word vector set and the standard word vector set, uses the similarity calculation results to annotate entity information in the original medical record text data.

[0029] The beneficial effects of the data segmentation method based on a pre-trained language model provided by this invention are as follows: Leveraging the powerful semantic understanding capabilities of pre-trained language models, this method accurately captures the deep semantic features of medical record texts and standard terms, thereby improving the accuracy and reliability of entity information annotation. Through N-gram segmentation and similarity calculation, it achieves refined segmentation and key information extraction of medical record texts, effectively identifying key information such as disease entities and symptoms, thus improving the efficiency and quality of medical record data processing. Furthermore, this method does not rely on manually defined rules, exhibiting high flexibility and scalability. It can adapt to medical record texts of different formats and contents, and the segmentation effect can be optimized by adjusting parameters such as the similarity threshold. This provides an efficient, flexible, and accurate solution for the processing and analysis of medical record data, contributing to the development of medical informatization and the effective utilization of medical data.

[0030] It should be noted that raw medical record text data refers to unprocessed patient medical record information obtained directly from the medical record system. This information usually exists in the form of natural language and includes detailed content such as the patient's condition description, diagnosis results, treatment process, and examination reports.

[0031] In another embodiment of this solution, the specific implementation process of S1 is as follows: The original medical record text data of patients is extracted from the Health Information System (HIS) or other electronic medical record storage systems. This data may include the chief complaint, present illness, past medical history, physical examination, auxiliary examinations, preliminary diagnosis, and treatment plan. Then, the extracted original medical record text data is preprocessed, including removing irrelevant characters (such as punctuation marks and spaces), standardizing the text format (such as converting all text to lowercase), and word segmentation, dividing continuous text into individual words or phrases. Next, the sliding window method is used to perform N-gram partitioning on the preprocessed medical record text. Specifically, a window of size n is selected, and starting from the beginning of the text, consecutive text segments of length n are selected sequentially to form an N-gram combination. The window is then slid forward one character or word, and the above process is repeated until the entire medical record text is covered. In this way, multiple text segments of different lengths can be generated, resulting in N data points. For example, for the medical record text "blood cell production, normal function", when n=2, the sliding window method can yield Bi-gram combinations such as "blood cells", "cell production", and "normal function"; when n=3, it can yield Tri-gram combinations such as "blood cell production", "cell production function", and "production function normal". These N generated data will serve as input for subsequent processing, used for further vectorization and similarity calculation, thereby achieving the extraction and segmentation of key information in the medical record text.

[0032] In another embodiment of this solution, the specific process of S2 is as follows: Choose a suitable pre-trained language model for medical text processing, such as BERT. This model typically consists of three main layers: an embedding layer, a multi-head attention mechanism layer, and a forward computation layer. For each data segment obtained through N-gram partitioning, it is input into the pre-trained language model. In the embedding layer, using the word embedding weight matrix already learned in the pre-trained model, each word or character in the input text segment is mapped to a fixed-dimensional (e.g., 768-dimensional) word vector. These word vectors are combined to form the embedding matrix representation of the input data. Subsequently, the multi-head attention mechanism layer is passed to the forward computation layer. This layer uses multiple attention heads to perform a weighted summation of the word vectors output from the embedding layer, capturing the dependencies between words at different positions in the text segment and generating word vector representations that incorporate attention information. Each attention head focuses on a different part of the text, thus enabling an understanding of the semantic information of the text from multiple perspectives. The word vectors processed by the multi-head attention mechanism layer are then passed to the forward computation layer, which contains two linear transformations and an activation function (e.g., ReLU), further performing non-linear transformations on the word vectors and outputting a 768-dimensional vector representation for each data segment. The vectors obtained after processing all N data segments as described above are collected to form a medical record word vector set. Each vector in this set contains deep semantic information of the corresponding data segment, providing a foundation for subsequent similarity calculation and entity information annotation.

[0033] In another embodiment of this solution, the specific process of S3 is as follows: A standard vocabulary database is compiled, containing commonly used medical terms with clear medical significance. This database may include disease names, symptom descriptions, examination items, and drug names. These standard terms are then fed into a pre-trained language model, with a structure consistent with that used for processing medical record text data. The model includes an embedding layer, a multi-head attention mechanism layer, and a forward computation layer. In the embedding layer, the word embedding weight matrix learned in the pre-trained model maps each word or character in the standard term to a fixed-dimensional (e.g., 768-dimensional) word vector. These word vectors are combined to form the embedding matrix representation of the standard term. Next, the multi-head attention mechanism layer performs a weighted summation of the word vectors output from the embedding layer using multiple attention heads. This captures the dependencies between words at different positions within the standard term, generating word vector representations that incorporate attention information. Each attention head focuses on a different part of the standard term, thus enabling a multi-faceted understanding of its semantic information. The word vectors, after being processed by the multi-head attention mechanism layer, are then passed to the forward computation layer. This layer contains two linear transformations and activation functions (such as ReLU) to further perform non-linear transformations on the word vectors, ultimately outputting a 768-dimensional vector representation of each standard word. All the vectors obtained from the above processing of the standard words are collected to form a standard word vector set. Each vector in this set contains deep semantic information of the corresponding standard word, providing an accurate benchmark for subsequent similarity calculations with the medical record word vector set.

[0034] In another embodiment of this solution, the specific process of S4 is as follows: Each medical record word vector is extracted from the medical record word vector set. These vectors are obtained by processing medical record text data using a pre-trained language model, and each vector represents the semantic information of a specific segment of the medical record text. Next, cosine similarity is calculated between each medical record word vector and all standard word vectors in the standard word vector set. The cosine similarity formula is the dot product of the two vectors divided by their modulus product, which quantifies the semantic similarity between the medical record word vector and the standard word vectors. After calculation, a similarity score list is obtained, where each score corresponds to a standard word. Then, each similarity score is checked against a preset similarity threshold, a value between 0 and 1, used to control the strictness of the annotation. If the similarity score between a medical record word vector and a standard word vector is greater than or equal to the preset threshold, the medical record text segment represented by the medical record word vector is considered semantically matched with the standard word, and it can be annotated as entity information corresponding to the standard word, such as disease name, symptom description, etc. All medical record text fragments that meet the criteria are labeled, and the labeling results are mapped back to the corresponding positions in the original medical record text data. This completes the labeling of entity information in the original medical record text data, enabling key medical information in the medical record text to be clearly identified and marked, providing accurate structured data support for subsequent medical data analysis and processing.

[0035] It should be noted that the annotation of key medical information in this solution provides clinicians with more precise decision support, helping them quickly locate important information in medical records, such as disease names, symptoms, and examination results, thereby improving diagnostic efficiency and accuracy. The annotated structured data facilitates statistical analysis and mining of medical data, supporting medical research institutions in conducting large-scale epidemiological studies, disease trend analysis, and treatment efficacy evaluation. Furthermore, for medical information systems, the annotated data enables better retrieval and sharing of medical record information, facilitating remote medical collaboration between different medical institutions and rapid transmission of patient information. Simultaneously, it provides high-quality annotated data for training medical machine learning models, contributing to the development of more intelligent medical auxiliary diagnostic systems and disease prediction models.

[0036] Furthermore, the process of segmenting the original medical record text data to obtain N data points is as follows: The original medical record text data is divided into N-grams using a sliding window to obtain N data points.

[0037] Furthermore, pre-trained language models include: Embedding layer, multi-head attention mechanism layer, and forward computation layer; The Embedding layer is used to perform matrix multiplication using the weight matrix and the ID mapped from the input data to obtain the embedding word vectors; The multi-head attention mechanism layer is used to determine the vector representation of each input data that incorporates attention information by matrix multiplication based on the embedding word vectors; The forward computation layer is used to generate output data after activation by two linear layers and an activation layer.

[0038] Furthermore, the process of annotating entity information in the original medical record text data based on similarity calculation results is as follows: For each vector in the medical record word vector set, calculate the cosine similarity with all vectors in the standard word vector set; The cosine similarity calculation result is compared with the preset similarity threshold one by one. When the cosine similarity calculation result is greater than the preset similarity threshold, the vector in the medical record word vector set corresponding to the cosine similarity calculation result is labeled to obtain the entity information in the original medical record text data.

[0039] Example 1: For the original text data of medical records, it is necessary to extract the key medical information. With the development of pre-trained models, a medical record data segmentation method based on pre-trained models and text similarity is proposed.

[0040] The method consists of three steps: first step: Text generation is performed using a BERT pre-trained language model, and the model is constructed as follows: The L1 and embedding layers perform matrix symmetry by mapping the embedding weight matrix and the IDs mapped to the input data, resulting in embedding word vectors as the embedding matrix representation of the input data, with a vector dimension of 768. L2, the multi-head attention mechanism layer, extracts matrix features from the 768-dimensional feature vector output by the embedding layer through three linear layers, and calculates the 768-dimensional vector representation of each input data that incorporates attention information through matrix multiplication; L3, the forward computation layer, takes each input from the multi-head attention mechanism layer and passes it through two linear layers, and after activation by the activation layer, outputs the final 768-dimensional vector representation of each data point. The original medical record text data is divided and combined according to 1 to n grams. The N-gram model is a language model (LM). A language model is a probability-based discriminative model. Its input is a sentence (the sequential sequence of words), and its output is the probability of the sentence, that is, the joint probability of these words.

[0041] An N-gram is a set of N words in a specific order, where the words are not required to be distinct. Commonly used N-grams (N=2) and Tri-grams (N=3) are generally sufficient. For example, the sentence "I love deep learning" can be broken down into N-grams and Tri-grams: Bi-gram: {I, love}, {love, deep}, {love, deep}, {deep, learning} Tri-gram: {I, love, deep}, {love, deep, learning}. For example, the original medical record text data AT is "blood cell production, normal function". By using the sliding window method, all adjacent text combinations in medical record A are obtained according to 1~n gram: ['blood', 'blood cells', 'blood cell production'... 'cells', 'cells', 'cell production', 'cell generation'...]. In this way, (n-1)n / 2 data can be generated for sentence AT. These data are then processed by the BERT model to obtain 768-dimensional vector representations. These vectors are stored as a medical record word vector set H. Step Two: All the standard terms in the medical field prepared in advance are processed by the BERT model. The structure of the BERT model is the same as above. The 768-dimensional vector representations of these standard terms are obtained and stored as a set of standard term vectors E. Step 3: For the already constructed medical record word vector set H and standard word vector set E, assuming that the medical record word vector set H contains a 768-dimensional vector representation of the medical record word A, performing a matrix multiplication operation between the 768-dimensional vector of A and the vector matrix in the standard word vector set E yields the cosine similarity between the current medical record word question A and all the standard words stored in the question-answering database E. The formula is: Where A is a 768-dimensional vector representation of word A in the medical record word vector set H, and B is a vector representation in the standard word vector set E.

[0042] The similarity scores between word A and all words in the standard vocabulary were calculated and ranged from 0 to 1. An adjustable similarity threshold of 0.8 was set. Words above the threshold can be regarded as having the same meaning as the standard words and are words of the same disease. They are identified as disease entities in the original medical records. In this way, the segmentation operation of the medical record text is achieved.

[0043] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0044] It should be noted that the beneficial effects of the data segmentation system based on a pre-trained language model provided in the above embodiments are the same as those of the data segmentation method based on a pre-trained language model described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0045] like Figure 2 As shown, an electronic device 300 according to an embodiment of the present invention includes a processor 320 coupled to a memory 310. The memory 310 stores at least one computer program 330, which is loaded and executed by the processor 320 to enable the electronic device 300 to implement any of the above-mentioned methods. Specifically: The electronic device 300 can vary considerably due to differences in configuration or performance. It may include one or more processors 320 (Central Processing Units, CPUs) and one or more memories 310. The one or more memories 310 store at least one computer program 330, which is loaded and executed by the one or more processors 320 to enable the electronic device 300 to implement the data segmentation method based on a pre-trained language model provided in the above embodiments. Of course, the electronic device 300 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The electronic device 300 may also include other components for implementing device functions, which will not be elaborated upon here.

[0046] An embodiment of the present invention provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to enable a computer to implement any of the above-described methods.

[0047] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0048] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform any of the methods described above.

[0049] It should be noted that the terms "first" and "second" in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0050] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0051] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0052] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A data segmentation method based on a pre-trained language model, characterized in that, The method comprises the following steps: Obtaining original medical record text data, and segmenting the original medical record text data to obtain N data; Processing the N data through a pre-trained language model to obtain a medical record word vector set; Processing the obtained medical field standard words through the pre-trained language model to obtain a standard word vector set; Based on the medical record word vector set and the standard word vector set, entity information in the original medical record text data is labeled through a similarity calculation result.

2. The data segmentation method based on a pre-trained language model according to claim 1, characterized in that, The process of segmenting the original medical record text data to obtain N data is specifically as follows: The original medical record text data is divided by N-gram through a sliding window to obtain N data.

3. The data segmentation method based on a pre-trained language model according to claim 1, characterized in that, The pre-trained language model comprises: An Embedding layer, a multi-head attention mechanism layer, and a forward calculation layer; The Embedding layer is used to perform matrix multiplication through a weight matrix and an ID mapped by input data to obtain an embedding word vector; The multi-head attention mechanism layer is used to determine a vector representation of each input data fused with attention information based on the embedding word vector through matrix multiplication; The forward calculation layer is used to generate output data after activation through two linear layers and an activation layer.

4. The data segmentation method based on a pre-trained language model according to claim 1, characterized in that, The process of labeling entity information in the original medical record text data through a similarity calculation result is specifically as follows: The cosine similarity of each vector in the medical record word vector set with all vectors in the standard word vector set is calculated; The size between the cosine similarity calculation result and a preset similarity threshold is judged one by one, and the vector in the medical record word vector set corresponding to the cosine similarity calculation result greater than the preset similarity threshold is labeled to obtain the entity information in the original medical record text data. 5.A data segmentation system based on a pre-trained language model, characterized in that, The method comprises the following steps: An obtaining module is configured to obtain original medical record text data, and segment the original medical record text data to obtain N data; A first processing module is configured to process the N data through a pre-trained language model to obtain a medical record word vector set; A second processing module is configured to process the obtained medical field standard words through the pre-trained language model to obtain a standard word vector set; A segmentation module is configured to label entity information in the original medical record text data based on the medical record word vector set and the standard word vector set through a similarity calculation result.

6. The data segmentation system based on a pre-trained language model according to claim 5, wherein, The process of segmenting the original medical record text data to obtain N data is specifically as follows: The original medical record text data is divided by N-gram through a sliding window to obtain N data.

7. The data segmentation system based on a pre-trained language model according to claim 5, wherein, The pre-trained language model comprises: An Embedding layer, a multi-head attention mechanism layer, and a forward calculation layer; The Embedding layer is used to perform matrix multiplication through a weight matrix and an ID mapped by input data to obtain an embedding word vector; The multi-head attention mechanism layer is used to determine a vector representation of each input data fused with attention information based on the embedding word vector through matrix multiplication; The forward calculation layer is used to generate output data after activation through two linear layers and an activation layer. The forward calculation layer is used to generate output data after two linear layers and an activation layer are activated.

8. The data segmentation system based on a pre-trained language model according to claim 5, wherein, The process of labeling the entity information in the original medical record text data through the similarity calculation result is specifically as follows: The cosine similarity between each vector in the medical record vector set and all vectors in the standard vector set is calculated. The size between the cosine similarity calculation result and the preset similarity threshold is judged one by one, and the vector in the medical record vector set corresponding to the cosine similarity calculation result greater than the preset similarity threshold is labeled to obtain the entity information in the original medical record text data.

9. An electronic device, comprising: The electronic device includes a processor coupled with a memory, and the memory stores at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the method of any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one computer program, which is loaded and executed by the processor to enable the computer to implement the method of any one of claims 1 to 4.

Images