Data labeling method and device, computer equipment and storage medium
By combining self-supervised learning technology and label generation methods, the challenges in accuracy and consistency of existing automated medical data annotation methods are solved, and efficient and accurate medical data annotation is achieved, reducing costs and improving labeling efficiency.
Patent Information
- Application Number
- CN202411364762.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2025-02-18
AI Technical Summary
The existing automated medical data annotation methods have challenges in the accuracy and consistency of label generation, resulting in unsatisfactory medical data annotation.
By combining self-supervised learning technology and label generation method, data representation information of medical data is obtained, and target data is output through pre-training neural network models to generate label information, realizing automatic labeling of medical data.
It reduces the annotation cost, improves the annotation efficiency and accuracy, and the generated labels have high accuracy and semantic consistency, and are suitable for medical data of different types and fields.
Smart Images
Figure CN119475010A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine learning technology, applied in the medical field, and in particular to a data labeling method, device, computer equipment and storage medium. Background Art
[0002] In the medical field, processing medical data (such as classification / segmentation) through large models can quickly complete the classification and organization of massive medical data. Data labeling is one of the key steps in training and applying machine learning models. However, traditional medical data labeling methods based on manual labeling require a lot of time and manpower costs. In addition, due to the complexity and diversity of medical data, manual labeling may also have problems of subjectivity and inconsistency, which will lead to unsatisfactory labeling results of the final medical data.
[0003] Currently, automated medical data annotation methods have attracted much attention. However, existing automatic annotation methods still face challenges in terms of accuracy and consistency of label generation.
[0004] Therefore, an innovative method is urgently needed to solve the problem of poor consistency in medical data annotation, reduce annotation costs, and improve annotation efficiency and accuracy. Summary of the invention
[0005] The present application provides a data annotation method, apparatus, computer device and storage medium, which aims to solve the current problem that automated medical data annotation methods have attracted much attention. However, existing automatic annotation methods are often limited to specific tasks and data types, and there are still challenges in the accuracy and consistency of label generation. The method provided combines self-supervised learning technology and label generation methods to achieve automatic annotation of medical data, thereby reducing annotation costs and improving annotation efficiency and accuracy.
[0006] In a first aspect, the present application provides a data annotation method, comprising:
[0007] Acquire a plurality of first medical data to be annotated; wherein the first medical data includes any one of medical text and medical image;
[0008] Parsing each of the first medical data based on a self-supervised learning method to obtain data representation information corresponding to each of the first medical data; wherein the data representation information includes any one of semantic information corresponding to the medical text and image features corresponding to the medical image;
[0009] Parse each of the data representation information and output the target data corresponding to each of the data representation information; wherein the target data corresponding to the medical text includes one or more of context prediction information, text filling information, disease / symptom category and entity information, and the target data corresponding to the medical image includes one or more of region segmentation information, location information, category information and key point information;
[0010] Generate label information corresponding to the first medical data according to each of the target data, and complete the labeling of the plurality of the first medical data.
[0011] In some embodiments, after completing the labeling of multiple first medical data, it also includes: generating training samples based on each of the first medical data and the corresponding label information; completing the training of the data classification model to be trained based on the training samples; wherein the data classification model includes any one of a text classification model and an image segmentation model; inputting the second medical data to be classified into the trained data classification model, the data classification model parses the second medical data, and outputs the classification result corresponding to the second medical data.
[0012] In some embodiments, parsing each of the data representation information and outputting target data corresponding to each of the data representation information includes: inputting each of the data representation information into a pre-trained first neural network model, the first neural network model parsing the data representation information, and outputting target data corresponding to each of the first medical data.
[0013] Exemplarily, the first neural network model includes one or more of a context prediction model, a category prediction model and an entity recognition model; if the data representation information is semantic information corresponding to the medical text, each of the data representation information is input into a pre-trained first neural network model, the first neural network model parses the data representation information, and outputs the target data corresponding to each of the first medical data, including: inputting the medical text and the semantic information into the context prediction model, the context prediction model parses the medical text and the semantic information, and outputs the context information and / or the text filling information corresponding to the medical text, so as to The label information is generated by using text information and / or text filling information; and / or, the medical text and semantic information are input into the category prediction model, the category prediction model parses the medical text and semantic information, and outputs the disease / symptom category corresponding to the medical text, so as to generate the label information according to the disease / symptom category; and / or, the medical text and semantic information are input into the entity recognition model, the entity recognition model parses the medical text and semantic information, and outputs the entity information corresponding to the medical text, so as to generate the label information according to the entity information; wherein the entity information includes at least one or more of the disease name and the drug name.
[0014] Exemplarily, the first neural network model includes one or more of an image segmentation model, an object detection model and a key point detection model; if the data representation information includes image features corresponding to the medical image, each of the data representation information is input into a pre-trained first neural network model, the first neural network model parses the data representation information, and outputs target data corresponding to each of the first medical data, including: inputting the medical image and image features into the image segmentation model, the image segmentation model parses the medical image and image features, and outputs region segmentation information corresponding to the medical image, so as to generate the label information according to the region segmentation information; and / or, inputting the medical image and image features into the object detection model, the object detection model parses the medical image and image features, and outputs position information and category information corresponding to the medical image, so as to generate the label information according to the position information and category information; and / or, inputting the medical image and image features into the key point detection model, the key point detection model parses the medical image and image features, and outputs key point information corresponding to the medical image, so as to generate the label information according to the key point information.
[0015] In some embodiments, parsing each of the first medical data and obtaining data representation information corresponding to each of the first medical data includes: inputting each of the first medical data into a second neural network model pre-trained based on self-supervised learning, the second neural network model parsing each of the first medical data according to pre-trained parameters, and outputting data representation information corresponding to each of the first medical data.
[0016] In some embodiments, before parsing each of the first medical data, it also includes: preprocessing the first medical data; the preprocessing includes at least data cleaning and normalization.
[0017] In a second aspect, the present application provides a data labeling device, comprising:
[0018] A data acquisition unit, configured to acquire a plurality of first medical data to be annotated; wherein the first medical data includes any one of a medical text and a medical image;
[0019] A data parsing unit, configured to parse each of the first medical data based on a self-supervised learning method, and obtain data representation information corresponding to each of the first medical data; wherein the data representation information includes any one of semantic information corresponding to the medical text and image features corresponding to the medical image;
[0020] A target generation unit, configured to parse each of the data representation information and output target data corresponding to each of the data representation information; wherein the target data corresponding to the medical text includes one or more of context prediction information, text filling information, disease / symptom category and entity information, and the target data corresponding to the medical image includes one or more of region segmentation information, location information, category information and key point information;
[0021] The labeling completion unit is used to generate label information corresponding to the first medical data according to each of the target data, and complete the labeling of multiple first medical data.
[0022] In a third aspect, the present application further provides a computer device, including:
[0023] Memory and processor;
[0024] The memory is used to store computer programs;
[0025] The processor is used to execute the computer program and implement the steps of the data labeling method described in the first aspect above when executing the computer program.
[0026] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the steps of the data labeling method described in the first aspect above.
[0027] The present application discloses a data annotation method, device, computer equipment and storage medium. First, multiple first medical data to be annotated are obtained; wherein the first medical data includes any one of medical text and medical image. Then, each first medical data is parsed based on a self-supervised learning method to obtain data representation information corresponding to each first medical data; wherein the data representation information includes any one of semantic information corresponding to the medical text and image features corresponding to the medical image. Then, each data representation information is parsed, and the target data corresponding to each data representation information is output; wherein the target data corresponding to the medical text includes one or more of context prediction information, text filling information, disease / symptom category and entity information, and the target data corresponding to the medical image includes one or more of region segmentation information, location information, category information and key point information. Finally, the label information corresponding to the corresponding first medical data is generated according to each target data, and the labeling of multiple first medical data is completed. The provided method realizes automatic annotation of medical data by combining self-supervised learning technology to obtain corresponding data representation information, target data and label generation method, thereby reducing annotation cost and improving annotation efficiency and accuracy.
[0028] At the same time, the provided method also has the following beneficial effects:
[0029] 1. Reduce costs: Compared with traditional manual annotation, the present invention greatly reduces the cost of annotation by using an automated medical data annotation method. Large-scale medical data annotation can be completed without a large amount of human resources and time investment, thus saving costs and resources.
[0030] 2. Improve efficiency: The present invention realizes the automatic labeling of medical data, making the labeling process more efficient. Using large neural network models for self-supervised learning, rich representations of data can be quickly learned, and innovative label generation methods can effectively generate high-quality labels, thereby improving the speed and productivity of labeling.
[0031] 3. High accuracy: Using a pre-trained large neural network model and an innovative label generation method, the labels generated by the present invention have high accuracy and semantic consistency. Compared with traditional manual labeling, the automatically generated labels are more objective and consistent, and can better reflect the content and characteristics of the original data.
[0032] 4. Strong versatility: The automated medical data annotation method proposed in the present invention has strong versatility. Whether processing text data or image data, whether for tasks such as classification, segmentation or detection, the label generation method can be adjusted and optimized to achieve automatic annotation of medical data of different types and fields, with wide applicability and flexibility.
[0033] 5. Improve the performance of medical models trained with labeled data: Automated medical data labeling can not only reduce costs and improve efficiency, but also improve the performance of medical models. The generated high-quality labels provide reliable data support for model training, improve the generalization ability and performance of the model, and thus promote the further application and development of medical models.
[0034] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0036] Figure 1 This is a schematic flowchart of the steps of a first data labeling method provided by an embodiment of the present application;
[0037] Figure 2 is a schematic flow chart of the steps of a second data labeling method provided in an embodiment of the present application;
[0038] Figure 3 is a schematic flow chart of the steps of a classification result output method provided in an embodiment of the present application;
[0039] Figure 4 It is a structural schematic diagram of a data labeling device provided in one embodiment of the present application;
[0040] Figure 5 It is a schematic block diagram of the structure of a computer device provided in one embodiment of the present application.
[0041] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. DETAILED DESCRIPTION
[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0043] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.
[0044] It should be understood that, in order to facilitate the clear description of the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, the words "first", "second", etc. are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the difference.
[0045] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0046] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0047] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0048] In data annotation tasks in the financial field, such as summarizing long documents such as insurance contracts and financial reports, it can help users and related practitioners quickly understand the overall content of the text and save them from reading too much text.
[0049] In the medical field, processing medical data (such as classification / segmentation) through large models can quickly complete the classification and organization of massive medical data. Data labeling is one of the key steps in training and applying machine learning models. However, traditional medical data labeling methods based on manual labeling require a lot of time and manpower costs. In addition, due to the complexity and diversity of medical data, manual labeling may also have problems of subjectivity and inconsistency, which will lead to less than ideal labeling results for the final medical data.
[0050] Currently, automated medical data annotation methods have attracted much attention. However, existing automatic annotation methods are often limited to specific tasks and data types, and there are still challenges in the accuracy and consistency of label generation.
[0051] Therefore, an innovative method is urgently needed to solve the problem of poor consistency in medical data annotation, reduce annotation costs, and improve annotation efficiency and accuracy.
[0052] To solve the above problems, please refer to Figure 1 , Figure 1 This is a schematic flow chart of a data labeling method provided by an embodiment of the present application. The data labeling method can be implemented by a computer device, which can be deployed on a single server or a server cluster. It can also be deployed on a handheld terminal, a laptop computer, a wearable device or a robot, etc.
[0053] It should be noted that the provided method can be applied not only to the medical field for automatic annotation of medical data (medical text and medical images), but also to other systems with a large amount of text and image data that need to be annotated (such as finance, education, etc.) This method can be used to automatically annotate massive data and use the annotated data to train the corresponding model. Therefore, the embodiments of this application do not limit the field of the provided method.
[0054] To solve the above problems, please refer to Figure 1 Specifically, Figure 1 As shown, the provided data labeling method includes steps S101 to S104. The details are as follows:
[0055] Step S101. Acquire a plurality of first medical data to be annotated; wherein the first medical data includes any one of medical texts and medical images.
[0056] Specifically, a large amount of medical data is extracted from the database or data source corresponding to the medical system to which the provided method is applied in accordance with relevant regulations. These data may include medical texts (such as medical records, diagnostic reports, medical papers, etc.) and medical images (such as X-rays, MRI, CT scans, etc.). At the same time, it is necessary to ensure that the acquired data is unlabeled and has a unified and complete format. This provides a data basis for subsequent analysis and labeling. At the same time, the diversity and coverage of the data are ensured to increase the generalization ability of the model finally trained based on the labeled first medical data.
[0057] Step S102. parse each first medical data based on a self-supervised learning method to obtain data representation information corresponding to each first medical data; wherein the data representation information includes any one of semantic information corresponding to the medical text and image features corresponding to the medical image.
[0058] Specifically, by converting raw data into a format that can be understood and processed by computers, parsing based on self-supervised learning methods can improve the accuracy of target data extraction in subsequent steps. For medical texts, texts can be converted into semantic information by using natural language processing (NLP) technologies such as word vectors (Word2Vec) and BERT. Key information such as keywords and sentence vectors are extracted. For medical images, computer vision technologies such as convolutional neural networks (CNN) are used to extract image features such as edge information, texture information, shape features, etc.
[0059] Step S103. Parse each data representation information and output the target data corresponding to each data representation information; wherein the target data corresponding to the medical text includes one or more of context prediction information, text filling information, disease / symptom category and entity information, and the target data corresponding to the medical image includes one or more of region segmentation information, location information, category information and key point information.
[0060] Specifically, by extracting useful structured information from the data representation information as target data, the foundation for data annotation is laid, and the accuracy and comprehensiveness of the annotated data can be enhanced, and the understanding and discrimination ability of the model can be improved. For medical text, the relationship between the text before and after is parsed by using context prediction. At the same time, the missing information is supplemented by filling technology to improve the completeness and accuracy of the text. The text is also classified into diseases / symptoms and entity information is extracted, such as drug names, disease names, symptoms, etc. For medical images, the image is divided into different areas through image segmentation technology, and the interest areas (such as lesions) are annotated. And the location information (such as coordinates, area) of each segmented area is extracted. It also identifies the category information in the image (such as tumor type), and then combines it with key point detection (such as human skeleton key points). In summary, the target data extraction of medical text and medical images can ensure the comprehensiveness of the data and the accuracy of subsequent label generation.
[0061] In some embodiments, parsing each data representation information and outputting target data corresponding to each data representation information includes: inputting each data representation information into a pre-trained first neural network model, the first neural network model parsing the data representation information, and outputting target data corresponding to each first medical data.
[0062] By inputting each data representation information (such as the high-dimensional feature vector output by the second neural network model pre-trained based on the self-supervised learning method) into the pre-trained first neural network model (such as the GPT model). The first neural network model parses these high-dimensional feature vectors according to the pre-trained parameters, and outputs the target data (such as label information) corresponding to each first medical data. Then, by efficiently and accurately parsing the data representation information based on the self-supervised learning method, the complex feature vectors are converted into useful target data (such as specific classification labels and attributes). The generalization ability and robustness of the model are improved, and the data representation information is parsed by the pre-trained first neural network model to obtain high-quality target data. This process not only enhances the model's ability to parse complex medical data, but also greatly improves the efficiency and accuracy of data processing. Whether it is text parsing or image parsing, it can provide strong support for clinical decision-making and medical research.
[0063] Exemplarily, the first neural network model includes one or more of a context prediction model, a category prediction model and an entity recognition model; if the data representation information is semantic information corresponding to a medical text, each data representation information is input into a pre-trained first neural network model, the first neural network model parses the data representation information, and outputs target data corresponding to each first medical data, including: inputting the medical text and the semantic information into the context prediction model, the context prediction model parses the medical text and the semantic information, and outputs the context information and / or text filling information corresponding to the medical text, so as to generate label information according to the context information and / or text filling information; and / or, inputting the medical text and the semantic information into the category prediction model, the category prediction model parses the medical text and the semantic information, and outputs the disease / symptom category corresponding to the medical text, so as to generate label information according to the disease / symptom category; and / or, inputting the medical text and the semantic information into the entity recognition model, the entity recognition model parses the medical text and the semantic information, and outputs the entity information corresponding to the medical text, so as to generate label information according to the entity information; wherein the entity information includes at least one or more of the disease name and the drug name.
[0064] By inputting medical text and semantic information into the context prediction model, the context information and / or text filling information corresponding to the medical text can be output, and label information (e.g., predicting missing words, the next sentence of the text, etc.) can be generated based on the context information and / or text filling information.
[0065] By inputting medical text and semantic information into the category prediction model, the disease / symptom category corresponding to the medical text can be output, and label information (such as diagnosis category) can be generated based on the predicted disease / symptom category.
[0066] The entity recognition model inputs: medical text and semantic information, and can output the entity information corresponding to the medical text (such as disease name, drug name). And generate label information (such as marking disease name, drug name) based on the recognized entity information.
[0067] The context prediction model can enhance the ability to understand the context, and help analyze the integrity of the text and fill in the missing information. The category prediction model can realize the automatic classification of diseases and symptoms, which is helpful for rapid diagnosis and clinical decision-making. The entity recognition model can also accurately identify key entities in the text, which is convenient for information extraction, case summary and the construction of medical knowledge base.
[0068] Exemplarily, the first neural network model includes one or more of an image segmentation model, an object detection model and a key point detection model; if the data representation information includes image features corresponding to the medical image, each data representation information is input into a pre-trained first neural network model, the first neural network model parses the data representation information, and outputs target data corresponding to each first medical data, including: inputting the medical image and image features into the image segmentation model, the image segmentation model parses the medical image and image features, and outputs region segmentation information corresponding to the medical image, so as to generate label information based on the region segmentation information; and / or, inputting the medical image and image features into the object detection model, the object detection model parses the medical image and image features, and outputs location information and category information corresponding to the medical image, so as to generate label information based on the location information and category information; and / or, inputting the medical image and image features into the key point detection model, the key point detection model parses the medical image and image features, and outputs key point information corresponding to the medical image, so as to generate label information based on the key point information.
[0069] By inputting medical images and image features into the image segmentation model, the region segmentation information corresponding to the medical image can be output, and label information (such as segmenting different anatomical structures or lesion areas) can be generated based on the region segmentation information.
[0070] By inputting medical images and image features into the object detection model, it can output: location information and category information corresponding to the medical image. And generate label information based on the location information and category information (such as locating the tumor location and classifying its type).
[0071] By inputting medical images and image features into the key point detection model, the key point information corresponding to the medical image can be output, and label information (such as identifying key points at specific anatomical locations) can be generated based on the key point information.
[0072] The image segmentation model can then segment images with high precision, making it easier to identify and label different regions of medical images and improve the quality of clinical decision-making. The object detection model can achieve automatic positioning and classification, improve the level of automated analysis of medical images, and reduce doctors' diagnosis time. The key point detection model can accurately identify important anatomical locations, which is helpful for further detailed analysis and diagnosis.
[0073] Step S104: Generate label information corresponding to the corresponding first medical data according to each target data, and complete the labeling of multiple first medical data.
[0074] Specifically, by generating corresponding label information for the first medical data corresponding to each medical text and medical image, these label information can be a direct representation or conversion form of the target data generated in the previous step. Automated tools and platforms (such as Label Img, Doccano) can also be used for annotation to accelerate the annotation process. This step can then generate structured, standardized annotation data, provide high-quality data sets for model training, improve annotation efficiency and consistency, and reduce human errors. Provide a basis for further model training, data mining and analysis.
[0075] At the same time, the provided method can efficiently convert large amounts of unlabeled medical data into labeled data, improve data quality and analysis accuracy, and ultimately help build more accurate and efficient medical models and applications.
[0076] In some embodiments, please refer to Figure 2 Before parsing each first medical data, step S105 is also included.
[0077] Step S105: Preprocess the first medical data; the preprocessing at least includes data cleaning and normalization.
[0078] Step S105 removes noise and irrelevant information through data cleaning, making the input data purer and more effective, thereby improving the training quality and prediction accuracy of the model. At the same time, it handles missing values and spelling errors, increases the integrity and consistency of the data, and avoids data bias. Data cleaning removes errors and noise, enhances the model's ability to handle abnormal and sudden situations, and improves the robustness and reliability of the model.
[0079] Step S105 also converts text and image data into a simplified format (such as vector, normalized image) by normalization, which can significantly reduce the computational complexity. The normalization operation ensures the uniform format and scale of the input data, reducing the sensitivity of the model to data of different scales. It also improves the convergence speed and stability of the model, because the unified data range can make the optimization process smoother and more efficient. At the same time, the normalized data is more suitable for efficient matrix operations, which improves the training and reasoning speed.
[0080] By,preprocessing steps ensuring that the data is optimized before parsing,,high-quality target data can be generated more efficiently, making the,generated labels more stable and reliable.
[0081] In summary, step S105 ensures high-quality, standardized input medical data through data cleaning and normalization, significantly improving the data processing and analysis capabilities of subsequent steps. This can not only improve the accuracy and stability of the model, but also simplify calculations and improve the efficiency and robustness of the overall system.
[0082] In some embodiments, parsing each first medical data and obtaining data representation information corresponding to each first medical data includes: inputting each first medical data into a second neural network model pre-trained based on self-supervised learning, the second neural network model parsing each first medical data according to pre-trained parameters, and outputting data representation information corresponding to each first medical data.
[0083] Each first medical data is parsed by using a pre-trained neural network model. The second neural network model based on self-supervised learning refers to a model that has been pre-trained on a large-scale data set (which may be the same as or different from the above-mentioned first neural network model) and is capable of extracting high-quality features. Each first medical data (whether it is a medical text or a medical image) is input into the second neural network model. The model can be of different types, such as BERT (input medical text) or ResNet (input medical image) or GPT (universal for text and images), depending on the type of medical data. The second neural network model automatically parses the input medical data according to its pre-trained parameters. The output data representation information will contain high-dimensional feature vectors that represent the semantic information of the medical text or the image features of the medical image.
[0084] Furthermore, by using pre-trained neural network models, complex data features can be extracted quickly and efficiently (such as pre-trained large-scale neural network models for self-supervised learning on medical data) without having to train the model from scratch. Pre-trained models have been trained on large-scale data sets and have strong generalization and robustness.
[0085] At the same time, the pre-trained model has learned effective representation information on data in related fields (such as medicine), and through transfer learning, it can be directly applied to new medical data, thereby improving the accuracy of analysis. It reduces the need for manual feature engineering and automatically extracts high-dimensional features through neural networks to avoid human bias.
[0086] The pre-trained second neural network model provided can be plug-and-play, greatly shortening the time for model development and optimization. Combined with its powerful nonlinear feature extraction capabilities, it can process complex, unstructured medical data, such as texts containing polysemous words and images with a lot of noise.
[0087] The provided method can also fine-tune the neural network model according to specific needs to better adapt it to specific medical tasks. In the future, the performance of the model can be continuously improved by continuing to learn and accumulate more medical data.
[0088] In summary, by applying the pre-trained second neural network model, the embodiment of step S102 can provide excellent technical advantages in efficiently and accurately analyzing medical data, greatly improve the quality of data representation information, and lay a solid foundation for subsequent annotation and model training. This not only reduces the workload of developers, but also significantly shortens the data processing cycle and improves the efficiency of the overall workflow.
[0089] Exemplarily, the first neural network model and the second neural network model can be the same neural network model, for example, a pre-trained large model neural network model is used to perform self-supervised learning on medical data. By maximizing the likelihood of the data or other self-supervised learning objectives, the model can learn the representation of the medical data. Specifically, the first neural network model and the second neural network model can achieve the following functions:
[0090] (1) Data representation learning: Use pre-trained large neural network models (such as the GPT model) to perform self-supervised learning on medical data. Through the pre-trained parameters of the model, rich semantic information and feature representations of large-scale medical text data or image data can be obtained.
[0091] (2) Target generation: In the process of self-supervised learning, a large model is used to generate target data. For medical text data, the GPT model can be used to generate labels related to the text content, such as generating disease names or symptom descriptions based on medical record descriptions. For medical image data, the GPT model can be used to generate image tags, such as generating tags for lesion areas in an image or segmenting an image.
[0092] (3) Automatic labeling: Use the generated target data to automatically label the medical data. Associate the generated labels with the original medical data to form a fully labeled data set. Since the large model has a strong ability to understand and express data, the generated labels have high accuracy and semantic consistency.
[0093] In some embodiments, please refer to Figure 3 After the labeling of the plurality of first medical data is completed, steps S106 to S108 are also included.
[0094] Step S106: Generate a training sample according to each first medical data and corresponding label information.
[0095] By pairing each first medical data (such as medical text or medical image) with previously generated label information, it is formed into a training sample. The training sample includes input data (medical text or image) and corresponding labels (classification information, feature information, etc.). The data and labels are organized into a format suitable for model training, such as tensors or data matrices. The training set and validation set are split to ensure that the data set is diverse and uniform.
[0096] Then, by generating formatted and annotated training samples, high-quality data input is provided for subsequent model training. It also ensures a one-to-one correspondence between data and labels, reducing confusion and errors in data annotation and model training. And by splitting the training and validation sets, the model performance can be evaluated in real time during the model training process to prevent overfitting.
[0097] Step S107. Complete the training of the data classification model to be trained according to the training samples; wherein the data classification model includes any one of a text classification model and an image segmentation model.
[0098] By selecting the appropriate model architecture according to the task requirements, such as BERT, CNN and other models for text classification tasks, U-Net, Mask R-CNN and other models for image segmentation tasks, or any other model selection based on requirements. By initializing the parameters of the model, set the training hyperparameters (such as learning rate, batch size). Use the training samples to train the model and optimize the model parameters to minimize the loss function (such as cross entropy, mean square error, etc.). At the same time, monitor the performance indicators (such as accuracy, F1 score) and loss values during the training process. And use the validation set to verify the model, determine whether the model is overfitting or underfitting, and adjust the hyperparameters.
[0099] In summary, through effective model training, a model that can accurately classify and segment medical data is built. A large amount of high-quality training data is used to improve the generalization ability and robustness of the model. By monitoring and adjusting the training process, the stability and optimizability of the model performance are ensured.
[0100] Step S108: Input the second medical data to be classified into the trained data classification model; the data classification model analyzes the second medical data and outputs the classification result corresponding to the second medical data.
[0101] The second medical data is preprocessed similarly to the first medical data (such as data cleaning and normalization) to ensure that the data format is consistent. The preprocessed second medical data is input into the trained data classification model. According to the results of the model analysis, the corresponding classification information (such as text classification labels, image segmentation results) is output.
[0102] Then, the trained model is used to classify and analyze new medical data, generate effective classification results, and provide data support for practical applications. At the same time, it can process new unlabeled data and improve the automation and efficiency of classification tasks. The output classification results can be used in various scenarios such as auxiliary diagnosis and medical research, improving the intelligent level of medical data processing.
[0103] In summary, by introducing steps S106 to S108, model training and application are added to the original data labeling and analysis process. By generating training samples, training models, and classifying new data, not only the entire data processing process is enriched, but also the practicality and intelligence level of the system are greatly improved. These steps are effectively combined to make the entire process from raw data to classification results more complete and efficient.
[0104] See also Figure 4 As shown, Figure 4 1 is a schematic diagram of the structure of the data labeling device 200 provided in an embodiment of the present application. The data labeling device 200 is used to perform the steps of the data labeling method shown in the above embodiments. The data labeling device 200 can be a single server or a server cluster, or the data labeling device 200 can be a terminal, which can be a handheld terminal, a laptop computer, a wearable device or a robot, etc.
[0105] like Figure 4 As shown, the data labeling device 200 includes:
[0106] A data acquisition unit 201 is used to acquire a plurality of first medical data to be annotated; wherein the first medical data includes any one of medical text and medical image;
[0107] A data parsing unit 202 is used to parse each of the first medical data to obtain data representation information corresponding to each of the first medical data; wherein the data representation information includes any one of semantic information corresponding to the medical text and image features corresponding to the medical image;
[0108] The target generation unit 203 is used to parse each of the data representation information and output the target data corresponding to each of the data representation information; wherein the target data corresponding to the medical text includes one or more of context prediction information, text filling information, disease / symptom category and entity information, and the target data corresponding to the medical image includes one or more of region segmentation information, position information, category information and key point information;
[0109] The labeling completion unit 204 is used to generate label information corresponding to the first medical data according to each of the target data, and complete the labeling of multiple first medical data.
[0110] In some embodiments, after completing the labeling of the plurality of the first medical data, the method further includes: a sample generation unit, configured to generate a training sample according to each of the first medical data and the corresponding label information; a model training unit, configured to complete the training of the data classification model to be trained according to the training sample; wherein the data classification model includes any one of a text classification model and an image segmentation model; and a classification completion unit, configured to input the second medical data to be classified into the trained data classification model, wherein the data classification model parses the second medical data and outputs the classification result corresponding to the second medical data.
[0111] In some embodiments, the target generation unit 203 is further used to: input each of the data representation information into a pre-trained first neural network model, the first neural network model parses the data representation information, and outputs target data corresponding to each of the first medical data.
[0112] Exemplarily, the first neural network model includes one or more of a context prediction model, a category prediction model and an entity recognition model; if the data representation information is semantic information corresponding to the medical text, the target generation unit 203 is also used to: input the medical text and the semantic information into the context prediction model, the context prediction model parses the medical text and the semantic information, and outputs the context information and / or the text filling information corresponding to the medical text, so as to generate the label information according to the context information and / or the text filling information; and / or, input the medical text and the semantic information into the category prediction model, the category prediction model parses the medical text and the semantic information, and outputs the disease / symptom category corresponding to the medical text, so as to generate the label information according to the disease / symptom category; and / or, input the medical text and the semantic information into the entity recognition model, the entity recognition model parses the medical text and the semantic information, and outputs the entity information corresponding to the medical text, so as to generate the label information according to the entity information; wherein the entity information includes at least one or more of the disease name and the drug name.
[0113] Exemplarily, the first neural network model includes one or more of an image segmentation model, an object detection model and a key point detection model; if the data representation information includes image features corresponding to the medical image, the target generation unit 203 is further used to: input the medical image and image features into the image segmentation model, the image segmentation model parses the medical image and image features, and outputs region segmentation information corresponding to the medical image, so as to generate the label information according to the region segmentation information; and / or, input the medical image and image features into the object detection model, the object detection model parses the medical image and image features, and outputs position information and category information corresponding to the medical image, so as to generate the label information according to the position information and category information; and / or, input the medical image and image features into the key point detection model, the key point detection model parses the medical image and image features, and outputs key point information corresponding to the medical image, so as to generate the label information according to the key point information.
[0114] In some embodiments, the data parsing unit 202 is also used to: input each of the first medical data into a second neural network model pre-trained based on self-supervised learning, the second neural network model parses each of the first medical data according to the pre-trained parameters, and outputs the data representation information corresponding to each of the first medical data.
[0115] In some embodiments, before parsing each of the first medical data, a preprocessing unit is further included, which is used to: preprocess the first medical data; the preprocessing at least includes data cleaning and normalization.
[0116] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the data labeling device and each module described above can refer to the corresponding process in the data labeling method embodiments described in the above embodiments, and will not be repeated here.
[0117] The above data annotation method can be implemented in the form of a computer program. Figure 4 Run on the device shown.
[0118] See also Figure 5 , Figure 5 1 is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device includes a processor, a memory and a network interface connected via a device bus, wherein the memory may include a storage medium and an internal memory.
[0119] The storage medium can store an operating device and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any data labeling method.
[0120] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.
[0121] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any data labeling method.
[0122] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal to which the scheme of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0123] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0124] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:
[0125] Acquire a plurality of first medical data to be annotated; wherein the first medical data includes any one of medical text and medical image;
[0126] Parsing each of the first medical data to obtain data representation information corresponding to each of the first medical data; wherein the data representation information includes any one of semantic information corresponding to the medical text and image features corresponding to the medical image;
[0127] Parse each of the data representation information and output the target data corresponding to each of the data representation information; wherein the target data corresponding to the medical text includes one or more of context prediction information, text filling information, disease / symptom category and entity information, and the target data corresponding to the medical image includes one or more of region segmentation information, location information, category information and key point information;
[0128] Generate label information corresponding to the first medical data according to each of the target data, and complete the labeling of the plurality of the first medical data.
[0129] In some embodiments, after completing the labeling of multiple first medical data, it also includes: generating training samples based on each of the first medical data and the corresponding label information; completing the training of the data classification model to be trained based on the training samples; wherein the data classification model includes any one of a text classification model and an image segmentation model; inputting the second medical data to be classified into the trained data classification model, the data classification model parses the second medical data, and outputs the classification result corresponding to the second medical data.
[0130] In some embodiments, parsing each of the data representation information and outputting target data corresponding to each of the data representation information includes: inputting each of the data representation information into a pre-trained first neural network model, the first neural network model parsing the data representation information, and outputting target data corresponding to each of the first medical data.
[0131] Exemplarily, the first neural network model includes one or more of a context prediction model, a category prediction model and an entity recognition model; if the data representation information is semantic information corresponding to the medical text, each of the data representation information is input into a pre-trained first neural network model, the first neural network model parses the data representation information, and outputs the target data corresponding to each of the first medical data, including: inputting the medical text and the semantic information into the context prediction model, the context prediction model parses the medical text and the semantic information, and outputs the context information and / or the text filling information corresponding to the medical text, so as to The label information is generated by using text information and / or text filling information; and / or, the medical text and semantic information are input into the category prediction model, the category prediction model parses the medical text and semantic information, and outputs the disease / symptom category corresponding to the medical text, so as to generate the label information according to the disease / symptom category; and / or, the medical text and semantic information are input into the entity recognition model, the entity recognition model parses the medical text and semantic information, and outputs the entity information corresponding to the medical text, so as to generate the label information according to the entity information; wherein the entity information includes at least one or more of the disease name and the drug name.
[0132] Exemplarily, the first neural network model includes one or more of an image segmentation model, an object detection model and a key point detection model; if the data representation information includes image features corresponding to the medical image, each of the data representation information is input into a pre-trained first neural network model, the first neural network model parses the data representation information, and outputs target data corresponding to each of the first medical data, including: inputting the medical image and image features into the image segmentation model, the image segmentation model parses the medical image and image features, and outputs region segmentation information corresponding to the medical image, so as to generate the label information according to the region segmentation information; and / or, inputting the medical image and image features into the object detection model, the object detection model parses the medical image and image features, and outputs position information and category information corresponding to the medical image, so as to generate the label information according to the position information and category information; and / or, inputting the medical image and image features into the key point detection model, the key point detection model parses the medical image and image features, and outputs key point information corresponding to the medical image, so as to generate the label information according to the key point information.
[0133] In some embodiments, parsing each of the first medical data and obtaining data representation information corresponding to each of the first medical data includes: inputting each of the first medical data into a second neural network model pre-trained based on self-supervised learning, the second neural network model parsing each of the first medical data according to pre-trained parameters, and outputting data representation information corresponding to each of the first medical data.
[0134] In some embodiments, before parsing each of the first medical data, it also includes: preprocessing the first medical data; the preprocessing includes at least data cleaning and normalization.
[0135] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement the steps of the data labeling method provided in the above embodiments of the present application.
[0136] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.
[0137] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A data labeling method, characterized in that: include: Acquire a plurality of first medical data to be annotated; wherein the first medical data includes any one of medical text and medical image; Parsing each of the first medical data based on a self-supervised learning method to obtain data representation information corresponding to each of the first medical data; wherein the data representation information includes any one of semantic information corresponding to the medical text and image features corresponding to the medical image; Parse each of the data representation information and output the target data corresponding to each of the data representation information; wherein the target data corresponding to the medical text includes one or more of context prediction information, text filling information, disease / symptom category and entity information, and the target data corresponding to the medical image includes one or more of region segmentation information, location information, category information and key point information; Generate label information corresponding to the first medical data according to each of the target data, and complete the labeling of the plurality of the first medical data.
2. The method according to claim 1, characterized in that After the labeling of the plurality of the first medical data is completed, the method further includes: Generate a training sample according to each of the first medical data and the corresponding label information; Complete the training of the data classification model to be trained according to the training samples; wherein the data classification model includes any one of a text classification model and an image segmentation model; The second medical data to be classified is input into the trained data classification model, and the data classification model analyzes the second medical data and outputs a classification result corresponding to the second medical data.
3. The method according to claim 1, characterized in that The parsing each of the data representation information and outputting the target data corresponding to each of the data representation information includes: Each of the data representation information is input into a pre-trained first neural network model, and the first neural network model parses the data representation information and outputs target data corresponding to each of the first medical data.
4. The method according to claim 3, characterized in that The first neural network model includes one or more of a context prediction model, a category prediction model, and an entity recognition model; if the data representation information is semantic information corresponding to the medical text, each of the data representation information is input into the pre-trained first neural network model, the first neural network model parses the data representation information, and outputs target data corresponding to each of the first medical data, including: The medical text and semantic information are input into the context prediction model, the context prediction model parses the medical text and semantic information, and outputs the context information and / or the text filling information corresponding to the medical text, so as to generate the label information according to the context information and / or the text filling information; and / or, The medical text and semantic information are input into the category prediction model, the category prediction model parses the medical text and semantic information, and outputs the disease / symptom category corresponding to the medical text, so as to generate the label information according to the disease / symptom category; and / or, The medical text and semantic information are input into the entity recognition model, and the entity recognition model parses the medical text and semantic information, and outputs the entity information corresponding to the medical text to generate the label information according to the entity information; wherein the entity information includes at least one or more of a disease name and a drug name.
5. The method according to claim 3, characterized in that: The first neural network model includes one or more of an image segmentation model, an object detection model, and a key point detection model; if the data representation information includes image features corresponding to the medical image, each of the data representation information is input into a pre-trained first neural network model, the first neural network model parses the data representation information, and outputs target data corresponding to each of the first medical data, including: inputting the medical image and the image features into the image segmentation model, the image segmentation model analyzing the medical image and the image features, and outputting the region segmentation information corresponding to the medical image, so as to generate the label information according to the region segmentation information; and / or, inputting the medical image and the image features into the object detection model, the object detection model parsing the medical image and the image features, outputting the position information and the category information corresponding to the medical image, so as to generate the label information according to the position information and the category information; and / or, The medical image and image features are input into the key point detection model, and the key point detection model parses the medical image and image features, and outputs key point information corresponding to the medical image to generate the label information according to the key point information.
6. The method according to claim 1, characterized in that The step of parsing each of the first medical data based on the self-supervised learning method to obtain data representation information corresponding to each of the first medical data includes: Each of the first medical data is input into a second neural network model pre-trained based on a self-supervised learning method. The second neural network model parses each of the first medical data according to pre-trained parameters and outputs data representation information corresponding to each of the first medical data.
7. The method according to claim 1, characterized in that: Before parsing each of the first medical data, the method further includes: The first medical data is preprocessed; the preprocessing at least includes data cleaning and normalization.
8. A data labeling device, characterized in that: include: A data acquisition unit, configured to acquire a plurality of first medical data to be annotated; wherein the first medical data includes any one of a medical text and a medical image; A data parsing unit, configured to parse each of the first medical data based on a self-supervised learning method, and obtain data representation information corresponding to each of the first medical data; wherein the data representation information includes any one of semantic information corresponding to the medical text and image features corresponding to the medical image; A target generation unit, configured to parse each of the data representation information and output target data corresponding to each of the data representation information; wherein the target data corresponding to the medical text includes one or more of context prediction information, text filling information, disease / symptom category and entity information, and the target data corresponding to the medical image includes one or more of region segmentation information, location information, category information and key point information; The labeling completion unit is used to generate label information corresponding to the first medical data according to each of the target data, and complete the labeling of multiple first medical data.
9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and implement the method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to implement the method according to any one of claims 1 to 7.