Three-dimensional medical image intelligent labeling method and system based on AI

Abstract

The invention provides an AI-based three-dimensional medical image intelligent labeling method and system. The accuracy and efficiency of medical image labeling are remarkably improved. By combining an image classifier of preliminary knowledge learning with a basic lesion labeling network, knowledge information from different sources can be synthesized, and a more comprehensive and reliable lesion semantic knowledge point sequence is generated. Furthermore, through three types of training image generation strategies, complementarity among different networks is effectively utilized, the recognition capability of common lesion areas is enhanced, and lesion areas which are respectively and uniquely found are explored, so that diversity and representativeness of training data are enriched. Finally, deep knowledge learning is carried out on the basic lesion labeling network based on the diversified training data, the recognition precision and generalization ability of the generated target lesion labeling network are remarkably improved, the workload of manual labeling of doctors is relieved, and the medical image analysis process is accelerated.

Description

Technical Field

[0001] The present application relates to the field of smart medical technology, and more specifically, to an AI-based three-dimensional medical image intelligent annotation method and system. Background Art

[0002] With the rapid development of medical imaging technology and the growing clinical needs, 3D medical images are playing an increasingly important role in clinical diagnosis, disease assessment and treatment plan formulation. However, traditional medical image annotation methods are highly dependent on doctors' experience and manual operations, which is not only time-consuming and labor-intensive, but also easily affected by subjective factors, making it difficult to ensure the consistency and accuracy of the annotation results.

[0003] In recent years, the rise of artificial intelligence (AI) technology has brought new solutions to medical image annotation. By training deep learning models, AI technology can automatically identify and annotate lesion areas in medical images, greatly improving annotation efficiency and accuracy. However, most existing AI medical image annotation methods rely on a single type of training data, which makes it difficult to fully cover different types of lesion areas and is easily affected by data bias. Summary of the invention

[0004] In view of this, the purpose of this application is to provide an AI-based three-dimensional medical image intelligent annotation method and system.

[0005] According to the first aspect of the present application, a method for intelligent annotation of three-dimensional medical images based on AI is provided, the method comprising:

[0006] Obtain sample three-dimensional medical scan images;

[0007] Loading the sample three-dimensional medical scan image into the image classifier of preliminary knowledge learning to generate a first lesion semantic knowledge point sequence, and loading the sample three-dimensional medical scan image into the basic lesion annotation network of preliminary knowledge learning to generate a second lesion semantic knowledge point sequence, wherein the basic lesion annotation network is used to represent the neural network generated by knowledge learning based on the basic network learning data, and the image classifier is used to represent the neural network generated by knowledge learning based on the general medical image library;

[0008] Generate a first type of training three-dimensional medical scan images, a second type of training three-dimensional medical scan images, and a third type of training three-dimensional medical scan images based on the first lesion semantic knowledge point sequence and the second lesion semantic knowledge point sequence, wherein the first lesion semantic knowledge point annotated on the first type of training three-dimensional medical scan images is located both in the first lesion semantic knowledge point sequence and in the second lesion semantic knowledge point sequence, the second lesion semantic knowledge point annotated on the second type of training three-dimensional medical scan images is located in the first lesion semantic knowledge point sequence and is not located in the second lesion semantic knowledge point sequence, and the third lesion semantic knowledge point annotated on the third type of training three-dimensional medical scan images is located in the second lesion semantic knowledge point sequence and is not located in the first lesion semantic knowledge point sequence;

[0009] The basic lesion annotation network is subjected to knowledge learning based on the basic network learning data, the first type of training three-dimensional medical scanning images, the second type of training three-dimensional medical scanning images, and the third type of training three-dimensional medical scanning images to generate a target lesion annotation network.

[0010] In a possible implementation of the first aspect, generating a first type of training three-dimensional medical scan image, a second type of training three-dimensional medical scan image, and a third type of training three-dimensional medical scan image based on the first lesion semantic knowledge point sequence and the second lesion semantic knowledge point sequence includes:

[0011] Determine the first lesion semantic knowledge point, the second lesion semantic knowledge point, and the third lesion semantic knowledge point based on the first lesion semantic knowledge point sequence and the second lesion semantic knowledge point sequence;

[0012] The classification probability value of the first lesion semantic knowledge point is defined as a first predefined value, and the first type of training three-dimensional medical scan images is generated, where the first type of training three-dimensional medical scan images is used to represent the classification probability value of the first lesion semantic knowledge point annotated on the sample three-dimensional medical scan image as the first predefined value;

[0013] The classification probability value of the second lesion semantic knowledge point is defined as a first estimated possibility, and the second type of training three-dimensional medical scan images is generated, wherein the first estimated possibility is used to characterize the degree of possibility of the basic lesion annotation network estimating whether the sample three-dimensional medical scan image is annotated with the second lesion semantic knowledge point, and the second type of training three-dimensional medical scan images is used to characterize the classification probability value of the sample three-dimensional medical scan image annotating the second lesion semantic knowledge point as the first estimated possibility;

[0014] The classification probability value of the third lesion semantic knowledge point is defined as a second estimated possibility, and the third type of training three-dimensional medical scanning images is generated. The second estimated possibility is less than the degree of possibility estimated by the basic lesion annotation network whether the sample three-dimensional medical scanning image is annotated with the third lesion semantic knowledge point. The classification probability value of the third type of training three-dimensional medical scanning images used to characterize the sample three-dimensional medical scanning images annotating the third lesion semantic knowledge point is the second estimated possibility.

[0015] In a possible implementation of the first aspect, defining the classification probability value of the third lesion semantic knowledge point as a second estimated possibility and generating the third type of training three-dimensional medical scan images includes:

[0016] Obtaining the possibility degree of whether the sample three-dimensional medical scan image is annotated with the third lesion semantic knowledge point estimated by the basic lesion annotation network;

[0017] The product of the degree of possibility and a second predefined numerical value is output as the second estimated possibility, the classification probability value of the third lesion semantic knowledge point is defined as the second estimated possibility, and the third type of training three-dimensional medical scanning image is generated, and the second predefined numerical value is a value greater than 0 and less than 1 / x.

[0018] In a possible implementation of the first aspect, the performing knowledge learning on the basic lesion annotation network based on the basic network learning data, the first type of training three-dimensional medical scan images, the second type of training three-dimensional medical scan images, and the third type of training three-dimensional medical scan images to generate a target lesion annotation network includes:

[0019] Performing knowledge learning on the basic lesion annotation network according to the basic network learning data, the first type of training three-dimensional medical scanning images, the second type of training three-dimensional medical scanning images, and the third type of training three-dimensional medical scanning images to generate a temporary lesion annotation network, wherein the training step length configured for the generated temporary lesion annotation network is the first training step length;

[0020] Advanced knowledge learning is performed on the temporary lesion annotation network based on the basic network learning data and the first type of training three-dimensional medical scan images to generate the target lesion annotation network. The training step size configured for the generated target lesion annotation network is the second training step size, and the second training step size is smaller than the first training step size.

[0021] In a possible implementation of the first aspect, performing advanced knowledge learning on the temporary lesion annotation network based on the basic network learning data and the first type of training three-dimensional medical scan images to generate the target lesion annotation network includes:

[0022] When the temporary lesion annotation network includes a basic image semantic feature extraction unit and a fully connected output unit, a second training step size is set for the basic image semantic feature extraction unit, and a third training step size is set for the fully connected output unit, wherein the third training step size is greater than the first training step size;

[0023] Advanced knowledge learning is performed on the temporary lesion labeling network according to the second training step size and the third training step size to generate the target lesion labeling network.

[0024] In a possible implementation of the first aspect, the performing knowledge learning on the basic lesion annotation network based on the basic network learning data, the first type of training three-dimensional medical scan images, the second type of training three-dimensional medical scan images, and the third type of training three-dimensional medical scan images to generate a target lesion annotation network includes:

[0025] Target network learning data is configured based on the basic network learning data, the first type of training three-dimensional medical scanning images, the second type of training three-dimensional medical scanning images, and the third type of training three-dimensional medical scanning images, wherein one target network learning data includes a three-dimensional medical scanning image, a lesion semantic knowledge point sequence, and a classification probability value distribution corresponding to the target lesion semantic knowledge point sequence;

[0026] Based on the following steps, the target network learning data is sequentially loaded into the basic lesion annotation network, and the basic lesion annotation network is cyclically learned to generate the target lesion annotation network. The three-dimensional medical scan images configured in the current batch in the current round of knowledge learning are used as reference three-dimensional medical scan images. Each knowledge learning stage uses the global target network learning data for knowledge learning, and each batch performs knowledge learning based on part of the three-dimensional medical scan images in the target network learning data.

[0027] Loading the reference three-dimensional medical scan image into a basic image semantic feature extraction unit to generate target image semantic features, wherein the basic lesion annotation network includes the basic image semantic feature extraction unit;

[0028] The target image semantic features are loaded into a fully connected output unit to generate reference lesion semantic knowledge points and reference classification probability values ​​corresponding to the reference lesion semantic knowledge points, wherein the basic lesion annotation network includes the fully connected output unit, and the reference lesion semantic knowledge points are used to characterize the lesion semantic knowledge points annotated by the reference three-dimensional medical scan image estimated by the basic lesion annotation network;

[0029] Calculating a target learning error based on the one lesion semantic knowledge point sequence corresponding to the reference three-dimensional medical scan image, the classification probability value distribution, and the reference lesion semantic knowledge point and the reference classification probability value;

[0030] The network parameter information of the basic lesion labeling network is updated according to the target learning error until the target learning error meets the convergence requirement, thereby generating the target lesion labeling network.

[0031] In a possible implementation of the first aspect, after generating the first type of training three-dimensional medical scan images, the second type of training three-dimensional medical scan images, and the third type of training three-dimensional medical scan images based on the first lesion semantic knowledge point sequence and the second lesion semantic knowledge point sequence, the method further includes:

[0032] Determine the number of images corresponding to each initial lesion semantic knowledge point in the initial lesion semantic knowledge point sequence based on the basic network learning data;

[0033] Outputting the initial lesion semantic knowledge points whose image quantity meets the set quantity condition as first initial lesion semantic knowledge points, and outputting the initial lesion semantic knowledge points whose image quantity does not meet the set quantity condition as second initial lesion semantic knowledge points;

[0034] The second category of training three-dimensional medical scanning images and the third category of training three-dimensional medical scanning images that are marked with the first initial lesion semantic knowledge point are deleted, and the second category of training three-dimensional medical scanning images and the third category of training three-dimensional medical scanning images that are marked with the second initial lesion semantic knowledge point are retained.

[0035] In a possible implementation of the first aspect, before loading the sample three-dimensional medical scan image into a basic lesion annotation network for preliminary knowledge learning to generate a second lesion semantic knowledge point sequence, the method further includes:

[0036] Loading the to-be-learned three-dimensional medical scan image in the basic network learning data into the basic lesion annotation network to generate a target estimation probability, wherein the target estimation probability is used to characterize the probability degree of the basic lesion annotation network estimating whether the to-be-learned three-dimensional medical scan image is annotated with a reference lesion semantic knowledge point, and the to-be-learned three-dimensional medical scan image is pre-annotated with whether the reference lesion semantic knowledge point is annotated;

[0037] Acquire a threshold value sequence, wherein a threshold value in the threshold value sequence is used to generate a target labeling determination result in collaboration with the target estimation possibility, and the target labeling determination result is used to indicate whether the three-dimensional medical scan image to be learned is labeled with the reference lesion semantic knowledge point;

[0038] Based on the target estimation possibility and the threshold value sequence, determining the target threshold value of the reference lesion semantic knowledge point, the target threshold value is the maximum balanced evaluation value in the threshold value sequence, and the balanced evaluation value is collaboratively generated by the target annotation determination result and the label data of whether the three-dimensional medical scan image to be learned is annotated with the reference lesion semantic knowledge point;

[0039] The target threshold value is defined as a target network parameter of the basic lesion annotation network, and the target threshold value is used to determine the second lesion semantic knowledge point sequence in the basic lesion annotation network.

[0040] In a possible implementation of the first aspect, after performing knowledge learning on the basic lesion annotation network based on the basic network learning data, the first type of training three-dimensional medical scan images, the second type of training three-dimensional medical scan images, and the third type of training three-dimensional medical scan images to generate a target lesion annotation network, the method further includes:

[0041] Acquire a target medical examination data stream, and perform three-dimensional medical image extraction on the target medical examination data stream to generate a target three-dimensional medical scan image;

[0042] Loading the target three-dimensional medical scan image into the target lesion annotation network to generate a target lesion semantic knowledge point sequence and a target classification probability value sequence corresponding to the target lesion semantic knowledge point sequence;

[0043] The target lesion annotation data is annotated for the target medical examination data stream based on the target lesion semantic knowledge point sequence and the target classification probability value sequence, wherein the target lesion annotation data represents the lesion semantic knowledge points annotated by the target medical examination data stream.

[0044] According to the second aspect of the present application, a three-dimensional medical image annotation service system is provided, wherein the three-dimensional medical image annotation service system comprises a machine-readable storage medium and a processor, wherein the machine-readable storage medium stores machine-executable instructions, and when the processor executes the machine-executable instructions, the three-dimensional medical image annotation service system implements the aforementioned AI-based three-dimensional medical image intelligent annotation method.

[0045] According to the third aspect of the present application, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the computer-executable instructions are executed, the aforementioned AI-based three-dimensional medical image intelligent labeling method is implemented.

[0046] According to any one of the above aspects, the technical effect of the present application is:

[0047] The embodiments of the present application significantly improve the accuracy and efficiency of medical image annotation. By combining the image classifier for preliminary knowledge learning with the basic lesion annotation network, the present invention can integrate knowledge information from different sources to generate a more comprehensive and reliable sequence of lesion semantic knowledge points. Furthermore, through the three types of training image generation strategies, the complementarity between different networks is effectively utilized, which not only strengthens the recognition ability of common lesion areas, but also explores the lesion areas uniquely discovered by each network, thereby enriching the diversity and representativeness of the training data. Finally, based on these diverse training data, the basic lesion annotation network is deeply learned, and the generated target lesion annotation network is significantly improved in recognition accuracy and generalization ability, which not only reduces the workload of doctors for manual annotation, but also accelerates the medical image analysis process. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 A schematic diagram of the process of the AI-based three-dimensional medical image intelligent annotation method provided in an embodiment of the present application is shown;

[0050] Figure 2 A schematic diagram of the component structure of a three-dimensional medical image annotation service system for implementing the above-mentioned AI-based three-dimensional medical image intelligent annotation method provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0051] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0052] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when an element is "connected" or "coupled" to another element, the element may be directly connected or coupled to the other element, or may refer to the element and the other element establishing a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling, and the term "and / or" used herein indicates at least one of the items defined by the term, such as "A and / or B" may be implemented as "A", or as "B", or as "A and B".

[0053] In order to make the purpose, technical solution and advantages of the present application clearer, the implementation mode of the present application will be further described in detail with reference to the accompanying drawings. The technical solution of the embodiment of the present application and the technical effect produced by the technical solution of the present application are explained below by describing several exemplary implementation modes. It should be pointed out that the following implementation modes can refer to, draw lessons from or combine with each other, and the same terms, similar features and similar implementation steps in different implementation modes are not described repeatedly.

[0054] Figure 1 The flowchart of the AI-based 3D medical image intelligent annotation method and system provided in the embodiment of the present application is shown. It should be understood that in other embodiments, the order of some steps in the AI-based 3D medical image intelligent annotation method of this embodiment can be shared with each other according to actual needs, or some steps can be omitted or maintained. The detailed steps of the AI-based 3D medical image intelligent annotation method include:

[0055] Step S110, obtaining a sample three-dimensional medical scan image.

[0056] In this embodiment, in the medical imaging center, the server receives a sample 3D medical scan image transmitted from the medical imaging device, which is a patient's lung CT scan data for diagnosing possible lesions. The server stores this image file in a local high-performance storage system for subsequent analysis and processing.

[0057] Step S120, loading the sample three-dimensional medical scanning image into the image classifier of preliminary knowledge learning to generate a first lesion semantic knowledge point sequence, and loading the sample three-dimensional medical scanning image into the basic lesion annotation network of preliminary knowledge learning to generate a second lesion semantic knowledge point sequence, wherein the basic lesion annotation network is used to characterize the neural network generated by knowledge learning based on the basic network learning data, and the image classifier is used to characterize the neural network generated by knowledge learning based on the general medical image library.

[0058] In this embodiment, the server first loads the sample 3D medical scan image into an image classifier that has undergone preliminary knowledge learning. The image classifier has been trained using a large number of general medical image libraries and has a certain ability to identify lesions. After processing, the image classifier outputs a first sequence of lesion semantic knowledge points, which includes several possible lesion areas identified and their corresponding lesion types, such as lung nodules, pneumonia, etc.

[0059] The server then loaded the sample 3D medical scan image into the basic lesion annotation network, which was generated after learning knowledge based on the basic network learning data and focuses on annotating lesion areas from medical scan images. After some calculations, the basic lesion annotation network outputs a second lesion semantic knowledge point sequence, which also contains the lesion information it has identified.

[0060] Step S130, based on the first lesion semantic knowledge point sequence and the second lesion semantic knowledge point sequence, generate a first type of training three-dimensional medical scanning image, a second type of training three-dimensional medical scanning image and a third type of training three-dimensional medical scanning image, the first lesion semantic knowledge point annotated by the first type of training three-dimensional medical scanning image is located in both the first lesion semantic knowledge point sequence and the second lesion semantic knowledge point sequence, the second lesion semantic knowledge point annotated by the second type of training three-dimensional medical scanning image is located in the first lesion semantic knowledge point sequence and is not located in the second lesion semantic knowledge point sequence, and the third lesion semantic knowledge point annotated by the third type of training three-dimensional medical scanning image is located in the second lesion semantic knowledge point sequence and is not located in the first lesion semantic knowledge point sequence.

[0061] In this embodiment, the server then compares the first lesion semantic knowledge point sequence with the second lesion semantic knowledge point sequence to find out the similarities and differences between the two. Based on this information, the server generates three types of training 3D medical scan images:

[0062] The first type of training 3D medical scan images annotated those lesion semantic knowledge points that appeared in both lesion semantic knowledge point sequences. These lesion semantic knowledge points were considered to be relatively certain lesion areas, and therefore were assigned higher classification probability values ​​in the training data, such as 0.95.

[0063] The second type of training 3D medical scan images annotate those lesion areas that only appear in the first lesion semantic knowledge point sequence. These lesion areas can be discovered uniquely by the image classifier, but are not recognized by the basic lesion annotation network. Therefore, when generating this type of training data, the server sets a relatively low classification probability value, such as 0.6, based on the estimated possibility of these lesion areas by the basic lesion annotation network.

[0064] The third type of training 3D medical scan images annotates those lesion areas that only appear in the second lesion semantic knowledge point sequence. These lesion areas may be unique findings of the basic lesion annotation network and are not recognized by the image classifier. For this type of data, the server will set a lower classification probability value, such as 0.4, because the basic lesion annotation network has relatively low confidence in identifying these lesion areas.

[0065] Step S140, performing knowledge learning on the basic lesion annotation network according to the basic network learning data, the first type of training three-dimensional medical scanning images, the second type of training three-dimensional medical scanning images, and the third type of training three-dimensional medical scanning images to generate a target lesion annotation network.

[0066] In this embodiment, finally, the server uses the three types of new training 3D medical scan images and the original basic network learning data to further learn the basic lesion annotation network. In this process, the basic lesion annotation network will continuously adjust its internal parameters to better adapt to these new training data.

[0067] Specifically, the server will load these training data into the basic lesion annotation network in batches. After each batch of data is loaded, the difference between the actual output of the basic lesion annotation network and the label in the training data (i.e., learning error) will be calculated. Then, the network parameters will be updated based on this error to reduce the error generated when processing similar data in the future. This process will be repeated until the network's performance reaches a preset standard or the learning error no longer decreases significantly.

[0068] After this series of knowledge learning processes, the basic lesion annotation network has evolved into a target lesion annotation network, which will be more accurate and efficient in identifying lesion areas in medical scan images. After that, when new medical scan images are input, this target lesion annotation network can quickly annotate the lesion areas in them, providing powerful assistance for doctors' diagnosis.

[0069] Based on the above steps, the embodiments of the present application significantly improve the accuracy and efficiency of medical image annotation. By combining the image classifier of preliminary knowledge learning with the basic lesion annotation network, the present invention can integrate knowledge information from different sources to generate a more comprehensive and reliable sequence of lesion semantic knowledge points. Furthermore, through the three types of training image generation strategies, the complementarity between different networks is effectively utilized, which not only strengthens the recognition ability of common lesion areas, but also explores the lesion areas discovered by each unique network, thereby enriching the diversity and representativeness of the training data. Finally, based on these diverse training data, the basic lesion annotation network is deeply learned, and the generated target lesion annotation network is significantly improved in recognition accuracy and generalization ability, which not only reduces the workload of doctors for manual annotation, but also accelerates the medical image analysis process.

[0070] In a possible implementation, step S130 includes:

[0071] Step S131, determining the first lesion semantic knowledge point, the second lesion semantic knowledge point and the third lesion semantic knowledge point based on the first lesion semantic knowledge point sequence and the second lesion semantic knowledge point sequence.

[0072] In this embodiment, the server has processed a sample 3D medical scan image through step S120 and obtained two lesion semantic knowledge point sequences: the first one is generated by the image classifier, and the second one is generated by the basic lesion annotation network. Now, the server needs to compare the two lesion semantic knowledge point sequences to determine the three types of lesion semantic knowledge points required for subsequent training.

[0073] The server first traverses each knowledge point (such as lung nodule, pneumonia, etc.) in the first lesion semantic knowledge point sequence and checks whether they also appear in the second lesion semantic knowledge point sequence.

[0074] In detail, for knowledge points that appear simultaneously in two lesion semantic knowledge point sequences, the server marks them as first lesion semantic knowledge points. These knowledge points are lesion areas that both networks are relatively confident about.

[0075] Knowledge points that only appear in the first lesion semantic knowledge point sequence are labeled as second lesion semantic knowledge points, which can be unique findings of the image classifier and failed to be identified by the underlying lesion annotation network.

[0076] Knowledge points that only appear in the second lesion semantic knowledge point sequence are labeled as third lesion semantic knowledge points, which can be unique findings of the underlying lesion annotation network and failed to be identified by the image classifier.

[0077] Step S132, defining the classification probability value of the first lesion semantic knowledge point as a first predefined value, generating the first type of training three-dimensional medical scanning images, wherein the first type of training three-dimensional medical scanning images is used to characterize the sample three-dimensional medical scanning images, and the classification probability value of the first lesion semantic knowledge point annotated is the first predefined value.

[0078] In this embodiment, the server assigns a first predefined numerical value as a classification probability value to each first lesion semantic knowledge point. This classification probability value is usually high, such as 0.95, indicating that these first lesion semantic knowledge points are highly confident in the training data.

[0079] Based on the sample 3D medical scan images, the server generates corresponding training 3D medical scan images for each first lesion semantic knowledge point. These training 3D medical scan images will clearly indicate the type of the lesion area and the corresponding classification probability value (0.95) when annotating.

[0080] Step S133, defining the classification probability value of the second lesion semantic knowledge point as a first estimated possibility, generating the second type of training three-dimensional medical scanning images, the first estimated possibility being used to characterize the degree of possibility of the basic lesion annotation network estimating whether the sample three-dimensional medical scanning image is annotated with the second lesion semantic knowledge point, and the second type of training three-dimensional medical scanning images being used to characterize the classification probability value of the sample three-dimensional medical scanning image annotating the second lesion semantic knowledge point as the first estimated possibility.

[0081] In this embodiment, for the second lesion semantic knowledge points, the server needs to generate training data to explore the recognition ability of the basic lesion annotation network for these second lesion semantic knowledge points, and may guide network learning by adjusting the classification probability value.

[0082] In detail, the server first uses the basic lesion annotation network to predict the sample three-dimensional medical scan image, and obtains the estimated possibility of the basic lesion annotation network for the second lesion semantic knowledge points. The possibility value reflects the current recognition confidence of the basic lesion annotation network for these second lesion semantic knowledge points.

[0083] Next, the server uses this estimated probability value as the first estimated probability to generate a classification probability value for the second type of training three-dimensional medical scan images. The classification probability value is usually lower than the first predefined value, such as 0.6, to reflect the network's lower confidence in these knowledge points.

[0084] Based on the sample three-dimensional medical scan image and the second lesion semantic knowledge point, the server generates a training three-dimensional medical scan image and indicates the type of these lesion areas and the corresponding classification probability value (first estimated possibility) when annotating.

[0085] Step S134, defining the classification probability value of the third lesion semantic knowledge point as a second estimated possibility, generating the third type of training three-dimensional medical scanning images, the second estimated possibility being less than the degree of possibility estimated by the basic lesion annotation network as to whether the sample three-dimensional medical scanning image is annotated with the third lesion semantic knowledge point, and the third type of training three-dimensional medical scanning images is used to characterize the classification probability value of the sample three-dimensional medical scanning image annotating the third lesion semantic knowledge point as the second estimated possibility.

[0086] In this embodiment, for the third lesion semantic knowledge points, these are areas uniquely identified by the basic lesion annotation network, and the server needs to generate training data to further verify and strengthen these findings.

[0087] Similarly, the server first obtains the estimated probability of the basic lesion annotation network for these third lesion semantic knowledge points. In order to further reduce the classification probability values ​​of these knowledge points in the training data, the server introduces a second predefined value (such as 0.2, a value greater than 0 and less than 1 / x, where x is an empirical value or an adjustable parameter), and multiplies the initial estimated probability by this value to obtain the second estimated probability. This operation further reduces the network's confidence in these knowledge points. Based on the sample three-dimensional medical scan image and the third lesion semantic knowledge point, the server generates a training three-dimensional medical scan image, and indicates the type of these lesion areas and the corresponding classification probability value (second estimated probability) when annotating.

[0088] In detail, in a specific example, suppose the sample 3D medical scan image shows a patient's lung CT scan results, the image classifier identifies two lesion areas, lung nodules and pneumonia, while the basic lesion annotation network identifies atelectasis in addition to lung nodules.

[0089] The first lesion semantic knowledge point: pulmonary nodule (appears in two lesion semantic knowledge point sequences at the same time).

[0090] The second type of training 3D medical scan images: generated for the pneumonia area, the classification probability value is set to 0.6 (assuming that the estimated probability of pneumonia by the basic lesion annotation network is 0.8, but lower than the confidence of the image classifier).

[0091] The third type of training three-dimensional medical scan images: generated for atelectasis areas, the classification probability value is set to 0.1 (assuming that the estimated possibility of atelectasis by the basic lesion annotation network is 0.5, multiplied by the second predefined value 0.2).

[0092] In this way, three types of training 3D medical scan images with different classification probability values ​​were generated, providing rich and targeted data support for the subsequent knowledge learning process. These training data will help the basic lesion annotation network to more comprehensively learn and identify different types of lesion areas, thereby improving its overall performance.

[0093] In a possible implementation, step S134 includes:

[0094] Step S1341, obtaining the possibility degree estimated by the basic lesion annotation network as to whether the sample three-dimensional medical scan image is annotated with the third lesion semantic knowledge point.

[0095] Step S1342, output the product of the degree of possibility and a second predefined numerical value as the second estimated possibility, define the classification probability value of the third lesion semantic knowledge point as the second estimated possibility, generate the third type of training three-dimensional medical scanning image, and the second predefined numerical value is a value greater than 0 and less than 1 / x.

[0096] In this embodiment, the server first needs to load the sample 3D medical scan image into the basic lesion annotation network to obtain the estimated possibility of each possible lesion area by the basic lesion annotation network. For this step, special attention is paid to those third lesion semantic knowledge points that are only identified in the basic lesion annotation network.

[0097] In detail, the server takes the sample 3D medical scan image as input and loads it into the trained basic lesion annotation network. The basic lesion annotation network processes the input sample 3D medical scan image to generate an estimated probability for each potential lesion area. These estimated probabilities reflect the confidence of the basic lesion annotation network in whether a specific lesion area exists in the sample 3D medical scan image. Then, the server extracts the probability values ​​corresponding to the third lesion semantic knowledge point from the estimated probability output by the basic lesion annotation network, and these probability values ​​will be used for the subsequent calculation of the second estimated probability.

[0098] After determining the estimated likelihood of the basic lesion annotation network for the third lesion semantic knowledge point, the server needs to multiply these likelihood values ​​by a second predefined value to generate a second estimated likelihood for the training data. This operation is intended to further reduce the basic lesion annotation network's confidence in these knowledge points during the training phase, thereby guiding the basic lesion annotation network to handle these areas more cautiously in subsequent learning.

[0099] In detail, the server determines a second predefined value based on the system configuration or previous experimental results, and the second predefined value is greater than 0 and less than 1 / x, where x is a parameter that can be adjusted according to the specific situation. For example, if x is set to 10, the second predefined value can be 0.1 (satisfying the condition of being greater than 0 and less than 0.1). The server multiplies the estimated possibility value of each third lesion semantic knowledge point by the second predefined value to obtain the corresponding second estimated possibility. For example, if the estimated possibility of a lesion area is 0.5 and the second predefined value is 0.1, the second estimated possibility is 0.05.

[0100] Finally, the server uses the calculated second estimated likelihood as the classification probability value to generate a third type of training 3D medical scan images. These third type of training 3D medical scan images will be used for subsequent knowledge learning of the basic lesion annotation network to improve its ability to identify the third lesion semantic knowledge points.

[0101] In detail, the server generates corresponding third-category training three-dimensional medical scans based on the sample three-dimensional medical scan images and the third lesion semantic knowledge points. When annotating these third-category training three-dimensional medical scans, the server sets the classification probability value of the third lesion semantic knowledge point to the second estimated probability calculated previously. The generated third-category training three-dimensional medical scan images and their annotation information are saved in the storage system of the server for subsequent use in the knowledge learning of the basic lesion annotation network.

[0102] In a specific example, assume that the estimated probability of the basic lesion annotation network for a pulmonary atelectasis area (third lesion semantic knowledge point) in the sample three-dimensional medical scan image is 0.6. The second predefined value set by the server is 0.2. Through calculation, the server obtains the second estimated probability of the pulmonary atelectasis area as 0.12 (0.6*0.2). Subsequently, the server generates a training three-dimensional medical scan image with an atelectasis area annotated and a classification probability value of 0.12 as part of the third category of training three-dimensional medical scan images. These third category training three-dimensional medical scan images will be used to optimize the performance of the basic lesion annotation network in the subsequent knowledge learning stage.

[0103] In a possible implementation, step S140 includes:

[0104] Step S141, performing knowledge learning on the basic lesion annotation network according to the basic network learning data, the first type of training three-dimensional medical scanning images, the second type of training three-dimensional medical scanning images, and the third type of training three-dimensional medical scanning images, to generate a temporary lesion annotation network, wherein the training step size configured for the generated temporary lesion annotation network is the first training step size.

[0105] Step S142, performing advanced knowledge learning on the temporary lesion annotation network based on the basic network learning data and the first type of training three-dimensional medical scanning images to generate the target lesion annotation network, wherein the training step size configured for the generated target lesion annotation network is the second training step size, and the second training step size is smaller than the first training step size.

[0106] In this embodiment, the server has acquired the basic network learning data, the first type of training 3D medical scan images (high confidence lesion areas), the second type of training 3D medical scan images (uniquely discovered by the image classifier, low confidence of the basic network), and the third type of training 3D medical scan images (uniquely discovered by the basic network, not recognized by the image classifier). Now, the server will use these data to perform a series of knowledge learning steps to optimize the performance of the basic lesion annotation network.

[0107] The server first uses the basic network learning data and all three types of training 3D medical scan images to perform preliminary knowledge learning on the basic lesion annotation network. This process is designed to allow the network to fully contact different types of lesion areas and preliminarily adjust its internal parameters to adapt to these data.

[0108] In detail, the server integrates all training data (basic network learning data + three types of training 3D medical scan images) into a large training data set. Then, the basic lesion annotation network is loaded into the server's memory and prepared for training. Then, some key parameters in the training process are configured, including learning rate, batch size, training rounds, etc. In particular, the training step size is set to the first training step size, which is predetermined according to the network structure and data scale, and is usually large to ensure that the network can quickly cover most of the data. According to the set training parameters, the server loads the training data into the network in batches, calculates the loss function, and updates the weights and biases of the network through the back propagation algorithm. This process is repeated for multiple rounds until the preset stop condition is met (such as loss function convergence, reaching the maximum training rounds, etc.). After preliminary knowledge learning, the server generates a temporary lesion annotation network, which has a certain recognition ability for different types of lesion areas, but further fine-tuning is required.

[0109] Based on the preliminary knowledge learning, the server then uses the basic network learning data and the first type of training 3D medical scan images (high-confidence lesion areas) to perform advanced knowledge learning on the temporary lesion annotation network. This process aims to enhance the network's recognition ability for high-confidence lesion areas and fine-tune the network's parameters by reducing the training step size.

[0110] In detail, basic network learning data and the first type of training 3D medical scan images can be selected from all training data as input data for advanced knowledge learning. Then, the temporary lesion annotation network generated after preliminary knowledge learning is loaded into the memory of the server. Next, the training parameters are reset, especially the training step size is adjusted to the second training step size. Since preliminary knowledge learning has been carried out at this time, a smaller step size can be used to fine-tune the parameters of the network to avoid excessive step sizes causing the network to oscillate near the optimal solution.

[0111] Thus, the temporary lesion annotation network can be advanced trained using the filtered data. Similar to preliminary knowledge learning, the server loads the data into the network in batches, calculates the loss function, and updates the weights and biases of the network through the back-propagation algorithm. However, due to the changes in the amount of data and the training step size, this process may be slower and more meticulous.

[0112] After advanced knowledge learning, the server generated a target lesion annotation network, which significantly improved the recognition ability of high-confidence lesion areas while maintaining the recognition ability of various lesion areas.

[0113] In a specific example, assume that the server used 100,000 training 3D medical scan images (including basic network learning data and three types of training 3D medical scan images) in the preliminary knowledge learning stage, and set the first training step to 1000. After multiple rounds of training, the server generated a temporary lesion annotation network. Subsequently, in the advanced knowledge learning stage, the server screened out 50,000 training 3D medical scan images of high-confidence lesion areas (basic network learning data and the first type of training 3D medical scan images) and adjusted the training step to 500. After further refined training, the server successfully generated a target lesion annotation network, which showed higher recognition accuracy and robustness in subsequent tests.

[0114] In a possible implementation, step S142 includes:

[0115] Step S1421, when the temporary lesion annotation network includes a basic image semantic feature extraction unit and a fully connected output unit, a second training step is set for the basic image semantic feature extraction unit, and a third training step is set for the fully connected output unit, and the third training step is greater than the first training step.

[0116] Step S1422: performing advanced knowledge learning on the temporary lesion labeling network according to the second training step size and the third training step size to generate the target lesion labeling network.

[0117] In this embodiment, the server has generated a temporary lesion annotation network through preliminary knowledge learning, and the temporary lesion annotation network includes a basic image semantic feature extraction unit and a fully connected output unit. Now, the server will use the basic network learning data and the first type of training 3D medical scan images (high confidence lesion areas) to perform advanced knowledge learning on the temporary network, aiming to optimize the performance of the network by setting different training steps for different network units.

[0118] Before advanced knowledge learning begins, the server needs to identify the different units in the temporary lesion labeling network and set appropriate training steps for them.

[0119] In detail, the server analyzes the structure of the temporary lesion annotation network and identifies the basic image semantic feature extraction unit and the fully connected output unit. The basic image semantic feature extraction unit is responsible for extracting useful feature information from the input 3D medical scan image, while the fully connected output unit generates the final lesion annotation result based on this feature information.

[0120] Next, the second training step can be set for the basic image semantic feature extraction unit. Since feature extraction is the basis for subsequent classification, this step should not be too large to ensure that the network can stably learn effective feature representations. The second training step may be less than or equal to the first training step in preliminary knowledge learning, depending on the current state of the network and the complexity of the data.

[0121] A third training step is set for the fully connected output unit, and the third training step is larger than the second training step (and may also be larger than the first training step in the preliminary knowledge learning). Since the fully connected output unit directly determines the final classification result, a larger step can be used to accelerate its convergence speed on high-confidence lesion areas, while allowing the network to explore possible better solutions to a certain extent.

[0122] After determining the training step size of different network units, the server begins to use the basic network learning data and the first type of training 3D medical scan images to perform advanced knowledge learning on the temporary lesion annotation network.

[0123] Specifically, the server integrates the basic network learning data and the first type of training 3D medical scan images into a training data set for advanced knowledge learning. During the training process, the server updates the parameters of the basic image semantic feature extraction unit and the fully connected output unit according to the set training step size. Specifically, for each batch of training data:

[0124] First, the three-dimensional medical scan image is input into the basic image semantic feature extraction unit, and the weight and bias of the unit are updated according to the second training step.

[0125] Then, the extracted feature information is passed to the fully connected output unit to generate the lesion annotation result, and the weight and bias of the unit are updated according to the third training step.

[0126] The above process is repeated until the preset stopping condition is met (such as loss function convergence, reaching the maximum number of training rounds, etc.).

[0127] During the training process, the server can periodically use the validation set to evaluate the performance of the network to ensure that the network is not overfitting on the training data and has good generalization ability to new data.

[0128] After a period of advanced knowledge learning, the performance of the temporary lesion annotation network has been significantly improved, and the server saves it as the target lesion annotation network.

[0129] In detail, the server saves the network parameters after advanced knowledge learning into the storage system and generates a target lesion annotation network, which not only contains the optimized weights and biases, but also reflects the server's fine adjustment of the network structure and clever use of the training step during the training process.

[0130] In a specific example, assume that the server sets the second training step of 500 for the basic image semantic feature extraction unit and the third training step of 1000 for the fully connected output unit in the advanced knowledge learning stage. Through phased training, the server found that the network's recognition accuracy on high-confidence lesion areas continued to improve, while the generalization ability on new data remained stable. Finally, the server generated a target lesion annotation network and verified its excellent performance in subsequent tests.

[0131] In another possible implementation, step S140 may further include:

[0132] Step A110, configuring target network learning data based on the basic network learning data, the first type of training three-dimensional medical scanning images, the second type of training three-dimensional medical scanning images, and the third type of training three-dimensional medical scanning images, wherein the target network learning data includes a three-dimensional medical scanning image, a lesion semantic knowledge point sequence, and a classification probability value distribution corresponding to the target lesion semantic knowledge point sequence.

[0133] Step A120, based on the following steps, the target network learning data is sequentially loaded into the basic lesion annotation network, and the basic lesion annotation network is subjected to cyclic knowledge learning to generate the target lesion annotation network. The three-dimensional medical scanning images configured for the current batch in the current round of knowledge learning are used as reference three-dimensional medical scanning images. Each knowledge learning stage uses the global target network learning data for knowledge learning, and each batch performs knowledge learning based on part of the three-dimensional medical scanning images in the target network learning data.

[0134] Step A130, loading the reference three-dimensional medical scan image into a basic image semantic feature extraction unit to generate target image semantic features, wherein the basic lesion annotation network includes the basic image semantic feature extraction unit.

[0135] Step A140, loading the target image semantic features into a fully connected output unit, generating reference lesion semantic knowledge points and reference classification probability values ​​corresponding to the reference lesion semantic knowledge points, the basic lesion annotation network including the fully connected output unit, the reference lesion semantic knowledge points being used to characterize the lesion semantic knowledge points annotated by the reference three-dimensional medical scan image estimated by the basic lesion annotation network.

[0136] Step A150, calculating a target learning error based on the lesion semantic knowledge point sequence corresponding to the reference three-dimensional medical scan image, the classification probability value distribution, the reference lesion semantic knowledge point and the reference classification probability value.

[0137] Step A160, updating the network parameter information of the basic lesion labeling network according to the target learning error until the target learning error meets the convergence requirement, thereby generating the target lesion labeling network.

[0138] In this embodiment, the server has collected and processed the basic network learning data, the first type of training 3D medical scan images (high confidence lesions), the second type of training 3D medical scan images (unique findings of the image classifier), and the third type of training 3D medical scan images (unique findings of the basic lesion annotation network). Now, the server will use these data to configure the target network learning data and perform cyclic knowledge learning on the basic lesion annotation network.

[0139] The server first needs to integrate and configure all training data into the format of target network learning data. Each data item contains a three-dimensional medical scan image, a sequence of lesion semantic knowledge points, and the corresponding classification probability value distribution.

[0140] In detail, the server traverses all training 3D medical scan images and assigns corresponding lesion semantic knowledge point sequences and classification probability value distributions to each training 3D medical scan image. These sequences and distributions are determined according to the image source (basic network learning data, first category, second category, third category) and annotation rules. Then, the integrated data is stored in the format of target network learning data for subsequent loading and learning.

[0141] The server starts to load the target network learning data in a loop and conducts knowledge learning on the basic lesion annotation network. The process is divided into multiple knowledge learning stages, and each stage contains multiple batches.

[0142] Specifically, at the beginning of each knowledge learning phase, the server loads the global target network learning data. Then, in each batch, the server randomly or sequentially selects a portion of 3D medical scan images from the global data as the training data for the current batch. Then, each 3D medical scan image in the current batch is used as a reference 3D medical scan image and sequentially loaded into the basic lesion annotation network.

[0143] Next, the server uses the basic image semantic feature extraction unit and the fully connected output unit in the basic lesion annotation network to process the reference 3D medical scan image to generate target image semantic features and reference lesion semantic knowledge points and their classification probability values.

[0144] Specifically, the reference 3D medical scan image can be loaded into the basic image semantic feature extraction unit, and the semantic features of the image can be extracted through operations such as convolution and pooling to generate the target image semantic features. Then, the target image semantic features are loaded into the fully connected output unit, and through structures such as the fully connected layer and activation function, a reference lesion semantic knowledge point sequence and its corresponding reference classification probability value are generated.

[0145] On this basis, the server calculates the target learning error according to the real lesion semantic knowledge point sequence and classification probability value distribution of the reference 3D medical scan image, as well as the reference lesion semantic knowledge points and reference classification probability values ​​generated by the network.

[0146] In detail, a loss function (such as cross entropy loss) is used to compare the difference between the real lesion semantic knowledge point sequence and classification probability value distribution and the reference lesion semantic knowledge point and reference classification probability value generated by the network, and the target learning error is calculated. According to the calculated target learning error, the server updates the parameter information of the basic lesion annotation network through the back propagation algorithm to reduce future learning errors.

[0147] Specifically, an optimization algorithm (such as SGD, Adam) can be used to perform gradient descent or similar update operations on network parameters according to the target learning error to adjust network parameters such as weights and biases.

[0148] Repeat the above process of loading data, extracting features, classifying output, calculating errors, and updating parameters until the target learning error meets the preset convergence requirements (such as the error value is less than a certain threshold, the error change for multiple consecutive rounds is less than a certain range, etc.). At this time, the network is considered to have converged and the target lesion annotation network is generated.

[0149] In a specific example, suppose that the server gradually optimizes the basic lesion annotation network through the above steps when processing a training data set containing lung CT scan images. In each knowledge learning stage, the server loads the global training data and randomly selects a portion of the images in each batch for processing. Through feature extraction and classification output, the server obtains the lesion semantic knowledge points and classification probability values ​​predicted by the network, and then compares them with the actual annotation information to calculate the learning error. According to the error value, the server continuously adjusts the parameters of the network until the error meets the convergence requirements. In the end, the server generates a target lesion annotation network with excellent performance, which can accurately annotate the lesion area in the image, providing strong support for the doctor's diagnosis.

[0150] In a possible implementation, after step S130, the method further includes:

[0151] Step B110, determining the number of images corresponding to each initial lesion semantic knowledge point in the initial lesion semantic knowledge point sequence based on the basic network learning data.

[0152] Step B120, outputting the initial lesion semantic knowledge points whose image quantity meets the set quantity condition as first initial lesion semantic knowledge points, and outputting the initial lesion semantic knowledge points whose image quantity does not meet the set quantity condition as second initial lesion semantic knowledge points.

[0153] Step B130, deleting the second category of training three-dimensional medical scanning images and the third category of training three-dimensional medical scanning images that are marked with the first initial lesion semantic knowledge point, and retaining the second category of training three-dimensional medical scanning images and the third category of training three-dimensional medical scanning images that are marked with the second initial lesion semantic knowledge point.

[0154] In this embodiment, the server has successfully generated the first, second and third types of training 3D medical scan images, which correspond to high-confidence lesion areas, lesion areas uniquely discovered by the image classifier and with low confidence by the basic network, and lesion areas uniquely discovered by the basic network and not identified by the image classifier. Now, the server will further screen and optimize the initial lesion semantic knowledge points based on the basic network learning data to ensure the balance and effectiveness of the training data.

[0155] The server first analyzes the basic network learning data and counts the number of images that appear in the dataset for each initial lesion semantic knowledge point.

[0156] Specifically, the server traverses all 3D medical scan images in the basic network learning data and records the lesion semantic knowledge points annotated in each image. For each initial lesion semantic knowledge point, the server counts the number of images that appear in the dataset and saves it in the corresponding data structure (such as a dictionary or database table).

[0157] Next, the server screens the initial lesion semantic knowledge points according to the set quantity conditions (such as the minimum image quantity threshold), and classifies those that meet the conditions as first initial lesion semantic knowledge points, and those that do not meet the conditions as second initial lesion semantic knowledge points.

[0158] In detail, the server sets a reasonable minimum image quantity threshold according to the training requirements and the characteristics of the data set. For each initial lesion semantic knowledge point, the server checks whether the corresponding image quantity meets the minimum threshold requirement. If it does, the initial lesion semantic knowledge point is divided into the first initial lesion semantic knowledge point; if it does not, it is divided into the second initial lesion semantic knowledge point.

[0159] Based on the screening results, the server optimizes the previously generated second and third category training 3D medical scanning images and deletes those images with too few labeled images (i.e., the second initial lesion semantic knowledge points) to maintain the balance and effectiveness of the training data.

[0160] For example, the server traverses the second and third types of training three-dimensional medical scan images, and checks whether the lesion semantic knowledge point annotated on each image belongs to the first initial lesion semantic knowledge point.

[0161] For images annotated with the second initial lesion semantic knowledge point, the server deletes them from the corresponding training data set. These images may not provide enough information to effectively train the network because the corresponding lesion semantic knowledge points appear too few times in the basic network learning data. The server retains the images annotated with the first initial lesion semantic knowledge point, which will be used in the subsequent knowledge learning process to optimize the performance of the basic lesion annotation network.

[0162] In a specific example, suppose that when the server is processing a CT scan image dataset containing a variety of lung lesions, it is found that the lesion semantic knowledge point "pulmonary nodule" has a large amount of image support in the basic network learning data, while "pulmonary atelectasis" has relatively few. According to the set minimum image number threshold, the server classifies "pulmonary nodule" as the first initial lesion semantic knowledge point, and "pulmonary atelectasis" as the second initial lesion semantic knowledge point. Subsequently, the server checked the second and third categories of training three-dimensional medical scan images generated previously, and deleted all images labeled "pulmonary atelectasis" because the lesion semantic knowledge points corresponding to these images were insufficiently supported in the basic data. In the end, the server retained a more balanced and effective training dataset for subsequent knowledge learning of the basic lesion annotation network.

[0163] In a possible implementation, before the sample three-dimensional medical scan image is loaded into the basic lesion annotation network for preliminary knowledge learning to generate the second lesion semantic knowledge point sequence, the method further includes:

[0164] Step C110, loading the three-dimensional medical scanning image to be learned in the basic network learning data into the basic lesion annotation network, generating a target estimation possibility, wherein the target estimation possibility is used to characterize the possibility degree of the basic lesion annotation network estimating whether the three-dimensional medical scanning image to be learned is annotated with reference lesion semantic knowledge points, and the three-dimensional medical scanning image to be learned is pre-annotated with whether the reference lesion semantic knowledge points are annotated.

[0165] Step C120, obtaining a threshold value sequence, wherein a threshold value in the threshold value sequence is used to generate a target labeling determination result in collaboration with the target estimation possibility, and the target labeling determination result is used to characterize whether the three-dimensional medical scan image to be learned is labeled with the reference lesion semantic knowledge point.

[0166] Step C130, based on the target estimation possibility and the threshold value sequence, determine the target threshold value of the reference lesion semantic knowledge point, the target threshold value is the maximum balance evaluation value in the threshold value sequence, and the balance evaluation value is collaboratively generated by the target annotation judgment result and the label data of whether the three-dimensional medical scan image to be learned is annotated with the reference lesion semantic knowledge point.

[0167] Step C140, defining the target threshold value as a target network parameter of the basic lesion annotation network, wherein the target threshold value is used to determine the second lesion semantic knowledge point sequence in the basic lesion annotation network.

[0168] In this embodiment, the server has prepared a series of basic network learning data, which includes a large number of 3D medical scan images, and each 3D medical scan image is pre-labeled to indicate whether it contains specific reference lesion semantic knowledge points (such as lung nodules, pneumonia, etc.). Now, the server will use these data to tune the basic lesion annotation network, especially to determine a suitable threshold value, so that it can more accurately determine whether it contains specific lesion semantic knowledge points when processing new images in the future.

[0169] The server first selects a portion of the 3D medical scan images to be learned from the basic network learning data. These 3D medical scan images to be learned have been pre-labeled to indicate whether they contain reference lesion semantic knowledge points. Then, the server loads these 3D medical scan images to be learned into the basic lesion annotation network to generate the corresponding target estimation probability.

[0170] In detail, the server selects the 3D medical scan images to be learned from the basic network learning data according to a preset sampling strategy (such as random sampling, stratified sampling, etc.). The selected 3D medical scan images to be learned are sequentially loaded into the basic lesion annotation network, and the basic lesion annotation network processes each 3D medical scan image to be learned to generate a corresponding target estimation probability, which reflects the probability that the basic lesion annotation network believes that the 3D medical scan image to be learned contains the reference lesion semantic knowledge point.

[0171] The server has a predefined sequence of threshold values, which are used to compare with the estimated probability of the target to generate the target labeling decision result. The decision result will be used to evaluate the effect of different threshold values ​​later.

[0172] In detail, the server generates a threshold value sequence based on experience or pre-set rules (such as equal interval division, division based on specific statistics, etc.). For each 3D medical scan image to be learned, the server compares its target estimation probability with each threshold value in the threshold value sequence. If the probability value is greater than the threshold value, it is determined that the 3D medical scan image to be learned contains the reference lesion semantic knowledge point; otherwise, it is determined that it does not contain it. In this way, for each 3D medical scan image to be learned and each threshold value, the server can generate a target labeling judgment result.

[0173] The server needs to find an optimal threshold value so that the basic lesion annotation network can achieve the best performance when judging whether the image contains the reference lesion semantic knowledge point. The optimal threshold value will be determined based on the target annotation judgment result and the real label data of the image.

[0174] In detail, for each threshold value in the threshold value sequence, the server calculates its corresponding balanced evaluation value. The balanced evaluation value usually takes into account the accuracy (such as true positive rate, false positive rate, etc.) and stability (such as performance consistency on different data sets) of the judgment result, and is quantified in some way (such as F1 score, area under ROC curve, etc.).

[0175] The server traverses the threshold value sequence and finds the threshold value with the largest balance evaluation value as the target threshold value, which performs best in terms of accuracy and stability of the balance determination result.

[0176] Once the target threshold is determined, the server sets it as an important parameter of the basic lesion annotation network. This parameter will play a key role in the subsequent processing of new images, helping the network to more accurately determine whether the image contains specific lesion semantic knowledge points.

[0177] Specifically, the server saves the target threshold value to the configuration file of the basic lesion annotation network, or directly sets it to the corresponding parameters of the network through the programming interface. After completing the parameter setting, the basic lesion annotation network is ready to process new three-dimensional medical scan images and generate a second lesion semantic knowledge point sequence.

[0178] In a specific example, suppose that when the server is processing a basic lesion annotation network for lung nodule detection, it first uses a portion of the images in the basic network learning data to tune the network. The server selected 1,000 images that were pre-labeled as to whether they contained lung nodules as the three-dimensional medical scan images to be learned, and loaded them into the network to generate target estimation possibilities. Then, the server used a sequence of 10 different threshold values ​​and generated corresponding target annotation judgment results for each threshold value. By calculating the balanced evaluation value of each threshold value (assuming that the F1 score is used as the evaluation indicator), the server found the threshold value with the largest balanced evaluation value as the target threshold value. Finally, the server sets this target threshold value as an important parameter of the network and prepares to use the tuned network to process new three-dimensional medical scan images.

[0179] In a possible implementation, after step S140, the method further includes:

[0180] Step S150 , obtaining a target medical examination data stream, and performing three-dimensional medical image extraction on the target medical examination data stream to generate a target three-dimensional medical scan image.

[0181] Step S160, loading the target three-dimensional medical scan image into the target lesion annotation network, generating a target lesion semantic knowledge point sequence and a target classification probability value sequence corresponding to the target lesion semantic knowledge point sequence.

[0182] Step S170, annotating target lesion annotation data for the target medical examination data stream based on the target lesion semantic knowledge point sequence and the target classification probability value sequence, wherein the target lesion annotation data represents the lesion semantic knowledge points annotated by the target medical examination data stream.

[0183] In this embodiment, the server has successfully used the basic network learning data, the first, second and third types of training 3D medical scan images to learn the basic lesion annotation network and generated a target lesion annotation network with excellent performance. Now, the server receives a new target medical examination data stream, which contains a series of unlabeled target 3D medical images. The server needs to use the target lesion annotation network to automatically annotate these target 3D medical images.

[0184] In detail, the server receives a target medical examination data stream from a medical imaging device or a database. The target medical examination data stream may contain various information such as the patient's personal information, examination time, examination type, etc., but the server is mainly concerned with the three-dimensional medical imaging data therein.

[0185] In detail, the server receives the target medical examination data stream through a network interface and stores it in a local or remote storage system. The server parses the target medical examination data stream and extracts three-dimensional medical image data from it. The three-dimensional medical image data can be original DICOM files, image slices that have been preliminarily processed, or image files in other formats. The extracted three-dimensional medical images can be pre-processed as needed, such as denoising, contrast enhancement, normalization, etc., to improve the accuracy of subsequent annotation.

[0186] The server takes the extracted and preprocessed (if necessary) target 3D medical image as input and loads it into the trained target lesion annotation network. The server loads the target lesion annotation network into memory and prepares for inference calculation. Then, the target 3D medical scan image is input into the network, and the target lesion annotation network processes and analyzes the target 3D medical scan image. After the target lesion annotation network processes the input target 3D medical scan image, it outputs a series of possible lesion semantic knowledge points and their corresponding classification probability values.

[0187] Among them, the basic image semantic feature extraction unit in the target lesion annotation network first extracts features from the target 3D medical scan image, and then the fully connected output unit classifies according to the extracted features to generate a target lesion semantic knowledge point sequence and a target classification probability value sequence. The server receives the network output results and saves them in an appropriate data structure for subsequent use.

[0188] The server generates corresponding lesion annotation data for the target medical examination data stream based on the target lesion semantic knowledge point sequence and the target classification probability value sequence, and these data will be used for subsequent medical diagnosis or research. In detail, the server generates annotation data for each image in the target medical examination data stream based on each lesion semantic knowledge point and its classification probability value. The annotation data may include information such as the type, location, size, and severity of the lesion. The server integrates the generated annotation data with the original medical examination data stream to form a medical examination report or data set containing complete annotation information. The server outputs the integrated results to a user interface, database, or other designated location for use by doctors or researchers.

[0189] In a specific example, suppose that the server receives a target medical examination data stream from a lung CT scan. The server first extracts the three-dimensional medical image data and performs necessary preprocessing on the image. Then, the server loads these images into the target lesion annotation network, and the network outputs possible lesion semantic knowledge points (such as lung nodules, pneumonia, etc.) and their corresponding classification probability values. Finally, based on these output results, the server generates detailed lesion annotation data for the target medical examination data stream, including information such as the location, size, and type of each lesion, and integrates this information into the medical examination report for doctors to conduct further diagnosis and analysis.

[0190] Figure 2 A three-dimensional medical image annotation service system 100 provided in an embodiment of the present application is shown, including a processor 1001 and a memory 1003 and a program code stored in the memory 1003. The processor 1001 executes the above program code to implement the steps of the AI-based three-dimensional medical image intelligent annotation method.

[0191] Figure 2 The three-dimensional medical image annotation service system 100 shown includes: a processor 1001 and a memory 1003. The processor 1001 and the memory 1003 are connected, such as through a bus 1002. Optionally, the three-dimensional medical image annotation service system 100 may also include a transceiver 1004, which can be used for data interaction between the three-dimensional medical image annotation service system and other three-dimensional medical image annotation service systems, such as data transmission and / or data reception. It should be noted that the transceiver 1004 is not limited to one in actual scheduling, and the structure of the three-dimensional medical image annotation service system 100 does not constitute a limitation on the embodiments of the present application.

[0192] Processor 1001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 1001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0193] The bus 1002 may include a path to transmit information between the above components. The bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 1002 may be divided into an address bus, a data bus, a control bus, etc.

[0194] The memory 1003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium, other magnetic storage devices, or any other medium that can be used to have or store program code and can be read by a computer, without limitation herein.

[0195] The memory 1003 is used to store program codes for executing the embodiments of the present application, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the program codes stored in the memory 1003 to implement the steps shown in the above method embodiments.

[0196] An embodiment of the present application provides a computer-readable storage medium having program code stored thereon. When the program code is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0197] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order represented by the arrows. Unless there is a clear explanation in this article, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders based on demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages according to the actual implementation scenario, and some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios at the execution time, the execution order of these sub-steps or stages can be flexibly configured based on demand, and the embodiment of the present application does not limit this.

[0198] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the scheme of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. A three-dimensional medical image intelligent annotation method based on AI, characterized in that: The method comprises: Obtain sample three-dimensional medical scan images; Loading the sample three-dimensional medical scan image into the image classifier of preliminary knowledge learning to generate a first lesion semantic knowledge point sequence, and loading the sample three-dimensional medical scan image into the basic lesion annotation network of preliminary knowledge learning to generate a second lesion semantic knowledge point sequence, wherein the basic lesion annotation network is used to represent the neural network generated by knowledge learning based on the basic network learning data, and the image classifier is used to represent the neural network generated by knowledge learning based on the general medical image library; Generate a first type of training three-dimensional medical scan images, a second type of training three-dimensional medical scan images, and a third type of training three-dimensional medical scan images based on the first lesion semantic knowledge point sequence and the second lesion semantic knowledge point sequence, wherein the first lesion semantic knowledge point annotated on the first type of training three-dimensional medical scan images is located both in the first lesion semantic knowledge point sequence and in the second lesion semantic knowledge point sequence, the second lesion semantic knowledge point annotated on the second type of training three-dimensional medical scan images is located in the first lesion semantic knowledge point sequence and is not located in the second lesion semantic knowledge point sequence, and the third lesion semantic knowledge point annotated on the third type of training three-dimensional medical scan images is located in the second lesion semantic knowledge point sequence and is not located in the first lesion semantic knowledge point sequence; The basic lesion annotation network is subjected to knowledge learning based on the basic network learning data, the first type of training three-dimensional medical scanning images, the second type of training three-dimensional medical scanning images, and the third type of training three-dimensional medical scanning images to generate a target lesion annotation network.

2. The AI-based 3D medical image intelligent annotation method according to claim 1, characterized in that: The generating of a first type of training three-dimensional medical scanning images, a second type of training three-dimensional medical scanning images, and a third type of training three-dimensional medical scanning images based on the first lesion semantic knowledge point sequence and the second lesion semantic knowledge point sequence comprises: Determine the first lesion semantic knowledge point, the second lesion semantic knowledge point, and the third lesion semantic knowledge point based on the first lesion semantic knowledge point sequence and the second lesion semantic knowledge point sequence; The classification probability value of the first lesion semantic knowledge point is defined as a first predefined value, and the first type of training three-dimensional medical scan images is generated, where the first type of training three-dimensional medical scan images is used to represent the classification probability value of the first lesion semantic knowledge point annotated on the sample three-dimensional medical scan image as the first predefined value; The classification probability value of the second lesion semantic knowledge point is defined as a first estimated possibility, and the second type of training three-dimensional medical scan images is generated, wherein the first estimated possibility is used to characterize the degree of possibility of the basic lesion annotation network estimating whether the sample three-dimensional medical scan image is annotated with the second lesion semantic knowledge point, and the second type of training three-dimensional medical scan images is used to characterize the classification probability value of the sample three-dimensional medical scan image annotating the second lesion semantic knowledge point as the first estimated possibility; The classification probability value of the third lesion semantic knowledge point is defined as a second estimated possibility, and the third type of training three-dimensional medical scanning images is generated. The second estimated possibility is less than the degree of possibility estimated by the basic lesion annotation network whether the sample three-dimensional medical scanning image is annotated with the third lesion semantic knowledge point. The classification probability value of the third type of training three-dimensional medical scanning images used to characterize the sample three-dimensional medical scanning images annotating the third lesion semantic knowledge point is the second estimated possibility.

3. The AI-based 3D medical image intelligent annotation method according to claim 2, characterized in that: The step of defining the classification probability value of the third lesion semantic knowledge point as a second estimated possibility and generating the third type of training three-dimensional medical scan image comprises: Obtaining the possibility degree of whether the sample three-dimensional medical scan image is annotated with the third lesion semantic knowledge point estimated by the basic lesion annotation network; The product of the degree of possibility and a second predefined numerical value is output as the second estimated possibility, the classification probability value of the third lesion semantic knowledge point is defined as the second estimated possibility, and the third type of training three-dimensional medical scanning image is generated, and the second predefined numerical value is a value greater than 0 and less than 1 / x.

4. The AI-based 3D medical image intelligent annotation method according to claim 1, characterized in that: The step of performing knowledge learning on the basic lesion annotation network based on the basic network learning data, the first type of training three-dimensional medical scanning images, the second type of training three-dimensional medical scanning images, and the third type of training three-dimensional medical scanning images to generate a target lesion annotation network includes: Performing knowledge learning on the basic lesion annotation network according to the basic network learning data, the first type of training three-dimensional medical scanning images, the second type of training three-dimensional medical scanning images, and the third type of training three-dimensional medical scanning images to generate a temporary lesion annotation network, wherein the training step length configured for the generated temporary lesion annotation network is the first training step length; Advanced knowledge learning is performed on the temporary lesion annotation network based on the basic network learning data and the first type of training three-dimensional medical scan images to generate the target lesion annotation network. The training step size configured for the generated target lesion annotation network is the second training step size, and the second training step size is smaller than the first training step size.

5. The AI-based 3D medical image intelligent annotation method according to claim 4, characterized in that: The step of performing advanced knowledge learning on the temporary lesion annotation network based on the basic network learning data and the first type of training three-dimensional medical scan images to generate the target lesion annotation network includes: When the temporary lesion annotation network includes a basic image semantic feature extraction unit and a fully connected output unit, a second training step size is set for the basic image semantic feature extraction unit, and a third training step size is set for the fully connected output unit, wherein the third training step size is greater than the first training step size; Advanced knowledge learning is performed on the temporary lesion labeling network according to the second training step size and the third training step size to generate the target lesion labeling network.

6. The AI-based 3D medical image intelligent annotation method according to claim 1, characterized in that: The step of performing knowledge learning on the basic lesion annotation network based on the basic network learning data, the first type of training three-dimensional medical scanning images, the second type of training three-dimensional medical scanning images, and the third type of training three-dimensional medical scanning images to generate a target lesion annotation network includes: Target network learning data is configured based on the basic network learning data, the first type of training three-dimensional medical scanning images, the second type of training three-dimensional medical scanning images, and the third type of training three-dimensional medical scanning images, wherein one target network learning data includes a three-dimensional medical scanning image, a lesion semantic knowledge point sequence, and a classification probability value distribution corresponding to the target lesion semantic knowledge point sequence; Based on the following steps, the target network learning data is sequentially loaded into the basic lesion annotation network, and the basic lesion annotation network is cyclically learned to generate the target lesion annotation network. The three-dimensional medical scan images configured in the current batch in the current round of knowledge learning are used as reference three-dimensional medical scan images. Each knowledge learning stage uses the global target network learning data for knowledge learning, and each batch performs knowledge learning based on part of the three-dimensional medical scan images in the target network learning data. Loading the reference three-dimensional medical scan image into a basic image semantic feature extraction unit to generate target image semantic features, wherein the basic lesion annotation network includes the basic image semantic feature extraction unit; The target image semantic features are loaded into a fully connected output unit to generate reference lesion semantic knowledge points and reference classification probability values ​​corresponding to the reference lesion semantic knowledge points, wherein the basic lesion annotation network includes the fully connected output unit, and the reference lesion semantic knowledge points are used to characterize the lesion semantic knowledge points annotated by the reference three-dimensional medical scan image estimated by the basic lesion annotation network; Calculating a target learning error based on the one lesion semantic knowledge point sequence corresponding to the reference three-dimensional medical scan image, the classification probability value distribution, and the reference lesion semantic knowledge point and the reference classification probability value; The network parameter information of the basic lesion labeling network is updated according to the target learning error until the target learning error meets the convergence requirement, thereby generating the target lesion labeling network.

7. The AI-based 3D medical image intelligent annotation method according to claim 1, characterized in that: After generating the first type of training three-dimensional medical scan images, the second type of training three-dimensional medical scan images, and the third type of training three-dimensional medical scan images based on the first lesion semantic knowledge point sequence and the second lesion semantic knowledge point sequence, the method further includes: Determine the number of images corresponding to each initial lesion semantic knowledge point in the initial lesion semantic knowledge point sequence based on the basic network learning data; Outputting the initial lesion semantic knowledge points whose image quantity meets the set quantity condition as first initial lesion semantic knowledge points, and outputting the initial lesion semantic knowledge points whose image quantity does not meet the set quantity condition as second initial lesion semantic knowledge points; The second category of training three-dimensional medical scanning images and the third category of training three-dimensional medical scanning images that are marked with the first initial lesion semantic knowledge point are deleted, and the second category of training three-dimensional medical scanning images and the third category of training three-dimensional medical scanning images that are marked with the second initial lesion semantic knowledge point are retained.

8. The AI-based 3D medical image intelligent annotation method according to claim 1, characterized in that: Before the sample three-dimensional medical scan image is loaded into the basic lesion annotation network for preliminary knowledge learning to generate a second lesion semantic knowledge point sequence, the method further includes: Loading the to-be-learned three-dimensional medical scan image in the basic network learning data into the basic lesion annotation network to generate a target estimation probability, wherein the target estimation probability is used to characterize the probability degree of the basic lesion annotation network estimating whether the to-be-learned three-dimensional medical scan image is annotated with a reference lesion semantic knowledge point, and the to-be-learned three-dimensional medical scan image is pre-annotated with whether the reference lesion semantic knowledge point is annotated; Acquire a threshold value sequence, wherein a threshold value in the threshold value sequence is used to generate a target labeling determination result in collaboration with the target estimation possibility, and the target labeling determination result is used to indicate whether the three-dimensional medical scan image to be learned is labeled with the reference lesion semantic knowledge point; Based on the target estimation possibility and the threshold value sequence, determining the target threshold value of the reference lesion semantic knowledge point, the target threshold value is the maximum balanced evaluation value in the threshold value sequence, and the balanced evaluation value is collaboratively generated by the target annotation determination result and the label data of whether the three-dimensional medical scan image to be learned is annotated with the reference lesion semantic knowledge point; The target threshold value is defined as a target network parameter of the basic lesion annotation network, and the target threshold value is used to determine the second lesion semantic knowledge point sequence in the basic lesion annotation network.

9. The AI-based 3D medical image intelligent annotation method according to claim 1, characterized in that: After performing knowledge learning on the basic lesion annotation network based on the basic network learning data, the first type of training three-dimensional medical scan images, the second type of training three-dimensional medical scan images, and the third type of training three-dimensional medical scan images to generate a target lesion annotation network, the method further includes: Acquire a target medical examination data stream, and perform three-dimensional medical image extraction on the target medical examination data stream to generate a target three-dimensional medical scan image; Loading the target three-dimensional medical scan image into the target lesion annotation network to generate a target lesion semantic knowledge point sequence and a target classification probability value sequence corresponding to the target lesion semantic knowledge point sequence; The target lesion annotation data is annotated for the target medical examination data stream based on the target lesion semantic knowledge point sequence and the target classification probability value sequence, wherein the target lesion annotation data represents the lesion semantic knowledge points annotated by the target medical examination data stream.

10. A three-dimensional medical image annotation service system, characterized in that: It includes a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by the processor, the AI-based three-dimensional medical image intelligent annotation method described in any one of claims 1 to 9 is implemented.

Images