Cancer detection method and apparatus based on medical image, and device, medium and product

Abstract

The present invention relates to the technical field of image recognition. Provided are a cancer detection method and apparatus based on a medical image, and a device, a medium and a product. The method comprises: inputting a medical image into an image classification model, so as to obtain an image classification result that is output by the image classification model, wherein the image classification result is used for representing a recognition result of performing cancer identification on the medical image; inputting the medical image into a semantic segmentation model, so as to obtain a semantic segmentation result that is output by the semantic segmentation model, wherein the semantic segmentation result is used for indicating whether there is a lesion region in the medical image; on the basis of the semantic segmentation result and the image classification result, determining whether it is necessary to perform detection on a corresponding organ part in the medical image; and when it is necessary to perform detection on the corresponding organ part in the medical image, outputting detection prompt information. The present invention can realize high-efficiency, high-sensitivity, and high-accuracy cancer detection.

Description

Cancer detection methods, devices, equipment, media, and products based on medical images

[0001] Cross-referencing

[0002] This application claims priority to Chinese Patent Application No. 2024119490839, filed on December 27, 2024, entitled “Cancer Detection Method, Apparatus, Device, Media and Product Based on Medical Images”, the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] This invention relates to the field of image recognition technology, and in particular to a method, apparatus, device, medium and product for cancer detection based on medical images. Background Technology

[0004] With the rapid development of artificial intelligence, its application in medical devices is becoming increasingly widespread. AI-based medical image recognition can assist in the diagnosis of tumors and cancers, improving the accuracy of these diagnoses.

[0005] Currently, medical images are analyzed for object detection. Based on the detection results, the presence of lesion areas is determined, specifically by outlining suspected cancerous lesions with rectangular boxes to assist doctors in further diagnosis. However, the resolution of acquired medical images varies; some are captured with close-up lenses, while others are captured with distant lenses. This reduces the accuracy of object detection using a single model, potentially leading to missed detections. In other words, the sensitivity of current cancer identification is not high. Furthermore, object detection uses rectangular boxes to select lesion areas, but lesion areas may not be rectangular. Therefore, the accuracy of lesion localization is not high, requiring doctors to manually determine the lesion area within the rectangular box. This means that current organ cancer detection is not very efficient. Summary of the Invention

[0006] This invention provides a cancer detection method, device, equipment, medium, and product based on medical images to address the shortcomings of low sensitivity and efficiency in cancer identification in existing technologies, and to achieve efficient and highly sensitive cancer detection.

[0007] This invention provides a cancer detection method based on medical images, comprising:

[0008] A medical image is input into an image classification model to obtain the image classification result output by the image classification model; the image classification result is used to characterize the cancer identification result of the medical image;

[0009] The medical image is input into a semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion region in the medical image.

[0010] Based on the semantic segmentation results and the image classification results, it is determined whether it is necessary to detect the organ parts corresponding to the medical image;

[0011] When it is necessary to detect the organ or part corresponding to the medical image, a detection prompt message is output.

[0012] According to the present invention, a cancer detection method based on medical images is provided, wherein the image classification result includes a first classification result or a second classification result, and the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result.

[0013] The step of determining whether to detect the organ region corresponding to the medical image based on the semantic segmentation result and the image classification result includes:

[0014] If, based on the semantic segmentation result, it is determined that there is a lesion region in the medical image, and the image classification result is the first classification result, it is determined that the organ part corresponding to the medical image needs to be detected.

[0015] If, based on the semantic segmentation result, it is determined that there is a lesion region in the medical image, and the image classification result is the second classification result, it is determined that it is not necessary to detect the organ part corresponding to the medical image;

[0016] If, based on the semantic segmentation results, it is determined that there are no lesion regions in the medical image, it is determined that there is no need to detect the organ parts corresponding to the medical image.

[0017] According to a cancer detection method based on medical images provided by the present invention, the output detection prompt information includes:

[0018] Based on the semantic segmentation results and the image classification results, detection prompt information is generated; the detection prompt information includes the lesion region in the medical image, the cancer identification result indicated by the image classification results, and auxiliary diagnostic suggestions;

[0019] Output the detection prompt information.

[0020] According to a cancer detection method based on medical images provided by the present invention, before inputting the medical image into an image classification model and obtaining the image classification result output by the image classification model, the method further includes:

[0021] The medical image is input into the feature extraction model to obtain the image features output by the feature extraction model;

[0022] Accordingly, the step of inputting the medical image into the image classification model and obtaining the image classification result output by the image classification model includes:

[0023] The image features are input into the image classification model to obtain the image classification result output by the image classification model;

[0024] Accordingly, the step of inputting the medical image into the semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model includes:

[0025] The image features are input into the semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model.

[0026] According to the cancer detection method based on medical images provided by the present invention, the semantic segmentation model is an integrated semantic segmentation model, which includes multiple different segmentation models;

[0027] For any pixel in the medical image, the integrated semantic segmentation model is used to determine multiple category prediction results for the pixel. Each category prediction result includes a lesion pixel or a non-lesion pixel. If any category prediction result among the multiple category prediction results is a lesion pixel, then the final category prediction result of the pixel is determined to be a lesion pixel. If all category prediction results among the multiple category prediction results are non-lesion pixels, then the final category prediction result of the pixel is determined to be a non-lesion pixel.

[0028] According to the present invention, a cancer detection method based on medical images is provided, wherein the image classification result includes a first classification result or a second classification result, and the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result.

[0029] Before determining whether it is necessary to detect the organ region corresponding to the medical image based on the semantic segmentation result and the image classification result, the method further includes:

[0030] If, based on the semantic segmentation result, it is determined that there is no lesion region in the medical image, and the image classification result is the first classification result, the segmentation threshold of the semantic segmentation model is adjusted to obtain and output the lesion region in the medical image;

[0031] Specifically, for any pixel in the medical image, the semantic segmentation model determines the category prediction result of the pixel based on the segmentation threshold, and the category prediction result of the pixel includes lesion pixels or non-lesion pixels.

[0032] According to a cancer detection method based on medical images provided by the present invention, adjusting the segmentation threshold of the semantic segmentation model includes:

[0033] The lowest segmentation threshold is set to 0, and the highest segmentation threshold is set as the current segmentation threshold of the semantic segmentation model.

[0034] Determine the sum of the lowest segmentation threshold and the highest segmentation threshold, and adjust the segmentation threshold of the semantic segmentation model to half of the sum.

[0035] The semantic segmentation result is updated based on the adjusted segmentation threshold;

[0036] If, based on the updated semantic segmentation results, it is determined that the proportion of the lesion region in the medical image is greater than the first preset proportion, the minimum segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the minimum segmentation threshold and the maximum segmentation threshold and adjusting the segmentation threshold of the semantic segmentation model to half of the sum is returned.

[0037] If, based on the updated semantic segmentation results, it is determined that the proportion of the lesion region in the medical image is less than the second preset proportion, the highest segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the lowest segmentation threshold and the highest segmentation threshold and adjusting the segmentation threshold of the semantic segmentation model to half of the sum is returned.

[0038] Specifically, for any pixel in the medical image, the semantic segmentation model outputs a predicted value for the pixel. If the predicted value of the pixel is greater than or equal to the segmentation threshold, the category prediction result of the pixel is determined to be a lesion pixel. The proportion of the lesion region is determined based on the ratio of the lesion region in the medical image to the area of ​​the medical image, and the first preset proportion is greater than the second preset proportion.

[0039] According to a cancer detection method based on medical images provided by the present invention, the method further includes:

[0040] The medical image is input into the organ location recognition model to obtain the organ location recognition result output by the organ location recognition model;

[0041] Output the prompt information corresponding to the organ location identification result to indicate the currently examined organ location;

[0042] The organ part recognition model is trained based on the sample medical image and the corresponding organ part recognition result label of the sample medical image.

[0043] The present invention also provides a cancer detection device based on medical images, comprising:

[0044] An image classification module is used to input medical images into an image classification model and obtain the image classification results output by the image classification model; the image classification results are used to characterize the cancer identification results of the medical images;

[0045] A semantic segmentation module is used to input the medical image into a semantic segmentation model and obtain the semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion region in the medical image;

[0046] The detection determination module is used to determine, based on the semantic segmentation result and the image classification result, whether it is necessary to detect the organ part corresponding to the medical image;

[0047] The prompt output module is used to output detection prompt information when it is necessary to detect the organ part corresponding to the medical image.

[0048] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the cancer detection method based on medical images as described above.

[0049] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cancer detection method based on medical images as described above.

[0050] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the cancer detection method based on medical images as described above.

[0051] This invention provides a cancer detection method, apparatus, device, medium, and product based on medical images. The method inputs a medical image into an image classification model to obtain the image classification result output by the model. It also inputs the medical image into a semantic segmentation model to obtain the semantic segmentation result output by the model. The image classification result characterizes the cancer identification result of the medical image, while the semantic segmentation result indicates the presence and location of lesion regions in the medical image. Therefore, fusing semantic segmentation and image classification can improve the accuracy and sensitivity of cancer identification. Furthermore, the image classification result and the semantic segmentation result... If any result indicates the presence of cancer, a preliminary diagnosis can be made, thereby improving the sensitivity of cancer identification, ensuring a certain biopsy positivity rate, and reducing false negatives. Furthermore, based on semantic segmentation and image classification results, it determines whether the organ corresponding to the medical image needs to be detected, assisting doctors in deciding whether to perform the detection, thus improving cancer detection efficiency and sensitivity, and reducing false negatives. Moreover, when detection of the organ corresponding to the medical image is necessary, detection prompts are output to encourage doctors to conduct further cancer detection, thereby improving cancer detection efficiency and sensitivity, and reducing false negatives. In summary, this invention can achieve efficient, highly sensitive, and highly accurate cancer detection. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0053] Figure 1 is one of the flowcharts of the cancer detection method based on medical images provided by the present invention.

[0054] Figure 2 is a second schematic flowchart of the cancer detection method based on medical images provided by the present invention.

[0055] Figure 3 is a schematic flowchart of the cancer detection method based on medical images provided by the present invention.

[0056] Figure 4 is a schematic flowchart of the cancer detection method based on medical images provided by the present invention.

[0057] Figure 5 is the fifth flowchart of the cancer detection method based on medical images provided by the present invention.

[0058] Figure 6 is a schematic diagram of the structure of the cancer detection device based on medical images provided by the present invention.

[0059] Figure 7 is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0061] Currently, medical images are analyzed for object detection. Based on the detection results, the presence of lesion areas is determined, specifically by outlining suspected cancerous lesions with rectangular boxes to assist doctors in further diagnosis. However, the resolution of acquired medical images varies; some are captured with close-up lenses, while others are captured with distant lenses. This reduces the accuracy of object detection using a single model, potentially leading to missed detections. In other words, the sensitivity of current cancer identification is not high. Furthermore, object detection uses rectangular boxes to select lesion areas, but lesion areas may not be rectangular. Therefore, the accuracy of lesion localization is not high, requiring doctors to manually determine the lesion area within the rectangular box. This means that current organ cancer detection is not very efficient.

[0062] For example, this method acquires real-time images of the stomach during electronic chromoendoscopy; identifies and labels lesion areas based on a pre-built early gastric cancer recognition model; and determines whether overlapping lesion areas belong to the same lesion area. This approach utilizes target detection technology to simultaneously achieve image recognition and location localization. However, the lesion area is selected using a rectangular bounding box, which is not precise enough for accurate location localization, as the lesion area may not be a rectangular region, requiring the doctor to make further judgments within the bounding box. Furthermore, using only one target detection model makes it difficult to apply simultaneously to both close-up and long-range imaging, resulting in insufficient sensitivity for early cancer recognition and many missed detections, causing economic losses or even life-threatening situations for patients.

[0063] Furthermore, image classification of medical images can help doctors determine whether an early-stage cancer is suspected, thus assisting in further diagnosis. However, cancer identification based solely on image classification results is not very accurate and may result in missed detections, meaning that current cancer identification is not very sensitive. Moreover, image classification cannot determine the lesion area, requiring doctors to manually locate the lesion area within the medical image, indicating that current cancer identification is not very efficient.

[0064] For example, a sample set of early gastric cancer images is obtained, and feature recognition is performed on each lesion image in the sample set to obtain the feature vector of each lesion image. Based on the sample set of early gastric cancer images and the feature vectors, a preset initial model is trained to obtain an early gastric cancer recognition model. However, this method only provides one way to recognize early gastric cancer and does not provide the function of locating the early cancer lesion.

[0065] To address the above problems, the present invention proposes the following embodiments. The cancer detection method based on medical images of the present invention is described below with reference to Figures 1-5.

[0066] Figure 1 is one of the flowcharts of the cancer detection method based on medical images provided by the present invention. As shown in Figure 1, the cancer detection method based on medical images includes the following steps 110, 120, 130 and 140.

[0067] Step 110: Input the medical image into the image classification model to obtain the image classification result output by the image classification model.

[0068] Here, the cancers to be detected can include, but are not limited to: digestive tract cancers (such as esophageal cancer, stomach cancer, colorectal cancer, etc.), lung cancer, liver cancer, and breast cancer, etc. Furthermore, it can identify early-stage, intermediate-stage, and late-stage cancers; for example, digestive tract cancers can be classified as early-stage digestive tract cancers and advanced-stage digestive tract cancers. This embodiment of the invention uses early-stage digestive tract cancer (early-stage malignant tumors of the digestive tract) as an example for illustration.

[0069] Here, medical images refer to images acquired from organs or parts of the body. For example, a digestive tract image (medical image) is an image acquired from the digestive tract; this image can be acquired using a digestive endoscope. As another example, a lung image (medical image) is an image acquired from the lungs; this image can be a CT (Computed Tomography) image.

[0070] For example, the digestive tract sites that can be captured by upper gastrointestinal endoscopy (gastroscopy) include: oropharynx, upper esophagus, middle esophagus, lower esophagus, esophagogastric junction, upper gastric body, posterior wall of upper gastric body, anterior wall of upper gastric body, greater curvature of upper gastric body, lesser curvature of upper gastric body, middle gastric body, posterior wall of middle gastric body, anterior wall of middle gastric body, greater curvature of middle gastric body, lesser curvature of middle gastric body, lower gastric body, posterior wall of lower gastric body, anterior wall of lower gastric body, greater curvature of lower gastric body, lesser curvature of lower gastric body, greater curvature of anterobody junction, gastric angle, posterior wall of gastric angle, anterior wall of gastric angle, gastric antrum, posterior wall of gastric antrum, anterior wall of gastric antrum, greater curvature of gastric antrum, lesser curvature of gastric antrum, pylorus, duodenal bulb, descending duodenum, gastric fundus, posterior wall of gastric fundus, anterior wall of gastric fundus, greater curvature of gastric fundus, lesser curvature of gastric fundus, and cardia. Colonoscopy can capture images of the following parts of the digestive tract: terminal ileum, appendiceal orifice, ileocecal valve, ascending colon, hepatic flexure of the colon, transverse colon, splenic flexure, descending colon, sigmoid colon, rectum, and colectomycorectal junction.

[0071] Furthermore, the acquired medical images undergo image preprocessing. In one embodiment, the acquired medical images are uniformly adjusted to a preset input size, for example, 512×512.

[0072] The image classification result is used to characterize the cancer identification result of the medical image. That is, the image classification model is used to identify cancer in the medical image. Specifically, the image classification model is used to classify the medical image into different cancer probabilities, thereby determining whether it is a suspected cancer.

[0073] In one embodiment, the image classification result includes a first classification result or a second classification result, where the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result. For example, the first classification result may indicate suspected cancer, while the second classification result may indicate no cancer.

[0074] In one embodiment, the image classification model is trained based on a sample medical image and the corresponding image classification result label.

[0075] Step 120: Input the medical image into the semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model.

[0076] The semantic segmentation result is used to indicate the presence and location of lesion regions in the medical image. The semantic segmentation result is pixel-level, meaning the semantic segmentation model determines the category prediction result for each pixel in the medical image, i.e., whether each pixel is a lesion pixel. This allows for the identification of irregular lesion regions, thus improving the accuracy of lesion region localization. Specifically, it helps doctors determine the location of lesions, thereby improving both the accuracy and efficiency of cancer identification.

[0077] Furthermore, this semantic segmentation model is an ensemble semantic segmentation model, which includes multiple different segmentation models. Compared to a single semantic segmentation model, the ensemble semantic segmentation model can improve the accuracy of semantic segmentation results, thereby improving the accuracy of cancer identification. Even further, the ensemble semantic segmentation model is composed of multiple single semantic segmentation models based on the bagging method.

[0078] In one embodiment, the semantic segmentation model is trained based on a sample medical image and the corresponding semantic segmentation result label.

[0079] Step 130: Based on the semantic segmentation result and the image classification result, determine whether it is necessary to detect the organ parts corresponding to the medical image.

[0080] It should be noted that when it is necessary to examine the organ or part corresponding to the medical image, it indicates that further cancer diagnosis is required.

[0081] It should be understood that image classification results are used to characterize the cancer identification results of medical images, while semantic segmentation results are used to indicate whether there are lesion areas in the medical images and to locate the location and range of the lesion areas. Therefore, the fusion of semantic segmentation and image classification can improve the accuracy of cancer identification. If either the image classification result or the semantic segmentation result indicates the presence of cancer, a preliminary judgment can be made that cancer is present, thereby improving the sensitivity of cancer identification, ensuring a certain biopsy positive rate, and reducing missed detections. Furthermore, based on the semantic segmentation result and the image classification result, it can be determined whether the corresponding organ part in the medical image needs to be detected, assisting doctors in judging whether detection is necessary. That is, it can assist doctors in further judging the diagnostic method based on the two results, thereby improving the efficiency of cancer detection.

[0082] In some embodiments, the image classification result includes a first classification result or a second classification result, wherein the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result. Based on this, step 130 has the following embodiments.

[0083] In one embodiment, if the semantic segmentation result determines that a lesion region exists in the medical image, and the image classification result is the first classification result, it is determined that the organ corresponding to the medical image needs to be detected; if the semantic segmentation result determines that a lesion region exists in the medical image, and the image classification result is the second classification result, it is determined that the organ corresponding to the medical image does not need to be detected; if the semantic segmentation result determines that no lesion region exists in the medical image, it is determined that the organ corresponding to the medical image does not need to be detected.

[0084] In another embodiment, if the semantic segmentation result determines that there is a lesion region in the medical image, or the image classification result is the first classification result, it is determined that the organ corresponding to the medical image needs to be detected; if the semantic segmentation result determines that there is no lesion region in the medical image, and the image classification result is the second classification result, it is determined that the organ corresponding to the medical image does not need to be detected.

[0085] It should be understood that if the semantic segmentation results indicate the presence of a lesion region in a medical image, or if the image classification result is a first-class classification result, it is determined that the corresponding organ in the medical image needs to be detected. That is, if either the image classification result or the semantic segmentation result indicates the presence of cancer, a preliminary judgment can be made that cancer exists, thus requiring detection of the corresponding organ in the medical image. This improves the sensitivity of cancer identification, ensures a certain biopsy positivity rate, and reduces false negatives. Conversely, if the semantic segmentation results indicate the absence of a lesion region in a medical image, and the image classification result is a second-class classification result, it is determined that the corresponding organ in the medical image does not need to be detected. That is, if both the image classification result and the semantic segmentation result indicate the absence of cancer, then it is determined that detection is not required. This improves the sensitivity of cancer identification, ensures a certain biopsy positivity rate, and reduces false negatives.

[0086] For example, as shown in Figure 2, the acquired medical image is preprocessed, and then features are extracted from the preprocessed medical image. These features are then input into a semantic segmentation model and an image classification model, respectively. Next, based on the semantic segmentation results, it is first determined whether a lesion region exists in the medical image. If a lesion region is found, the corresponding organ needs to be detected, and corresponding auxiliary diagnostic prompts are provided. If no lesion region is found, the image classification model's image classification results are used for further judgment. If the image classification result is a first classification result, the corresponding organ needs to be detected, and corresponding auxiliary diagnostic prompts are provided. If the semantic segmentation result determines that no lesion region exists in the medical image, and the image classification result is a second classification result, the corresponding organ does not need to be detected, and no prompts are provided.

[0087] Image classification models can output different probabilities. Based on this, if the probability output by the image classification model is greater than the auxiliary diagnostic probability threshold, the image classification result is the first classification result; if the probability output by the image classification model is less than or equal to the auxiliary diagnostic probability threshold, the image classification result is the second classification result. Therefore, the judgment logic for determining whether to detect the corresponding organ part in the medical image also differs depending on the different probabilities output by the image classification model.

[0088] For example, as shown in Figure 3, the acquired medical image is preprocessed, and then features are extracted from the preprocessed medical image. These features are then input into a semantic segmentation model and an image classification model, respectively. Next, based on the semantic segmentation results, it is first determined whether a lesion region exists in the medical image. If a lesion region is found, the corresponding organ needs to be detected, and an auxiliary diagnostic prompt is given accordingly. If no lesion region is found, the output probability of the image classification model is used for further judgment. If the output probability is greater than the auxiliary diagnostic probability threshold, the corresponding organ needs to be detected, and an auxiliary diagnostic prompt is given accordingly. If no lesion region is found and the output probability is not greater than the auxiliary diagnostic probability threshold, the corresponding organ does not need to be detected, and no prompt is given.

[0089] Step 140: If it is necessary to detect the organ or part corresponding to the medical image, output detection prompt information.

[0090] Here, the detection prompts are used to suggest further cancer detection, i.e., further cancer diagnosis. Furthermore, the detection prompts include auxiliary diagnostic suggestions (such as biopsy recommendations) to suggest further cancer diagnosis, such as further suggestion to perform a biopsy. Even further, the detection prompts include lesion areas in medical images, cancer identification results indicated by image classification results, and auxiliary diagnostic suggestions to provide more comprehensive prompts and improve cancer detection efficiency.

[0091] It should be understood that directly outputting detection prompts can prompt doctors to conduct further tests and assist them in making further diagnoses, that is, to help doctors determine whether further cancer testing is needed, thereby improving the efficiency of cancer detection.

[0092] The cancer detection method based on medical images provided in this invention inputs a medical image into an image classification model to obtain the image classification result output by the image classification model, and inputs the medical image into a semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model. The image classification result is used to characterize the cancer identification result of the medical image, and the semantic segmentation result is used to indicate whether there is a lesion region in the medical image. Therefore, the fusion of semantic segmentation and image classification can improve the accuracy of cancer identification, thereby improving the sensitivity of cancer identification. When either the image classification result or the semantic segmentation result indicates the presence of cancer, a preliminary judgment can be made that cancer is present, thereby improving the sensitivity of cancer identification, ensuring a certain biopsy positive rate, and reducing false negatives. Furthermore, based on the semantic segmentation result and the image classification result, it is determined whether the organ corresponding to the medical image needs to be detected, assisting the doctor in judging whether detection is necessary, thereby improving the efficiency of cancer detection, increasing the sensitivity of cancer identification, and reducing false negatives. When the organ corresponding to the medical image needs to be detected, detection prompt information is output to prompt the doctor to conduct further cancer detection, assisting the doctor in making further diagnoses, thereby improving the efficiency of cancer detection, increasing the sensitivity of cancer identification, and reducing false negatives. In summary, this invention can achieve efficient, highly sensitive, and highly accurate cancer detection.

[0093] Based on any of the above embodiments, the image classification result includes a first classification result or a second classification result, wherein the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result. For example, the first classification result is suspected cancer, and the second classification result is no cancer.

[0094] Accordingly, step 130 above includes:

[0095] If, based on the semantic segmentation result, it is determined that there is a lesion region in the medical image, and the image classification result is the first classification result, it is determined that the organ part corresponding to the medical image needs to be detected.

[0096] If, based on the semantic segmentation result, it is determined that there is a lesion region in the medical image, and the image classification result is the second classification result, it is determined that it is not necessary to detect the organ part corresponding to the medical image;

[0097] If, based on the semantic segmentation results, it is determined that there are no lesion regions in the medical image, it is determined that there is no need to detect the organ parts corresponding to the medical image.

[0098] Considering that it is only necessary to determine whether the corresponding organ or part of the medical image needs to be detected, it is only necessary to perform binary classification on the medical image, that is, the image classification model is a binary classification model.

[0099] Considering that semantic segmentation models are based on pixel-level semantic segmentation, meaning that semantic segmentation results are more sensitive, if it is determined from the semantic segmentation results that there are no lesion areas in the medical image, it can be directly determined that there is no need to detect the corresponding organ parts in the medical image.

[0100] For example, as shown in Figure 4, the acquired medical image undergoes image preprocessing, followed by feature extraction. The image features are then input into a semantic segmentation model and an image classification model. Next, based on the semantic segmentation results, it is determined whether a lesion region exists in the medical image. If no lesion region is found, detection of the corresponding organ is not required, and no prompt is given. If a lesion region is found, subsequent judgment is made based on the image classification results. If a lesion region is found and the image classification result is a first classification result, detection of the corresponding organ is required, and auxiliary diagnostic prompts are given. If a lesion region is found and the image classification result is a second classification result, detection of the corresponding organ is not required, and no prompt is given.

[0101] Image classification models can output different probabilities. Based on this, if the probability output by the image classification model is greater than the auxiliary diagnostic probability threshold, the image classification result is the first classification result; if the probability output by the image classification model is less than or equal to the auxiliary diagnostic probability threshold, the image classification result is the second classification result. Therefore, the judgment logic for determining whether to detect the corresponding organ part in the medical image also differs depending on the different probabilities output by the image classification model.

[0102] For example, as shown in Figure 5, the acquired medical image undergoes image preprocessing, followed by feature extraction. The image features are then input into a semantic segmentation model and an image classification model, respectively. Next, based on the semantic segmentation results, it is first determined whether a lesion region exists in the medical image. If no lesion region is found, detection of the corresponding organ is not required, and no prompt is given. If a lesion region is found, the output probability of the image classification model is used for subsequent judgment. If a lesion region is found and the output probability is greater than the auxiliary diagnostic probability threshold, detection of the corresponding organ is required, and an auxiliary diagnostic prompt is given. If a lesion region is found and the output probability is not greater than the auxiliary diagnostic probability threshold, detection of the corresponding organ is not required, and no prompt is given.

[0103] The cancer detection method based on medical images provided in this invention improves the accuracy of determining whether to detect the corresponding organ or part of the medical image by determining, as described above, thereby improving the accuracy and efficiency of cancer detection. Furthermore, it determines that detection of the corresponding organ or part of the medical image is necessary only when semantic segmentation results indicate the presence of a lesion region in the medical image and the image classification result is a first classification result, thus reducing false positives and improving the specificity of cancer identification. Even if semantic segmentation results indicate the presence of a lesion region in the medical image, as long as the image classification result is a second classification result, detection of the corresponding organ or part of the medical image is not required, thus reducing false positives and improving the specificity of cancer identification. Finally, if semantic segmentation results indicate the absence of a lesion region in the medical image, it is not necessary to determine the image classification result, thus reducing false positives and improving the specificity of cancer identification.

[0104] Based on any of the above embodiments, the output detection prompt information in this method includes:

[0105] Based on the semantic segmentation results and the image classification results, detection prompt information is generated; the detection prompt information includes the lesion region in the medical image, the cancer identification result indicated by the image classification results, and auxiliary diagnostic suggestions;

[0106] Output the detection prompt information.

[0107] Specifically, the system determines the lesion region in the medical image based on the semantic segmentation result, determines the cancer identification result based on the image classification result, and generates corresponding auxiliary diagnostic suggestions when it is necessary to detect the corresponding organ part of the medical image. Thus, based on the three, it generates detection prompt information including the lesion region in the medical image, the cancer identification result indicated by the image classification result, and the auxiliary diagnostic suggestions.

[0108] For example, if the medical image is a lung CT image, the detection prompt information includes lung nodule area (lesion area), malignant lung nodule (cancer identification result), and biopsy suggestion (adjunctive diagnostic suggestion).

[0109] The cancer detection method based on medical images provided in this invention generates detection prompts based on semantic segmentation results and image classification results. These prompts include lesion areas in the medical images, cancer identification results indicated by the image classification results, and auxiliary diagnostic suggestions. This provides a more comprehensive approach to prompting doctors to conduct further cancer detection, thereby improving the comprehensiveness and accuracy of cancer detection, increasing the sensitivity of cancer identification, reducing missed detections, and assisting doctors in making further diagnoses, thus improving the efficiency of cancer detection.

[0110] Based on any of the above embodiments, before step 110, the method further includes: inputting the medical image into a feature extraction model to obtain image features output by the feature extraction model.

[0111] Accordingly, step 110 includes: inputting the image features into an image classification model to obtain the image classification result output by the image classification model.

[0112] Accordingly, step 120 includes: inputting the image features into the semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model.

[0113] In other words, the semantic segmentation model and the image classification model share a single feature extraction model. For example, the semantic segmentation model and the image classification model share a backbone network, and two detection heads are connected to the backbone network to achieve semantic segmentation and image classification respectively.

[0114] Furthermore, the feature extraction model is used to extract high-level abstract features from medical images. Compared with low-level features, high-level abstract features have stronger semantic expressive power and stronger robustness, and can be applied to various complex organ image acquisition scenarios, thus achieving higher accuracy and better generalization ability. In particular, it performs better for image classification models, ultimately improving the accuracy of cancer identification.

[0115] The cancer detection method based on medical images provided in this invention uses a semantic segmentation model and an image classification model to share a single feature extraction model, thereby effectively reducing the computational load of model inference and improving the efficiency of cancer detection. It also reduces the storage space occupied by the model, making it suitable for devices with limited hardware resources.

[0116] Based on any of the above embodiments, in this method, the semantic segmentation model is an integrated semantic segmentation model, which includes multiple different segmentation models.

[0117] For any pixel in the medical image, the integrated semantic segmentation model is used to determine multiple category prediction results for the pixel. Each category prediction result includes a lesion pixel or a non-lesion pixel. If any of the multiple category prediction results indicates a lesion pixel, then the final category prediction result for the pixel is determined to be a lesion pixel. If all of the multiple category prediction results indicate a non-lesion pixel, then the final category prediction result for the pixel is determined to be a non-lesion pixel. That is, this embodiment of the invention uses the union of the results from multiple single semantic segmentation models to determine the final result.

[0118] Existing bagging ensemble learning methods use voting to determine the majority result. This invention, considering the greater harm of missed cancer detections, assigns a positive result to any single positive result. That is, if any of the multiple category predictions identifies a pixel as a lesion, the final category prediction for that pixel is determined to be a lesion pixel, thereby improving the sensitivity of cancer identification and reducing missed detections.

[0119] The cancer detection method based on medical images provided in this invention uses an ensemble semantic segmentation model, which includes multiple different segmentation models. Compared to a single semantic segmentation model, the ensemble semantic segmentation model can improve the accuracy of semantic segmentation results, thereby improving the accuracy of cancer identification. For any pixel in a medical image, the ensemble semantic segmentation model is used to determine multiple category prediction results for the pixel. Each category prediction result includes lesion pixels or non-lesion pixels. If any category prediction result among the multiple category prediction results is a lesion pixel, then the final category prediction result of the pixel is determined to be a lesion pixel. If all category prediction results among the multiple category prediction results are non-lesion pixels, then the final category prediction result of the pixel is determined to be a non-lesion pixel, thereby improving the sensitivity of cancer identification and reducing missed detections.

[0120] Based on any of the above embodiments, in this method, the image classification result includes a first classification result or a second classification result, wherein the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result. For example, the first classification result is suspected cancer, and the second classification result is no cancer.

[0121] Accordingly, considering that the image classification result is the first classification result, i.e., a high probability of cancer, the semantic segmentation model fails to identify the lesion region, thus requiring further localization by the doctor, leading to a decrease in cancer detection efficiency; based on this, before step 130 above, the method further includes:

[0122] If, based on the semantic segmentation result, it is determined that there is no lesion region in the medical image, and the image classification result is the first classification result, the segmentation threshold of the semantic segmentation model is adjusted to obtain and output the lesion region in the medical image.

[0123] Specifically, for any pixel in the medical image, the semantic segmentation model determines the category prediction result of the pixel based on the segmentation threshold, and the category prediction result of the pixel includes lesion pixels or non-lesion pixels.

[0124] For example, in cases where an image classification model identifies a suspected cancerous area but the semantic segmentation model does not output the lesion region, the threshold of the semantic segmentation model is dynamically adjusted. By dynamically adjusting the segmentation threshold of the semantic segmentation model, the lesion region is output, allowing doctors to locate and analyze the lesion region.

[0125] The cancer detection method based on medical images provided in this invention, through the above-described method, even if the semantic segmentation model fails to identify the lesion region when the image classification result is the first classification result, i.e., a high probability of cancer, can adjust the segmentation threshold of the semantic segmentation model to obtain and output the lesion region in the medical image. This assists doctors in further accurately locating the lesion region, thereby improving the efficiency of cancer detection, increasing the sensitivity of cancer identification, and reducing missed detections.

[0126] Based on any of the above embodiments, in this method, adjusting the segmentation threshold of the semantic segmentation model includes:

[0127] The lowest segmentation threshold is set to 0, and the highest segmentation threshold is set as the current segmentation threshold of the semantic segmentation model.

[0128] Determine the sum of the lowest segmentation threshold and the highest segmentation threshold, and adjust the segmentation threshold of the semantic segmentation model to half of the sum.

[0129] The semantic segmentation result is updated based on the adjusted segmentation threshold;

[0130] If, based on the updated semantic segmentation results, it is determined that the proportion of the lesion region in the medical image is greater than the first preset proportion, the minimum segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the minimum segmentation threshold and the maximum segmentation threshold and adjusting the segmentation threshold of the semantic segmentation model to half of the sum is returned.

[0131] If, based on the updated semantic segmentation results, it is determined that the proportion of the lesion region in the medical image is less than the second preset proportion, the highest segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the lowest segmentation threshold and the highest segmentation threshold and adjusting the segmentation threshold of the semantic segmentation model to half of the sum is returned.

[0132] Specifically, for any pixel in the medical image, the semantic segmentation model outputs a predicted value for the pixel. If the predicted value of the pixel is greater than or equal to the segmentation threshold, the category prediction result of the pixel is determined to be a lesion pixel. The proportion of the lesion region is determined based on the ratio of the lesion region in the medical image to the area of ​​the medical image, and the first preset proportion is greater than the second preset proportion.

[0133] Here, the highest segmentation threshold is greater than the lowest segmentation threshold. Both the highest and lowest segmentation thresholds can be updated dynamically.

[0134] Considering that the current segmentation threshold of the semantic segmentation model does not output the lesion region, while if the segmentation threshold is 0, the semantic segmentation model outputs the lesion region of the entire medical image, the initial minimum segmentation threshold is set to 0, and the initial maximum segmentation threshold is the current segmentation threshold of the semantic segmentation model.

[0135] Here, the first and second preset percentages can be set according to the actual situation. For example, according to statistical analysis, the lesion area in a positive medical image generally accounts for 5%-80% of the entire positive medical image. Based on this, the first preset percentage is 80%, and the second preset percentage is 5%.

[0136] For example, assuming the current segmentation threshold of the semantic segmentation model is threshold_init, the first preset percentage is 80%, and the second preset percentage is 5%, the segmentation threshold of the semantic segmentation model is adjusted within the open interval (0, threshold_init) until the semantic segmentation model outputs the lesion region. Furthermore, using a binary search approach can improve the accuracy and reasonableness of lesion region determination. Specifically, the steps are as follows: First, set the minimum segmentation threshold threshold_low = 0 and the maximum segmentation threshold threshold_high = threshold_init. Second, set the segmentation threshold of the semantic segmentation model to threshold = (threshold_low + threshold_high) / 2. Third, based on the adjusted segmentation threshold, update the semantic segmentation result, i.e., obtain the lesion region according to the new segmentation threshold and determine the area ratio P of the lesion region to the entire medical image. Fourth, if 0.05 ≤ P ≤ 0.80, determine the current threshold as a reasonable segmentation threshold and output the lesion region corresponding to this segmentation threshold. If P > 0.80, then threshold_low = threshold, and return to the second step, until the finally obtained lesion region satisfies 0.05 ≤ P ≤ 0.80. If P < 0.05, then threshold_high = threshold, and return to the second step, until the finally obtained lesion region satisfies 0.05 ≤ P ≤ 0.80.

[0137] The cancer detection method based on medical images provided in this invention determines the sum of the minimum and maximum segmentation thresholds through the above-described method. The segmentation threshold of the semantic segmentation model is then adjusted to half of this sum, allowing for accurate dynamic adjustment of the segmentation threshold. This improves the sensitivity of lesion region localization, thereby increasing the cancer detection rate. Furthermore, it ensures that the proportion of lesion regions falls between a first and second preset proportion, further enhancing the accuracy of lesion region localization and thus improving the overall accuracy of cancer detection.

[0138] Based on any of the above embodiments, the method further includes:

[0139] The medical image is input into the organ location recognition model to obtain the organ location recognition result output by the organ location recognition model;

[0140] Output the prompt information corresponding to the organ location identification result to indicate the organ location currently being examined.

[0141] The organ part recognition model is trained based on the sample medical image and the corresponding organ part recognition result label of the sample medical image.

[0142] For example, if the organ location identification result is the upper anterior wall of the stomach body, the corresponding prompt message can be "The digestive tract location currently being examined is the upper anterior wall of the stomach body".

[0143] Furthermore, the organ part recognition model can share a feature extraction model with the image classification model and the semantic segmentation model. That is, the image features of the medical image are input into the organ part recognition model to obtain the organ part recognition result output by the organ part recognition model.

[0144] The cancer detection method based on medical images provided in this invention inputs medical images into an organ location recognition model to obtain organ location recognition results output by the organ location recognition model; it outputs prompt information corresponding to the organ location recognition results to indicate the currently examined organ location, thereby assisting doctors in further judging the currently examined organ location and improving the detection efficiency of organ cancer.

[0145] Based on the above embodiments, examination quality control reports (such as upper gastrointestinal endoscopic tumor screening quality control reports) can be generated for doctors or patients to read later.

[0146] Based on the above embodiments, the principle for selecting hyperparameters during the training phase of each model is to maximize sensitivity while ensuring specificity is not less than 90%. For ease of understanding, assume that the number of positive samples in the test set is P, and the number of negative samples is F; the number of true positive samples determined by the semantic segmentation result of the semantic segmentation model is STP, and the number of false positive samples is SFP; the number of true positive samples determined by the image classification result of the image classification model is CTP, and the number of false positive samples is CFP; based on this, the sensitivity is (STP+CTP) / P, and the specificity is (F-SFP-CFP) / F.

[0147] Based on the above embodiments, the present invention can significantly improve the sensitivity of cancer identification while ensuring a certain biopsy positive rate, and at the same time achieve targeted localization of suspected lesion areas.

[0148] The cancer detection device based on medical images provided by the present invention will be described below. The cancer detection device based on medical images described below can be referred to in correspondence with the cancer detection method based on medical images described above.

[0149] Figure 6 is a schematic diagram of the structure of the cancer detection device based on medical images provided by the present invention. As shown in Figure 6, the cancer detection device based on medical images includes an image classification module 610, a semantic segmentation module 620, a detection determination module 630, and a prompt output module 640.

[0150] The image classification module 610 is used to input medical images into an image classification model and obtain the image classification result output by the image classification model; the image classification result is used to characterize the cancer identification result of the medical image.

[0151] The semantic segmentation module 620 is used to input the medical image into the semantic segmentation model and obtain the semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion region in the medical image.

[0152] The detection determination module 630 is used to determine, based on the semantic segmentation result and the image classification result, whether it is necessary to detect the organ part corresponding to the medical image.

[0153] The prompt output module 640 is used to output detection prompt information when it is necessary to detect the organ part corresponding to the medical image.

[0154] The cancer detection device based on medical images provided in this invention inputs a medical image into an image classification model to obtain an image classification result, and then inputs the medical image into a semantic segmentation model to obtain a semantic segmentation result. The image classification result characterizes the cancer identification result of the medical image, while the semantic segmentation result indicates the presence of lesion regions in the medical image. Therefore, fusing semantic segmentation and image classification can improve the accuracy of cancer identification, thereby increasing the sensitivity of cancer detection. Furthermore, if either the image classification result or the semantic segmentation result indicates the presence of cancer, a preliminary judgment can be made that cancer is present. This invention improves the sensitivity of cancer detection, ensures a certain biopsy positivity rate, and reduces false negatives. Furthermore, based on semantic segmentation and image classification results, it determines whether to perform detection on the corresponding organ or site in the medical image, assisting doctors in deciding whether detection is necessary. This helps doctors further determine the diagnostic approach based on the two results, thereby improving cancer detection efficiency and sensitivity, and reducing false negatives. When detection of the corresponding organ or site in the medical image is required, it outputs detection prompts to guide doctors to further cancer detection, assisting in further diagnosis and thus improving cancer detection efficiency and sensitivity, and reducing false negatives. In summary, this invention can achieve efficient, highly sensitive, and highly accurate cancer detection.

[0155] Figure 7 illustrates a schematic diagram of the physical structure of an electronic device. As shown in Figure 7, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740. The processor 710, communication interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a cancer detection method based on medical images. This method includes: inputting a medical image into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used to characterize the cancer identification result of the medical image; inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether a lesion region exists in the medical image; based on the semantic segmentation result and the image classification result, determining whether it is necessary to detect the organ corresponding to the medical image; and outputting detection prompt information if it is necessary to detect the organ corresponding to the medical image.

[0156] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the cancer detection method based on medical images provided by the above methods. The method includes: inputting a medical image into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used to characterize the cancer identification result of the medical image; inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion region in the medical image; based on the semantic segmentation result and the image classification result, determining whether it is necessary to detect the organ corresponding to the medical image; and outputting detection prompt information when it is necessary to detect the organ corresponding to the medical image.

[0158] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the cancer detection method based on medical images provided by the methods described above. The method includes: inputting a medical image into an image classification model to obtain an image classification result output by the image classification model; the image classification result being used to characterize the cancer identification result of the medical image; inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result being used to indicate whether a lesion region exists in the medical image; determining, based on the semantic segmentation result and the image classification result, whether it is necessary to detect the organ corresponding to the medical image; and outputting detection prompt information if it is necessary to detect the organ corresponding to the medical image.

[0159] In one specific embodiment, to verify the significant effect of this technical solution (as shown in Figure 5) in improving the specificity of cancer detection, a comparative experiment was conducted using the publicly available multicenter colorectal cancer pathology dataset (DigestPath2019). This dataset originated from four medical centers and included 250 malignant tissue sections and 410 benign tissue sections obtained at 20x magnification after hematoxylin and eosin (H&E) staining. Whole-slice images (WSI) were randomly divided into a training set and an internal test set.

[0160] Based on the technical solution shown in Figure 5 (i.e., fusing a semantic segmentation model and an image classification model, and making collaborative decisions through an auxiliary diagnostic probability threshold), a colorectal cancer pathology recognition model (hereinafter referred to as the "fusion model") was trained. As a control, a colorectal cancer pathology recognition model (hereinafter referred to as the "pure classification model") that does not include a semantic segmentation branch and relies solely on image classification was trained using the same training set.

[0161] The performance comparison of the two models on the same internal test set is as follows:

[0162] 1. Fusion model: Sensitivity is 100.00%, specificity is 81.84%.

[0163] 2. Pure classification model: sensitivity is 100.00%, specificity is only 20.72%.

[0164] Statistical analysis (P<0.001) showed that the specificity of the fusion model was significantly higher than that of the pure classification model. These experimental results demonstrate that this invention, by fusing semantic segmentation and image classification results and performing collaborative decision-making, can significantly improve detection specificity while maintaining extremely high sensitivity (avoiding missed detections), effectively reducing false positive results. This significantly improves the accuracy and clinical applicability of cancer auxiliary diagnosis, overcoming the shortcomings of existing technologies that rely solely on classification models, resulting in high false positive rates and low diagnostic efficiency.

[0165] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0166] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Industrial applicability

[0168] This invention provides a method, apparatus, device, medium, and product for cancer detection based on medical images, relating to the field of image recognition technology. The method includes: inputting a medical image into an image classification model to obtain an image classification result output by the model; the image classification result characterizes the cancer identification result of the medical image; inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the model; the semantic segmentation result indicates whether a lesion region exists in the medical image; based on the semantic segmentation result and the image classification result, determining whether it is necessary to detect the organ corresponding to the medical image; and outputting a detection prompt if it is necessary to detect the organ corresponding to the medical image. This invention can achieve efficient, highly sensitive, and highly accurate cancer detection, possessing good economic value and application prospects.

Claims

1. A cancer detection method based on medical images, characterized by, include: The medical image is input into the image classification model to obtain the image classification result output by the image classification model; The image classification results are used to characterize the cancer identification results of the medical images; The medical image is input into a semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion region in the medical image. Based on the semantic segmentation results and the image classification results, it is determined whether it is necessary to detect the organ parts corresponding to the medical image; When it is necessary to detect the organ or part corresponding to the medical image, a detection prompt message is output.

2. The method of claim 1, wherein, The image classification result includes a first classification result or a second classification result, wherein the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result. The step of determining whether to detect the organ region corresponding to the medical image based on the semantic segmentation result and the image classification result includes: If, based on the semantic segmentation result, it is determined that there is a lesion region in the medical image, and the image classification result is the first classification result, it is determined that the organ part corresponding to the medical image needs to be detected. If, based on the semantic segmentation result, it is determined that there is a lesion region in the medical image, and the image classification result is the second classification result, it is determined that it is not necessary to detect the organ part corresponding to the medical image; If, based on the semantic segmentation results, it is determined that there are no lesion regions in the medical image, it is determined that there is no need to detect the organ parts corresponding to the medical image. 3.The medical image-based cancer detection method of claim 1, wherein, The output detection prompt information includes: Based on the semantic segmentation results and the image classification results, detection prompt information is generated; the detection prompt information includes the lesion region in the medical image, the cancer identification result indicated by the image classification results, and auxiliary diagnostic suggestions; Output the detection prompt information. 4.The method of claim 1, wherein, Before inputting the medical image into the image classification model and obtaining the image classification result output by the image classification model, the method further includes: The medical image is input into the feature extraction model to obtain the image features output by the feature extraction model; Accordingly, the step of inputting the medical image into the image classification model and obtaining the image classification result output by the image classification model includes: The image features are input into the image classification model to obtain the image classification result output by the image classification model; Accordingly, the step of inputting the medical image into the semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model includes: The image features are input into the semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model.

5. The method of claim 1 to 4, wherein, The semantic segmentation model is an integrated semantic segmentation model, which includes multiple different segmentation models. For any pixel in the medical image, the integrated semantic segmentation model is used to determine multiple category prediction results for the pixel. Each category prediction result includes a lesion pixel or a non-lesion pixel. If any category prediction result among the multiple category prediction results is a lesion pixel, then the final category prediction result of the pixel is determined to be a lesion pixel. If all category prediction results among the multiple category prediction results are non-lesion pixels, then the final category prediction result of the pixel is determined to be a non-lesion pixel.

6. The method of claim 1 to 4, wherein, The image classification result includes a first classification result or a second classification result, wherein the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result. Before determining whether it is necessary to detect the organ region corresponding to the medical image based on the semantic segmentation result and the image classification result, the method further includes: If, based on the semantic segmentation result, it is determined that there is no lesion region in the medical image, and the image classification result is the first classification result, the segmentation threshold of the semantic segmentation model is adjusted to obtain and output the lesion region in the medical image; Specifically, for any pixel in the medical image, the semantic segmentation model determines the category prediction result of the pixel based on the segmentation threshold, and the category prediction result of the pixel includes lesion pixels or non-lesion pixels. 7.The medical image-based cancer detection method according to claim 6, wherein, Adjusting the segmentation threshold of the semantic segmentation model includes: The lowest segmentation threshold is set to 0, and the highest segmentation threshold is set as the current segmentation threshold of the semantic segmentation model. Determine the sum of the lowest segmentation threshold and the highest segmentation threshold, and adjust the segmentation threshold of the semantic segmentation model to half of the sum. The semantic segmentation result is updated based on the adjusted segmentation threshold; If, based on the updated semantic segmentation results, it is determined that the proportion of the lesion region in the medical image is greater than the first preset proportion, the minimum segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the minimum segmentation threshold and the maximum segmentation threshold and adjusting the segmentation threshold of the semantic segmentation model to half of the sum is returned. If, based on the updated semantic segmentation results, it is determined that the proportion of the lesion region in the medical image is less than the second preset proportion, the highest segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the lowest segmentation threshold and the highest segmentation threshold and adjusting the segmentation threshold of the semantic segmentation model to half of the sum is returned. Specifically, for any pixel in the medical image, the semantic segmentation model outputs a predicted value for the pixel. If the predicted value of the pixel is greater than or equal to the segmentation threshold, the category prediction result of the pixel is determined to be a lesion pixel. The proportion of the lesion region is determined based on the ratio of the lesion region in the medical image to the area of ​​the medical image, and the first preset proportion is greater than the second preset proportion.

8. The method of claim 1 to 4, wherein, The method further includes: The medical image is input into the organ location recognition model to obtain the organ location recognition result output by the organ location recognition model; Output the prompt information corresponding to the organ location identification result to indicate the currently examined organ location; The organ part recognition model is trained based on the sample medical image and the corresponding organ part recognition result label of the sample medical image.

9. A cancer detection apparatus based on medical images, characterized by, include: The image classification module is used to input medical images into the image classification model and obtain the image classification results output by the image classification model. The image classification results are used to characterize the cancer identification results of the medical images; A semantic segmentation module is used to input the medical image into a semantic segmentation model and obtain the semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion region in the medical image; The detection determination module is used to determine, based on the semantic segmentation result and the image classification result, whether it is necessary to detect the organ part corresponding to the medical image; The prompt output module is used to output detection prompt information when it is necessary to detect the organ part corresponding to the medical image.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the cancer detection method based on medical images as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the cancer detection method based on medical images as described in any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the cancer detection method based on medical images as described in any one of claims 1 to 8.

Images