Cancer detection method and device based on medical image, equipment, medium and product

Abstract

The invention provides a cancer detection method, device and equipment based on a medical image, a medium and a product, and relates to the technical field of image recognition. The method comprises the following steps: inputting a medical image into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used for representing an identification result of performing cancer identification on the medical image; inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used for indicating whether a focus area exists in the medical image or not; based on the semantic segmentation result and the image classification result, determining whether an organ part corresponding to the medical image needs to be detected; and outputting detection prompt information under the condition that the organ part corresponding to the medical image needs to be detected. According to the invention, high-efficiency, high-sensitivity and high-accuracy cancer detection can be realized.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a cancer detection method, device, equipment, medium and product based on medical images. Background Art

[0002] With the rapid development of artificial intelligence, artificial intelligence technology is being used more and more widely in medical devices. Medical image recognition based on artificial intelligence can assist in the diagnosis of tumor cancer and improve the accuracy of tumor cancer diagnosis.

[0003] At present, by performing target detection on medical images, it is determined whether there is a lesion area in the medical image based on the target detection result, that is, the lesion area suspected of cancer is framed with a rectangular frame to assist doctors in making further diagnosis. However, the resolution of the collected medical images is different. Some are collected by close-up lenses, and some are collected by long-distance lenses. This leads to a decrease in the accuracy of target detection through a target detection model, which may result in missed detections. In other words, the sensitivity of current cancer recognition is not high; and the target detection result is to select the lesion area with a rectangular frame, but the lesion area may not be a rectangular area, so the positioning accuracy of the lesion area is not high, resulting in the doctor still needing to determine the lesion area within the rectangular frame. That is, the efficiency of current organ cancer detection is not high. Summary of the invention

[0004] The present invention provides a cancer detection method, device, equipment, medium and product based on medical images, which are used to solve the defects of low sensitivity and efficiency of cancer recognition in the prior art and realize efficient and highly sensitive cancer detection.

[0005] The present invention provides a cancer detection method based on medical images, comprising: Inputting the medical image into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used to represent a recognition result of cancer recognition for the medical image; Inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion area in the medical image; Based on the semantic segmentation result and the image classification result, determining whether it is necessary to detect the organ part corresponding to the medical image; When it is necessary to detect the organ part corresponding to the medical image, detection prompt information is output.

[0006] According to a cancer detection method based on medical images provided by the present invention, the image classification result includes a first classification result or a second classification result, and the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result; The determining whether it is necessary to detect the organ part corresponding to the medical image based on the semantic segmentation result and the image classification result includes: If it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the image classification result is the first classification result, it is determined that the organ part corresponding to the medical image needs to be detected; If it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the image classification result is the second classification result, it is determined that there is no need to detect the organ part corresponding to the medical image; If it is determined based on the semantic segmentation result that no lesion area exists in the medical image, it is determined that there is no need to detect the organ part corresponding to the medical image.

[0007] According to a cancer detection method based on medical images provided by the present invention, the outputting of detection prompt information includes: Based on the semantic segmentation result and the image classification result, generating detection prompt information; the detection prompt information includes the lesion area in the medical image, the cancer recognition result indicated by the image classification result, and auxiliary diagnosis suggestions; The detection prompt information is output.

[0008] According to a cancer detection method based on medical images provided by the present invention, before inputting the medical image into the image classification model to obtain the image classification result output by the image classification model, the method further includes: Inputting the medical image into a feature extraction model to obtain image features output by the feature extraction model; Accordingly, the step of inputting the medical image into the image classification model to obtain the image classification result output by the image classification model includes: Inputting the image features into an image classification model to obtain an image classification result output by the image classification model; Accordingly, the step of inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model includes: The image features are input into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model.

[0009] According to a cancer detection method based on medical images provided by the present invention, the semantic segmentation model is an integrated semantic segmentation model, and the integrated semantic segmentation model includes multiple different segmentation models; For any pixel of the medical image, the integrated semantic segmentation model is used to determine multiple category prediction results of the pixel, any of the category prediction results including lesion pixels or non-lesion pixels, if any category prediction result among the multiple category prediction results is a lesion pixel, then the final category prediction result of the pixel is determined as a lesion pixel, if all category prediction results among the multiple category prediction results are non-lesion pixels, then the final category prediction result of the pixel is determined as a non-lesion pixel.

[0010] According to a cancer detection method based on medical images provided by the present invention, the image classification result includes a first classification result or a second classification result, and the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result; Before determining whether it is necessary to detect the organ part corresponding to the medical image based on the semantic segmentation result and the image classification result, the method further includes: If it is determined based on the semantic segmentation result that there is no lesion area in the medical image, and the image classification result is the first classification result, adjusting the segmentation threshold of the semantic segmentation model to obtain and output the lesion area in the medical image; Among them, for any pixel of the medical image, the semantic segmentation model determines the category prediction result of the pixel based on the segmentation threshold, and the category prediction result of the pixel includes a lesion pixel or a non-lesion pixel.

[0011] According to a cancer detection method based on medical images provided by the present invention, the step of adjusting the segmentation threshold of the semantic segmentation model includes: The lowest segmentation threshold is determined as 0, and the highest segmentation threshold is determined as the current segmentation threshold of the semantic segmentation model; Determine a sum of the minimum segmentation threshold and the maximum segmentation threshold, and adjust the segmentation threshold of the semantic segmentation model to half of the sum; Based on the adjusted segmentation threshold, updating the semantic segmentation result; If it is determined based on the updated semantic segmentation result that the proportion of the lesion area of ​​the medical image is greater than the first preset proportion, the minimum segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the minimum segmentation threshold and the maximum segmentation threshold is returned to adjust the segmentation threshold of the semantic segmentation model to half of the sum; If it is determined based on the updated semantic segmentation result that the proportion of the lesion area of ​​the medical image is less than the second preset proportion, the highest segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the lowest segmentation threshold and the highest segmentation threshold is returned to adjust the segmentation threshold of the semantic segmentation model to half of the sum; Among them, for any pixel of the medical image, the semantic segmentation model outputs the predicted value of the pixel. If the predicted value of the pixel is greater than or equal to the segmentation threshold, the category prediction result of the pixel is determined to be a lesion pixel; the lesion area ratio is determined based on the area ratio of the lesion area in the medical image to the area of ​​the medical image, and the first preset ratio is greater than the second preset ratio.

[0012] According to a cancer detection method based on medical images provided by the present invention, the method further includes: Inputting the medical image into an organ part recognition model to obtain an organ part recognition result output by the organ part recognition model; Outputting prompt information corresponding to the organ part recognition result to indicate the organ part currently being examined; The organ part recognition model is obtained by training based on sample medical images and organ part recognition result labels corresponding to the sample medical images.

[0013] The present invention also provides a cancer detection device based on medical images, comprising: An image classification module, used to input a medical image into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used to represent a recognition result of cancer recognition for the medical image; A semantic segmentation module, used for inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used for indicating whether there is a lesion area in the medical image; A detection determination module, used to determine whether it is necessary to detect the organ part corresponding to the medical image based on the semantic segmentation result and the image classification result; The prompt output module is used to output detection prompt information when it is necessary to detect the organ part corresponding to the medical image.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the medical image-based cancer detection method described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the computer program implements any of the above-mentioned medical image-based cancer detection methods.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned medical image-based cancer detection methods.

[0017] The cancer detection method, device, equipment, medium and product based on medical images provided by the present invention input the medical image into an image classification model to obtain the image classification result output by the image classification model, and input the medical image into a semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model, and the image classification result is used to characterize the recognition result of cancer recognition in the medical image, and the semantic segmentation result is used to indicate whether there is a lesion area in the medical image and locate the position range of the lesion area, so that the fusion of semantic segmentation and image classification can improve the recognition accuracy of cancer, thereby improving the sensitivity of cancer recognition, and the image classification result and the semantic segmentation result are When any result indicates the presence of cancer, it can be preliminarily determined that cancer exists, thereby improving the sensitivity of cancer identification, ensuring a certain biopsy positivity rate, and reducing missed detections; and based on the semantic segmentation results and image classification results, it is determined whether it is necessary to detect the organ part corresponding to the medical image, and assist the doctor in determining whether the detection is necessary, thereby improving the efficiency of cancer detection, and improving the sensitivity of cancer identification, and reducing missed detections; and when it is necessary to detect the organ part corresponding to the medical image, the detection prompt information is output to prompt the doctor to further perform cancer detection, thereby improving the efficiency of cancer detection, and improving the sensitivity of cancer identification, and reducing missed detections. In summary, the present invention can achieve efficient, highly sensitive, and highly accurate cancer detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is one of the flow charts of the cancer detection method based on medical images provided by the present invention.

[0020] Figure 2 This is the second flow chart of the cancer detection method based on medical images provided by the present invention.

[0021] Figure 3This is the third flow chart of the cancer detection method based on medical images provided by the present invention.

[0022] Figure 4 This is the fourth flow chart of the cancer detection method based on medical images provided by the present invention.

[0023] Figure 5 This is the fifth flow chart of the cancer detection method based on medical images provided by the present invention.

[0024] Figure 6 It is a schematic structural diagram of a cancer detection device based on medical images provided by the present invention.

[0025] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] At present, by performing target detection on medical images, it is determined whether there is a lesion area in the medical image based on the target detection result, that is, the lesion area suspected of cancer is framed with a rectangular frame to assist doctors in making further diagnosis. However, the resolution of the collected medical images is different. Some are collected by close-up lenses, and some are collected by long-distance lenses. This leads to a decrease in the accuracy of target detection through a target detection model, which may result in missed detections. In other words, the sensitivity of current cancer recognition is not high; and the target detection result is to select the lesion area with a rectangular frame, but the lesion area may not be a rectangular area, so the positioning accuracy of the lesion area is not high, resulting in the doctor still needing to determine the lesion area within the rectangular frame. That is, the efficiency of current organ cancer detection is not high.

[0028] For example, obtain the real-time gastric image collected during the electronic staining endoscopy; identify and annotate the lesion area based on the pre-built gastric early cancer recognition model; when the identified lesion areas overlap, determine whether they belong to the same lesion area. This method uses target detection technology to achieve image recognition and position positioning at the same time, but the lesion area is selected using a rectangular frame, and the position positioning is not accurate enough. The lesion area may not be a rectangular area, and the doctor needs to make a judgment within the rectangular frame. And using only one target detection model, it is difficult to apply it to both close-up and long-distance lenses. The sensitivity of early cancer recognition is not high enough, and many missed detections will occur, causing economic losses or even life-threatening consequences to patients.

[0029] In addition, by classifying medical images, whether it is suspected early cancer is determined based on the image classification results, thereby assisting doctors in making further diagnoses. However, the accuracy of cancer identification based solely on image classification results is not high, which may result in missed detections, that is, the sensitivity of current cancer identification is not high; and image classification cannot determine the lesion area, resulting in the need for doctors to determine the lesion area in the medical image themselves, that is, the efficiency of current cancer identification is not high.

[0030] For example, a sample set of early gastric cancer images is obtained, and feature recognition is performed on each lesion image in the sample set of early gastric cancer images to obtain a feature vector of each lesion image; a preset initial model is trained based on the sample set of early gastric cancer images and the feature vector to obtain an early gastric cancer recognition model. However, this method only provides a method for early gastric cancer recognition and does not provide a function for locating the location of early cancer lesions.

[0031] In view of the above problems, the present invention proposes the following embodiments. Figure 1-Figure 5 The present invention describes a method for cancer detection based on medical images.

[0032] Figure 1 is one of the flow charts of the cancer detection method based on medical images provided by the present invention, such as Figure 1 As shown, the cancer detection method based on medical images includes the following steps 110, 120, 130 and 140.

[0033] Step 110: input the medical image into an image classification model to obtain an image classification result output by the image classification model.

[0034] Here, the cancer to be detected may include, but is not limited to: digestive tract cancer (such as esophageal cancer, gastric cancer, colorectal cancer, etc.), lung cancer, liver cancer, breast cancer, etc. And early cancer, intermediate cancer, late cancer, etc. can be identified. For example, digestive tract cancer can be early digestive tract cancer and advanced digestive tract cancer, etc. The embodiment of the present invention is described by taking early digestive tract cancer (early digestive tract malignant tumor) as an example.

[0035] Here, a medical image is an image acquired by collecting images of an organ part. For example, a digestive tract image (medical image) is an image acquired by collecting images of a digestive tract part; the digestive tract image can be acquired by digestive tract endoscopy. For another example, a lung image (medical image) is an image acquired by collecting images of the lungs; the lung image can be a CT (Computed Tomography) image.

[0036] Exemplarily, the parts of the digestive tract that can be collected by an upper gastrointestinal endoscope (gastroscopy) include: oropharynx, upper esophagus, middle esophagus, lower esophagus, esophagogastric junction, upper gastric body, posterior wall of upper gastric body, anterior wall of upper gastric body, greater curvature of upper gastric body, lesser curvature of upper gastric body, middle gastric body, posterior wall of middle gastric body, anterior wall of middle gastric body, greater curvature of middle gastric body, lesser curvature of middle gastric body, lower gastric body, posterior wall of lower gastric body, anterior wall of lower gastric body, greater curvature of lower gastric body, lesser curvature of lower gastric body, greater curvature of antrum-body junction, gastric angle, posterior wall of gastric angle, anterior wall of gastric angle, gastric antrum, posterior wall of gastric antrum, anterior wall of gastric antrum, greater curvature of gastric antrum, lesser curvature of gastric antrum, pylorus, duodenal bulb, descending duodenum, gastric fundus, posterior wall of gastric fundus, anterior wall of gastric fundus, greater curvature of gastric fundus, lesser curvature of gastric fundus and cardia. The parts of the digestive tract that can be sampled by colonoscopy include: the terminal ileum, appendix orifice, ileocecal valve, ascending colon, hepatic flexure of the colon, transverse colon, splenic flexure, descending colon, sigmoid colon, rectum, and rectal junction.

[0037] Further, the collected medical images are subjected to image preprocessing. In one embodiment, the collected medical images are uniformly adjusted to medical images of a preset input size, for example, the preset input size is .

[0038] The image classification result is used to characterize the recognition result of cancer recognition of the medical image. That is, the image classification model is used to recognize cancer in the medical image. Specifically, the image classification model is used to classify the medical image into classification results with different cancer probabilities, and then determine whether it is suspected cancer.

[0039] In one embodiment, the image classification result includes a first classification result or a second classification result, and the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result. For example, the first classification result is suspected cancer, and the second classification result is no cancer.

[0040] In one embodiment, the image classification model is trained based on sample medical images and image classification result labels corresponding to the sample medical images.

[0041] Step 120: input the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model.

[0042] The semantic segmentation result is used to indicate whether there is a lesion area in the medical image and locate the position range of the lesion area. The semantic segmentation result is a pixel-level segmentation result, that is, the semantic segmentation model is used to determine the category prediction result of each pixel in the medical image, that is, to determine whether each pixel in the medical image is a lesion pixel, so that irregular lesion areas can be determined, that is, the actual lesion area can be determined, thereby improving the positioning accuracy of the lesion area, that is, accurately locating the position range of the lesion area, assisting doctors in determining the lesion area, thereby improving the accuracy of cancer recognition and improving the efficiency of cancer recognition.

[0043] Furthermore, the semantic segmentation model is an integrated semantic segmentation model, which includes multiple different segmentation models. Compared with a single semantic segmentation model, the integrated semantic segmentation model can improve the accuracy of semantic segmentation results, thereby improving the accuracy of cancer recognition. Furthermore, the integrated semantic segmentation model is composed of multiple single semantic segmentation models based on a bagging method.

[0044] In one embodiment, the semantic segmentation model is trained based on a sample medical image and a semantic segmentation result label corresponding to the sample medical image.

[0045] Step 130: Determine whether it is necessary to detect the organ part corresponding to the medical image based on the semantic segmentation result and the image classification result.

[0046] It should be noted that when the organ part corresponding to the medical image needs to be detected, it means that further cancer diagnosis is needed.

[0047] It should be understood that the image classification result is used to characterize the recognition result of cancer identification in medical images, and the semantic segmentation result is used to indicate whether there is a lesion area in the medical image and locate the position range of the lesion area, so that the fusion of semantic segmentation and image classification can improve the accuracy of cancer recognition, and when any one of the image classification result and the semantic segmentation result indicates the presence of cancer, it can be preliminarily judged that cancer exists, thereby improving the sensitivity of cancer identification, ensuring a certain biopsy positivity rate, and reducing missed detections; and based on the semantic segmentation result and the image classification result, it is determined whether it is necessary to detect the organ part corresponding to the medical image, to assist doctors in determining whether detection is needed, that is, to assist doctors in further determining the diagnosis method based on the two results, thereby improving the efficiency of cancer detection.

[0048] In some embodiments, the image classification result includes a first classification result or a second classification result, and the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result. Based on this, step 130 has the following embodiments.

[0049] In one embodiment, if it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the image classification result is the first classification result, it is determined that the organ part corresponding to the medical image needs to be detected; if it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the image classification result is the second classification result, it is determined that there is no need to detect the organ part corresponding to the medical image; if it is determined based on the semantic segmentation result that there is no lesion area in the medical image, it is determined that there is no need to detect the organ part corresponding to the medical image.

[0050] In another embodiment, if it is determined based on the semantic segmentation result that there is a lesion area in the medical image, or the image classification result is the first classification result, it is determined that the organ part corresponding to the medical image needs to be detected; if it is determined based on the semantic segmentation result that there is no lesion area in the medical image, and the image classification result is the second classification result, it is determined that there is no need to detect the organ part corresponding to the medical image.

[0051] It should be understood that if it is determined based on the semantic segmentation result that there is a lesion area in the medical image, or the image classification result is the first classification result, it is determined that the organ part corresponding to the medical image needs to be detected, that is, if any one of the image classification results and the semantic segmentation results indicates the presence of cancer, it can be preliminarily judged that there is cancer, and thus it is necessary to detect the organ part corresponding to the medical image, thereby improving the sensitivity of cancer recognition, ensuring a certain biopsy positivity rate, and reducing missed detections; and if it is determined based on the semantic segmentation result that there is no lesion area in the medical image, and the image classification result is the second classification result, it is determined that there is no need to detect the organ part corresponding to the medical image, that is, if all the results in the image classification results and the semantic segmentation results indicate the absence of cancer, it is determined that no detection is required, thereby improving the sensitivity of cancer recognition, ensuring a certain biopsy positivity rate, and reducing missed detections.

[0052] For example, Figure 2As shown, the collected medical image is subjected to image preprocessing, and then the feature extraction is performed on the medical image after the image preprocessing, and then the image features of the medical image are respectively input into the semantic segmentation model and the image classification model; then, it is determined whether there is a lesion area in the medical image based on the semantic segmentation result. If it is determined based on the semantic segmentation result that there is a lesion area in the medical image, it is determined that the organ part corresponding to the medical image needs to be detected, and an auxiliary diagnosis prompt is performed accordingly; if it is determined based on the semantic segmentation result that there is no lesion area in the medical image, a subsequent judgment is performed based on the image classification result of the image classification model; if the image classification result is the first classification result, it is determined that the organ part corresponding to the medical image needs to be detected, and an auxiliary diagnosis prompt is performed accordingly; if it is determined based on the semantic segmentation result that there is no lesion area in the medical image, and the image classification result is the second classification result, it is determined that there is no need to detect the organ part corresponding to the medical image, and no prompt is given accordingly.

[0053] The image classification model can output different probabilities. Based on this, if the probability output by the image classification model is greater than the auxiliary diagnosis probability threshold, the image classification result is the first classification result. If the probability output by the image classification model is less than or equal to the auxiliary diagnosis probability threshold, the image classification result is the second classification result. Therefore, according to the different output probabilities of the image classification model, the judgment logic for determining whether it is necessary to detect the corresponding organ part of the medical image is also different.

[0054] For example, Figure 3 As shown, the collected medical image is subjected to image preprocessing, and then the feature extraction is performed on the medical image after the image preprocessing, and then the image features of the medical image are respectively input into the semantic segmentation model and the image classification model; then, it is determined whether there is a lesion area in the medical image based on the semantic segmentation result. If it is determined based on the semantic segmentation result that there is a lesion area in the medical image, it is determined that the organ part corresponding to the medical image needs to be detected, and an auxiliary diagnosis prompt is performed accordingly; if it is determined based on the semantic segmentation result that there is no lesion area in the medical image, subsequent judgment is performed based on the output probability of the image classification model; if the output probability is greater than the auxiliary diagnosis probability threshold, it is determined that the organ part corresponding to the medical image needs to be detected, and an auxiliary diagnosis prompt is performed accordingly; if it is determined based on the semantic segmentation result that there is no lesion area in the medical image, and the output probability is not greater than the auxiliary diagnosis probability threshold, it is determined that there is no need to detect the organ part corresponding to the medical image, and no prompt is given accordingly.

[0055] Step 140: output detection prompt information when it is necessary to detect the organ part corresponding to the medical image.

[0056] Here, the detection prompt information is used to prompt further cancer detection, that is, further cancer diagnosis. Further, the detection prompt information includes auxiliary diagnosis suggestions (such as biopsy suggestions) to prompt further cancer diagnosis, such as further prompting to perform a biopsy. Further, the detection prompt information includes the lesion area in the medical image, the cancer recognition result indicated by the image classification result, and the auxiliary diagnosis suggestion, so as to provide more comprehensive prompts and improve the efficiency of cancer detection.

[0057] It should be understood that directly outputting the detection prompt information is to prompt the doctor to conduct further testing and assist the doctor in making further diagnosis, that is, to assist the doctor in diagnosing whether further cancer testing is needed, thereby improving the efficiency of cancer detection.

[0058] The cancer detection method based on medical images provided by the embodiment of the present invention inputs the medical image into an image classification model to obtain an image classification result output by the image classification model, and inputs the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model, and the image classification result is used to characterize the recognition result of cancer recognition for the medical image, and the semantic segmentation result is used to indicate whether there is a lesion area in the medical image, so that the fusion of semantic segmentation and image classification can improve the recognition accuracy of cancer, and further improve the sensitivity of cancer recognition, and when any one of the image classification result and the semantic segmentation result indicates the presence of cancer, it can be preliminarily determined that cancer exists, thereby improving the sensitivity of cancer recognition, ensuring a certain biopsy positive rate, and reducing missed detection; and based on the semantic segmentation result and the image classification result, it is determined whether it is necessary to detect the organ part corresponding to the medical image, and the doctor is assisted in determining whether the detection is necessary, thereby improving the detection efficiency of cancer, and improving the sensitivity of cancer recognition, and reducing missed detection; and when it is necessary to detect the organ part corresponding to the medical image, a detection prompt information is output to prompt the doctor to further perform cancer detection, and assist the doctor in making further diagnosis, thereby improving the detection efficiency of cancer, and improving the sensitivity of cancer recognition, and reducing missed detection. In summary, the present invention can achieve efficient, high-sensitivity and high-accuracy cancer detection.

[0059] Based on any of the above embodiments, the image classification result includes a first classification result or a second classification result, and the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result. For example, the first classification result is suspected cancer, and the second classification result is no cancer.

[0060] Accordingly, the above step 130 includes: If it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the image classification result is the first classification result, it is determined that the organ part corresponding to the medical image needs to be detected; If it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the image classification result is the second classification result, it is determined that there is no need to detect the organ part corresponding to the medical image; If it is determined based on the semantic segmentation result that no lesion area exists in the medical image, it is determined that there is no need to detect the organ part corresponding to the medical image.

[0061] Considering that it is only necessary to determine whether the organ part corresponding to the medical image needs to be detected, based on this, it is only necessary to perform binary classification on the medical image, that is, the image classification model is a binary classification model.

[0062] Considering that the semantic segmentation model performs semantic segmentation at the pixel level, that is, the semantic segmentation result is more sensitive, based on this, if it is determined based on the semantic segmentation result that there is no lesion area in the medical image, it is directly determined that there is no need to detect the corresponding organ part of the medical image.

[0063] For example, Figure 4 As shown, the collected medical image is subjected to image preprocessing, and then the feature extraction is performed on the medical image after the image preprocessing, and then the image features of the medical image are respectively input into the semantic segmentation model and the image classification model; then, it is determined whether there is a lesion area in the medical image based on the semantic segmentation result. If it is determined based on the semantic segmentation result that there is no lesion area in the medical image, it is determined that there is no need to detect the organ part corresponding to the medical image, and accordingly no prompt is given; if it is determined based on the semantic segmentation result that there is a lesion area in the medical image, a subsequent judgment is made based on the image classification result of the image classification model; if it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the image classification result is the first classification result, it is determined that there is a need to detect the organ part corresponding to the medical image, and accordingly an auxiliary diagnosis prompt is given; if it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the image classification result is the second classification result, it is determined that there is no need to detect the organ part corresponding to the medical image, and accordingly no prompt is given.

[0064] The image classification model can output different probabilities. Based on this, if the probability output by the image classification model is greater than the auxiliary diagnosis probability threshold, the image classification result is the first classification result. If the probability output by the image classification model is less than or equal to the auxiliary diagnosis probability threshold, the image classification result is the second classification result. Therefore, according to the different output probabilities of the image classification model, the judgment logic for determining whether it is necessary to detect the corresponding organ part of the medical image is also different.

[0065] For example, Figure 5As shown, the collected medical image is subjected to image preprocessing, and then the feature extraction is performed on the medical image after the image preprocessing, and then the image features of the medical image are respectively input into the semantic segmentation model and the image classification model; then, it is determined whether there is a lesion area in the medical image based on the semantic segmentation result. If it is determined based on the semantic segmentation result that there is no lesion area in the medical image, it is determined that there is no need to detect the organ part corresponding to the medical image, and accordingly no prompt is given; if it is determined based on the semantic segmentation result that there is a lesion area in the medical image, subsequent judgment is performed based on the output probability of the image classification model; if it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the output probability is greater than the auxiliary diagnosis probability threshold, it is determined that there is a need to detect the organ part corresponding to the medical image, and accordingly an auxiliary diagnosis prompt is given; if it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the output probability is not greater than the auxiliary diagnosis probability threshold, it is determined that there is no need to detect the organ part corresponding to the medical image, and accordingly no prompt is given.

[0066] The cancer detection method based on medical images provided by the embodiment of the present invention determines whether it is necessary to detect the organ part corresponding to the medical image in the above-mentioned manner, which can improve the accuracy of determining whether detection is necessary, thereby improving the accuracy and efficiency of cancer detection; and only when it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the image classification result is the first classification result, it is determined that the organ part corresponding to the medical image needs to be detected, thereby reducing false positives and improving the specificity of cancer recognition; and even if it is determined based on the semantic segmentation result that there is a lesion area in the medical image, as long as the image classification result is the second classification result, it is determined that there is no need to detect the organ part corresponding to the medical image, thereby reducing false positives and improving the specificity of cancer recognition; and as long as it is determined based on the semantic segmentation result that there is no lesion area in the medical image, there is no need to determine what the image classification result is, and it is determined that there is no need to detect the organ part corresponding to the medical image, thereby reducing false positives and improving the specificity of cancer recognition.

[0067] Based on any of the above embodiments, in the method, the outputting detection prompt information includes: Based on the semantic segmentation result and the image classification result, generating detection prompt information; the detection prompt information includes the lesion area in the medical image, the cancer recognition result indicated by the image classification result, and auxiliary diagnosis suggestions; The detection prompt information is output.

[0068] Specifically, the lesion area in the medical image is determined based on the semantic segmentation result, and the cancer recognition result is determined based on the image classification result, and when it is necessary to detect the organ part corresponding to the medical image, the corresponding auxiliary diagnosis suggestion is generated, thereby generating detection prompt information including the lesion area in the medical image, the cancer recognition result indicated by the image classification result and the auxiliary diagnosis suggestion.

[0069] For example, if the medical image is a lung CT image, the detection prompt information includes lung nodule area (lesion area), malignant lung nodule (cancer identification result) and biopsy suggestion (auxiliary diagnosis suggestion).

[0070] The medical image-based cancer detection method provided by the embodiment of the present invention generates detection prompt information based on semantic segmentation results and image classification results, so that the detection prompt information includes the lesion area in the medical image, the cancer recognition result indicated by the image classification result and auxiliary diagnosis suggestions, so as to more comprehensively prompt doctors to further perform cancer detection, thereby improving the comprehensiveness and accuracy of cancer detection, and further improving the sensitivity of cancer recognition, reducing missed detections, and assisting doctors in making further diagnoses, thereby improving the efficiency of cancer detection.

[0071] Based on any of the above embodiments, before the above step 110, the method further includes: inputting the medical image into a feature extraction model to obtain image features output by the feature extraction model.

[0072] Correspondingly, the above step 110 includes: inputting the image features into an image classification model to obtain an image classification result output by the image classification model.

[0073] Correspondingly, the above step 120 includes: inputting the image features into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model.

[0074] In other words, the semantic segmentation model and the image classification model share a feature extraction model. For example, the semantic segmentation model and the image classification model share a backbone network (trunk grid), and then two detection heads are connected on the basis of the backbone network to realize semantic segmentation and image classification respectively.

[0075] Furthermore, the feature extraction model is used to extract high-level abstract features of medical images. High-level abstract features have stronger semantic expression capabilities and stronger robustness than low-level features, and can be applied to various complex organ image acquisition scenarios, thus having higher accuracy and better generalization capabilities, especially for image classification models, and ultimately improving the accuracy of cancer recognition.

[0076] In the cancer detection method based on medical images provided by the embodiment of the present invention, the semantic segmentation model and the image classification model share a feature extraction model, thereby effectively reducing the amount of computational complexity of model reasoning, thereby improving the efficiency of cancer detection, and reducing the storage space occupied by the model, so as to be suitable for devices with limited hardware resources.

[0077] Based on any of the above embodiments, in the method, the semantic segmentation model is an integrated semantic segmentation model, and the integrated semantic segmentation model includes multiple different segmentation models.

[0078] For any pixel of the medical image, the integrated semantic segmentation model is used to determine multiple category prediction results of the pixel, any of which includes lesion pixels or non-lesion pixels. If any of the multiple category prediction results is a lesion pixel, the final category prediction result of the pixel is determined as a lesion pixel. If all of the multiple category prediction results are non-lesion pixels, the final category prediction result of the pixel is determined as a non-lesion pixel. That is, the embodiment of the present invention uses the union of multiple single semantic segmentation model results to determine the final result.

[0079] The existing bagging ensemble learning method uses voting to take the majority as the final result. The embodiment of the present invention takes into account that missed cancer detection is more harmful, so as long as there is a positive result, the final result is positive, that is, if any of the multiple category prediction results is a lesion pixel, the final category prediction result of the pixel is determined as a lesion pixel, thereby improving the sensitivity of cancer recognition and reducing missed detection.

[0080] In the cancer detection method based on medical images provided by an embodiment of the present invention, the semantic segmentation model is an integrated semantic segmentation model, that is, the integrated semantic segmentation model includes multiple different segmentation models. Compared with a single semantic segmentation model, the integrated semantic segmentation model can improve the accuracy of determining the semantic segmentation results, thereby improving the accuracy of cancer recognition; for any pixel of the medical image, the integrated semantic segmentation model is used to determine multiple category prediction results of the pixel, and any category prediction result includes a lesion pixel or a non-lesion pixel. If any category prediction result among the multiple category prediction results is a lesion pixel, the final category prediction result of the pixel is determined as a lesion pixel. If all category prediction results among the multiple category prediction results are non-lesion pixels, the final category prediction result of the pixel is determined as a non-lesion pixel, thereby improving the sensitivity of cancer recognition and reducing missed detections.

[0081] Based on any of the above embodiments, in the method, the image classification result includes a first classification result or a second classification result, and the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result. For example, the first classification result is suspected cancer, and the second classification result is no cancer.

[0082] Accordingly, considering that the image classification result is the first classification result, that is, in the case of a high probability of cancer, since the semantic segmentation model does not determine the lesion area, the doctor needs to further locate the lesion area, resulting in a decrease in the efficiency of cancer detection; based on this, before the above step 130, the method further includes: If it is determined based on the semantic segmentation result that there is no lesion area in the medical image, and the image classification result is the first classification result, the segmentation threshold of the semantic segmentation model is adjusted to obtain and output the lesion area in the medical image.

[0083] Among them, for any pixel of the medical image, the semantic segmentation model determines the category prediction result of the pixel based on the segmentation threshold, and the category prediction result of the pixel includes a lesion pixel or a non-lesion pixel.

[0084] For example, when the image classification model determines that the image is suspected of cancer but the semantic segmentation model does not output the lesion area, the threshold of the semantic segmentation model is dynamically adjusted. By dynamically adjusting the segmentation threshold of the semantic segmentation model, the lesion area is output for the doctor to locate, observe and analyze the lesion area.

[0085] The cancer detection method based on medical images provided by an embodiment of the present invention, through the above-mentioned method, when the image classification result is the first classification result, that is, when the probability of cancer is high, even if the semantic segmentation model has not determined the lesion area, the segmentation threshold of the semantic segmentation model can be adjusted to obtain and output the lesion area in the medical image, to assist doctors in further accurately locating the lesion area, thereby improving the efficiency of cancer detection, and improving the sensitivity of cancer recognition, reducing missed detections.

[0086] Based on any of the above embodiments, in the method, adjusting the segmentation threshold of the semantic segmentation model includes: The lowest segmentation threshold is determined as 0, and the highest segmentation threshold is determined as the current segmentation threshold of the semantic segmentation model; Determine a sum of the minimum segmentation threshold and the maximum segmentation threshold, and adjust the segmentation threshold of the semantic segmentation model to half of the sum; Based on the adjusted segmentation threshold, updating the semantic segmentation result; If it is determined based on the updated semantic segmentation result that the proportion of the lesion area of ​​the medical image is greater than the first preset proportion, the minimum segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the minimum segmentation threshold and the maximum segmentation threshold is returned to adjust the segmentation threshold of the semantic segmentation model to half of the sum; If it is determined based on the updated semantic segmentation result that the proportion of the lesion area of ​​the medical image is less than the second preset proportion, the highest segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the lowest segmentation threshold and the highest segmentation threshold is returned to adjust the segmentation threshold of the semantic segmentation model to half of the sum.

[0087] Among them, for any pixel of the medical image, the semantic segmentation model outputs the predicted value of the pixel. If the predicted value of the pixel is greater than or equal to the segmentation threshold, the category prediction result of the pixel is determined to be a lesion pixel; the lesion area ratio is determined based on the area ratio of the lesion area in the medical image to the area of ​​the medical image, and the first preset ratio is greater than the second preset ratio.

[0088] Here, the highest segmentation threshold is greater than the lowest segmentation threshold. Both the highest segmentation threshold and the lowest segmentation threshold can be dynamically updated.

[0089] Taking into account that the current segmentation threshold based on the semantic segmentation model does not output the lesion area, and if the segmentation threshold is 0, the lesion area output by the semantic segmentation model is the entire medical image, based on this, the initial minimum segmentation threshold is set to 0, and the initial maximum segmentation threshold is the current segmentation threshold of the semantic segmentation model.

[0090] Here, the first preset ratio and the second preset ratio can be set according to actual conditions. For example, according to data statistical analysis, the lesion area of ​​a positive medical image generally accounts for 5%-80% of the entire positive medical image. Based on this, the first preset ratio is 80% and the second preset ratio is 5%.

[0091] Exemplarily, assuming that the current segmentation threshold of the semantic segmentation model is threshold_init, the first preset proportion is 80%, and the second preset proportion is 5%. Based on this, the segmentation threshold of the semantic segmentation model is adjusted in the open interval (0, threshold_init) until the semantic segmentation model outputs the lesion area. Furthermore, the use of binary search can improve the accuracy and rationality of determining the lesion area; specifically, in the first step, set the lowest segmentation threshold threshold_low=0, the highest segmentation threshold threshold_high=threshold_init, in the second step, set the segmentation threshold threshold of the semantic segmentation model threshold=(threshold_low + threshold_high) / 2, in the third step, based on the adjusted segmentation threshold, update the semantic segmentation result, that is, according to the new segmentation threshold, obtain the lesion area, and determine the area ratio P of the lesion area to the entire medical image, in the fourth step, if 0.05≤P≤0.80, determine the current threshold as a reasonable segmentation threshold and output the lesion area corresponding to this segmentation threshold, if P>0.80, then threshold_low=threshold, and return to the second step until the final lesion area satisfies 0.05≤P≤0.80, if P<0.05, then threshold_high=threshold, and return to the second step until the final lesion area satisfies 0.05≤P≤0.80.

[0092] The medical image-based cancer detection method provided by an embodiment of the present invention determines the sum of the minimum segmentation threshold and the maximum segmentation threshold in the above manner, and adjusts the segmentation threshold of the semantic segmentation model to half of the sum, so as to accurately and dynamically adjust the segmentation threshold, thereby improving the positioning sensitivity of the lesion area, thereby improving the detection rate of cancer; and ensuring that the proportion of the lesion area is between the first preset proportion and the second preset proportion, thereby further improving the positioning accuracy of the lesion area, thereby further improving the detection accuracy of cancer.

[0093] Based on any of the above embodiments, the method further includes: Inputting the medical image into an organ part recognition model to obtain an organ part recognition result output by the organ part recognition model; Output the prompt information corresponding to the organ part recognition result to prompt the organ part currently being examined.

[0094] The organ part recognition model is obtained by training based on sample medical images and organ part recognition result labels corresponding to the sample medical images.

[0095] For example, if the organ part recognition result is the upper anterior wall of the gastric body, the corresponding prompt information may be "the digestive tract part currently being examined is the upper anterior wall of the gastric body".

[0096] Furthermore, the organ part recognition model can share a feature extraction model with the image classification model and the semantic segmentation model, that is, the image features of the medical image are input into the organ part recognition model to obtain the organ part recognition results output by the organ part recognition model.

[0097] The medical image-based cancer detection method provided by the embodiment of the present invention inputs the medical image into an organ part recognition model to obtain an organ part recognition result output by the organ part recognition model; outputs prompt information corresponding to the organ part recognition result to prompt the organ part currently being examined, thereby assisting the doctor to further judge the organ part currently to be examined and improving the detection efficiency of organ cancer.

[0098] Based on the above embodiments, an inspection quality control report (such as an upper gastrointestinal endoscopic tumor screening quality control report) can be generated for subsequent reading by doctors or patients.

[0099] Based on the above embodiments, the principle of hyperparameter selection in the training stage of the above models is: to maximize the sensitivity while ensuring that the specificity is not less than 90%. For ease of understanding, assume that the number of positive samples in the sample test set is P, and the number of negative samples is F; the number of true positive samples STP and the number of false positive samples SFP determined by the semantic segmentation results of the semantic segmentation model; the number of true positive samples CTP and the number of false positive samples CFP determined by the image classification results of the image classification model; based on this, the sensitivity is (STP+CTP) / P, and the specificity is (F-SFP-CFP) / F.

[0100] Based on the above embodiments, the present invention can significantly improve the sensitivity of cancer identification while ensuring a certain biopsy positivity rate, and at the same time achieve targeted positioning of suspected lesion areas.

[0101] The following is a description of a cancer detection device based on medical images provided by the present invention. The cancer detection device based on medical images described below and the cancer detection method based on medical images described above can refer to each other.

[0102] Figure 6 is a schematic diagram of the structure of a cancer detection device based on medical images provided by the present invention, such as Figure 6 As shown, the cancer detection device based on medical images includes an image classification module 610 , a semantic segmentation module 620 , a detection determination module 630 and a prompt output module 640 .

[0103] The image classification module 610 is used to input the medical image into the image classification model to obtain the image classification result output by the image classification model; the image classification result is used to represent the recognition result of cancer recognition for the medical image.

[0104] The semantic segmentation module 620 is used to input the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion area in the medical image.

[0105] The detection determination module 630 is used to determine whether it is necessary to detect the organ part corresponding to the medical image based on the semantic segmentation result and the image classification result.

[0106] The prompt output module 640 is used to output detection prompt information when it is necessary to detect the organ part corresponding to the medical image.

[0107] The cancer detection device based on medical images provided by the embodiment of the present invention inputs the medical image into an image classification model to obtain an image classification result output by the image classification model, and inputs the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model, and the image classification result is used to characterize the recognition result of cancer recognition for the medical image, and the semantic segmentation result is used to indicate whether there is a lesion area in the medical image, so that the fusion of semantic segmentation and image classification can improve the recognition accuracy of cancer, and further improve the sensitivity of cancer recognition, and when any one of the image classification result and the semantic segmentation result indicates the presence of cancer, it can be preliminarily determined that cancer exists. Thereby improving the sensitivity of cancer identification, ensuring a certain biopsy positivity rate, and reducing missed detections; and determining whether it is necessary to detect the organ part corresponding to the medical image based on the semantic segmentation results and the image classification results, assisting the doctor in determining whether the detection is necessary, that is, assisting the doctor in further determining the diagnosis method based on the two results, thereby improving the efficiency of cancer detection, and improving the sensitivity of cancer identification, and reducing missed detections; and in the case where it is necessary to detect the organ part corresponding to the medical image, outputting detection prompt information to prompt the doctor to further perform cancer detection, assisting the doctor in making further diagnosis, thereby improving the efficiency of cancer detection, and improving the sensitivity of cancer identification, and reducing missed detections. In summary, the present invention can achieve efficient, highly sensitive, and highly accurate cancer detection.

[0108] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute a cancer detection method based on a medical image, the method comprising: inputting a medical image into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used to characterize the recognition result of cancer recognition for the medical image; inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion area in the medical image; based on the semantic segmentation result and the image classification result, determining whether it is necessary to detect the organ part corresponding to the medical image; and outputting detection prompt information when it is necessary to detect the organ part corresponding to the medical image.

[0109] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0110] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the cancer detection method based on medical images provided by the above-mentioned methods, and the method includes: inputting the medical image into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used to characterize the recognition result of cancer recognition for the medical image; inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion area in the medical image; based on the semantic segmentation result and the image classification result, determining whether it is necessary to detect the organ part corresponding to the medical image; if it is necessary to detect the organ part corresponding to the medical image, outputting detection prompt information.

[0111] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the medical image-based cancer detection method provided by the above-mentioned methods, the method comprising: inputting a medical image into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used to characterize the recognition result of cancer recognition for the medical image; inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion area in the medical image; based on the semantic segmentation result and the image classification result, determining whether it is necessary to detect the organ part corresponding to the medical image; and if it is necessary to detect the organ part corresponding to the medical image, outputting detection prompt information.

[0112] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cancer detection method based on medical images, characterized in that: include: Inputting the medical image into the image classification model to obtain the image classification result output by the image classification model; The image classification result is used to represent the recognition result of cancer recognition on the medical image; Inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion area in the medical image; Based on the semantic segmentation result and the image classification result, determining whether it is necessary to detect the organ part corresponding to the medical image; When it is necessary to detect the organ part corresponding to the medical image, detection prompt information is output.

2. The cancer detection method based on medical images according to claim 1, characterized in that: The image classification result includes a first classification result or a second classification result, and the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result; The determining whether it is necessary to detect the organ part corresponding to the medical image based on the semantic segmentation result and the image classification result includes: If it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the image classification result is the first classification result, it is determined that the organ part corresponding to the medical image needs to be detected; If it is determined based on the semantic segmentation result that there is a lesion area in the medical image, and the image classification result is the second classification result, it is determined that there is no need to detect the organ part corresponding to the medical image; If it is determined based on the semantic segmentation result that no lesion area exists in the medical image, it is determined that there is no need to detect the organ part corresponding to the medical image.

3. The cancer detection method based on medical images according to claim 1, characterized in that: The output detection prompt information includes: Based on the semantic segmentation result and the image classification result, generating detection prompt information; the detection prompt information includes the lesion area in the medical image, the cancer recognition result indicated by the image classification result, and auxiliary diagnosis suggestions; The detection prompt information is output.

4. The cancer detection method based on medical images according to claim 1, characterized in that: Before inputting the medical image into the image classification model to obtain the image classification result output by the image classification model, the method further includes: Inputting the medical image into a feature extraction model to obtain image features output by the feature extraction model; Accordingly, the step of inputting the medical image into the image classification model to obtain the image classification result output by the image classification model includes: Inputting the image features into an image classification model to obtain an image classification result output by the image classification model; Accordingly, the step of inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model includes: The image features are input into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model.

5. The method for cancer detection based on medical images according to any one of claims 1 to 4, characterized in that: The semantic segmentation model is an integrated semantic segmentation model, and the integrated semantic segmentation model includes a plurality of different segmentation models; For any pixel of the medical image, the integrated semantic segmentation model is used to determine multiple category prediction results of the pixel, any of the category prediction results including lesion pixels or non-lesion pixels, if any category prediction result among the multiple category prediction results is a lesion pixel, then the final category prediction result of the pixel is determined as a lesion pixel, if all category prediction results among the multiple category prediction results are non-lesion pixels, then the final category prediction result of the pixel is determined as a non-lesion pixel.

6. The method for cancer detection based on medical images according to any one of claims 1 to 4, characterized in that: The image classification result includes a first classification result or a second classification result, and the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result; Before determining whether it is necessary to detect the organ part corresponding to the medical image based on the semantic segmentation result and the image classification result, the method further includes: If it is determined based on the semantic segmentation result that there is no lesion area in the medical image, and the image classification result is the first classification result, adjusting the segmentation threshold of the semantic segmentation model to obtain and output the lesion area in the medical image; Among them, for any pixel of the medical image, the semantic segmentation model determines the category prediction result of the pixel based on the segmentation threshold, and the category prediction result of the pixel includes a lesion pixel or a non-lesion pixel.

7. The method for cancer detection based on medical images according to claim 6, characterized in that: The adjusting the segmentation threshold of the semantic segmentation model includes: The lowest segmentation threshold is determined as 0, and the highest segmentation threshold is determined as the current segmentation threshold of the semantic segmentation model; Determine a sum of the minimum segmentation threshold and the maximum segmentation threshold, and adjust the segmentation threshold of the semantic segmentation model to half of the sum; Based on the adjusted segmentation threshold, updating the semantic segmentation result; If it is determined based on the updated semantic segmentation result that the proportion of the lesion area of ​​the medical image is greater than the first preset proportion, the minimum segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the minimum segmentation threshold and the maximum segmentation threshold is returned to adjust the segmentation threshold of the semantic segmentation model to half of the sum; If it is determined based on the updated semantic segmentation result that the proportion of the lesion area of ​​the medical image is less than the second preset proportion, the highest segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the lowest segmentation threshold and the highest segmentation threshold is returned to adjust the segmentation threshold of the semantic segmentation model to half of the sum; Among them, for any pixel of the medical image, the semantic segmentation model outputs the predicted value of the pixel. If the predicted value of the pixel is greater than or equal to the segmentation threshold, the category prediction result of the pixel is determined to be a lesion pixel; the lesion area ratio is determined based on the area ratio of the lesion area in the medical image to the area of ​​the medical image, and the first preset ratio is greater than the second preset ratio.

8. The method for cancer detection based on medical images according to any one of claims 1 to 4, characterized in that: The method further comprises: Inputting the medical image into an organ part recognition model to obtain an organ part recognition result output by the organ part recognition model; Outputting prompt information corresponding to the organ part recognition result to indicate the organ part currently being examined; The organ part recognition model is obtained by training based on sample medical images and organ part recognition result labels corresponding to the sample medical images.

9. A cancer detection device based on medical images, characterized in that: include: An image classification module, used to input a medical image into an image classification model to obtain an image classification result output by the image classification model; The image classification result is used to represent the recognition result of cancer recognition on the medical image; A semantic segmentation module, used for inputting the medical image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used for indicating whether there is a lesion area in the medical image; A detection determination module, used to determine whether it is necessary to detect the organ part corresponding to the medical image based on the semantic segmentation result and the image classification result; The prompt output module is used to output detection prompt information when it is necessary to detect the organ part corresponding to the medical image.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the medical image-based cancer detection method according to any one of claims 1 to 8 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the medical image-based cancer detection method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the medical image-based cancer detection method according to any one of claims 1 to 8 is implemented.

Images