Image processing method and computer-aided diagnosis method for cancer

Abstract

Provided in the present disclosure are an image processing method and a computer-aided diagnosis method for cancer. The image processing method comprises: receiving an image processing task, the image processing task carrying a plurality of target images corresponding to a target detection region, and the image processing task being used for detecting whether there is an abnormal object in the target detection region; and inputting the plurality of target images into an image processing model to obtain a detection result corresponding to a target detection region, wherein the image processing model generates object feature information and focus feature information on the basis of the target images, generates object fusion feature information on the basis of the object feature information and the focus feature information, determines object classification information on the basis of the object fusion feature information, determines focus classification information and focus segmentation information on the basis of the focus feature information, and generates a detection result on the basis of the object classification information, the focus classification information and the focus segmentation information. The method can improve the accuracy of image detection.

Description

Image processing methods, computer-aided diagnosis methods for cancer

[0001] This disclosure claims priority to Chinese Patent Application No. 202410927473.X, filed on July 10, 2024, entitled “Image Processing Method, Computer-Aided Diagnosis Method for Cancer”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of computer technology, and in particular to an image processing method. Background Technology

[0003] As people's living standards improve, more and more people are paying attention to their health. Cancer, as a significant factor affecting people's health, plays a positive role in early diagnosis and treatment. With the continuous development of computer technology, various learning models are gradually being applied to prediction in various application scenarios. Image processing models have also achieved significant success in computer-aided diagnosis (CAD) tasks in medical imaging. Pathological analysis and classification from medical images is an important topic in computer-aided diagnosis.

[0004] Currently, tumor screening technology is mainly based on contrast-enhanced CT. While contrast-enhanced CT offers high accuracy, it exposes patients to contrast agents, which may cause allergic reactions or organ failure. Furthermore, limitations in technology and equipment prevent 24-hour availability in all regions, impacting its convenience. Plain CT, due to its lower resolution, makes lesions visually difficult to identify, resulting in a slightly lower detection rate compared to contrast-enhanced CT. Marking lesions on plain CT also presents challenges. Therefore, it is crucial to explore how to utilize plain CT for tumor screening and improve its accuracy. Summary of the Invention

[0005] In view of the above, this disclosure provides an image processing method. One or more embodiments of this specification also relate to a CT image processing method, a computer-aided diagnosis method for cancer, a computer-aided diagnosis method for liver cancer, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of this disclosure, an image processing method is provided, comprising:

[0007] The image processing task is received, wherein the image processing task carries multiple target images corresponding to the target detection area, and the image processing task is used to detect whether there is an abnormal object in the target detection area;

[0008] The multiple target images are input into an image processing model to obtain the detection results corresponding to the target detection regions. The image processing model generates object feature information and lesion feature information based on each target image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates detection results based on the object classification information, the lesion classification information, and the lesion segmentation information.

[0009] According to a second aspect of this disclosure, a CT image processing method is provided, comprising:

[0010] Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormal object in the target detection area;

[0011] The multiple plain CT images are input into an image processing model to obtain the detection results corresponding to the target detection area. The image processing model generates object feature information and lesion feature information based on each plain CT image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates detection results based on the object classification information, the lesion classification information, and the lesion segmentation information.

[0012] According to a third aspect of this disclosure, a computer-aided diagnostic method for cancer is provided, comprising:

[0013] Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is a tumor in the target detection area;

[0014] The multiple plain CT images are input into an image processing model to obtain the detection results corresponding to the target detection area. The image processing model generates object feature information and lesion feature information based on each plain CT image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates detection results based on the object classification information, the lesion classification information, and the lesion segmentation information.

[0015] According to a fourth aspect of this disclosure, a computer-aided diagnostic method for liver cancer is provided, comprising:

[0016] Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the liver region, and the CT image processing task is used to detect whether there is a tumor in the liver region;

[0017] The multiple plain CT images are input into an image processing model to obtain the detection results corresponding to the liver region. The image processing model generates object feature information and lesion feature information based on each plain CT image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates detection results based on the object classification information, the lesion classification information, and the lesion segmentation information.

[0018] According to a fifth aspect of this disclosure, a computer-aided diagnostic system for cancer is provided, comprising a client and a server, wherein:

[0019] The client is used to send a CT image processing task to the server, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is a tumor in the target detection area;

[0020] The server is used to input the multiple plain CT images into an image processing model to obtain the detection result corresponding to the target detection area. The image processing model generates object feature information and lesion feature information based on each plain CT image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates the detection result based on the object classification information, the lesion classification information, and the lesion segmentation information.

[0021] According to a sixth aspect of this disclosure, a computing device is provided, comprising:

[0022] Memory and processor;

[0023] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above method.

[0024] According to a seventh aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0025] According to the eighth aspect of this disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0026] One embodiment of this specification provides an image processing method, including receiving an image processing task, wherein the image processing task carries multiple target images corresponding to a target detection region, and the image processing task is used to detect whether there is an abnormal object in the target detection region; inputting the multiple target images into an image processing model to obtain a detection result corresponding to the target detection region, wherein the image processing model generates object feature information and lesion feature information based on each target image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates a detection result based on the object classification information, the lesion classification information, and the lesion segmentation information.

[0027] The image processing method provided in this disclosure inputs multiple target images into an image processing model. In the image processing model, object feature information and lesion feature information are generated based on each target image. At the same time, the object feature information and lesion feature information are fused to generate object fusion feature information. This allows the model to use not only object feature information but also lesion feature information when predicting the final object classification information, thus fusing global and local information and improving the detection capability of small lesions. Attached Figure Description

[0028] Figure 1 is a flowchart of an image processing method provided in one embodiment of this specification;

[0029] Figure 2 is a schematic diagram of the model structure of an image processing model provided in one embodiment of this specification;

[0030] Figure 3 is a schematic diagram of data processing of an object branch network provided in an embodiment of this specification;

[0031] Figure 4 is a schematic diagram of data processing of a lesion branch network provided in one embodiment of this specification;

[0032] Figure 5 is a schematic diagram of data processing of a fusion network provided in one embodiment of this specification;

[0033] Figure 6 is a flowchart of an image processing model training method provided in one embodiment of this specification;

[0034] Figure 7 is a flowchart of a CT image processing method provided in one embodiment of this specification;

[0035] Figure 8 is a flowchart illustrating a computer-aided diagnosis method for cancer according to an embodiment of this specification;

[0036] Figure 9 is a flowchart illustrating a computer-aided diagnostic method for liver cancer according to an embodiment of this specification;

[0037] Figure 10 is an architecture diagram of a computer-aided diagnosis system for cancer provided in one embodiment of this specification;

[0038] Figure 11 is a schematic diagram of an image processing apparatus provided in one embodiment of this specification;

[0039] Figure 12 is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0040] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0041] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0042] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0043] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0044] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0045] CT (Computed Tomography): Computed tomography is a scanning technology that uses a precisely collimated X-ray beam and a highly sensitive detector to scan a specific part of the body one section after another. It features fast scanning time and clear images and can be used to examine a variety of diseases.

[0046] Plain CT scan: also known as a regular scan, refers to a scan performed intravenously without the administration of contrast agents.

[0047] Enhanced CT: This refers to a scanning method that involves injecting a contrast agent into a blood vessel before scanning. The purpose is to increase the density difference between the diseased tissue and normal tissue, so as to show lesions that are not shown or are not clearly shown on plain CT. The presence or absence of enhancement and the type of enhancement help to characterize the lesion.

[0048] CAD (computer-aided diagnosis): Computer-aided diagnosis refers to the use of imaging, medical image processing technology, and other possible physiological and biochemical methods, combined with computer analysis and calculation, to assist in the detection of lesions and improve the accuracy of diagnosis.

[0049] As people's living standards improve, more and more people are paying attention to their health. Cancer is one of the major factors affecting people's health. Identifying tumors in medical images requires professional doctors to rely on their experience. However, due to the limitations of doctors' experience, image recognition and analysis using medical images has become an important topic.

[0050] Current cancer screening technologies are primarily based on contrast-enhanced CT scans. While contrast-enhanced CT offers high accuracy, it exposes patients to contrast agents, which can potentially cause allergic reactions or organ failure. Furthermore, limitations in technology and equipment restrict the availability of contrast-enhanced CT scans to 24 / 7 operation in all regions, impacting their convenience. Some screening techniques are based on ultrasound, but their sensitivity is low, leading to missed diagnoses. Other techniques are based on plain CT scans, but these have lower resolution, making lesions on organs difficult to visually identify, resulting in a slightly lower detection rate compared to contrast-enhanced CT. Therefore, improving tumor detection remains a pressing issue for researchers.

[0051] Based on this, an image processing method is provided in this specification. One or more embodiments of this specification also relate to a CT image processing method, a computer-aided diagnosis method for cancer, a computer-aided diagnosis method for liver cancer, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0052] Referring to Figure 1, Figure 1 shows a flowchart of an image processing method provided according to an embodiment of this specification, which specifically includes the following steps.

[0053] Step 102: Receive an image processing task, wherein the image processing task carries multiple target images corresponding to the target detection area, and the image processing task is used to detect whether there are abnormal objects in the target detection area.

[0054] In practical applications, image processing tasks sent by users can be received through either a server or a client.

[0055] Specifically, an image processing task can be understood as a task to detect whether there are abnormal objects within a target detection region. The image processing task carries multiple target images corresponding to the target detection region. Furthermore, the target detection region can be understood as a partition used to detect the presence of abnormal objects, and the abnormal object can be understood as a foreign object within the target detection region. For example, an abnormal object could be a tumor within the human body.

[0056] The target detection area can be any organ within the human body, such as the liver, lungs, stomach, or esophagus. By predicting the presence of abnormal objects within the target detection area, the state of the object to be detected can be further determined based on the prediction results, thereby aiding in the localization and precise treatment of abnormal objects.

[0057] The object to be detected can be understood as the object described in the target detection area. For example, if the target image is a CT image of Zhang San's stomach, then the target detection area is the stomach, the CT image is the target image, and the object to be detected is Zhang San. This image processing task is used to detect whether there is a tumor in the stomach area. In practical applications, the object to be detected can be a person or other living organisms, and this is not limited in one or more specific embodiments provided in this specification.

[0058] It should be noted that, in one or more embodiments of this specification, the image processing task can be applied to the recognition of various types of medical images, and to determine whether there are abnormal objects in the target detection area of ​​the medical image based on image features. For example, in the application scenario of gastric cancer detection, the presence of a tumor in the stomach region can be predicted based on a medical image of the stomach region, thereby helping doctors to accurately locate abnormal areas; in the application scenario of liver cancer detection, the presence of a tumor in the liver can be predicted based on a medical image of the liver region, thereby helping doctors to accurately locate abnormal areas, facilitating subsequent treatment.

[0059] For example, in a liver cancer detection scenario, the target image acquired is an image of the liver. Specifically, multiple target images acquired are plain CT images of the liver, which can be combined to form a 3D image of the liver region. Anomalies can be understood as tumors within the liver. By acquiring multiple plain CT images corresponding to the liver region and performing image detection processing on these images, the presence of tumors in the liver can be detected.

[0060] In practical applications, the abnormal object can be a certain type of cell, a certain type of tissue structure, etc., such as a malignant tumor, a benign tumor, proliferating tissue, etc. This is not limited in the one or more embodiments provided in this specification.

[0061] By receiving an image processing task, multiple target images corresponding to the target detection area carried in the image processing task can be used as input to detect whether there are abnormal objects within the target detection area.

[0062] In one specific embodiment provided in this specification, before receiving the image processing task, the method further includes:

[0063] Receive an image segmentation task, wherein the image segmentation task carries multiple initial images corresponding to a target detection region, and the image segmentation task is used to extract the target image corresponding to the target detection region;

[0064] Each initial image is input into a pre-trained image segmentation model to obtain the target image corresponding to each initial image output by the image segmentation model.

[0065] In the embodiments provided in this disclosure, the target image can be understood as a close-up image corresponding to the target detection region. However, in practical applications, multiple received images may contain other regions besides the target detection region, and these other regions can affect the target detection region. Therefore, the method provided in this disclosure also performs image segmentation processing.

[0066] Specifically, the first step is to obtain an image segmentation task from multiple initial images. These initial images include both target detection regions and regions that influence the target detection regions. This image segmentation task is used to extract the target images corresponding to the target detection regions from each initial image.

[0067] Multiple initial images are input into a pre-trained image segmentation model for processing. The image segmentation model is trained to identify target detection regions in the initial images, extract the target detection regions from the initial images, and generate target images corresponding to the target detection regions.

[0068] Taking plain CT images as an example, the method provided in this manual first obtains initial plain CT images. These initial plain CT images are those that meet the image quality requirements. The sources of these initial plain CT images can be multiple CT scanners or the same CT scanner; this manual does not impose any limitations on this. After obtaining the initial plain CT images, the format of each initial plain CT image must be standardized. The image segmentation model can be 3DUNet.

[0069] 3DUNet is a deep learning architecture for image segmentation in three-dimensional space. It is an extended version of the U-Net model, which was originally designed for semantic segmentation of two-dimensional biomedical images and has gained widespread recognition for its excellent performance and high accuracy in segmenting small objects. 3DUNet applies this idea to three-dimensional datasets, such as medical images (e.g., CT, MRI scans), which is very useful in many medical fields because the data in these fields often have rich three-dimensional structural information.

[0070] 3DUNet includes an encoder-decoder structure to capture global contextual information, recover lost spatial details, and generate accurate pixel-level segmentation labels. In the 3DUNet structure, 3D convolutional kernels replace 2D convolutional kernels, allowing simultaneous processing of input data features in three dimensions (length, width, and height). Furthermore, 3DUNet effectively fuses 3D features at different levels, facilitating the extraction of complex shape and structural information. This framework demonstrates good performance in medical image segmentation. In the method provided in this specification, the initial CT image is segmented using a preprocessing strategy based on the 3DUNet structure to obtain the target image corresponding to the target detection region for subsequent processing.

[0071] In this disclosure, the method for detecting the liver region is used as an example for explanation. The initial plain CT image is resampled to a size of 0.7*0.7*5mm in the x, y, and z directions. Using the 3DUNet segmentation model, the liver and several surrounding organs are masked on the initial plain CT image. The surrounding organs include the gallbladder, intrahepatic vessels, spleen, stomach, pancreas, right kidney, inferior vena cava, etc.

[0072] Next, based on the masks of the liver and spleen, their corresponding bounding boxes are determined. Then, the bounding boxes are expanded outward by a preset range of pixel distance to obtain the target image. The purpose of obtaining the regions of the liver and spleen is to use the spleen as a reference organ for the liver, so that when the subsequent model processes the image, it can learn not only the features of the target detection region (liver) but also the features of the reference detection region (spleen). Thus, the features of the liver need to be considered, while the features of the spleen do not need to be considered.

[0073] Step 104: Input the multiple target images into the image processing model to obtain the detection results corresponding to the target detection regions. The image processing model generates object feature information and lesion feature information based on each target image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates detection results based on the object classification information, the lesion classification information, and the lesion segmentation information.

[0074] After receiving an image processing task, multiple target images carried by the task can be obtained. By inputting these multiple target images into the image processing model, the detection results corresponding to the target detection area output by the image processing model can be obtained. The detection results specifically include the object classification information of the object to be detected, the lesion segmentation information and lesion classification information on the target detection area.

[0075] The object classification information of the object to be detected can be understood as classification information at the object level, which includes whether there are abnormal objects, the type of abnormal objects, and the probability of abnormal objects corresponding to each category.

[0076] Lesion segmentation information can be understood as mask information for abnormal objects on the image to be detected. In the method provided in this disclosure, lesion segmentation information includes global mask information and local mask information. Global mask information can be understood as the mask information of all abnormal objects in the target detection region, and local mask information can be understood as the mask information for each type of abnormal object. For example, if there are 3 abnormal objects in a certain target detection region, the lesion segmentation information includes 4 parts: the complete mask information of the three abnormal objects, and the mask information corresponding to each of the three abnormal objects.

[0077] Lesion classification information can be understood as determining the type of lesion in a single step after image processing model recognition, provided lesion segmentation information exists. Furthermore, lesion classification information includes lesion category information and organ category information. In the liver tumor detection provided in this disclosure, lesions are divided into 9 categories (hepatocellular carcinoma, cholangiocarcinoma, metastatic tumors, hemangiomas, focal nodular hyperplasia, cysts, calcifications, other malignant tumors, and other benign tumors), and organs are divided into 8 categories (liver, gallbladder, intrahepatic vessels, spleen, stomach, pancreas, right kidney, and inferior vena cava).

[0078] In the method provided in this disclosure, after receiving multiple target images, the image processing model processes the multiple target images to generate object feature information and lesion feature information. The object feature information can be understood as feature information used to generate object classification information, and the lesion feature information can be understood as feature information used to generate lesion classification information and lesion segmentation information. Specifically, the lesion feature information includes lesion sub-feature information and organ sub-feature information. The lesion sub-feature information is used to determine the lesion type, and the organ sub-feature information is used to determine the organ type.

[0079] In the method provided in this disclosure, although object feature information can be used to generate object classification information, lesion feature information is fused with object feature information to improve prediction accuracy. Based on this, object feature information and lesion feature information can be fused to obtain object fused feature information. The object fused feature information is still used for predicting object classification information; compared to object feature information, it includes relevant content related to lesion feature information, making subsequent predictions more accurate.

[0080] Finally, the image processing model determines object classification information based on object fusion feature information, and determines lesion classification information and lesion segmentation information based on lesion feature information, thus forming the final detection result.

[0081] The method provided in this disclosure further explains the model structure of the image processing model. Specifically, the image processing model includes an image processing backbone network, an object branch network, a lesion branch network, and a fusion network.

[0082] Referring to Figure 2, which shows a schematic diagram of the model structure of an image processing model provided in an embodiment of this specification, the image processing model includes an image processing backbone network, an object branch network, a lesion branch network, and a fusion network.

[0083] The plurality of target images are input into an image processing model to obtain the detection results corresponding to the target detection regions, including S1042-S1048:

[0084] S1042. Input the multiple target images into the image processing backbone network, and extract the feature information of the object to be processed and the feature information of the lesion to be processed from the image processing backbone network.

[0085] Multiple target images can be combined to form a 3D detection region for the target detection area. By inputting multiple target images into the image processing backbone network for processing, the feature information of the object to be processed and the feature information of the lesion to be processed generated by the image processing backbone network during the processing of multiple target images can be extracted.

[0086] It is important to note that both the object feature information and the lesion feature information are intermediate parameter features generated by the image processing model during the processing of multiple target images. Based on their respective uses, they are categorized as object feature information and lesion feature information. If a parameter feature is used to generate both object feature information and lesion feature information, then that parameter feature belongs to both categories.

[0087] Specifically, the image processing backbone network includes an image encoder and an image decoder;

[0088] The multiple target images are input into the image processing backbone network, and the feature information of the object to be processed and the feature information of the lesion to be processed are extracted from the image processing backbone network, including:

[0089] The plurality of target images are input into the image encoder to obtain target image coding features and at least one multi-scale image coding feature;

[0090] The target image encoding features are input into the image decoder to obtain at least one first multi-scale image decoding feature, at least one second multi-scale image decoding feature, and the target image decoding feature;

[0091] The at least one multi-scale image coding feature and the at least one first multi-scale image decoding feature are determined as the feature information of the object to be processed, and the at least one second multi-scale image decoding feature and the target image decoding feature are determined as the lesion feature information to be processed.

[0092] The method provided in this disclosure uses the nnU-Net framework to build an image processing model. The image processing backbone network includes an image encoder and an image decoder. In this embodiment, the U-Net Pixel Encoder is used as the image encoder, and the feature pyramid network is used as the image decoder.

[0093] When multiple target images are input into an image processing model, they are first encoded by an image encoder to obtain target image encoding features and at least one multi-scale image encoding feature. The target image encoding features are the encoding features output by the image encoder, and the at least one multi-scale image encoding feature is the encoding parameter feature generated by the image encoder during the processing of multiple target images.

[0094] The target image encoding features are input into the image decoder to obtain at least one first multi-scale image decoding feature, at least one second multi-scale image decoding feature, and the final target image decoding feature generated by the image decoder during the decoding process.

[0095] At this point, the multi-scale image coding features and the first multi-scale image decoding features are used as the feature information of the object to be processed, and the second multi-scale image decoding features and the target image decoding features are used as the feature information of the lesion to be processed.

[0096] The image encoder includes multiple sequentially connected image coding layers;

[0097] The plurality of target images are input into the image encoder to obtain target image coding features and at least one multi-scale image coding feature, including:

[0098] The multiple target images are input into the image encoder to obtain the target image encoding features output by the image encoder;

[0099] At least one target image coding layer is determined among the plurality of image coding layers, and the multi-scale image coding features output by each target image coding layer are obtained.

[0100] It's important to note that the target image coding layer doesn't refer to a specific image coding layer, but rather to the image coding layer from which image coding features need to be extracted. For example, if there are six image coding layers in total, and the multi-scale image coding features output from the 1st, 3rd, and 5th image coding layers are needed, then the 1st, 3rd, and 5th image coding layers are the target image coding layers. Similarly, if there are twelve image coding layers in total, and the multi-scale image coding features output from the 9th, 10th, 11th, and 12th image coding layers are needed, then the 9th, 10th, 11th, and 12th image coding layers are the target image coding layers.

[0101] Referring to Figure 2, the image processing backbone network shown in Figure 2 will be used as an example for explanation. The image encoder includes six image coding layers. Multiple target images are input into the image encoder for processing. The 5th and 6th image coding layers encode the target images, thus obtaining the target image coding features output by the image encoder, and the multi-scale image coding features output by the 5th and 6th image coding layers. and

[0102] In another specific embodiment provided in this disclosure, the image decoder includes a plurality of sequentially connected image decoding layers;

[0103] The target image encoding features are input into the image decoder to obtain at least one first multi-scale image decoding feature and at least one second multi-scale image decoding feature, including:

[0104] At least one first image decoding layer and at least one second image decoding layer are determined among the plurality of image decoding layers;

[0105] Obtain the first multi-scale image decoding features output by the first image decoding layer, and obtain the second multi-scale image decoding features output by the second image decoding layer.

[0106] Similar to the target image coding layer mentioned above, the first image decoding layer and the second image decoding layer in the image decoding layer do not refer to a specific image decoding layer, but rather to the image decoding layer used to determine the first multi-scale image decoding features and the second multi-scale image decoding features.

[0107] Referring to Figure 2, and taking the image processing backbone network shown in Figure 2 as an example for explanation, the image decoder includes 6 image decoding layers. The target image encoded features are input into the image decoder. It is determined that the output features of the 5th and 6th image decoding layers are the first multi-scale image decoding features, and the output features of the 2nd, 3rd, and 4th image decoding layers are the second multi-scale image decoding features. Therefore, the first multi-scale image decoding features output by the 5th and 6th image decoding layers are obtained. and and the second multi-scale image decoding features output from the 2nd, 3rd, and 4th image decoding layers. and and the target image decoding features output by the final image decoder

[0108] Taking Figure 2 as an example, multi-scale image coding features and First multi-scale image decoding features and The feature information of the object to be processed, the second multi-scale image decoding features and and target image decoding features The information constitutes the characteristic features of the lesion to be treated.

[0109] S1044. Input the feature information of the object to be processed into the object branch network to obtain object feature information, and input the feature information of the lesion to be processed into the lesion branch network to obtain lesion feature information and image feature information to be segmented.

[0110] The object branch network is used to generate object classification information for the object to be processed. After obtaining the feature information of the object to be processed in the above steps, the feature information of the object to be processed is input into the object branch network and processed in the object branch network to obtain the object feature information.

[0111] Referring to Figure 3, Figure 3 illustrates a data processing diagram of an object branching network provided in one embodiment of this specification. As shown in Figure 3, the feature information of the object to be processed is iteratively input into a 4-layer Dual-path Transformer Block (DPB) for processing to obtain the object feature information. The Dual-path Transformer Block is a special Transformer structure that optimizes performance and reduces computational complexity by decomposing the processing into two parallel paths (usually called local paths and global paths), thereby improving processing efficiency and effectiveness. These two paths typically focus on different types of features; the local path focuses more on fine-grained local features, while the global path focuses more on scene-level global features.

[0112] The lesion branch network is used to generate segmentation and classification information for lesions. After obtaining the lesion feature information in the above steps, the lesion feature information is input into the lesion branch network for processing to obtain lesion feature information and image feature information to be segmented.

[0113] Specifically, the lesion feature information to be processed includes at least one second multi-scale image decoding feature and a target image decoding feature;

[0114] The lesion feature information to be processed is input into the lesion branch network to obtain lesion feature information and image feature information to be segmented, including:

[0115] Feature decoding is performed on the decoding features of each second multi-scale image to obtain lesion feature information;

[0116] The lesion feature information and the target image decoding features are fused to generate image feature information to be segmented.

[0117] In the lesion branch network, lesion feature information and image feature information to be segmented are generated simultaneously. The lesion feature information is used to determine lesion classification information, and the image feature information to be segmented is used to determine lesion segmentation information.

[0118] Referring to Figure 4, Figure 4 illustrates a data processing diagram of a lesion branch network provided in an embodiment of this specification. As shown in Figure 4, the lesion branch network includes a Transformer Decoder structure. Fifty randomly initialized queries are input, and the Transformer Decoder performs decoding processing on the second multi-scale image decoding features. As shown in Figure 4, the second multi-scale image decoding features... and The 50 queries are input into the Transformer Decoder for processing, obtaining the lesion feature information output by the Transformer Decoder. Simultaneously, to better segment abnormal objects at the pixel level, the lesion feature information and the target image decoding features output by the image decoder are combined. The images are fused to obtain feature information of the image to be segmented, which is then used to mask abnormal objects in the detection area at the pixel level.

[0119] Thus, the object feature information generated in the object branch network and the lesion feature information and image feature information to be segmented generated in the lesion branch network have been obtained.

[0120] S1046. Input the object feature information and the lesion feature information into the fusion network to generate object fusion feature information.

[0121] A fusion network can be understood as a network used to combine object feature information and lesion feature information to generate object fusion feature information. Object fusion feature information enables image processing models to combine global information of objects and local information of lesions, which can improve the detection capability of smaller abnormal objects.

[0122] In one specific embodiment provided in this disclosure, the object feature information and the lesion feature information are input into the fusion network to generate object fusion feature information, including:

[0123] The object feature information and the lesion feature information are input into the fusion network. The lesion sub-feature information in the lesion feature information is extracted in the fusion network, and the object feature information and the lesion sub-feature information are spliced ​​together to generate object fusion feature information.

[0124] In practical applications, lesion feature information includes lesion sub-feature information and organ sub-feature information. In order to make the lesion features referential to the object features, the lesion sub-feature information and the object feature information can be fused to generate object fused feature information.

[0125] Referring to Figure 5, which illustrates a data processing diagram of a fusion network provided in an embodiment of this specification, as shown in Figure 5, lesion sub-feature information for determining the lesion type is extracted from the lesion feature information. After concatenating the lesion sub-feature information with the object feature information, object fusion feature information is obtained.

[0126] Object fusion feature information is the feature information used to finally determine the object classification information. It includes both object feature information and lesion feature information, making the final prediction result more accurate.

[0127] S1048. Generate lesion classification information based on the lesion feature information, generate lesion segmentation information based on the image feature information to be segmented, generate object classification information based on the object fusion feature information, and generate detection results based on the object classification information, the lesion classification information, and the lesion segmentation information.

[0128] After the above steps, lesion classification information, image feature information to be segmented, and object fusion feature information are obtained. Then, corresponding lesion classification information, lesion segmentation information, and object classification information are generated based on these three pieces of information. Finally, the final detection result is composed of the lesion classification information, lesion segmentation information, and object classification information.

[0129] In practical applications, three different classifiers are used to process different information. Specifically, in a specific embodiment provided in this specification, lesion classification information is generated based on the lesion feature information, and lesion segmentation information is generated based on the image feature information to be segmented, including:

[0130] The lesion feature information is input into the lesion classifier to generate lesion classification information;

[0131] The feature information of the image to be segmented is input into the segmenter to generate lesion segmentation information.

[0132] Based on the object fusion feature information, object classification information is generated, including:

[0133] The object fusion feature information is input into the object classifier to generate object classification information.

[0134] In practical applications, lesion feature information is input into a lesion classifier for processing to obtain lesion classification information output by the lesion classifier; lesion segmentation information is input into a segmenter for processing to obtain lesion segmentation information output by the segmenter; and object fusion feature information is input into an object classifier for processing to obtain object classification information output by the object classifier.

[0135] The image processing method provided in this disclosure inputs multiple target images into an image processing model. In the image processing model, object feature information and lesion feature information are generated based on each target image. At the same time, the object feature information and lesion feature information are fused to generate object fusion feature information. This allows the model to use not only object feature information but also lesion feature information when predicting the final object classification information, thus fusing global and local information and improving the detection capability of small lesions.

[0136] The image processing model provided in this disclosure is a pre-trained image processing model. Referring to Figure 6, Figure 6 shows a flowchart of an image processing model training method provided in an embodiment of this specification. As shown in Figure 6, the image processing model is trained and generated through the following steps:

[0137] Step 602: Obtain multiple sample target images and the corresponding sample detection results, wherein the sample detection results include sample object classification information, sample lesion classification information and sample lesion segmentation information.

[0138] Specifically, the image processing model training method provided in this specification uses supervised training, which includes training sample pairs. Each training sample pair comprises multiple sample target images targeting the target detection region, and corresponding sample detection results for these multiple sample target images. These multiple sample target images can be combined to form a three-dimensional image of the target detection region. The sample detection results specifically include sample object classification information, sample lesion classification information, and sample lesion segmentation information.

[0139] Sample object classification information includes whether there are abnormal objects in the target detection area of ​​the object, the type of abnormal object, and the probability of the abnormal object corresponding to each category. Sample lesion classification information can be understood as the lesion classification for the lesion corresponding to the target detection area, and the organ classification of the organ where the lesion is located. Sample lesion segmentation information can be understood as the mask information for abnormal objects in the target detection area.

[0140] In the method provided in this disclosure, the image processing model is trained in a supervised manner using multiple sample target images and the sample detection results corresponding to the multiple sample target images.

[0141] In one specific embodiment provided in this specification, acquiring multiple sample target images and corresponding sample detection results includes:

[0142] Identify the sample detection object and obtain the sample enhancement image and sample target image corresponding to the sample detection object, wherein the sample enhancement image and sample target image corresponding to the sample detection object correspond to the same target detection region of the sample detection object;

[0143] A reference sample detection result is generated on the enhanced sample image for the target detection region.

[0144] Based on the detection results of the reference sample, the sample detection results are labeled on the sample target image.

[0145] Specifically, a sample-enhanced image refers to an image whose contrast has been enhanced for the target detection region, while a sample target image refers to a regular image of the target detection region. The sample-enhanced image and the sample target image correspond to the same target detection region of the same sample detection object.

[0146] For example, taking Zhang San as the sample detection object and the liver as the target detection area, after determining the sample detection object, the corresponding sample enhancement image and sample target image are obtained based on the target detection area of ​​the sample detection object.

[0147] In practical applications, a sample-enhanced image is an image whose contrast has been enhanced for the target detection region. This allows for better acquisition of the detection results for the target detection region within the sample-enhanced image. In this disclosure, the detection results for the target detection region in the sample-enhanced image serve as the reference sample detection results. The reference sample detection results are then mapped to the target image through registration to obtain the corresponding sample detection results for the target image.

[0148] The process of generating reference sample detection results for target detection regions on augmented images is usually done through manual annotation. However, manual annotation requires experienced technicians, which is time-consuming, labor-intensive, and costly. In an optional embodiment provided in this disclosure, technicians can annotate a portion of the augmented image. Training sample pairs of augmented images and reference sample detection results are formed. A augmented image annotation model is then trained using these training sample pairs. After the model is trained, unannotated augmented images are input into the model for annotation, thereby generating reference sample detection results for the augmented image.

[0149] For example, in assistive medical scenarios, with user permission, paired enhanced CT images and plain CT images of acceptable quality are obtained from the radiology department. The paired enhanced CT images and plain CT images are CT images of the same sample subject taken at the same time and from the same angle. Enhanced CT images include arterial phase enhanced CT, venous phase enhanced CT, and delayed phase enhanced CT, etc.

[0150] Experienced radiologists and experts were invited to annotate mask information for abnormal objects in the target detection area on some enhanced CT images. Based on the gold standard of pathology for the sample objects, lesion classification information and object classification information for the abnormal objects were provided. The mask information, lesion classification information, and object classification information constituted the reference sample detection results.

[0151] An enhanced CT model is trained using labeled enhanced CT images and reference sample detection results, enabling the model to predict and annotate detection results based on enhanced CT images. Unlabeled enhanced CT images are then input into this model for processing. The model processes the unlabeled enhanced CT images to generate reference sample detection results corresponding to the unlabeled enhanced CT images.

[0152] In the aforementioned steps, since the enhanced CT image and the plain CT image are registered, the detection results of the reference sample on the enhanced CT image can be transferred to the plain CT image as the sample detection results of the plain CT image.

[0153] After the above processing, we can obtain the target image and the corresponding sample detection results. It's important to note that the first part of the target image and the corresponding sample detection results are obtained by registering manually annotated enhanced images and reference detection results, while the second part is obtained by registering model-annotated reference detection results. In subsequent processing, when dividing the target image and the corresponding sample detection results into training and testing sets, to ensure the accuracy of the training data, the testing set is selected from the first part of the manually annotated data for subsequent model performance testing.

[0154] Step 604: Input the multiple sample target images into the image processing model to obtain the prediction detection results and training contrast loss values ​​output by the image processing model.

[0155] After obtaining the target image and the corresponding sample detection results, the image processing model can be trained. At this point, the image processing model is not yet trained and needs to be trained using the target image and the corresponding sample detection results.

[0156] During training, the image processing model outputs predicted detection results and training comparison loss values ​​based on the sample target images. In the specific implementation provided in this specification, the concept of comparison loss value is introduced, that is, a two-stage class-balanced comparison loss function is constructed, and the comparison loss function is processed in the object branch network and the lesion branch network of the image processing model respectively, thereby obtaining the training comparison loss value during the training process.

[0157] In one specific embodiment provided in this disclosure, the image processing model includes an image processing backbone network, an object branch network, a lesion branch network, and a fusion network;

[0158] The multiple sample target images are input into an image processing model to obtain the predicted detection results and training contrastive loss values ​​output by the image processing model, including:

[0159] The multiple sample target images are input into the image processing backbone network to extract the feature information of the sample object to be processed and the feature information of the sample lesion to be processed from the image processing backbone network;

[0160] The feature information of the sample object to be processed is input into the object branch network to obtain the predicted object feature information, and the feature information of the sample lesion to be processed is input into the lesion branch network to obtain the predicted lesion feature information and the predicted image feature information to be segmented.

[0161] In the object branch network, the object comparison loss value is obtained based on the sample object classification information corresponding to the sample target image; in the lesion branch network, the lesion comparison loss value is obtained based on the sample lesion classification information corresponding to the sample target image.

[0162] The predicted object feature information and the predicted lesion feature information are input into the fusion network to generate sample object fusion feature information;

[0163] First object classification information is generated based on the predicted object feature information; second object classification information is generated based on the sample object fusion feature information; predicted lesion classification information is generated based on the predicted lesion feature information; and predicted lesion segmentation information is generated based on the predicted image to be segmented feature information.

[0164] A prediction detection result is generated based on the first object classification information, the second object classification information, the predicted lesion classification information, and the predicted lesion segmentation information. A training comparison loss value is generated based on the object comparison loss value and the lesion comparison loss value.

[0165] During the model training phase, the model structure of the image processing model is the same as that of the model application phase described above. For details regarding the model structure and data processing flow of the image processing model, please refer to the relevant sections of the model application phase described above, which will not be repeated here.

[0166] During the model training phase, the image processing model first processes multiple sample target images in the image processing backbone network. It then extracts the feature information of the sample objects to be processed and the feature information of the sample lesions to be processed generated in the image processing backbone network. The feature information of the sample objects to be processed is input into the object branch network to obtain predicted object feature information; the feature information of the sample lesions to be processed is input into the lesion branch network to obtain predicted lesion feature information and predicted image segmentation feature information. The data processing procedures of the image processing backbone network, object branch network, and lesion branch network during the model training phase are the same as those in the application phase described above, and will not be repeated here.

[0167] It is important to note that a two-stage class-balanced contrastive loss function is introduced during model training. In the object branch network, based on the obtained feature information of the sample object to be processed, a two-class contrastive learning loss function is introduced to determine whether there are abnormal objects in the sample object. In the lesion branch network, based on the obtained lesion feature information of the sample lesion to be processed, a contrastive learning loss function for several abnormal objects is introduced. Through contrastive learning, the image processing model learns the differences in the feature information of the sample objects to be processed between different training batches, and the differences in the feature information of the sample lesions to be processed between different training batches. The contrastive loss function yields the object contrastive loss value and the lesion contrastive loss value.

[0168] In another specific embodiment provided in this disclosure, during the training process of the image processing model provided in this disclosure, if the batch size is too small, it may easily cause the model to collapse. Therefore, the training method of this disclosure uses a method that references historical sample object feature information. Specifically, the sample object feature information is input into the object branch network to obtain predicted object feature information, including:

[0169] Obtain feature information of historical sample objects;

[0170] The historical object feature information and the sample object feature information to be processed are input into the object branch network to obtain the predicted object feature information;

[0171] Accordingly, the lesion feature information of the sample to be processed is input into the lesion branch network to obtain predicted lesion feature information and predicted image feature information to be segmented, including:

[0172] Obtain lesion feature information from historical samples;

[0173] The lesion feature information of the historical samples and the lesion feature information of the samples to be processed are input into the lesion branch network to obtain the predicted lesion feature information and the predicted image feature information to be segmented.

[0174] In this implementation, a memory bank strategy is introduced, adding object feature storage blocks and lesion feature storage blocks to the object branch network and lesion branch network, respectively. The object feature storage block stores historical sample object feature information, while the lesion feature storage block stores historical sample lesion feature information. Both feature storage blocks have the same capacity. However, due to the different data types in the classification problem, there may be a long-tail problem in some classification information, meaning that the amount of data for some categories is insufficient. To address the long-tail problem, the number of features for each category stored in the two feature storage blocks is balanced. During the updating of each feature storage block, the number of features already stored for each category is also considered to ensure that the number of features for each category remains balanced.

[0175] In addition to introducing a contrastive loss function, the training method provided in this disclosure also generates first object classification information based on the predicted object feature information, and generates second object classification information based on the sample object fusion feature information. That is, during the model training process, two object classification information will be generated. One is the first object classification information generated based on the predicted object feature information, and the other is the second object classification information generated based on the predicted object feature information and the predicted lesion feature information. Both will be used as the prediction results of the model during the model training stage and used to calculate the model loss value in the subsequent calculation.

[0176] At the same time, similar to the application stage mentioned above, during the model training stage, predicted lesion classification information will be generated based on the predicted lesion feature information, and predicted lesion segmentation information will be generated based on the predicted image feature information.

[0177] Finally, the first object classification information, the second object classification information, the predicted lesion classification information, and the predicted lesion segmentation information are determined to generate the predicted detection result. The object alignment loss value and the lesion alignment loss value are determined to be the training contrastive loss value.

[0178] Step 606: Calculate the predicted loss value based on the predicted detection results and the sample detection results.

[0179] After obtaining the predicted detection results, the predicted detection results are compared with the sample detection results to calculate the predicted loss value. At this point, the image processing model is not yet trained. By calculating the predicted loss value, the difference between the predicted results and the sample results is determined, thereby further adjusting the parameters of the image processing model and achieving the training of the image processing model.

[0180] Specifically, in one embodiment provided in this specification, calculating the predicted loss value based on the predicted detection result and the sample detection result includes:

[0181] Calculate the first object loss value based on the first object classification information and the sample object classification information;

[0182] Calculate the second object loss value based on the second object classification information and the sample object classification information;

[0183] Calculate the lesion classification loss value based on the predicted lesion classification information and the sample lesion classification information;

[0184] The lesion segmentation loss value is calculated based on the predicted lesion segmentation information and the sample lesion segmentation information.

[0185] In practical applications, the predicted detection results include first object classification information and second object classification information. Both are used to calculate loss values ​​along with the sample object classification information. Specifically, the first object loss value is calculated using the first object classification information and the sample object classification information, and the second object loss value is calculated using the second object classification information and the sample object classification information. Similarly, the lesion classification loss value is calculated based on the predicted lesion classification information and the sample lesion classification information, and the lesion segmentation loss value is calculated based on the predicted lesion segmentation information and the predicted lesion classification information. There are many methods for calculating model loss values, such as the cross-entropy loss function, the maximum loss function, the average loss function, etc. This specification does not limit the specific method of the loss function; the actual application shall prevail.

[0186] Step 608: Adjust the model parameters of the image processing model according to the predicted loss value and the training contrast loss value, and continue to train the image processing model until the model training stops.

[0187] After obtaining the predicted loss value and the training comparison loss value, the model parameters of the image processing model can be adjusted by backpropagation based on these two types of loss values.

[0188] Specifically, in one embodiment provided in this disclosure, adjusting the model parameters of the image processing model based on the predicted loss value and the training contrastive loss value includes:

[0189] The model loss value is calculated based on the first object loss value, the second object loss value, the lesion classification loss value, the lesion segmentation loss value, the object comparison loss value, and the lesion comparison loss value;

[0190] Adjust the model parameters of the image processing model based on the model loss value.

[0191] In practical applications, the predicted loss value and the training comparison loss value are fused to obtain a new loss value. Furthermore, according to the preset loss weights, the loss values ​​of the first object, the second object, the lesion classification, the lesion segmentation, the object comparison, and the lesion comparison can be fused to obtain a new model loss value. The model parameters of the image processing model are then adjusted based on the model loss value until the model training stopping condition is met, and a trained image processing model is obtained.

[0192] The image processing model training method disclosed herein proposes to train a sample augmentation image annotation model using sample augmentation images with partial manual annotation. This allows the sample augmentation image annotation model to annotate sample augmentation images that have not been manually annotated, saving annotation time. Furthermore, by registering the sample augmentation image and the target image, the detection results of reference samples on the sample augmentation image are transferred to the target image to obtain the sample detection results of the target image. This solves the problems of difficult and missing annotations on the target image, reducing annotation costs.

[0193] In addition, a two-stage class-balanced contrastive learning loss function was introduced during model training, which enhanced the image processing model's ability to identify and diagnose problems. This improved classification accuracy in both the object branch network and the lesion branch network, especially for long-tail categories with limited feature data, where it achieved better classification and recognition accuracy.

[0194] The image processing method provided in this specification will be further explained below with reference to Figure 7, taking the application of the image processing method in CT image processing as an example. Figure 7 shows a flowchart of a CT image processing method according to an embodiment of this specification, specifically including the following steps.

[0195] Step 702: Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormal object in the target detection area.

[0196] Step 704: Input the multiple plain CT images into the image processing model to obtain the detection result corresponding to the target detection area. The image processing model generates object feature information and lesion feature information based on each plain CT image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates the detection result based on the object classification information, the lesion classification information, and the lesion segmentation information.

[0197] It should be noted that the implementation methods of steps 702 to 704 are the same as those of steps 102 to 104 above, and will not be repeated here.

[0198] Specifically, in this embodiment, the method is further explained using a CT image processing task to detect the presence of abnormal objects within a target detection area. The CT image processing task includes taking a plain CT image of the target detection area and inputting it into an image processing model for identification. The image processing model can identify whether abnormal objects exist in the plain CT image, and if so, the mask information of the abnormal objects. It can also predict the object classification information of the object to be detected, i.e., whether an abnormal object exists in the target detection area of ​​the object to be detected, and the type of the abnormal object. This method solves the problem of poor detection effect and low accuracy in current applications where plain CT images cannot be used to detect target detection areas.

[0199] Referring to Figure 8, Figure 8 shows a flowchart of a computer-aided diagnosis method for cancer according to an embodiment of this specification, specifically including:

[0200] Step 802: Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is a tumor in the target detection area.

[0201] Step 804: Input the multiple plain CT images into the image processing model to obtain the detection result corresponding to the target detection area. The image processing model generates object feature information and lesion feature information based on each plain CT image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates the detection result based on the object classification information, the lesion classification information, and the lesion segmentation information.

[0202] The computer-aided cancer diagnosis method provided in this embodiment is applicable to screening for the presence of tumors in various target detection areas, including but not limited to the presence of tumors in organs throughout the body such as the pancreas, esophagus, liver, lungs, breast, intestines, stomach, and lymph nodes. The detection results include user classification information (i.e., whether a tumor exists, and whether the tumor is malignant), lesion classification information (what type of tumor it is), and lesion segmentation information (mask information marking the tumor on the CT image). The method provided in this disclosure can offer guidance to doctors, helping to improve their diagnostic accuracy and providing data support for their diagnostic results.

[0203] Referring to Figure 9, which shows a flowchart of a computer-aided diagnostic method for liver cancer according to an embodiment of this specification, specifically including:

[0204] Step 902: Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the liver region, and the CT image processing task is used to detect whether there is a tumor in the liver region.

[0205] Step 904: Input the multiple plain CT images into the image processing model to obtain the detection results corresponding to the liver region. The image processing model generates object feature information and lesion feature information based on each plain CT image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates detection results based on the object classification information, the lesion classification information, and the lesion segmentation information.

[0206] The computer-aided diagnostic method for liver cancer provided in this embodiment can be used to screen for diseases such as hepatocellular carcinoma, cholangiocarcinoma, metastatic tumors, hemangiomas, focal nodular hyperplasia, cysts, and calcifications. It provides guidance to doctors, helps improve their diagnostic accuracy, and provides data support for doctors to give diagnostic results.

[0207] Referring to Figure 10, which shows an architecture diagram of a computer-aided cancer diagnosis system provided in one embodiment of this specification, the computer-aided cancer diagnosis system may include a client 100 and a server 200.

[0208] Client 100 is used to send a CT image processing task to server 200, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is a tumor in the target detection area;

[0209] Server 200 is used to input the multiple plain CT images into an image processing model to obtain the detection result corresponding to the target detection area. The image processing model generates object feature information and lesion feature information based on each plain CT image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, generates the detection result based on the object classification information, the lesion classification information, and the lesion segmentation information, and sends the detection result corresponding to the target detection area to client 100.

[0210] Client 100 is also used to receive the detection results sent by server 200.

[0211] A computer-aided diagnosis system for cancer may include multiple clients 100 and a server 200. Clients 100 can be referred to as edge devices, and the server 200 as cloud devices. Multiple clients 100 can establish communication connections through the server 200. In the context of computer-aided diagnosis for cancer, the server 200 is used to provide computer-aided diagnosis services for cancer among the multiple clients 100. Each client 100 can act as either a sender or a receiver, communicating through the server 200.

[0212] Users can interact with server 200 through client 100 to receive data sent by other clients 100, or send data to other clients 100, etc. In the scenario of computer-aided diagnosis of cancer, users can publish data streams to server 200 through client 100, server 200 can generate test results based on the data stream, and push the test results to other clients that have established communication.

[0213] In this system, client 100 and server 200 establish a connection via a network. The network provides the medium for communication between client 100 and server 200. The network can include various connection types, such as wired or wireless communication links or fiber optic cables. Data transmitted by client 100 may need to undergo encoding, transcoding, compression, or other processing before being published to server 200.

[0214] Client 100 can be a browser, an app (application), a web application such as an H5 (HyperText Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. Client 100 can be developed based on the software development kit (SDK) of the corresponding service provided by server 200, such as a real-time communication (RTC) SDK. Client 100 can be deployed on electronic devices and depends on the device or certain apps on the device to run. Electronic devices may have displays and support information browsing, such as personal mobile terminals like mobile phones, tablets, and personal computers. Various other types of applications can also be configured on electronic devices, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.

[0215] Server 200 may include servers providing various services, such as servers providing communication services to multiple clients, servers supporting backend training of models used on clients, and servers processing data sent by clients. It should be noted that server 200 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server integrated with blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0216] It is worth noting that the computer-aided diagnosis method for cancer provided in this disclosure is generally executed by the server. However, in other embodiments of this specification, the client may also have similar functions to the server, thereby executing the computer-aided diagnosis method for cancer provided in this disclosure. In other embodiments, the computer-aided diagnosis method for cancer provided in this disclosure may also be executed jointly by the client and the server.

[0217] Corresponding to the above method embodiments, this specification also provides an image processing apparatus embodiment. Figure 11 shows a schematic diagram of the structure of an image processing apparatus provided in one embodiment of this specification. As shown in Figure 11, the apparatus includes:

[0218] The receiving module 1102 is configured to receive an image processing task, wherein the image processing task carries multiple target images corresponding to the target detection area, and the image processing task is used to detect whether there is an abnormal object in the target detection area.

[0219] The detection module 1104 is configured to input the plurality of target images into an image processing model to obtain detection results corresponding to the target detection regions. The image processing model generates object feature information and lesion feature information based on each target image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates detection results based on the object classification information, the lesion classification information, and the lesion segmentation information.

[0220] Optionally, the image processing model includes an image processing backbone network, an object branch network, a lesion branch network, and a fusion network;

[0221] The detection module 1104 is further configured as follows:

[0222] The multiple target images are input into the image processing backbone network to extract the feature information of the object to be processed and the feature information of the lesion to be processed from the image processing backbone network;

[0223] The feature information of the object to be processed is input into the object branch network to obtain object feature information, and the feature information of the lesion to be processed is input into the lesion branch network to obtain lesion feature information and image feature information to be segmented;

[0224] The object feature information and the lesion feature information are input into the fusion network to generate object fusion feature information;

[0225] Based on the lesion feature information, lesion classification information is generated; based on the image feature information to be segmented, lesion segmentation information is generated; based on the object fusion feature information, object classification information is generated; and based on the object classification information, the lesion classification information, and the lesion segmentation information, a detection result is generated.

[0226] Optionally, the image processing backbone network includes an image encoder and an image decoder;

[0227] The detection module 1104 is further configured as follows:

[0228] The plurality of target images are input into the image encoder to obtain target image coding features and at least one multi-scale image coding feature;

[0229] The target image encoding features are input into the image decoder to obtain at least one first multi-scale image decoding feature, at least one second multi-scale image decoding feature, and the target image decoding feature;

[0230] The at least one multi-scale image coding feature and the at least one first multi-scale image decoding feature are determined as the feature information of the object to be processed, and the at least one second multi-scale image decoding feature and the target image decoding feature are determined as the lesion feature information to be processed.

[0231] Optionally, the image encoder includes multiple sequentially connected image coding layers;

[0232] The detection module 1104 is further configured as follows:

[0233] The multiple target images are input into the image encoder to obtain the target image encoding features output by the image encoder;

[0234] At least one target image coding layer is determined among the plurality of image coding layers, and the multi-scale image coding features output by each target image coding layer are obtained.

[0235] Optionally, the image decoder includes multiple sequentially connected image decoding layers;

[0236] The detection module 1104 is further configured as follows:

[0237] At least one first image decoding layer and at least one second image decoding layer are determined among the plurality of image decoding layers;

[0238] Obtain the first multi-scale image decoding features output by the first image decoding layer, and obtain the second multi-scale image decoding features output by the second image decoding layer.

[0239] Optionally, the lesion feature information to be processed includes at least one second multi-scale image decoding feature and a target image decoding feature;

[0240] The detection module 1104 is further configured as follows:

[0241] Feature decoding is performed on the decoding features of each second multi-scale image to obtain lesion feature information;

[0242] The lesion feature information and the target image decoding features are fused to generate image feature information to be segmented.

[0243] Optionally, the detection module 1104 is further configured to:

[0244] The lesion feature information is input into the lesion classifier to generate lesion classification information;

[0245] The feature information of the image to be segmented is input into the segmenter to generate lesion segmentation information.

[0246] Optionally, the detection module 1104 is further configured to:

[0247] The object feature information and the lesion feature information are input into the fusion network. The lesion sub-feature information in the lesion feature information is extracted in the fusion network, and the object feature information and the lesion sub-feature information are spliced ​​together to generate object fusion feature information.

[0248] Optionally, the detection module 1104 is further configured to:

[0249] The object fusion feature information is input into the object classifier to generate object classification information.

[0250] Optionally, the device further includes a training module configured to:

[0251] Acquire multiple sample target images and corresponding sample detection results for the multiple sample target images, wherein the sample detection results include sample object classification information, sample lesion classification information, and sample lesion segmentation information;

[0252] The multiple sample target images are input into the image processing model to obtain the prediction detection results and training contrast loss values ​​output by the image processing model.

[0253] Calculate the predicted loss value based on the predicted detection results and the sample detection results;

[0254] The model parameters of the image processing model are adjusted based on the predicted loss value and the training contrast loss value, and the image processing model is trained until the model training stops.

[0255] Optionally, the training module is further configured to:

[0256] Identify the sample detection object and obtain the sample enhancement image and sample target image corresponding to the sample detection object, wherein the sample enhancement image and sample target image corresponding to the sample detection object correspond to the same target detection region of the sample detection object;

[0257] A reference sample detection result is generated on the enhanced sample image for the target detection region.

[0258] Based on the detection results of the reference sample, the sample detection results are labeled on the sample target image.

[0259] Optionally, the image processing model includes an image processing backbone network, an object branch network, a lesion branch network, and a fusion network;

[0260] The training module is further configured as follows:

[0261] The multiple sample target images are input into an image processing model to obtain the predicted detection results and training contrastive loss values ​​output by the image processing model, including:

[0262] The multiple sample target images are input into the image processing backbone network to extract the feature information of the sample object to be processed and the feature information of the sample lesion to be processed from the image processing backbone network;

[0263] The feature information of the sample object to be processed is input into the object branch network to obtain the predicted object feature information, and the feature information of the sample lesion to be processed is input into the lesion branch network to obtain the predicted lesion feature information and the predicted image feature information to be segmented.

[0264] In the object branch network, the object comparison loss value is obtained based on the sample object classification information corresponding to the sample target image; in the lesion branch network, the lesion comparison loss value is obtained based on the sample lesion classification information corresponding to the sample target image.

[0265] The predicted object feature information and the predicted lesion feature information are input into the fusion network to generate sample object fusion feature information;

[0266] First object classification information is generated based on the predicted object feature information; second object classification information is generated based on the sample object fusion feature information; predicted lesion classification information is generated based on the predicted lesion feature information; and predicted lesion segmentation information is generated based on the predicted image to be segmented feature information.

[0267] A prediction detection result is generated based on the first object classification information, the second object classification information, the predicted lesion classification information, and the predicted lesion segmentation information. A training comparison loss value is generated based on the object comparison loss value and the lesion comparison loss value.

[0268] Optionally, the training module is further configured to:

[0269] Calculate the first object loss value based on the first object classification information and the sample object classification information;

[0270] Calculate the second object loss value based on the second object classification information and the sample object classification information;

[0271] Calculate the lesion classification loss value based on the predicted lesion classification information and the sample lesion classification information;

[0272] The lesion segmentation loss value is calculated based on the predicted lesion segmentation information and the sample lesion segmentation information.

[0273] Optionally, the training module is further configured to:

[0274] Obtain feature information of historical sample objects;

[0275] The historical object feature information and the sample object feature information to be processed are input into the object branch network to obtain the predicted object feature information;

[0276] Obtain lesion feature information from historical samples;

[0277] The lesion feature information of the historical samples and the lesion feature information of the samples to be processed are input into the lesion branch network to obtain the predicted lesion feature information and the predicted image feature information to be segmented.

[0278] Optionally, the training module is further configured to:

[0279] The model loss value is calculated based on the first object loss value, the second object loss value, the lesion classification loss value, the lesion segmentation loss value, the object comparison loss value, and the lesion comparison loss value;

[0280] Adjust the model parameters of the image processing model based on the model loss value.

[0281] The image processing apparatus provided in this disclosure inputs multiple target images into an image processing model. In the image processing model, object feature information and lesion feature information are generated based on each target image. At the same time, the object feature information and lesion feature information are fused to generate object fusion feature information. This allows the model to use not only object feature information but also lesion feature information when predicting the final object classification information, thus fusing global and local information and improving the detection capability of small lesions.

[0282] Furthermore, the training module provided in this disclosure proposes training a sample augmentation image annotation model using partially manual annotation of the augmented images. This allows the model to annotate sample augmentation images that have not been manually annotated, saving annotation time. Additionally, by registering the augmented images with the target images, the detection results of reference samples from the augmented images are transferred to the target images to obtain the target image's detection results. This solves the problems of difficult and missing annotations on target images, reducing annotation costs.

[0283] Meanwhile, a two-stage class-balanced contrastive learning loss function was introduced during model training, which enhanced the image processing model's ability to identify and diagnose problems. This improved classification accuracy in both the object branch network and the lesion branch network, especially for long-tail categories with limited feature data, where it demonstrated better classification and recognition accuracy.

[0284] The above is an illustrative scheme of an image processing apparatus according to this embodiment. It should be noted that the technical solution of this image processing apparatus and the technical solution of the image processing method described above belong to the same concept. For details not described in detail in the technical solution of the image processing apparatus, please refer to the description of the technical solution of the image processing method described above.

[0285] Figure 12 shows a structural block diagram of a computing device 1200 according to an embodiment of this specification. The components of the computing device 1200 include, but are not limited to, a memory 1210 and a processor 1220. The processor 1220 is connected to the memory 1210 via a bus 1230, and a database 1250 is used to store data.

[0286] The computing device 1200 also includes an access device 1240, which enables the computing device 1200 to communicate via one or more networks 1260. Examples of such networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1240 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0287] In one embodiment of this specification, the aforementioned components of the computing device 1200, as well as other components not shown in FIG. 12, may be interconnected, for example, via a bus. It should be understood that the block diagram of the computing device shown in FIG. 12 is merely illustrative and not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0288] The computing device 1200 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1200 can also be a mobile or stationary server.

[0289] The processor 1220 is configured to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-mentioned image processing method, CT image processing method, computer-aided diagnosis method for cancer, or computer-aided diagnosis method for liver cancer.

[0290] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the computing device embodiments are basically similar to the image processing method, CT image processing method, computer-aided diagnosis method for cancer, or computer-aided diagnosis method for liver cancer embodiments, so the description is relatively simple. Relevant parts can be referred to the descriptions of the image processing method, CT image processing method, computer-aided diagnosis method for cancer, or computer-aided diagnosis method for liver cancer embodiments.

[0291] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described image processing method, CT image processing method, computer-aided diagnosis method for cancer, or computer-aided diagnosis method for liver cancer.

[0292] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the computer-readable storage medium embodiments are relatively simple in description because they are substantially similar to the image processing method, CT image processing method, computer-aided diagnosis method for cancer, or computer-aided diagnosis method for liver cancer embodiments. Relevant parts can be referred to the descriptions of the image processing method, CT image processing method, computer-aided diagnosis method for cancer, or computer-aided diagnosis method for liver cancer embodiments.

[0293] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described image processing method, CT image processing method, computer-aided diagnosis method for cancer, or computer-aided diagnosis method for liver cancer.

[0294] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solutions of the image processing method, CT image processing method, computer-aided diagnosis method for cancer, or computer-aided diagnosis method for liver cancer described above. For details not described in detail in the technical solution of the computer program product, please refer to the descriptions of the technical solutions of the image processing method, CT image processing method, computer-aided diagnosis method for cancer, or computer-aided diagnosis method for liver cancer described above.

[0295] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0296] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0297] It should be noted that the above description describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0298] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0299] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this disclosure. These embodiments have been selected and specifically described in this specification to better explain the principles and practical applications of this disclosure, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. An image processing method, comprising: The image processing task is received, wherein the image processing task carries multiple target images corresponding to the target detection area, and the image processing task is used to detect whether there is an abnormal object in the target detection area; The multiple target images are input into an image processing model to obtain the detection results corresponding to the target detection regions. The image processing model generates object feature information and lesion feature information based on each target image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates detection results based on the object classification information, the lesion classification information, and the lesion segmentation information.

2. The method as described in claim 1, wherein the image processing model comprises an image processing backbone network, an object branch network, a lesion branch network, and a fusion network; The plurality of target images are input into an image processing model to obtain the detection results corresponding to the target detection regions, including: The multiple target images are input into the image processing backbone network to extract the feature information of the object to be processed and the feature information of the lesion to be processed from the image processing backbone network; The feature information of the object to be processed is input into the object branch network to obtain object feature information, and the feature information of the lesion to be processed is input into the lesion branch network to obtain lesion feature information and image feature information to be segmented; The object feature information and the lesion feature information are input into the fusion network to generate object fusion feature information; Based on the lesion feature information, lesion classification information is generated; based on the image feature information to be segmented, lesion segmentation information is generated; based on the object fusion feature information, object classification information is generated; and based on the object classification information, the lesion classification information, and the lesion segmentation information, a detection result is generated.

3. The method as described in claim 2, wherein the image processing backbone network comprises an image encoder and an image decoder; The multiple target images are input into the image processing backbone network, and the feature information of the object to be processed and the feature information of the lesion to be processed are extracted from the image processing backbone network, including: The plurality of target images are input into the image encoder to obtain target image coding features and at least one multi-scale image coding feature; The target image encoding features are input into the image decoder to obtain at least one first multi-scale image decoding feature, at least one second multi-scale image decoding feature, and the target image decoding feature; The at least one multi-scale image coding feature and the at least one first multi-scale image decoding feature are determined as the feature information of the object to be processed, and the at least one second multi-scale image decoding feature and the target image decoding feature are determined as the lesion feature information to be processed.

4. The method of claim 3, wherein the image encoder comprises a plurality of sequentially connected image coding layers; The plurality of target images are input into the image encoder to obtain target image coding features and at least one multi-scale image coding feature, including: The multiple target images are input into the image encoder to obtain the target image encoding features output by the image encoder; At least one target image coding layer is determined among the plurality of image coding layers, and the multi-scale image coding features output by each target image coding layer are obtained.

5. The method of claim 3 or 4, wherein the image decoder comprises a plurality of sequentially connected image decoding layers; The target image encoding features are input into the image decoder to obtain at least one first multi-scale image decoding feature and at least one second multi-scale image decoding feature, including: At least one first image decoding layer and at least one second image decoding layer are determined among the plurality of image decoding layers; Obtain the first multi-scale image decoding features output by the first image decoding layer, and obtain the second multi-scale image decoding features output by the second image decoding layer.

6. The method according to any one of claims 2 to 5, wherein the lesion feature information to be processed includes at least one second multi-scale image decoding feature and a target image decoding feature; The lesion feature information to be processed is input into the lesion branch network to obtain lesion feature information and image feature information to be segmented, including: Feature decoding is performed on the decoding features of each second multi-scale image to obtain lesion feature information; The lesion feature information and the target image decoding features are fused to generate image feature information to be segmented.

7. The method according to any one of claims 2 to 6, wherein generating lesion classification information based on the lesion feature information and generating lesion segmentation information based on the image feature information to be segmented, comprises: The lesion feature information is input into the lesion classifier to generate lesion classification information; The feature information of the image to be segmented is input into the segmenter to generate lesion segmentation information.

8. The method according to any one of claims 2 to 7, wherein the object feature information and the lesion feature information are input into the fusion network to generate object fusion feature information, comprising: The object feature information and the lesion feature information are input into the fusion network. The lesion sub-feature information in the lesion feature information is extracted in the fusion network, and the object feature information and the lesion sub-feature information are spliced ​​together to generate object fusion feature information.

9. The method according to any one of claims 2 to 8, wherein generating object classification information based on the object fusion feature information comprises: The object fusion feature information is input into the object classifier to generate object classification information.

10. The method according to any one of claims 1 to 9, wherein the image processing model is generated by training through the following steps: Acquire multiple sample target images and the corresponding sample detection results for the multiple sample target images, where, The sample detection results include sample object classification information, sample lesion classification information, and sample lesion segmentation information; The multiple sample target images are input into the image processing model to obtain the prediction detection results and training contrast loss values ​​output by the image processing model. Calculate the predicted loss value based on the predicted detection results and the sample detection results; The model parameters of the image processing model are adjusted based on the predicted loss value and the training contrast loss value, and the image processing model is trained until the model training stops.

11. The method of claim 10, wherein obtaining multiple sample target images and sample detection results corresponding to the multiple sample target images includes: Identify the sample detection object and obtain the sample enhancement image and sample target image corresponding to the sample detection object, wherein the sample enhancement image and sample target image corresponding to the sample detection object correspond to the same target detection region of the sample detection object; A reference sample detection result is generated on the enhanced sample image for the target detection region. Based on the detection results of the reference sample, the sample detection results are labeled on the sample target image.

12. The method as described in claim 10 or 11, wherein the image processing model comprises an image processing backbone network, an object branch network, a lesion branch network, and a fusion network; The multiple sample target images are input into an image processing model to obtain the predicted detection results and training contrastive loss values ​​output by the image processing model, including: The multiple sample target images are input into the image processing backbone network to extract the feature information of the sample object to be processed and the feature information of the sample lesion to be processed from the image processing backbone network; The feature information of the sample object to be processed is input into the object branch network to obtain the predicted object feature information, and the feature information of the sample lesion to be processed is input into the lesion branch network to obtain the predicted lesion feature information and the predicted image feature information to be segmented. In the object branch network, the object comparison loss value is obtained based on the sample object classification information corresponding to the sample target image; in the lesion branch network, the lesion comparison loss value is obtained based on the sample lesion classification information corresponding to the sample target image. The predicted object feature information and the predicted lesion feature information are input into the fusion network to generate sample object fusion feature information; First object classification information is generated based on the predicted object feature information; second object classification information is generated based on the sample object fusion feature information; predicted lesion classification information is generated based on the predicted lesion feature information; and predicted lesion segmentation information is generated based on the predicted image to be segmented feature information. A prediction detection result is generated based on the first object classification information, the second object classification information, the predicted lesion classification information, and the predicted lesion segmentation information. A training comparison loss value is generated based on the object comparison loss value and the lesion comparison loss value.

13. The method of claim 12, wherein calculating the predicted loss value based on the predicted detection result and the sample detection result includes: Calculate the first object loss value based on the first object classification information and the sample object classification information; Calculate the second object loss value based on the second object classification information and the sample object classification information; Calculate the lesion classification loss value based on the predicted lesion classification information and the sample lesion classification information; The lesion segmentation loss value is calculated based on the predicted lesion segmentation information and the sample lesion segmentation information.

14. The method as described in claim 12 or 13, wherein the feature information of the sample object to be processed is input into the object branch network to obtain the predicted object feature information, comprising: Obtain feature information of historical sample objects; The historical object feature information and the sample object feature information to be processed are input into the object branch network to obtain the predicted object feature information; Accordingly, the lesion feature information of the sample to be processed is input into the lesion branch network to obtain predicted lesion feature information and predicted image feature information to be segmented, including: Obtain lesion feature information from historical samples; The lesion feature information of the historical samples and the lesion feature information of the samples to be processed are input into the lesion branch network to obtain the predicted lesion feature information and the predicted image feature information to be segmented.

15. The method according to any one of claims 12 to 14, wherein adjusting the model parameters of the image processing model based on the predicted loss value and the training contrastive loss value comprises: The model loss value is calculated based on the first object loss value, the second object loss value, the lesion classification loss value, the lesion segmentation loss value, the object comparison loss value, and the lesion comparison loss value; Adjust the model parameters of the image processing model based on the model loss value.

16. A CT image processing method, comprising: Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormal object in the target detection area; The multiple plain CT images are input into an image processing model to obtain the detection results corresponding to the target detection area. The image processing model generates object feature information and lesion feature information based on each plain CT image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates detection results based on the object classification information, the lesion classification information, and the lesion segmentation information.

17. A computer-aided diagnostic method for cancer, comprising: Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is a tumor in the target detection area; The multiple plain CT images are input into an image processing model to obtain the detection results corresponding to the target detection area. The image processing model generates object feature information and lesion feature information based on each plain CT image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates detection results based on the object classification information, the lesion classification information, and the lesion segmentation information.

18. A computer-aided diagnostic method for liver cancer, comprising: Receive CT image processing tasks, where... The CT image processing task carries multiple plain CT images corresponding to the liver region, and the CT image processing task is used to detect whether there is a tumor in the liver region; The multiple plain CT images are input into an image processing model to obtain the detection results corresponding to the liver region. The image processing model generates object feature information and lesion feature information based on each plain CT image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates detection results based on the object classification information, the lesion classification information, and the lesion segmentation information.

19. A computer-aided diagnosis system for cancer, comprising a client and a server, wherein, The client is used to send a CT image processing task to the server, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is a tumor in the target detection area; The server is used to input the multiple plain CT images into an image processing model to obtain the detection result corresponding to the target detection area. The image processing model generates object feature information and lesion feature information based on each plain CT image, generates object fusion feature information based on the object feature information and the lesion feature information, determines object classification information based on the object fusion feature information, determines lesion classification information and lesion segmentation information based on the lesion feature information, and generates the detection result based on the object classification information, the lesion classification information, and the lesion segmentation information.

20. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 18.

21. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 18.

22. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 18.

Images