Image detection method and device, equipment and storage medium

Abstract

The invention provides an image detection method and device, equipment and a storage medium, and the method comprises the steps: obtaining a detection image obtained through plain-scan CT, and extracting a target body part image from the detection image; performing first image classification segmentation processing on the target body part image through a first image detection model to determine that a first target lesion type and a lesion area corresponding to the first target lesion type exist in the target body part image; and performing second image classification segmentation processing on the target body part image through a second image detection model to determine a second target lesion type and a lesion area existing in the target body part image, the second target lesion type being a sub-category of the first target lesion type. Through cooperation of the two image detection models, refined detection of an image containing a lesion type and a lesion area can be realized.

Description

[0001] This invention patent application is a divisional application of the Chinese invention patent application with application date of May 24, 2022, application number 202210575258.9, and name “Image detection method, device, equipment and storage medium”. Technical Field

[0002] The present invention relates to the field of artificial intelligence technology, and in particular to an image detection method, device, equipment and storage medium. Background Art

[0003] Pancreatic cancer is a malignant tumor with a high mortality rate. It is not easy to detect through screening. Once discovered, it is often in the late stage of cancer, and the opportunity for surgery is lost. The 5-year survival rate is very low.

[0004] Pancreatic cancer and other pancreatic diseases are currently diagnosed through computed tomography (CT) scans. Commonly used CT scans include contrast-enhanced CT and plain CT. Contrast-enhanced CT requires the injection of contrast agents, which carries the risk of allergic reactions and increases costs. Furthermore, the multi-phase scans expose patients to increased radiation.

[0005] The CT images above contain a lot of valuable information, and it is of great significance to perform detailed detection on these images as needed. Summary of the Invention

[0006] Embodiments of the present invention provide an image detection method, apparatus, device, and storage medium for achieving refined detection of images containing lesion information.

[0007] In a first aspect, an embodiment of the present invention provides an image detection method, the method comprising:

[0008] acquiring a detection image obtained by plain scan computed tomography;

[0009] extracting a target body part image corresponding to the target body part from the detection image;

[0010] performing a first image classification and segmentation process on the target body part image using a first image detection model to determine whether a first target lesion type and a lesion area corresponding to the first target lesion type exist in the target body part image;

[0011] The target body part image is subjected to a second image classification and segmentation process through a second image detection model to determine a second target lesion type and lesion area present in the target body part image, where the second target lesion type is a subcategory of the first target lesion type.

[0012] In a second aspect, an embodiment of the present invention provides an image detection device, comprising:

[0013] An acquisition module, used for acquiring a detection image obtained by plain scan electronic computer tomography;

[0014] a cutting module, configured to extract a target body part image corresponding to the target body part from the detection image;

[0015] a first detection module, configured to perform a first image classification and segmentation process on the target body part image using a first image detection model to determine whether a first target lesion type and a lesion area corresponding to the first target lesion type exist in the target body part image;

[0016] The second detection module is used to perform a second image classification and segmentation processing on the target body part image through a second image detection model to determine a second target lesion type and lesion area present in the target body part image, where the second target lesion type is a subcategory of the first target lesion type.

[0017] In a third aspect, an embodiment of the present invention provides an electronic device comprising: a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the image detection method as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the image detection method described in the first aspect.

[0019] In a fifth aspect, an embodiment of the present invention provides an image detection method, the method comprising:

[0020] Receiving a request triggered by a user device by invoking an image detection service, wherein the request includes a detection image obtained by performing a plain scan electronic computed tomography scan on a human body;

[0021] Utilize the processing resources corresponding to the image detection service to perform the following steps:

[0022] receiving a request triggered by a user device by invoking an image detection service, wherein the request includes a detection image obtained by plain scan electronic computed tomography;

[0023] Utilize the processing resources corresponding to the image detection service to perform the following steps:

[0024] extracting a target body part image corresponding to the target body part from the detection image;

[0025] performing a first image classification and segmentation process on the target body part image using a first image detection model to determine whether a first target lesion type and a lesion area corresponding to the first target lesion type exist in the target body part image;

[0026] performing a second image classification and segmentation process on the target body part image using a second image detection model to determine a second target lesion type and lesion area present in the target body part image, where the second target lesion type is a subcategory of the first target lesion type;

[0027] Feedback the detection image marked with the second target lesion type and lesion area to the user device.

[0028] In a sixth aspect, an embodiment of the present invention provides an image detection method, applied to an augmented reality device, the method comprising:

[0029] acquiring a detection image obtained by plain scan computed tomography;

[0030] extracting a target body part image corresponding to the target body part from the detection image;

[0031] performing a first image classification and segmentation process on the target body part image using a first image detection model to determine whether a first target lesion type and a lesion area corresponding to the first target lesion type exist in the target body part image;

[0032] performing a second image classification and segmentation process on the target body part image using a second image detection model to determine a second target lesion type and lesion area present in the target body part image, where the second target lesion type is a subcategory of the first target lesion type;

[0033] The detection image marked with the second target lesion type and lesion area is displayed.

[0034] In an embodiment of the present invention, taking the detection of lesions in the pancreas as an example, the chest or abdomen of the user can be scanned by plain CT to obtain a detection image, without relying on enhanced CT, and thus without the need to inject contrast agent into the user (non-interventional). Afterwards, a target body part image corresponding to the target body part (such as the pancreatic area) is segmented from the detection image. Afterwards, the target body part image is first classified and segmented by a first image detection model to determine whether a first target lesion type and a lesion area corresponding to the first lesion type exist in the target body part image. If so, a second image classification and segmentation process is performed on the target body part image by a second image detection model to determine a second target lesion type and a lesion area present in the target body part image, wherein the second target lesion type is a subcategory of the first target lesion type.

[0035] As can be seen, the first image detection model is used to detect whether a certain type or types of diseases and corresponding lesion areas are present in the target body part image. The core of the model is to determine whether the target body part image contains lesions that need to be further identified. Based on this, the second image detection model is used to further accurately detect the specific lesion type and lesion area contained in the target body part image. The use of these two image detection models can provide auxiliary image information for locating related diseases in different parts of the body, such as the pancreas, lungs, and stomach. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 A flowchart of an image detection method provided by an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of an application of a pancreatic disease screening process provided by an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of the composition of a first image detection model provided by an embodiment of the present invention;

[0040] Figure 4 A schematic diagram of the composition of a second image detection model provided by an embodiment of the present invention;

[0041] Figure 5 A schematic diagram of an application of an image detection method provided by an embodiment of the present invention;

[0042] Figure 6 A schematic structural diagram of an image detection device provided by an embodiment of the present invention;

[0043] Figure 7 The following is a schematic diagram of the structure of an electronic device provided in this embodiment. DETAILED DESCRIPTION

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0045] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0046] In real life, for example, pancreatic cancer has no officially recommended screening method. Most pancreatic cancer patients are diagnosed in the late stages, missing surgical options, and have a very low five-year survival rate. Early detection and adjuvant chemotherapy after surgery could significantly improve the five-year survival rate.

[0047] The first-line diagnostic imaging option for pancreatic cancer is contrast-enhanced CT, which involves injecting a contrast agent into the user (in this context, the person whose disease is being detected, typically the patient). However, contrast agents carry the risk of allergic reactions, increase costs, and expose the patient to increased radiation due to the multiple-phase scan.

[0048] Plain chest CT scans are widely used for physical examinations, and their coverage already includes most of the pancreas. However, the low contrast of plain CT images makes it difficult for doctors to visually determine whether a pancreatic tumor or cancer is present. In fact, missing pancreatic tumors during physical examinations is common, which is one reason why they are often discovered at an advanced stage.

[0049] In real life, pancreatic diseases include not only pancreatic ductal adenocarcinoma (PDAC), a malignant pancreatic cancer, but also primitive neuroectodermal tumor (PNET), intraductal papillary mucinous neoplasm (IPMN), serous cystic neoplasm (SCN), mucinous cystic neoplasm (MCN), solid-pseudopapillary tumor of pancreas (SPT), chronic pancreatitis (CP), and other pancreatic diseases.

[0050] Taking the above-mentioned pancreatic-related diseases as an example, in an embodiment of the present invention, the detection of common lesion types in the pancreas can be divided into two classification and segmentation tasks: the first is to identify the three categories of PDAC, non-PDAC diseases (including PNET, SPT, IPMN, MCN, CP, SCN, etc.) and no disease based on plain CT images, as well as the lesion area corresponding to each lesion type; the second is if the output result of the first task is a non-PDAC type or PDAC, then further subtype classification and identification is performed based on the plain CT image.

[0051] It is very important to distinguish PDAC from other non-PDAC types of diseases here, because PDAC accounts for 90% of pancreatic cancer and is the most malignant type of pancreatic cancer.

[0052] The above is a brief introduction to two classification and segmentation tasks using the pancreas as an example. In reality, the same principles apply to the detection of other body parts. In other words, the image detection solution provided by the embodiments of the present invention is not limited to detecting specific diseases in specific body parts, but rather provides an image detection solution applicable to a wide range of lesion types in many body parts. For example, lung cancer, tuberculosis, pulmonary edema, and so on.

[0053] It should be noted that the image detection solution provided in the embodiment of the present invention only learns the lesion features presented in the image through a neural network model to achieve accurate detection of the lesion categories and lesion areas that may be contained in the image. The detection results are provided to doctors as an intermediate result information.

[0054] The image detection method provided by the embodiment of the present invention is described in detail below.

[0055] Figure 1 A flowchart of an image detection method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes the following steps:

[0056] 101. Acquire a detection image obtained by plain scan CT, and extract a target body part image corresponding to the target body part from the detection image.

[0057] 102. Perform a first image classification and segmentation process on the target body part image using a first image detection model to determine whether a first target lesion type and a lesion area corresponding to the first target lesion type exist in the target body part image.

[0058] 103. Perform a second image classification and segmentation process on the target body part image through a second image detection model to determine a second target lesion type and lesion area present in the target body part image, where the second target lesion type is a subcategory of the first target lesion type.

[0059] In real life, many diseases require the assistance of medical imaging, and CT is a common auxiliary method. In the embodiments of the present invention, when a user needs to be diagnosed for a certain disease, only a plain CT scan is required for the user, without the need for enhanced CT. The image obtained by the plain CT scan is called the detection image.

[0060] When acquiring CT images, the entire body area, such as the chest or abdomen, is often scanned. However, the diagnosis of some diseases often requires focusing on only a specific body part (the target body part), such as one or more organs. For example, in the case of pancreatic disease screening, the target body part of interest is the pancreas. Therefore, after obtaining the test image, to facilitate the subsequent image recognition process, it is first necessary to extract the image region corresponding to the target body part, such as the pancreas region, from the test image. This is called the target body part image.

[0061] In practical applications, a segmentation model can be pre-trained to segment the target body part image from the detection image.

[0062] Taking the pancreas as an example, in simple terms, during the training phase of the segmentation model, a large number of labeled training sample images can be obtained. These training sample images are images containing the pancreatic region, and the labeled information is the corresponding location of the pancreatic region in the image, usually marked as a polygonal box, indicating that all pixels within the polygonal box correspond to the pancreatic region. The segmentation model can be trained based on these labeled training sample images. Once the segmentation model is trained to convergence, it will be able to locate the pancreatic region.

[0063] Based on this, after the above-mentioned detection image is input into the segmentation model, the segmentation model can predict the category label corresponding to each pixel in the detection image: whether it is located in the pancreatic region. Assuming that the category label 1 is used to indicate that it is located in the pancreatic region, and the category label 0 is used to indicate that it is not located in the pancreatic region, then based on the category label corresponding to each pixel, the continuous region composed of the pixels corresponding to the category label 1 can be determined as the pancreatic region, that is, the image region corresponding to the target body part. Among them, "continuous region" means that even if there are a particularly small number of pixels with the category label 0 among a large number of pixels corresponding to the category label 1, the category label 0 of these pixels will be ignored. The specific method for determining the continuous region can be implemented with reference to the existing related technology and will not be repeated in this embodiment.

[0064] As mentioned above, taking the pancreatic disease screening scenario as an example, the pancreatic disease screening task is divided into two classification and segmentation tasks. The first classification and segmentation task is used to perform classification processing and image segmentation processing on the target body part image for the first group of lesion types; the second classification and segmentation task is used to perform classification processing and image segmentation processing on the target body part image for the second group of lesion types.

[0065] The image segmentation processing here refers to locating the lesion area corresponding to the corresponding lesion type in the target body part image, and the classification processing refers to classifying and identifying the lesion type.

[0066] For pancreatic diseases, the first group of lesion types may include PDAC type, non-PDAC type and no lesion (ie normal), and the second group of lesion types may include PNET, SPT, IPMN, MCN, CP, SCN and other types, and may even include PDAC.

[0067] In an optional embodiment, the second group of lesion types can be considered a subcategory of the non-PDAC type. Therefore, in actual applications, when the first classification segmentation task outputs that the corresponding lesion type of the target body part in the first group of lesion types is a non-PDAC type, the second classification segmentation task can be performed.

[0068] It should be noted that it can also be configured to execute the second classification segmentation task when the output result of the first classification segmentation task indicates that PDAC or non-PDAC type (i.e., the presence of disease) exists in the target body part. At this time, the second classification segmentation task needs to provide the classification function of PDAC and non-PDAC type subtypes (subcategories).

[0069] Based on the pancreatic disease example above, to summarize, the first image detection model is used to detect a first set of lesion types. This first set of lesion types includes: a third target lesion type, which is categorized by disease severity corresponding to the target body part; the first target lesion type; and no lesion. The first target lesion type is a general term for all lesion types except the third target lesion type. In other words, the first target lesion type does not refer to a specific disease, but rather indicates the presence of an abnormal condition related to a disease other than the third target lesion type.

[0070] In practical applications, the above two classification and segmentation tasks can be completed by respectively training two image detection models, namely the first image detection model and the second image detection model.

[0071] It is understandable that if the first image detection model detects the target body part image as containing the third target lesion type and its corresponding lesion area, then the processing of the second image detection model may be optionally omitted.

[0072] Dividing the image detection task into the two classification and segmentation tasks mentioned above has the following advantages:

[0073] First, compared to using a single model to complete the image detection task, the two image detection models can each have their own focus, making it easier to train a model with good performance and a stronger disease focus, thereby reducing the impact of sample imbalance on model performance.

[0074] Secondly, diseases involving the same organ are divided into the above two groups: the first group includes the most serious diseases, no diseases, and a general term for other diseases, ensuring that the first image detection model can fully learn the characteristics of the most serious disease and the characteristics of the organ when it is normal (no disease), and can ensure the accuracy of detection of the most serious disease, thereby ensuring the timely discovery of such malignant diseases. The first classification segmentation task is equivalent to a coarse classification segmentation task, which completes the identification process of whether the patient has a disease or whether he has certain types of diseases. The second classification segmentation task is equivalent to a fine classification segmentation task, and the corresponding second image detection model is trained to learn fine-grained difference features of more types of diseases to ensure more accurate image detection results.

[0075] Third, considering practical application needs: Since the first image detection model completes the classification and segmentation of the first group of lesion types, not only can the classification results be used to determine whether the patient suffers from the most serious disease mentioned above, but the segmentation results can also be used to determine the patient's lesion area.

[0076] Those skilled in the art will appreciate that the second image detection model provided in the above embodiment may be selected for use based on actual needs.

[0077] In summary, in practical applications, after locating the target body part image from the detection image, the target body part image is input into the first image detection model, and the first image detection model is used to perform classification processing and image segmentation processing on the target body part image for the first group of lesion types to obtain the lesion types and lesion areas present in the target body part image in the first group of lesion types. If the target body part image contains a first target lesion type (referring to a specific one or several lesion types) included in the first group of lesion types, the target body part image is subjected to classification processing and image segmentation processing of the second group of lesion types through the second image detection model to determine the second target lesion type and lesion area present in the target body part image in the second group of lesion types.

[0078] It can be understood that since the above two image detection models need to provide classification and image segmentation capabilities, during the model training process, it is necessary to obtain training sample images with two types of annotation information. One type of annotation information is the lesion type contained in the training sample image (contained in a corresponding group of lesion types), and the other is the position of the lesion area corresponding to the lesion type in the training sample image.

[0079] In an optional embodiment, both the first image detection model and the second image detection model can be trained using the following cross-validation training method. For ease of description, the first image detection model and the second image detection model are used as target image detection models, respectively. The training process includes:

[0080] Obtain a training sample set for training the target image detection model;

[0081] Constructing a plurality of training sample subsets corresponding to the training sample set;

[0082] Multiple target image detection models are trained using multiple training sample subsets respectively.

[0083] For example, suppose there are 100 sample images in the training sample set of the target image detection model, and a total of 5 target image detection models (different image detection models) are trained. Among them, the training sample subset corresponding to each target image detection model contains 80 sample images sampled from these 100 sample images (the training sample subset corresponding to each target image detection model is different), and the remaining 20 sample images are used as the test set of the corresponding target image detection model.

[0084] Through the above training method, multiple different first image detection models and multiple different second image detection models can be obtained. Based on this, the first image classification and segmentation processing can be performed on the target body part image using the multiple first image detection models to obtain the output results of each of the multiple first image detection models. Then, based on the output results of each of the multiple first image detection models, it is determined whether the target body part image has a lesion area corresponding to the first lesion type.

[0085] Among them, optionally, the output result with the highest proportion can be selected from the output results of multiple first image detection models as the final output result. For example, 4 of the 5 first image detection models output the classification result: PDAC, and the lesion area output by each is the same, then the lesion type present in the image of the target body part is determined to be PDAC, and the lesion area is the lesion area output by these 4 first image detection models.

[0086] Similarly, the second lesion type and lesion area present in the target body part image are determined through multiple second image detection models.

[0087] To summarize, in combination with the image detection solution provided by the embodiment of the present invention, taking the pancreatic disease screening scenario as an example, through the coordinated cooperation of the above two image detection models, based on the first image detection model trained to have the ability to recognize a small number of serious diseases, accurate detection of serious diseases can be achieved, and other types of diseases can be detected based on the second image detection model, thereby achieving comprehensive and accurate recognition of pancreatic images for various lesion types. Figure 2 The pancreas image detection process implemented in combination with the detection solution provided in the embodiment of the present invention is illustrated.

[0088] The structure and working process of the first image detection model and the second image detection model are introduced below.

[0089] For the first image detection model:

[0090] From a structural perspective, the first image detection model includes a first feature extraction sub-model and a first classification and segmentation sub-model, wherein the first feature extraction sub-model includes a first encoding module, a first decoding module, and a jump layer between the first encoding module and the first decoding module.

[0091] From the perspective of work process, the work process is as follows:

[0092] Extracting a first feature map group corresponding to the target body part image through a first encoding module, where the first feature map group is composed of feature maps at multiple scales;

[0093] Inputting the first feature map group into the first decoding module through the skip layer;

[0094] Obtaining a second feature map group corresponding to the target body part image through the first decoding module, the second feature map group consisting of feature maps of multiple scales;

[0095] The second feature map group is input into the first classification segmentation sub-model to fuse the feature maps contained in the second feature map group through the first classification segmentation sub-model, and based on the fused feature maps, it is determined whether the first target lesion type and the lesion area corresponding to the first target lesion type exist in the target body part image.

[0096] For ease of understanding, combined Figure 3 To illustrate the composition of the first image detection model.

[0097] exist Figure 3 In the figure, the left branch corresponds to the first feature extraction sub-model, and the right branch corresponds to the first classification and segmentation sub-model. Assume that the first encoding module is composed of 6 convolution blocks, and correspondingly, the first decoding module is also composed of 6 convolution blocks, where each convolution block is represented as: 2xConv3D, which means that it is composed of 2 serial three-dimensional (3D) convolution layers. The scale corresponding to each convolution block is different, and is used to extract feature maps of the corresponding scale, such as 5x8x5, 10x16x10, 20x32x20, ..., as shown in the figure, where C = 320, 256, 128..., C represents the number of channels. The directed connection lines between the convolution blocks corresponding to the same scale in the first encoding module and the first decoding module shown in the figure represent the jump layers between the convolution blocks of the same scale.

[0098] based on Figure 3 The structure of the first feature extraction sub-model shown in the figure is that when the target body part image is input into the first encoding module, it undergoes convolution and downsampling through the six convolutional blocks mentioned above, and six feature maps of decreasing scales are obtained in sequence. The output of the previous convolutional block serves as the input of the next convolutional block. The output of the last convolutional block in the first encoding module is input into the first decoding module, which then performs upsampling and deconvolution through the six convolutional blocks layer by layer, and outputs six feature maps of increasing scales in sequence. However, after a convolutional block in the first decoding module upsamples the feature map of scale i input by the previous convolutional block to obtain a feature map of scale j, it is necessary to use a skip layer to splice the feature map of scale j extracted from the first encoding module with the feature map before performing subsequent convolution operations. In other words, the role of the skip layer is to splice the feature maps of the same scale obtained by downsampling (encoding module) and upsampling (decoding module) in the channel dimension.

[0099] The feature maps at multiple scales extracted by the first encoding module are actually shallow features, forming the first feature map group; while the feature maps at multiple scales extracted by the first decoding module are actually deep features, forming the second feature map group. Feature fusion is achieved through jump connection.

[0100] like Figure 3 As shown in , the first classification segmentation sub-model may include multiple pooling layers (such as the global max pooling layer shown in the figure), a fully connected layer (FC) connected to each pooling layer, and a feature fusion layer. The inputs corresponding to the feature maps of multiple scales included in the second feature map group are compressed into the multiple pooling layers.

[0101] Finally, the compressed feature maps at multiple scales are input into the feature fusion layer for feature fusion (splicing). Based on the fused feature maps, it is determined whether there is a lesion area corresponding to the first target lesion type in the image of the target body part, such as the recognition results of the three categories of PDAC / non-PDAC / normal shown in the figure, as well as the lesion area segmentation results corresponding to PDAC / non-PDAC, to enhance interpretability.

[0102] For the second image detection model:

[0103] From a structural perspective, the second image detection model includes a first feature extraction sub-model, a second classification and segmentation sub-model, and a pooling module. The second classification and segmentation sub-model includes a memory unit and an attention module. The memory unit is trained to store the positions and visual features corresponding to different lesion types included in the first target lesion type in the target body part. The memory unit is configured to store the positions and visual features using a target number of memory vectors.

[0104] From the perspective of work process, the work process is as follows:

[0105] extracting a third feature map group corresponding to the target body part image by the second feature extraction sub-model, where the third feature map group is composed of feature maps at multiple scales;

[0106] For the target feature maps in the feature maps of the multiple scales, the following steps are performed: a pooling module is used to perform pooling processing on the target feature map to compress the target feature map into a target number of feature vectors; an attention module is used to perform cross-attention processing on the target number of reference vectors and the target number of feature vectors, and self-attention processing is performed on the target number of reference vectors, and the cross-attention processing results and the self-attention processing results are added; wherein, when the target feature map is the first of the feature maps of the multiple scales, the reference vector is the memory vector; when the target feature map is not the first of the feature maps of the multiple scales, the reference vector is the sum of the cross-attention processing result and the self-attention processing result corresponding to the previous target feature map; the target feature map is any one of the third feature map group;

[0107] According to the sum of the cross-attention processing result and the self-attention processing result corresponding to the last target feature map, the second target lesion type and lesion area present in the target body part image are determined.

[0108] In fact, similar to the first feature extraction sub-model in the first image detection model, the second feature extraction sub-model also includes a second encoding module, a second decoding module, and a jump layer between the second encoding module and the second decoding module. Based on this, optionally, extracting the third feature map group corresponding to the target body part image through the second feature extraction sub-model can be implemented as follows:

[0109] extracting a fourth feature map group corresponding to the target body part image through a second encoding module;

[0110] Inputting the fourth feature map group into the second decoding module through the skip layer;

[0111] Obtaining a fifth feature map group corresponding to the target body part image through a second decoding module;

[0112] It is determined that some feature maps included in the fourth feature map group and some feature maps included in the fifth feature map group constitute a third feature map group.

[0113] In addition, in an optional embodiment, the second image detection model may further include a position embedding module. Based on this, the attention module performs cross-attention processing on the reference vector of the target number and the feature vector of the target number, which can be implemented as follows:

[0114] Superimposing corresponding position embedding vectors on the target number of feature vectors respectively, wherein the position embedding vector superimposed on any feature vector is used to represent the position information corresponding to the any feature vector in the target number of feature vectors;

[0115] The attention module performs cross-attention processing on the reference vector of the target number and the feature vector of the position embedding vector corresponding to the superposition of the target number.

[0116] The introduction of position embedding vectors essentially introduces the spatial location features of the disease on the target body part, which helps to achieve more accurate recognition results.

[0117] For ease of understanding, combined Figure 4 To illustrate the composition of the second image detection model.

[0118] exist Figure 4 In the figure, the left branch corresponds to the second feature extraction sub-model, the middle branch corresponds to the position embedding module, and the right branch corresponds to the second classification and segmentation sub-model.

[0119] Among them, Figure 3 Similarly, assuming the second encoding module consists of six convolutional blocks, the second decoding module also consists of six convolutional blocks. The composition and related parameters of each convolutional block are shown in the figure. The directed connecting lines between convolutional blocks of the same scale in the second encoding module and the second decoding module in the figure represent skip layers between convolutional blocks of the same scale.

[0120] The working process of the first feature extraction sub-model is consistent with the above introduction. Figure 4 In the structure of the second feature extraction sub-model shown in FIG, when the target body part image is input into the second encoding module, the six convolutional blocks are sequentially used for convolution and downsampling to obtain feature maps of six scales, forming the fourth feature map group. The second decoding module sequentially uses the six convolutional blocks for upsampling and deconvolution layer by layer to output feature maps of six scales, forming the fifth feature map group.

[0121] Afterwards, if Figure 4 As shown in , some feature maps are selected from the fourth feature map group and the fifth feature map group respectively to form the third feature map group.

[0122] It should be noted that, in theory, the third feature map group can also be composed of all the feature maps included in the fourth and fifth feature map groups, but this will increase the computational complexity. In addition, since the fourth and fifth feature map groups correspond to shallow and deep features, respectively, selecting some feature maps from the fourth and fifth feature map groups can take into account both shallow and deep features, as well as the computational complexity.

[0123] Afterwards, for each feature map in the third feature map group, the pooling module performs pooling processing on each feature map to compress the feature map into a target number of feature vectors. The pooling module can provide adaptive average pooling processing (Adaptive Average Pooling) as shown in the figure. Figure 4 In the example, we assume that the number of targets is 5x8x5=200, which matches the size of the 200 320-dimensional vectors stored in the memory unit.

[0124] Among them, Figure 4 Among the feature maps of multiple scales shown in the figure, the smallest scale is 5x8x5, and the other scales are integer multiples of this scale. Therefore, the feature maps of other scales can be compressed based on the 5x8x5 scale, and the compression results are represented by 200 feature vectors.

[0125] In an optional embodiment, after each feature map in the third feature map is compressed into the target number of feature vectors, a corresponding position embedding vector (pos) may be superimposed on each feature vector therein, wherein the position embedding vector superimposed on any feature vector is used to represent the position information corresponding to any feature vector in the target number of feature vectors.

[0126] Afterwards, if Figure 4 As shown in , the attention module performs cross-attention processing on the reference vector of the target number and the feature vector of the position embedding vector corresponding to the superposition of the target number, and performs self-attention processing on the reference vector of the target number, and adds the cross-attention processing results and the self-attention processing results.

[0127] The reference vector of the target quantity is derived from the target quantity memory vector stored in the memory unit. The target quantity memory vector stored in the memory unit is updated and learned during the training process of the second image detection model. When the model converges, the target quantity memory vector stored in the memory unit will be finally stored for later use.

[0128] like Figure 4 As shown in the figure, after the compression and position embedding processes described above, each feature map in the third feature map group forms a vector sequence, each of which is sequentially input into the attention module. Therefore, the attention module can be considered to be composed of multiple attention units, as shown in the figure, each of which provides both self-attention and cross-attention mechanisms.

[0129] For ease of description and understanding, the feature vectors of the target number after pooling of the four feature maps in the third feature map group shown in the figure are represented as C1, C2, C3, and C4 respectively, the corresponding four position embedding vectors are represented as P1, P2, P3, and P4 respectively, and the memory vector of the target number stored in the memory unit is represented as M0. Based on this, the self-attention processing of the target number memory vector by the first attention unit is represented as: self-attention(M0)=M 01 , the first attention unit's self-attention processing of the feature vector C1 of the corresponding position embedding vector on the superposition of the target number memory vector and the target number is expressed as: cross-attention(C1+P1,M0)=M 02 , where C1+P1 represents the superposition of the target number of feature vectors C1 and their respective corresponding position embedding vectors.

[0130] The sum of the self-attention processing result and the cross-attention processing result is: M 01 +M 02 =M1, then M1 is the input of the next attention unit: the reference vector of the target number.

[0131] The second attention unit's self-attention processing of the reference vector M1 of the target quantity at this time is expressed as: self-attention(M1)=M 11 , the second attention unit's self-attention processing of the feature vector C2 of the corresponding position embedding vector on the superposition of the reference vector of the target number and the target number is expressed as: cross-attention(C2+P2,M1)=M 12 , the sum of the self-attention processing result and the cross-attention processing result is recorded as: M 11 +M 12 =M2.

[0132] Similarly, the self-attention processing of the third attention unit on the reference vector M2 of the target quantity at this time is expressed as: self-attention(M2)=M 21 , the self-attention processing of the third attention unit on the feature vector C3 of the corresponding position embedding vector on the superposition of the reference vector of the target number and the target number is expressed as: cross-attention(C3+P3,M2)=M 22 , the sum of the self-attention processing result and the cross-attention processing result is recorded as: M 21 +M 22 =M3. The fourth attention unit's self-attention processing of the reference vector M3 of the target quantity at this time is expressed as: self-attention(M3)=M 31, the fourth attention unit's self-attention processing of the feature vector C4 of the corresponding position embedding vector on the superposition of the reference vector of the target number and the target number is expressed as: cross-attention(C4+P4,M3)=M 32 , the sum of the self-attention processing result and the cross-attention processing result is recorded as: M 31 +M 32 =M4.

[0133] Then, the second target lesion type and lesion area in the target body part image are determined according to M4. For example, Figure 4 The response module shown in the figure performs some kind of pooling processing on the 200 320-dimensional vectors corresponding to M4, selects the feature value from each vector, inputs it to the subsequent fully connected layer (FC), and then outputs the prediction results of the lesion type and lesion area.

[0134] In this embodiment, a memory unit is used to store a set of globally shared model parameters. During the model training phase, the initial values ​​in the memory unit can be randomly determined. Subsequently, the parameters are updated with each iteration of model training. The memory unit is designed to learn global contextual information and location information, such as the relative position of a pancreatic tumor within the pancreas, thereby providing a distinguishable descriptor for each pancreatic disease type included in the first target lesion type (i.e., the second group of lesion types). In other words, the memory unit is designed to store feature information such as location (spatial) and texture (visual) of different pancreatic diseases. The updating and construction of this feature information is achieved through self-attention and cross-attention mechanisms.

[0135] The image detection method provided by the embodiment of the present invention can be executed in the cloud, and a number of computing nodes can be deployed in the cloud, each of which has computing, storage and other processing resources. In the cloud, multiple computing nodes can be organized to provide a certain service. Of course, a computing node can also provide one or more services. The way to provide the service in the cloud can be to provide a service interface to the outside world, and the user calls the service interface to use the corresponding service. The service interface includes a software development kit (Software Development Kit, referred to as SDK), an application programming interface (Application Programming Interface, referred to as API), and the like.

[0136] For the solution provided by the embodiments of the present invention, the cloud can provide a service interface for image detection services. Users can invoke this service interface through their user devices to trigger an image detection request to the cloud. This request includes a detection image obtained through a plain scan computed tomography scan. The cloud determines a computing node that responds to the request and uses the processing resources of this computing node to perform the following steps:

[0137] extracting a target body part image corresponding to the target body part from the detection image;

[0138] performing a first image classification and segmentation process on the target body part image using a first image detection model to determine whether a first target lesion type and a lesion area corresponding to the first target lesion type exist in the target body part image;

[0139] performing a second image classification and segmentation process on the target body part image using a second image detection model to determine a second target lesion type and lesion area present in the target body part image, where the second target lesion type is a subcategory of the first target lesion type;

[0140] Feedback the detection image marked with the second target lesion type and lesion area to the user device.

[0141] The above execution process can refer to the relevant descriptions in the aforementioned other embodiments and will not be repeated here.

[0142] For ease of understanding, combined Figure 5 To illustrate this, users can Figure 5 User device E1, shown in the figure, calls the image detection service to upload a service request containing a detection image. In the cloud, as shown in the figure, in addition to several computing nodes, a management node E2 running a control service is also deployed. After receiving the service request from user device E1, management node E2 determines the computing node E3 that responds to the service request. After receiving the service request, computing node E3 performs the above-described calculation process to obtain a detection image marked with the second target lesion type and lesion area. Computing node E3 then sends the detection image with this marking information to user device E1, which displays the detection image. The user can then perform further editing and other operations based on the image.

[0143] The image detection devices of one or more embodiments of the present invention are described in detail below. Those skilled in the art will appreciate that these devices can be constructed using commercially available hardware components and configured according to the steps taught in this solution.

[0144] Figure 6 A schematic diagram of the structure of an image detection device provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the device includes: a collection module 11, a cutting module 12, a first detection module 13, and a second detection module 14.

[0145] The acquisition module 11 is used to acquire a detection image obtained by plain scan electronic computed tomography.

[0146] The cutting module 12 is configured to extract a target body part image corresponding to the target body part from the detection image.

[0147] The first detection module 13 is used to perform a first image classification and segmentation process on the target body part image through a first image detection model to determine whether a first target lesion type and a lesion area corresponding to the first target lesion type exist in the target body part image.

[0148] The second detection module 14 is used to perform a second image classification and segmentation process on the target body part image through a second image detection model to determine a second target lesion type and lesion area present in the target body part image, where the second target lesion type is a subcategory of the first target lesion type.

[0149] Optionally, the first image detection model is used to detect a first group of lesion types, which include: a third target lesion type, the first target lesion type, and no lesion, which are divided in sequence according to the severity of the disease corresponding to the target body part, wherein the first target lesion type refers to a general term for lesion types other than the third target lesion type.

[0150] Optionally, the first image detection model includes a first feature extraction sub-model and a first classification segmentation sub-model, and the first feature extraction sub-model includes a first encoding module, a first decoding module, and a jump layer between the first encoding module and the first decoding module. Based on this, the first detection module 13 is used to: extract a first feature map group corresponding to the target body part image through the first encoding module, the first feature map group consisting of feature maps of multiple scales; input the first feature map group to the first decoding module through the jump layer; obtain a second feature map group corresponding to the target body part image through the first decoding module, the second feature map group consisting of feature maps of multiple scales; input the second feature map group to the first classification segmentation sub-model, so as to fuse the feature maps contained in the second feature map group through the first classification segmentation sub-model, and determine whether the first target lesion type and the lesion area corresponding to the first target lesion type exist in the target body part image based on the fused feature maps.

[0151] Optionally, the second image detection model includes a second feature extraction sub-model, a second classification and segmentation sub-model and a pooling module, and the second classification and segmentation sub-model includes a memory unit and an attention module; wherein, the memory unit is trained to store the positions and visual features corresponding to different lesion types contained in the first target lesion type in the target body part, and the memory unit is configured to store the positions and visual features with a target number of memory vectors. Based on this, the second detection module 14 is used to: extract a third feature map group corresponding to the target body part image through the second feature extraction sub-model, and the third feature map group is composed of feature maps of multiple scales; for the target feature map in the feature maps of the multiple scales in turn: perform pooling processing on the target feature map through the pooling module to compress the target feature map into the feature vector of the target number; and perform cross-attention processing on the reference vector of the target number and the feature vector of the target number through the attention module, perform self-attention processing on the reference vector of the target number, and add the cross-attention processing results and the self-attention processing results; wherein, when the target feature map is the first of the feature maps of the multiple scales, the reference vector is the memory vector, and when the target feature map is not the first of the feature maps of the multiple scales, the reference vector is the sum of the cross-attention processing result and the self-attention processing result corresponding to the previous target feature map; the target feature map is any one of the feature maps of the multiple scales; according to the sum of the cross-attention processing result and the self-attention processing result corresponding to the last target feature map, determine the second target lesion type and lesion area present in the target body part image.

[0152] Optionally, the second image detection model includes a position embedding module, based on which the second detection module 14 is also used to: superimpose corresponding position embedding vectors on the feature vectors of the target number respectively, wherein the position embedding vector superimposed on any feature vector is used to characterize the position information corresponding to any feature vector in the feature vector of the target number; and perform cross-attention processing on the reference vector of the target number and the feature vectors of the corresponding position embedding vectors superimposed on the target number through the attention module.

[0153] Optionally, the second feature extraction sub-model includes a second encoding module, a second decoding module and a jump layer between the second encoding module and the second decoding module. Based on this, the second detection module 14 is used to: extract the fourth feature map group corresponding to the target body part image through the second encoding module; input the fourth feature map group to the second decoding module through the jump layer; obtain the fifth feature map group corresponding to the target body part image through the second decoding module; and determine that some feature maps contained in the fourth feature map group and some feature maps contained in the fifth feature map group constitute the third feature map group.

[0154] Optionally, the first image detection model and the second image detection model are respectively used as target image detection models, and the device also includes: a training module for obtaining a training sample set for training the target image detection model; constructing multiple training sample subsets corresponding to the training sample set; and training multiple target image detection models respectively through the multiple training sample subsets.

[0155] Based on this, the first detection module 13 is specifically used to: perform first image classification and segmentation processing on the target body part image through multiple first image detection models respectively to obtain the output results of each of the multiple first image detection models; and determine whether the first target lesion type and the lesion area corresponding to the first target lesion type exist in the target body part image based on the output results of each of the multiple first image detection models. The second classification module detection is specifically used to: perform second image classification and segmentation processing on the target body part image through multiple second image detection models respectively to obtain the output results of each of the multiple second image detection models; and determine the second target lesion type and lesion area present in the target body part image based on the output results of each of the multiple second image detection models.

[0156] Figure 6 The device shown can execute the steps in the aforementioned embodiments. For detailed execution process and technical effects, please refer to the description in the aforementioned embodiments and will not be repeated here.

[0157] In one possible design, the above Figure 6 The structure of the image detection device shown can be realized as an electronic device. Figure 7 As shown, the electronic device may include: a processor 21, a memory 22, and a communication interface 23. The memory 22 stores executable code, and when the executable code is executed by the processor 21, the processor 21 can at least implement the image detection method provided in the above embodiment.

[0158] In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the image detection method provided in the aforementioned embodiment.

[0159] In an optional embodiment, the electronic device used to execute the image detection method provided by the embodiment of the present invention may be an extended reality (XR) device. XR is a general term for various forms such as virtual reality and augmented reality. At this time, the detection image obtained by plain scan CT can be input into the extended reality device, which is provided with the above-mentioned first image detection model and second image detection model. The extended reality device uses these two models to perform the above-mentioned image classification and segmentation processing on the target body part image extracted from the detection image, and finally determines the second target lesion type and lesion area in the target body part image, and displays the detection image marked with the second target lesion type and lesion area. In this way, doctors and users can see these marking information through the extended reality device.

[0160] In actual applications, for ease of viewing, optionally, after receiving the initially collected detection image, or the detection image with the above-mentioned marking information, the extended reality device can generate a virtual environment that is convenient for viewing the detection image more clearly, and render and display the detection image in the virtual environment. In addition, doctors and users can also input interactive operations to the extended reality device when watching, such as rotating the detection image, enlarging the detection image, etc. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art can understand and implement it without paying any creative labor.

[0161] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by a combination of hardware and software. Based on this understanding, the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An image detection method, characterized in that: include: acquiring a detection image obtained by plain scan computed tomography; extracting a target body part image corresponding to the target body part from the detection image; extracting a second feature map group corresponding to the target body part image by using a first feature extraction submodel in the first image detection model, and performing a first image classification and segmentation process on the second feature map group to determine whether a first target lesion type and a lesion area corresponding to the first target lesion type exist in the target body part image; A third feature map group corresponding to the target body part image is extracted through the second feature extraction submodel in the second image detection model, and a second image classification and segmentation processing is performed on the third feature map group to determine the second target lesion type and lesion area present in the target body part image, where the second target lesion type is a subcategory of the first target lesion type.

2. The method according to claim 1, characterized in that The first image detection model is used to detect a first group of lesion types, which include: a third target lesion type, the first target lesion type, and no lesion, which are divided in sequence according to the severity of the disease corresponding to the target body part, wherein the first target lesion type refers to a general term for lesion types other than the third target lesion type.

3. The method according to claim 1, characterized in that The first image detection model further includes a first classification and segmentation sub-model, and the first feature extraction sub-model includes a first encoding module, a first decoding module, and a jump layer between the first encoding module and the first decoding module; The extracting a second feature map group corresponding to the target body part image by using the first feature extraction sub-model in the first image detection model, and performing a first image classification and segmentation process on the second feature map group includes: extracting, by the first encoding module, a first feature map group corresponding to the target body part image, wherein the first feature map group is composed of feature maps at multiple scales; Inputting the first feature map group into the first decoding module through the skip layer; obtaining, by the first decoding module, a second feature map group corresponding to the target body part image, wherein the second feature map group is composed of feature maps at multiple scales; The second feature map group is input into the first classification segmentation sub-model to fuse the feature maps contained in the second feature map group through the first classification segmentation sub-model, and based on the fused feature maps, it is determined whether the first target lesion type and the lesion area corresponding to the first target lesion type exist in the target body part image.

4. The method according to claim 1, wherein The third feature map group is composed of feature maps of multiple scales. The second image detection model further includes a second classification and segmentation sub-model and a pooling module. The second classification and segmentation sub-model includes a memory unit and an attention module. The memory unit is trained to store the positions and visual features corresponding to different lesion types included in the first target lesion type in the target body part. The memory unit is configured to store the positions and visual features using a target number of memory vectors. The performing a second image classification and segmentation process on the third feature map group includes: For the target feature maps in the feature maps of the multiple scales, the pooling module is used to perform pooling processing on the target feature map to compress the target feature map into the feature vector of the target number; the attention module is used to perform cross-attention processing on the reference vector of the target number and the feature vector of the target number, and self-attention processing on the reference vector of the target number, and the cross-attention processing results and the self-attention processing results are added; wherein, when the target feature map is the first of the feature maps of the multiple scales, the reference vector is the memory vector, and when the target feature map is not the first of the feature maps of the multiple scales, the reference vector is the sum of the cross-attention processing result and the self-attention processing result corresponding to the previous target feature map; the target feature map is any one of the feature maps of the multiple scales; According to the sum of the cross-attention processing result and the self-attention processing result corresponding to the last target feature map, the second target lesion type and lesion area present in the target body part image are determined.

5. The method according to claim 4, characterized in that The second image detection model includes a position embedding module; the attention module performs cross attention processing on the reference vector of the target number and the feature vector of the target number, including: Superimposing corresponding position embedding vectors on the target number of feature vectors respectively, wherein the position embedding vector superimposed on any feature vector is used to represent the position information corresponding to the any feature vector in the target number of feature vectors; The attention module performs cross-attention processing on the reference vector of the target quantity and the feature vector of the position embedding vector corresponding to each other on the superposition of the target quantity.

6. The method according to claim 4, characterized in that The second feature extraction sub-model includes a second encoding module, a second decoding module, and a jump layer between the second encoding module and the second decoding module; The extracting a third feature map group corresponding to the target body part image by the second feature extraction submodel in the second image detection model includes: extracting a fourth feature map group corresponding to the target body part image by the second encoding module; Inputting the fourth feature map group into the second decoding module through the skip layer; Acquire a fifth feature map group corresponding to the target body part image through the second decoding module; It is determined that some feature maps included in the fourth feature map group and some feature maps included in the fifth feature map group constitute the third feature map group.

7. The method according to claim 1, characterized in that The first image detection model and the second image detection model are respectively used as target image detection models, and the method further includes: Obtaining a training sample set for training the target image detection model; Constructing a plurality of training sample subsets corresponding to the training sample set; Multiple target image detection models are trained using the multiple training sample subsets respectively.

8. The method according to claim 7, characterized in that The step of extracting a second feature map group corresponding to the target body part image by using a first feature extraction sub-model in the first image detection model, and performing a first image classification and segmentation process on the second feature map group to determine whether a lesion area corresponding to a first target lesion type exists in the target body part image, includes: extracting a second feature map group corresponding to the target body part image using first feature extraction sub-models in the plurality of first image detection models, respectively, and performing first image classification and segmentation processing on the second feature map group to obtain output results of each of the plurality of first image detection models; determining, based on output results of each of the plurality of first image detection models, whether a first target lesion type and a lesion region corresponding to the first target lesion type exist in the target body part image; The extracting a third feature map group corresponding to the target body part image by the second feature extraction submodel in the second image detection model, and performing a second image classification and segmentation process on the third feature map group to determine a second target lesion type and lesion area present in the target body part image includes: extracting a third feature map group corresponding to the target body part image using the second feature extraction sub-models in the plurality of second image detection models, and performing a second image classification and segmentation process on the third feature map group to obtain output results of each of the plurality of second image detection models; According to the output results of each of the multiple second image detection models, the second target lesion type and lesion area present in the target body part image are determined.

9. A computer-aided diagnosis method for cancer, characterized in that: include: acquiring a detection image obtained by plain scan computed tomography; extracting a target body part image corresponding to the target body part from the detection image; extracting a second feature map group corresponding to the target body part image by using a first feature extraction submodel in the first image detection model, and performing a first image classification and segmentation process on the second feature map group to determine whether a first target lesion type and a lesion area corresponding to the first target lesion type exist in the target body part image; A third feature map group corresponding to the target body part image is extracted through the second feature extraction submodel in the second image detection model, and a second image classification and segmentation processing is performed on the third feature map group to determine the second target lesion type and lesion area present in the target body part image, where the second target lesion type is a subcategory of the first target lesion type.

10. A computer-aided diagnosis method for pancreatic cancer, characterized in that: include: acquiring a detection image obtained by plain scan computed tomography; extracting a pancreatic region image corresponding to the pancreas from the detection image; extracting a second feature map group corresponding to the pancreatic region image by using a first feature extraction sub-model in the first image detection model, and performing a first image classification and segmentation process on the second feature map group to determine whether a first target lesion type and a lesion region corresponding to the first target lesion type exist in the pancreatic region image; A third feature map group corresponding to the pancreatic region image is extracted through a second feature extraction submodel in a second image detection model, and a second image classification and segmentation process is performed on the third feature map group to determine a second target lesion type and lesion area present in the pancreatic region image, where the second target lesion type is a subcategory of the first target lesion type.

11. An image detection method, characterized in that: Applied to extended reality devices, including: acquiring a detection image obtained by plain scan computed tomography; extracting a target body part image corresponding to the target body part from the detection image; extracting a second feature map group corresponding to the target body part image by using a first feature extraction submodel in the first image detection model, and performing a first image classification and segmentation process on the second feature map group to determine whether a first target lesion type and a lesion area corresponding to the first target lesion type exist in the target body part image; extracting a third feature map group corresponding to the target body part image by a second feature extraction submodel in a second image detection model, and performing a second image classification and segmentation process on the third feature map group to determine a second target lesion type and lesion area present in the target body part image, where the second target lesion type is a subcategory of the first target lesion type; The detection image marked with the second target lesion type and lesion area is displayed.

12. A computer-aided diagnosis system for cancer, characterized in that: Including client and server, The client is configured to call a service interface of an image detection service provided by the server to trigger an image detection request to the server, wherein the image detection request includes a detection image obtained by plain scan electronic computed tomography; The server is used to extract a target body part image corresponding to the target body part from the detection image; extract a second feature map group corresponding to the target body part image through a first feature extraction submodel in a first image detection model, and perform a first image classification and segmentation process on the second feature map group to determine whether a first target lesion type and a lesion area corresponding to the first target lesion type exist in the target body part image; extract a third feature map group corresponding to the target body part image through a second feature extraction submodel in a second image detection model, and perform a second image classification and segmentation process on the third feature map group to determine whether a second target lesion type and a lesion area exist in the target body part image, wherein the second target lesion type is a subcategory of the first target lesion type; and feed back the detection image marked with the second target lesion type and lesion area to the client.

13. The system according to claim 12, wherein: The client includes a user device, and the server includes a cloud.

14. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 11.

15. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to perform the steps of the method according to any one of claims 1 to 11.

16. A computer program product, characterized in that include: A computer program or instruction, when executed by a processor of an electronic device, causes the processor to perform the steps of the method according to any one of claims 1 to 11.

Images