Image processing model training method and image processing method
By training an image processing model and using multiple feature extraction networks and image registration techniques to process plain CT images, the problems of high cost of endoscopic screening and insufficient sensitivity of plain CT were solved, achieving efficient and low-cost gastric cancer screening and improving the detection rate of early gastric cancer.
Patent Information
- Application Number
- PCT/CN2026/087813
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-15
- Filing Date
- 2026-03-31
- Publication Date
- 2026-11-19
AI Technical Summary
In most countries with a high incidence of gastric cancer, large-scale gastric cancer screening of the general population via endoscopy is not feasible or cost-effective, resulting in high mortality rates. Existing plain CT images are not sensitive enough to detect early gastric cancer and are difficult to clearly show tumor boundaries.
By training an image processing model and using multiple feature extraction networks to process plain CT images, and combining masked regions of enhanced sample images with image registration techniques, accurate detection results are generated to achieve cancer screening in the stomach region.
It improves the detection rate of gastric cancer, reduces screening costs, provides a non-invasive, efficient and reliable gastric cancer screening method, and enhances the ability to detect early gastric cancer.
Smart Images

Figure CN2026087813_19112026_PF_FP_ABST
Abstract
Description
Image processing model training methods, image processing methods
[0001] This disclosure claims priority to Chinese Patent Application No. 202510629379.0, filed with the China Patent Office on May 15, 2025, entitled “Image Processing Model Training Method, Image Processing Method”, the entire contents of which are incorporated herein by reference. Technical Field
[0002] This disclosure relates to the field of computer technology, and in particular to an image processing model training method and an image processing method. Background Technology
[0003] In most countries with a high incidence of gastric cancer, large-scale gastric cancer screening in the general population via endoscopy is either impractical or cost-inefficient, resulting in high mortality rates. This is mainly due to the high cost of endoscopy, low detection rate (the proportion of gastric cancer cases detected out of the total number of individuals screened), and invasiveness, leading to low patient compliance.
[0004] Currently, enhanced CT (Computed Tomography) is used to assess gastric cancer, including depth of invasion, lymph node metastasis, and distant metastasis. However, plain CT, as a low-cost and widely used imaging protocol, is more commonly used in health check-up centers and hospitals (especially in resource-constrained areas), but detecting gastric cancer in plain CT images may be more challenging. Therefore, there is an urgent need for a method using plain CT for gastric cancer screening to effectively reduce costs and improve the detection rate of gastric cancer. Summary of the Invention
[0005] In view of this, embodiments of this disclosure provide an image processing model training method. One or more embodiments of this disclosure also relate to an image processing method, a CT image processing method, a computer-aided diagnosis method for cancer, a computer-aided diagnosis method for gastric cancer, a computer-aided diagnosis system for cancer, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.
[0006] According to a first aspect of the present disclosure, an image processing model training method is provided, comprising:
[0007] Obtain training sample pairs, wherein the training sample pairs include multiple sample target images and sample detection results corresponding to the target detection region;
[0008] The plurality of sample target images are input into an initial image processing model to obtain a predicted detection result corresponding to the target detection region. The initial image processing model includes at least two feature extraction networks. The predicted detection result is generated based on the features of at least two sample images. The sample image features are obtained by processing the plurality of sample target images through the feature extraction networks.
[0009] Calculate the model loss value based on the predicted detection results and the sample detection results;
[0010] The model parameters of the initial image processing model are adjusted based on the model loss value until the model training stops, thereby obtaining the target image processing model with adjusted model parameters.
[0011] According to a second aspect of the present disclosure, an image processing method is provided, comprising:
[0012] Receive an image processing task, wherein the image processing task carries multiple target images corresponding to a target detection region, and the image processing task is used to detect whether the target detection region is abnormal;
[0013] The multiple target images are input into the target image processing model to obtain the detection results of the target detection region output by the target image processing model, wherein the target image processing model is trained by the above-described image processing model training method.
[0014] According to a third aspect of the present disclosure, a CT image processing method is provided, comprising:
[0015] Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area;
[0016] The multiple plain CT images are input into a CT image processing model to obtain the detection results of the target detection area output by the CT image processing model. The CT image processing model is trained by the above-described image processing model training method.
[0017] According to a fourth aspect of the present disclosure, a computer-aided diagnosis method for cancer is provided, comprising:
[0018] Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is a tumor in the target detection area;
[0019] The multiple plain CT images are input into a CT image processing model to obtain the detection results of the target detection area output by the CT image processing model. The CT image processing model is trained by the above-described image processing model training method.
[0020] According to a fifth aspect of the present disclosure, a computer-aided diagnostic method for gastric cancer is provided, comprising:
[0021] The system receives a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the stomach region, and the CT image processing task is used to detect whether there is a tumor in the stomach region;
[0022] The multiple plain CT images are input into the CT image processing model to obtain the detection results of the stomach region output by the CT image processing model. The CT image processing model is trained by the above-described image processing model training method.
[0023] According to a sixth aspect of the present disclosure, a computer-aided diagnostic system for cancer is provided, comprising a client and a server, wherein,
[0024] The client is used to send a CT image processing task to the server, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is a tumor in the target detection area;
[0025] The server is used to input the multiple plain CT images into the CT image processing model, obtain the detection results of the target detection area output by the CT image processing model, and send the detection results to the client. The CT image processing model is trained by the above-described image processing model training method.
[0026] According to a seventh aspect of the present disclosure, a computing device is provided, comprising:
[0027] Memory and processor;
[0028] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above method.
[0029] According to an eighth aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.
[0030] According to a ninth aspect of the present disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.
[0031] This disclosure provides an embodiment of an image processing model training method, which obtains training sample pairs, including multiple sample target images and sample detection results corresponding to a target detection region. An initial image processing model is created using these multiple sample target images. At least two feature extraction networks within the initial image processing model are then used to process the multiple sample target images, obtaining sample image features output by each feature extraction network. This yields at least two sample image features of different scales and diverse characteristics. Predicted detection results corresponding to the target detection region are obtained using these at least two sample image features. A model loss value is calculated based on the predicted detection results and the sample detection results. The model parameters of the initial image processing model are then adjusted based on the model loss value to achieve supervised training of the initial image processing model until the model training stops, resulting in a target image processing model with adjusted model parameters. This enables the target image processing model to process target images and obtain accurate detection results for the target detection region when subsequently applied. Attached Figure Description
[0032] Figure 1 is a schematic diagram of a scenario of an image processing model training method provided in an embodiment of this disclosure;
[0033] Figure 2 is a flowchart of an image processing model training method provided in an embodiment of this disclosure;
[0034] Figure 3 is a flowchart of an image processing method provided in an embodiment of this disclosure;
[0035] Figure 4 is a flowchart of a CT image processing method provided in an embodiment of this disclosure;
[0036] Figure 5 is a schematic diagram of the processing procedure of a CT image processing method for gastric cancer detection provided in an embodiment of this disclosure;
[0037] Figure 6 is a flowchart of a computer-aided diagnosis method for cancer provided in an embodiment of this disclosure;
[0038] Figure 7 is a flowchart of a computer-aided diagnostic method for gastric cancer provided in an embodiment of this disclosure;
[0039] Figure 8 is a system architecture diagram of a computer-aided diagnosis system for cancer provided in an embodiment of this disclosure;
[0040] Figure 9 is a schematic diagram of the structure of an image processing model training device provided in an embodiment of this disclosure;
[0041] Figure 10 is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of the present disclosure;
[0042] Figure 11 is a structural block diagram of a computing device provided in an embodiment of this disclosure. Detailed Implementation
[0043] Numerous specific details are set forth in the following description to provide a full understanding of this disclosure. However, this disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this disclosure. Therefore, this disclosure is not limited to the specific implementations disclosed below.
[0044] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this disclosure refers to and includes any or all possible combinations of one or more associated listed items.
[0045] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this disclosure, and similarly, second may also be referred to as first. Depending on the context, the word “if” as used herein may be interpreted as “when”, “in response to a determination”, or “when…”.
[0046] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.
[0047] First, the terms and concepts involved in one or more embodiments of this disclosure will be explained.
[0048] CT (Computed Tomography): Computed tomography is a scanning technology that uses a precisely collimated X-ray beam and a highly sensitive detector to scan a specific part of the body one section after another. It features fast scanning time and clear images and can be used to examine a variety of diseases.
[0049] Plain CT scan: also known as conventional CT, refers to a scan performed without intravenous contrast agent administration.
[0050] Enhanced CT: This refers to a scanning method that involves injecting a contrast agent into a blood vessel before scanning. The purpose is to increase the density difference between the diseased tissue and normal tissue, so as to show lesions that are not shown or are not clearly shown on plain CT. The presence or absence of enhancement and the type of enhancement help to characterize the lesion.
[0051] CAD (computer-aided diagnosis): Computer-aided diagnosis refers to the use of imaging, medical image processing technology, and other possible physiological and biochemical methods, combined with computer analysis and calculation, to assist in the detection of lesions and improve the accuracy of diagnosis.
[0052] MOE: Mixture of experts, is a neural network architecture designed to improve overall model performance by integrating the predictions of multiple models or "experts".
[0053] Stomach cancer is the fifth most common cancer worldwide and the fourth leading cause of cancer-related death. Nearly 75% of new cases and related deaths occur in Asian countries, particularly China, Japan, and South Korea. Effective screening is crucial for reducing mortality, as studies show that the 5-year survival rate for early-stage resectable stomach cancer is 95%–99%, while the survival rate for late-stage stomach cancer is less than 30%. Endoscopy is the standard diagnostic test for stomach cancer because it allows direct visualization of the gastric mucosa and the opportunity to perform a biopsy for histological evaluation. Endoscopy has improved early diagnosis rates and reduced mortality.
[0054] However, in most countries with a high incidence of gastric cancer, large-scale gastric cancer screening in the general population via endoscopy is either impractical or cost-effective, leading to high mortality rates. This is primarily due to the high cost, low detection rate (the proportion of gastric cancer cases detected out of the total number of individuals screened), and invasive nature of endoscopy, resulting in low patient compliance. Screening high-risk individuals is preferable; if a previous screening technique yields a positive result (cancer diagnosis), the individual is more likely to undergo endoscopic screening for further confirmation. Therefore, there is an urgent need to develop new non-invasive, low-cost, efficient, and reliable screening technologies to identify individuals at high risk for gastric cancer.
[0055] Currently, contrast-enhanced CT is used to assess gastric cancer, including depth of invasion, lymph node metastasis, and distant metastasis. Plain CT is a low-cost and widely used imaging protocol, commonly used in health check-up centers and hospitals (especially in resource-constrained areas). Chest CT (used for lung cancer screening) and abdominal CT (routinely used in emergency imaging and community hospitals) scans primarily or completely cover the stomach; however, while plain CT is a valuable tool for opportunistic screening, it lacks sensitivity for early gastric cancer. Early lesions are small and have low contrast, and the lack of contrast agent enhancement on plain CT makes it difficult to clearly show tumor boundaries. Therefore, plain CT faces significant challenges in detecting early gastric cancer.
[0056] Based on this, this disclosure provides an image processing model training method. One or more embodiments of this disclosure also relate to an image processing method, a CT image processing method, a computer-aided diagnosis method for cancer, a computer-aided diagnosis method for gastric cancer, a computer-aided diagnosis system for cancer, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.
[0057] Referring to Figure 1, Figure 1 shows a scene diagram of an image processing model training method provided according to an embodiment of the present disclosure.
[0058] Specifically, Figure 1 includes an image processing system, which may include a client 100 and a server 200;
[0059] Client 100 is used to send an image processing model training task to server 200, wherein the image processing model training task carries a training sample pair, wherein the training sample pair includes multiple sample target images and sample detection results corresponding to the target detection region;
[0060] Server 200 is used to obtain training sample pairs; input the multiple sample target images into an initial image processing model to obtain the predicted detection result corresponding to the target detection region, wherein the initial image processing model includes at least two feature extraction networks, the predicted detection result is generated based on the features of at least two sample images, and the sample image features are obtained by processing the multiple sample target images through the feature extraction networks; calculate the model loss value based on the predicted detection result and the sample detection result; adjust the model parameters of the initial image processing model based on the model loss value until the model training stopping condition is reached to obtain the target image processing model with adjusted model parameters.
[0061] Client 100 is also used to receive the target image processing model or the calling interface of the target image processing model sent by server 200.
[0062] An image processing system may include multiple clients 100 and a server 200. Clients 100 can be referred to as edge devices, and the server 200 can be referred to as cloud devices. Multiple clients 100 can establish communication connections through the server 200. In an image processing scenario, the server 200 is used to provide image processing services between the multiple clients 100. Each client 100 can act as a sender or receiver, communicating through the server 200.
[0063] Users can interact with the server 200 through client 100 to receive data sent by other clients 100, or send data to other clients 100, etc.
[0064] In this system, client 100 and server 200 establish a connection via a network. The network provides the medium for communication between client 100 and server 200. The network can include various connection types, such as wired or wireless communication links or fiber optic cables. Data transmitted by client 100 may need to undergo encoding, transcoding, compression, or other processing before being published to server 200.
[0065] Client 100 can be a browser, an app (application), a web application such as an H5 (HyperText Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. Client 100 can be developed based on the software development kit (SDK) of the corresponding service provided by server 200, such as a real-time communication (RTC) SDK. Client 100 can be deployed on electronic devices and depends on the device or certain apps on the device to run. Electronic devices may have displays and support information browsing, such as personal mobile terminals like mobile phones, tablets, and personal computers. Various other types of applications can also be configured on electronic devices, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.
[0066] Server 200 may include servers providing various services, such as servers providing communication services to multiple clients, servers supporting backend training of models used on clients, and servers processing data sent by clients. It should be noted that server 200 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server integrated with blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.
[0067] It is worth noting that the image processing method provided in the embodiments of this disclosure is generally executed by the server. However, in other embodiments of this disclosure, the client may also have similar functions to the server, thereby executing the image processing method provided in the embodiments of this disclosure. In other embodiments, the image processing method provided in the embodiments of this disclosure may also be executed jointly by the client and the server.
[0068] Referring to Figure 2, Figure 2 shows a flowchart of an image processing model training method provided in an embodiment of the present disclosure, which specifically includes the following steps.
[0069] Step 202: Obtain training sample pairs, wherein the training sample pairs include multiple sample target images and sample detection results corresponding to the target detection region.
[0070] Specifically, the image processing model training method provided in this disclosure uses supervised training, which includes multiple training sample pairs. Each training sample pair includes sample target images and sample detection results for a target detection region. The sample target image can be understood as an image of the target region. In practical applications, the target detection region is a three-dimensional region; therefore, there are multiple sample target images for the target detection region, and these multiple sample target images can form a three-dimensional object for the target detection region. The sample detection result can be understood as anomaly annotation information for the target detection region. For example, the sample detection result includes images annotated with anomaly location information in the target detection region and / or text indicating whether anomalies exist in the target detection region, etc.
[0071] For example, in assisted medical scenarios, training sample pairs include CT images of the target detection area and the corresponding detection results. By training an initial image processing model, the trained target image processing model can determine the corresponding detection results for the target detection area based on the plain CT image. The CT images in the training sample pairs are plain CT images. Since the plain CT image lacks contrast agent, the detection results corresponding to the target detection area cannot be clearly determined based on the plain CT image. Therefore, the plain CT image is processed by using the annotation information in the enhanced CT image corresponding to the target detection area to obtain the sample detection results.
[0072] In one or more embodiments of this disclosure, detection is performed on a sample detection object to obtain a sample enhancement image and a sample target image corresponding to the sample detection object. The sample enhancement image and the sample target image contain the same target detection region of the sample detection object. The sample target image is processed using the annotation information corresponding to the sample enhancement image to obtain the sample detection result corresponding to the target sample image. Specific implementation methods are as follows:
[0073] The process of obtaining training sample pairs includes:
[0074] A sample detection object is determined, and multiple sample target images and multiple sample enhancement images corresponding to the sample detection object are obtained. The sample enhancement images and sample target images corresponding to the sample detection object correspond to the same target detection region of the sample detection object. The multiple sample enhancement images generate mask annotation regions based on the target detection region.
[0075] When the multiple sample target images and the multiple sample enhanced images are image registered, the sample target images are processed using the mask annotation regions in the sample enhanced images to obtain sample mask images;
[0076] Based on the sample mask image and the sample classification results corresponding to the multiple sample augmentation images, the sample detection results are obtained, and based on the multiple sample target images and the sample detection results, the training sample pairs are obtained.
[0077] Here, the sample detection object specifically refers to the object corresponding to the target detection region. For example, when the target detection region is the stomach, the sample detection object is the organism corresponding to the target detection region, such as a human, horse, or cow. After determining the sample detection object, the sample enhancement image and sample target image corresponding to the sample detection object are further obtained. The sample enhancement image and sample target image are images targeting the same target detection region.
[0078] The mask annotation area can be understood as the area in the target detection area where the contour of abnormal tissue is accurately marked, such as marking abnormal tissue by using a mask. In the context of assisted medical care, the mask annotation area can be understood as the area generated by mask annotation for the tumor area. Based on this mask annotation area, the location, shape, and size of the tumor can be clearly marked, forming a contrast with the surrounding normal tissue.
[0079] In practical applications, anomalous information in the augmented sample image is more obvious than in the target sample image. Therefore, mask annotation regions can be generated in the augmented sample image for the target detection area. Specifically, technicians can manually annotate the augmented sample image, or the augmented sample image can be input into an annotation model, which will then recognize the augmented sample image and generate mask annotation regions. There is no limitation on this.
[0080] In one specific embodiment provided in this disclosure, a sample augmentation image can be input into an information annotation model, which then annotates the sample augmentation image. Specifically, the information annotation model annotates mask annotation regions in the sample augmentation image and labels the sample classification results for the sample augmentation image. The mask annotation regions specifically refer to the segmentation mask information for abnormal regions in the target detection region, and the sample classification results specifically refer to the abnormal classification information for the target detection region.
[0081] It should be noted that the target detection area may have different characteristics at different time points. To ensure data consistency between the enhanced sample image and the target sample image, the method provided in this disclosure refers to the enhanced sample image and the target sample image as images of the same target detection area at the same time point. For example, taking a patient's CT image as an example, if the target detection area is the patient's stomach, then the target sample image and the enhanced sample image are the plain CT image and the enhanced CT image taken when the patient underwent examination at a certain time point. Both the enhanced CT image and the plain CT image are images of the stomach area, and both images must be from the same examination at the same time.
[0082] When the mask annotation region is determined and multiple sample target images are registered with multiple sample augmented images, annotations can be performed in the sample target images based on the mask annotation region. That is, since the abnormal information for abnormal regions is more obvious in the sample augmented images, the mask annotation region can be directly annotated; however, the abnormal information for abnormal regions in the sample target images is not obvious and cannot be annotated using annotation models or manual annotation methods. In this case, since the sample augmented images and sample target images are captured at the same time for the same target detection area, the mask annotation region in the sample augmented image can be used to annotate the sample target image. For example, if the annotated region in the mask annotation region is range 1, then the sample target image can be annotated based on range 1 to obtain the sample mask image corresponding to the sample target image. This sample mask image then includes mask annotations for range 2, where range 1 and range 2 are the same region.
[0083] In practical applications, image registration can be used to annotate the target image with masked regions in the enhanced image, thus determining the masked regions in the target image and obtaining the target image. Image registration refers to the process of matching and overlaying two or more images. The purpose of image registration is to find the spatial transformation relationship between two or more images, aligning them spatially, thereby enabling further image analysis or processing.
[0084] Image registration methods and techniques vary depending on the application field and specific needs. In medical image processing, image registration techniques are used to achieve spatial consistency between corresponding points on a medical image. In the method provided in this disclosure, masked annotation regions are marked in the enhanced sample image. Through image registration, the masked annotation regions in the enhanced sample image are transferred to the target sample image. Similarly, masked annotation regions are also performed in the target sample image. That is, through large-scale data annotation and the registration method from enhanced CT images to plain CT images, effective network supervision information can be obtained.
[0085] When a sample augmentation image is obtained, the sample classification result for the target detection area can be determined based on the sample augmentation image (e.g., the classification result includes normal and abnormal). In medical assistance scenarios, the sample classification result can be understood as cancer or non-cancer. For example, if the target detection area is the stomach area, the sample classification result is stomach cancer or non-stomach cancer.
[0086] Based on the obtained sample mask image and sample classification results, the sample detection results for the target detection area are determined.
[0087] The image processing model training method provided in this embodiment determines the mask annotation region in the sample target image to obtain the sample mask image. Specifically, the mask annotation region is delineated in the sample enhancement image, and the sample enhancement image and the sample target image corresponding to the sample detection object are aligned through an image registration algorithm, thereby migrating the mask annotation region in the sample enhancement image to the sample target image and accurately performing mask annotation on the mask annotation region in the sample target image.
[0088] In practical applications, CT images captured by different CT machines may have inconsistent sizes. In one or more embodiments of this disclosure, after obtaining multiple initial sample images containing the target detection region, the initial sample images can be processed using a region segmentation model to obtain the target sample image. Specifically, the region segmentation model is used to crop the corresponding region of interest (which includes the corresponding target detection region) from the initial sample images. By preprocessing the initial sample images and focusing on the region of interest, image processing in subsequent stages can be simplified and the processing efficiency of subsequent stages can be improved. Specific implementation methods are described below:
[0089] The process of obtaining training sample pairs includes:
[0090] Acquire multiple initial sample images containing the target detection region, and input the multiple initial sample images into the region segmentation model to obtain the target sample images corresponding to the multiple initial sample images;
[0091] The training sample pairs are obtained based on the multiple sample target images and sample detection results.
[0092] The initial sample image can be understood as the original captured image. This initial sample image may contain multiple non-target detection regions, which need to be further extracted from the initial sample image using a region segmentation model. The region segmentation model can be understood as an image segmentation model (such as U-Net, Mask R-CNN, etc.) used to accurately crop out the sub-image (i.e., the target image) of the target detection region from the initial sample image. The target image obtained after processing by the region segmentation model contains the target detection region and removes irrelevant background or other interference.
[0093] In assisted medical scenarios, the initial sample image can be a chest CT scan (including organs such as the lungs, heart, and stomach). If the target detection area is determined to be the stomach, the region segmentation model is a stomach segmentation model. The chest CT scan is input into the trained stomach segmentation model, which can automatically identify and crop out the stomach area to generate the corresponding stomach CT scan image.
[0094] In practical applications, the region segmentation model can be a segmentation network based on nnUNet (Adaptive Segmentation U-Net), implemented in full 3D resolution. It can locate the target detection region in CT images. For example, when the target detection region is the stomach region, it outputs a stomach segmentation mask. This stomach segmentation mask is used to generate a 3D bounding box of the stomach region, which is convenient for subsequent cropping. The cropped image is obtained as a plain CT image of the stomach. Thus, when detecting whether there is a tumor in the stomach, it can focus on the detection and processing of the stomach region, improve the processing efficiency of image processing, and improve the detection accuracy after removing irrelevant background and the influence of other organs.
[0095] Step 204: Input the plurality of sample target images into the initial image processing model to obtain the predicted detection result corresponding to the target detection region. The initial image processing model includes at least two feature extraction networks. The predicted detection result is generated based on the features of at least two sample images. The sample image features are obtained by processing the plurality of sample target images through the feature extraction networks.
[0096] The initial image processing model can be understood as an image processing model that has not yet been trained. The image processing model is constructed to process the input sample target image, identify and segment abnormal information on the target detection area, and generate the predicted detection result corresponding to the target detection area.
[0097] The predicted detection result can be understood as the anomaly information predicted on the target detection region after the initial image processing model identifies and segments the sample target image. The predicted detection result includes a sample mask image that segments the anomaly information, and the sample classification result obtained by classifying and recognizing the anomaly information. In practical applications, the sample target image is input into the initial image processing model, and the initial image processing model can output the sample mask image corresponding to the target detection region and the sample classification result based on the sample target image.
[0098] In one specific embodiment provided in this disclosure, the sample target image can be multiple CT images targeting the target detection region. The CT images are input into an initial image processing model, which is used to segment and identify the CT images to determine the predicted detection result corresponding to the target detection region.
[0099] When multiple sample target images are input into an initial image processing model, at least two feature extraction networks in the initial image processing model are used to process the sample target images to obtain the sample image features of multiple sample target images output by each feature extraction network, thereby generating a prediction detection result based on at least two sample image features.
[0100] In one or more embodiments of this disclosure, at least two feature extraction networks of an initial image processing model are used to extract at least two sample image features from the target image. These at least two sample image features are then fused to obtain fused image features, thereby obtaining a prediction detection result based on the fused image features. Specific implementation methods are described below:
[0101] The step of inputting the plurality of sample target images into an initial image processing model to obtain the predicted detection result corresponding to the target detection region includes:
[0102] The plurality of sample target images are input into at least two feature extraction networks of the initial image processing model to obtain at least two sample image features output by the at least two feature extraction networks, wherein one feature extraction network outputs one sample image feature;
[0103] Feature fusion is performed on the features of the at least two sample images to obtain fused image features, and the predicted detection result corresponding to the target detection region is obtained based on the fused image features.
[0104] The initial image processing model contains at least two independent feature extraction networks for extracting sample image features from different angles. The feature extraction networks are used to extract high-level semantic features from the input sample target image. Different feature extraction networks have different structures or parameters to capture complementary features. The high-dimensional feature representation output by each feature extraction network contains key information of the image.
[0105] Feature fusion can be understood as merging the sample image features output by multiple feature extraction networks into a unified feature representation through fusion methods such as splicing, weighted summation, or attention mechanisms.
[0106] Taking the auxiliary medical scenario as an example, after obtaining a cropped plain CT scan of the stomach, the plain CT scan of the stomach is simultaneously input into two feature extraction networks of the initial image processing model. The first feature extraction network is used to obtain the first sample image features containing local texture information, and the second feature extraction network is used to obtain the second sample image features that capture the overall shape of the stomach. The first sample image features and the second sample image features are merged in the channel dimension to obtain the fused image features. The fused image features are used to output the lesion location (i.e., the sample mask image) and the lesion probability (i.e., the sample classification result).
[0107] In practical applications, the initial image processing model can be understood as a hybrid expert model that adopts a multi-vision MOE architecture. For example, CNN (convolutional neural network), Transformer (a deep learning architecture based on multi-head attention), and Mamba (a deep learning architecture based on selective state space model) can be used as multiple experts in the MOE, i.e., the feature extraction network in the initial image processing model.
[0108] In practice, CNNs tend to extract local texture information, while Transformer and Mamba both obtain sample image features of global information. However, since Transformer and Mamba have different computational principles, the sample image features they extract are also different. In this way, diverse sample image features are obtained to determine information at different scales of the target detection area.
[0109] The image processing model training method provided in this disclosure obtains diverse image features corresponding to sample target images through at least two feature extraction networks. By using diverse sample image features, local details and global context information of the target detection region can be extracted. Thus, joint judgment of the target detection region is performed through feature fusion, avoiding the omission of key information by a single feature.
[0110] In one or more embodiments of this disclosure, the initial image processing model includes a gated selection network; this gated selection network is a separate neural network that outputs a vector when a sample target image is input to it, storing the weights of each expert network that subsequently passes through it, with the sum of all weights being 1. Specific implementation details are as follows:
[0111] The feature fusion of the features of the at least two sample images to obtain fused image features includes:
[0112] The multiple sample target images are input into the gated selection network to obtain the feature weights corresponding to the at least two feature extraction networks;
[0113] Based on the feature weights corresponding to the at least two feature extraction networks, the features of the at least two sample images are subjected to feature weighting processing to obtain the fused image features.
[0114] The gated selection network can be understood as a dynamic weight allocation network that automatically calculates the importance weights of different feature extraction networks through attention mechanisms or learnable parameters. Feature weights can be understood as scalar values (typically in the range of [0,1]) output by the gated selection network, representing the reliability or importance of the corresponding feature extraction network. For example, the feature weight of the first feature extraction network (used to capture detailed features) is 0.7, and the feature weight of the second feature extraction network (used to capture global features) is 0.3.
[0115] Continuing with the previous example, the stomach CT image is input into a gating network. The gating network determines the weights of each subsequent expert network (i.e., feature extraction network), for example, assigning a weight of 0.7 to the first feature extraction network and 0.4 to the second. The weights of the sample image features output from at least two feature extraction networks are summed, and then the fused image features are obtained through feature concatenation. Compared to simple feature concatenation, this method can suppress interference from low-quality features.
[0116] The image processing model training method provided in this disclosure allows the gated selection network to dynamically adjust weights based on the input sample target image, thereby determining higher weights for high-quality features, improving the quality of fused image features, and increasing prediction accuracy.
[0117] In one or more embodiments of this disclosure, the initial image processing model further includes a prediction network, which has two branches: one branch for outputting a predicted mask image and the other branch for outputting a predicted classification result. Specific implementation details are as follows:
[0118] The step of obtaining the predicted detection result corresponding to the target detection region based on the fused image features includes:
[0119] The fused image features are input into the prediction network of the initial image processing model to obtain the prediction detection results output by the prediction network, wherein the prediction detection results include the prediction mask image and the prediction classification results.
[0120] The prediction network can be understood as the network responsible for the final prediction in the initial image processing model. It typically contains two branches: a segmentation branch that outputs a pixel-level segmentation mask (predictive mask image), and a classification branch that outputs class probabilities (predictive classification result). The predictive mask image can be understood as a binary / multi-channel matrix of the same size as the sample target image, marking whether each pixel belongs to an abnormal region (e.g., 1 for abnormal regions, 0 for background or normal regions). The predicted classification result can be understood as a qualitative judgment of the target detection region (e.g., the probability distribution of categories such as "cancer" and "non-cancer").
[0121] Specifically, the fused image features are input into the prediction network, and the segmentation branch outputs a segmentation mask. For example, in the output image, white areas are used to represent tumor locations and black areas are used to represent normal tissue. In the classification branch, the category probability can be calculated through fully connected layers. For example, the output of the classification branch is malignant: 85%, benign: 12%, normal: 3%. Of course, the output of the classification branch can also be divided into two categories, namely cancer or normal, which is not limited here.
[0122] The image processing model training method provided in this disclosure uses a segmentation branch of the prediction network to constrain the prediction network to focus on abnormal or lesion areas, and a classification branch of the prediction network to guide the prediction network to distinguish between malignant / benign / normal (or cancer / normal) classification results. When the prediction network can generate segmentation and classification outputs simultaneously, the segmentation output provides a pixel-level visual image, which helps to understand and confirm the classification results. Thus, it can not only provide the patient's gastric cancer risk, but also has rich interpretability.
[0123] In one or more embodiments of this disclosure, at least two prediction networks can be trained in the initial image processing model. These at least two prediction networks have the same network structure but different training data. A classification score is determined by averaging the classification probabilities output by the at least two prediction networks, and a segmentation output is determined using pixel-level voting of the outputs of the at least two prediction networks. Specific implementation methods are described below:
[0124] The step of obtaining the predicted detection result corresponding to the target detection region based on the fused image features includes:
[0125] The fused image features are respectively input into at least two prediction networks of the initial image processing model to obtain at least two initial detection results output by the at least two prediction networks;
[0126] By averaging the at least two initial detection results, a predicted detection result corresponding to the target detection region is obtained.
[0127] Specifically, the at least two prediction networks are trained in the following manner:
[0128] The training sample pairs are grouped to determine at least two training sample groups;
[0129] The training sample group in the at least two training sample groups is determined as the validation sample group in turn, and at least two training data groups are obtained consisting of the validation sample group and the training sample group.
[0130] The at least two prediction networks are obtained by training using the at least two sets of training data.
[0131] Specifically, the training methods for at least two prediction networks are explained in detail. After obtaining multiple training sample pairs, the multiple training sample pairs are divided into at least two training sample groups. For example, if 1000 training sample pairs are obtained, the 1000 training sample pairs are evenly divided into 2 training sample groups, with each training sample group containing 500 training sample pairs.
[0132] One training sample group is selected as the validation sample group from at least two training sample groups, thus obtaining a training data group consisting of the validation sample group and the remaining training sample groups. This process is repeated by selecting each of the at least two training sample groups as a validation sample group once, resulting in at least two training data groups. For example, continuing the previous example, with a first training sample group and a second training sample group, the first training sample group is selected as the first validation sample group, thus forming the first training data group together with the second training sample group. Then, the second training sample group is selected as the second validation sample group, thus forming the second training data group together with the first training sample group and the second validation sample group. Subsequently, a first prediction network is trained using the first training data group, and a second prediction network is trained using the second training data group.
[0133] When training the prediction network using the training data set, the prediction network is trained using the training sample set in the training data set and validated using the validation sample set in the training data set.
[0134] In fact, the prediction network in this embodiment adopts the five-fold cross-validation method, that is, the training sample pairs are divided into five parts, four of which are used as training data each time, and the remaining part is used for validation. In this way, a total of five prediction networks with the same architecture but different training data can be trained.
[0135] In the application phase, the obtained fused image features are input into at least two prediction networks of the initial image processing model. The at least two prediction networks will make predictions independently, thereby obtaining at least two initial detection results output by the at least two prediction networks. The predicted detection result corresponding to the target detection region is obtained by averaging the at least two initial detection results.
[0136] In practice, when the prediction network includes both segmentation and classification branches, ensemble learning is applied to determine the final classification score (i.e., the sample classification result) by averaging the classification probabilities of at least two prediction networks, and to determine the final segmentation output (i.e., the sample mask image) by using pixel-level voting from at least two prediction networks.
[0137] The image processing model training method provided in this disclosure, through the above-mentioned cross-validation, can avoid overfitting caused by a small sample size and is conducive to eliminating the randomness of a single data partition (such as poor quality of a certain group of samples). Furthermore, the collective decision-making of multiple prediction networks is often better than that of a single prediction network, thus reducing the risk of erroneous predictions caused by the failure of a single prediction network. In assisted medical scenarios, in order to obtain more rigorous prediction results, using at least two prediction networks can reduce the risk of misdiagnosis.
[0138] Step 206: Calculate the model loss value based on the predicted detection results and the sample detection results.
[0139] Specifically, the initial image processing model is an untrained model. Therefore, the initial image processing model is trained by adjusting the model parameters by calculating the model loss value. Using the sample detection results as the ground truth, and given the predicted detection results output by the initial image processing model, the model loss value is calculated based on the predicted detection results and the sample detection results. This model loss value is used to quantify the difference between the predicted detection results (model predictions) and the sample detection results (ground truth).
[0140] In specific implementation, the sample detection result includes a sample mask image and a sample classification result, and the prediction detection result includes a prediction mask image and a prediction classification result. Therefore, the image loss value is calculated based on the sample mask image and the sample classification result, and the classification loss value is calculated based on the sample classification result and the prediction classification result. Thus, the overall model loss value corresponding to the initial image processing model is determined based on the image loss value and the classification loss value. The specific implementation method is as follows:
[0141] The step of calculating the model loss value based on the predicted detection result and the sample detection result includes:
[0142] Calculate the image loss value based on the sample mask image and the predicted mask image;
[0143] Calculate the classification loss value based on the sample classification result and the predicted classification result;
[0144] The model loss value is determined based on the image loss value and the classification loss value.
[0145] Specifically, for the segmentation branch, the predicted mask image output by the segmentation branch is compared with the sample mask image annotated by the doctor (or annotated using an annotation model) to determine the image loss value, which represents the difference between the sample mask image and the predicted mask image; for the classification branch, the predicted classification result is compared with the sample classification result (true label) to determine the classification loss value, which represents the difference between the predicted classification result and the sample classification result.
[0146] In practical applications, given the image loss value and classification loss value, the model loss value can be determined through a weighted fusion strategy, thereby jointly optimizing the loss of the segmentation branch and the classification branch. This approach can meet clinical needs, as doctors need to know both the location of the lesion (segmentation branch) and its nature (classification branch). Furthermore, accurate segmentation can help the classification branch focus on relevant regions, and accurate classification feedback can guide segmentation optimization.
[0147] The image processing model training method provided in this disclosure can jointly optimize the loss of the segmentation branch and the classification branch by determining the model loss value through the image loss value and the classification loss value, thereby improving the interpretability of the image processing model.
[0148] Step 208: Adjust the model parameters of the initial image processing model according to the model loss value until the model training stopping condition is met, and obtain the target image processing model after the model parameters are adjusted.
[0149] The target image processing model can be understood as the image processing model obtained after the initial image processing model reaches the model training stopping condition. The model structure of the target image processing model is the same as that of the initial image processing model.
[0150] Specifically, once the model loss value is obtained, the model parameters of the initial image processing model can be adjusted based on this value. After adjusting the model parameters, the above steps can be repeated to continue training the initial image processing model until the model training stopping condition is met. In practical applications, the model training stopping condition includes, but is not limited to, the model loss value being less than a preset threshold, and / or the number of training epochs reaching a preset number.
[0151] During the training of the initial image processing model, the training stopping condition can be set to the model loss value being less than a preset threshold, or the training stopping condition can be set to the number of training rounds reaching a preset number of training rounds, such as 1000 iterations.
[0152] The image processing model training method provided in this disclosure obtains training sample pairs, which include multiple sample target images and sample detection results corresponding to a target detection region. An initial image processing model is created using these multiple sample target images. At least two feature extraction networks in the initial image processing model are then used to process the multiple sample target images, obtaining sample image features output by each feature extraction network. This yields at least two sample image features of different scales and diverse features. Predicted detection results corresponding to the target detection region are obtained using these at least two sample image features. A model loss value is calculated based on the predicted detection results and the sample detection results. The model parameters of the initial image processing model are then adjusted based on the model loss value to achieve supervised training of the initial image processing model until the model training stops, resulting in a target image processing model with adjusted model parameters. This enables the target image processing model to process target images and obtain accurate detection results for the target detection region when subsequently applied.
[0153] Referring to Figure 3, Figure 3 shows a flowchart of an image processing method provided in an embodiment of the present disclosure, which specifically includes the following steps.
[0154] Step 302: Receive an image processing task, wherein the image processing task carries multiple target images corresponding to the target detection region, and the image processing task is used to detect whether the target detection region is abnormal.
[0155] In this context, the image processing task can be understood as a task to detect whether anomalies exist within a target detection region. The image processing task carries multiple target images corresponding to the target detection region. Further, the target detection region can be understood as a partition used to predict whether anomalies exist. For example, the target detection region can be any organ, such as the liver, spleen, lungs, stomach, etc. By predicting whether anomalies exist in the target detection region, the prediction results can further assist in judging the state of the object to be detected, thus helping to determine the state of the object. Detecting whether anomalies exist in the target detection region can be understood as detecting whether a tumor exists in an organ, or whether an embolism exists in a blood vessel, etc.
[0156] It should be noted that, in one or more embodiments of this disclosure, the image processing task can be used for the recognition of plain CT images. For example, in the process of identifying and segmenting gastric tumors, the stomach can be identified and segmented based on CT images, and the presence of a tumor in the stomach can be predicted. This helps doctors determine whether there are any abnormalities in the target detection area, thereby facilitating subsequent treatment.
[0157] In one specific embodiment provided in this disclosure, the detection of the presence of a tumor in the stomach is used as an example for explanation. Plain CT images of the stomach are acquired; that is, multiple target images are plain CT images including the stomach region. These multiple plain CT images constitute a 3D image of the stomach. By performing image detection on these multiple plain CT images, the presence of a tumor in the stomach is detected.
[0158] Step 304: Input the plurality of target images into the target image processing model to obtain the detection results of the target detection region output by the target image processing model, wherein the target image processing model is trained by the above image processing model training method.
[0159] In practical applications, after receiving an image processing task, multiple target images carried by the task are obtained and input into a target image processing model for processing. The detection results for the target detection region output by the target image processing model can be obtained. The detection results include the identified and segmented mask image and the classification results.
[0160] It should be noted that the target image processing model in the image processing method of this embodiment is trained by the image processing model training method in the above embodiments. This target image processing model can detect not only target images containing all target detection regions, but also target images containing only some target detection regions.
[0161] In practical applications, image processing tasks can be triggered locally on the terminal. This means multiple target images are stored locally on the terminal, and the terminal triggers the image processing task to initiate subsequent image processing steps. Alternatively, image processing tasks can be sent by the user. The terminal receives an image processing request from the user, which includes the image processing task. After processing the image using the image processing methods mentioned in the above embodiments and obtaining the detection results, the terminal returns the detection results to the user so that the user can perform appropriate subsequent processing based on the results.
[0162] In addition, the image processing task can also be triggered on the cloud-side device. That is, the user sends the image processing request to the cloud-side device through the terminal. With the powerful computing capabilities of the cloud-side device, the corresponding image processing task can be completed faster. After obtaining the detection results, the detection results are returned to the user so that the user can perform corresponding follow-up processing based on the detection results.
[0163] The image processing method provided in this disclosure combines plain CT scans with deep learning to screen target detection areas, effectively reducing costs and improving the detection rate of abnormal information. Furthermore, the target image processing model adopts a dual-channel joint classification-segmentation framework, and the output detection results include mask images and classification results, thus providing both patient disease risk information and rich interpretability.
[0164] Referring to Figure 4, Figure 4 shows a flowchart of a CT image processing method provided in an embodiment of the present disclosure, which specifically includes the following steps.
[0165] Step 402: Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area.
[0166] Step 404: Input the multiple plain CT images into the CT image processing model to obtain the detection results of the target detection area output by the CT image processing model, wherein the CT image processing model is trained by the above-described image processing model training method.
[0167] It should be noted that the implementation methods of steps 402 to 404 are the same as those of steps 302 to 304 described above, and will not be repeated in this embodiment.
[0168] Specifically, in the method provided in this embodiment, the detection of gastric tumors using CT image processing is taken as an example for further explanation. Based on this, the CT image processing task includes plain CT images of the gastric region. The plain CT images are input into the CT image processing model for identification. The CT image processing model can identify whether there is a tumor in the stomach in the plain CT images, which solves the problem that current algorithms cannot detect gastric cancer based on plain CT images, reduces screening costs, and identifies individuals at high risk of gastric cancer through the detection results output by the CT image processing model. This allows for priority consideration of endoscopic examinations for high-risk individuals, improves the cost-effectiveness of large-scale screening projects, and ultimately increases the detection rate of gastric cancer and reduces gastric cancer-related mortality.
[0169] Referring to Figure 5, Figure 5 shows a schematic diagram of the processing procedure of a CT image processing method for gastric cancer detection provided in an embodiment of the present disclosure.
[0170] Taking the use of an image processing model for tumor detection in the stomach region as an example, the image processing model aims to analyze three-dimensional plain CT scans to detect and segment gastric cancer. This image processing model will generate two outputs: one is a pixel-level segmentation mask outlining the stomach and potential tumors, and the other is a classification score that categorizes the patient as either a gastric cancer patient or a normal control group.
[0171] Before inputting the plain CT scan of the stomach into the image processing model, a stomach segmentation network (i.e., a region cutting network) is used to locate the stomach region throughout the CT scan. The stomach segmentation mask output by the stomach segmentation network is used to generate the 3D bounding box of the stomach region, which is then cropped to obtain the plain CT scan of the stomach. This cropping improves the processing efficiency of subsequent stages and simplifies the processing of CT images because tumor segmentation focuses on the region of interest. In fact, the stomach segmentation network is obtained through a semi-supervised self-training method, specifically using a training set that includes manually labeled stomach segmentation masks.
[0172] A plain CT scan of the stomach was used as input to an image processing model for tumor detection. The model employed a joint classification and segmentation network (i.e., a prediction network) with two branches. The segmentation branch focused on segmenting the tumor within the identified gastric region, while the classification branch integrated multi-level feature maps from the segmentation branch to classify the patient as either a gastric cancer patient or a normal control group.
[0173] The training process for the prediction network employs a five-fold cross-validation method. Specifically, the training set is divided into five parts, with four parts used as training data each time, and the remaining part used for validation. This allows for the training of a total of five prediction networks with the same architecture but different training data. During the testing phase, ensemble learning is applied. The output of the classification branch is determined by averaging the classification probabilities of the five prediction networks, and the output of the segmentation branch is determined by pixel-level voting among the five prediction networks. In the classification branch, the category with the higher probability (cancer, non-cancer) is selected as the final classification result for each test case. In the case of a two-branch prediction network, the prediction network can simultaneously generate the outputs of both the segmentation and classification branches. The segmentation branch output provides a pixel-level visual map (i.e., a mask image), which helps in understanding and confirming the classification result of the classification branch.
[0174] The image processing model provided in this embodiment adopts a multi-vision MOE architecture, simultaneously using a convolutional neural network (CNN), a visual transformer (Transformer), and a visual Mamba as multiple experts in the MOE to perceive information about the stomach at different scales on a plain CT scan. CNN focuses on local texture information, while Transformer and Mamba obtain global image features. However, the computational principles of Transformer and Mamba differ, resulting in different extracted image features and thus diverse image features. In fact, the image processing model includes a gating network. The plain CT scan of the stomach is input into the gating network of the image processing model to obtain the weights corresponding to each expert. Given the three image features output by the CNN, Transformer, and Mamba respectively, the weights determined by the gating network are used to fuse the three image features. The fused image features are then input into a prediction network, which obtains a detection result including a mask image and classification results.
[0175] The CT image processing method provided in this disclosure adopts a dual-channel joint classification-segmentation framework combined with a multi-vision MOE architecture, which can not only provide the patient's gastric cancer risk, but also has rich interpretability; the screening method based on plain CT and deep learning is used for gastric cancer risk stratification, effectively reducing screening costs and ultimately improving the detection rate of gastric cancer.
[0176] Referring to Figure 6, Figure 6 shows a flowchart of a computer-aided diagnosis method for cancer provided in an embodiment of the present disclosure, which specifically includes the following steps.
[0177] Step 602: Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is a tumor in the target detection area;
[0178] Step 604: Input the plurality of plain CT images into the CT image processing model to obtain the detection results of the target detection area output by the CT image processing model, wherein the CT image processing model is trained by the above-described image processing model training method.
[0179] For specific implementation details, please refer to the above embodiments, which will not be repeated here.
[0180] Referring to Figure 7, which shows a flowchart of a computer-aided diagnostic method for gastric cancer according to an embodiment of the present disclosure, the method includes the following steps.
[0181] Step 702: Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the stomach region, and the CT image processing task is used to detect whether there is a tumor in the stomach region;
[0182] Step 704: Input the multiple plain CT images into the CT image processing model to obtain the detection results of the stomach region output by the CT image processing model, wherein the CT image processing model is trained by the above-described image processing model training method.
[0183] For specific implementation details, please refer to the above embodiments, which will not be repeated here.
[0184] Corresponding to the above method embodiments, this disclosure also provides an embodiment of a computer-aided diagnosis system for cancer. Figure 8 shows a system architecture diagram of a computer-aided diagnosis system for cancer provided in one embodiment of this disclosure. As shown in Figure 8, the system includes a client 802 and a server 804, wherein...
[0185] The client 802 is used to send a CT image processing task to the server, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is a tumor in the target detection area;
[0186] The server 804 is used to input the multiple plain CT images into the CT image processing model, obtain the detection results of the target detection area output by the CT image processing model, and send the detection results to the client. The CT image processing model is trained by the above-described image processing model training method.
[0187] Corresponding to the above method embodiments, this disclosure also provides an embodiment of an image processing model training device. Figure 9 shows a schematic diagram of the structure of an image processing model training device provided in one embodiment of this disclosure. As shown in Figure 9, the device includes:
[0188] The module 902 is configured to obtain training sample pairs, wherein the training sample pairs include multiple sample target images and sample detection results corresponding to the target detection region;
[0189] The prediction module 904 is configured to input the plurality of sample target images into an initial image processing model to obtain a prediction detection result corresponding to the target detection region. The initial image processing model includes at least two feature extraction networks. The prediction detection result is generated based on the features of at least two sample images. The sample image features are obtained by processing the plurality of sample target images through the feature extraction networks.
[0190] The calculation module 906 is configured to calculate the model loss value based on the predicted detection result and the sample detection result;
[0191] The training module 908 is configured to adjust the model parameters of the initial image processing model according to the model loss value until the model training stops, thereby obtaining the target image processing model with adjusted model parameters.
[0192] Optionally, the prediction module 904 is further configured to:
[0193] The plurality of sample target images are input into at least two feature extraction networks of the initial image processing model to obtain at least two sample image features output by the at least two feature extraction networks, wherein one feature extraction network outputs one sample image feature;
[0194] Feature fusion is performed on the features of the at least two sample images to obtain fused image features, and the predicted detection result corresponding to the target detection region is obtained based on the fused image features.
[0195] Optionally, the prediction module 904 is further configured to:
[0196] The fused image features are input into the prediction network of the initial image processing model to obtain the prediction detection results output by the prediction network, wherein the prediction detection results include the prediction mask image and the prediction classification results.
[0197] Optionally, the prediction module 904 is further configured to:
[0198] The fused image features are respectively input into at least two prediction networks of the initial image processing model to obtain at least two initial detection results output by the at least two prediction networks;
[0199] By averaging the at least two initial detection results, a predicted detection result corresponding to the target detection region is obtained.
[0200] Optionally, the prediction module 904 is further configured to:
[0201] The multiple sample target images are input into the gated selection network to obtain the feature weights corresponding to the at least two feature extraction networks;
[0202] Based on the feature weights corresponding to the at least two feature extraction networks, the features of the at least two sample images are subjected to feature weighting processing to obtain the fused image features.
[0203] Optionally, the computing module 906 is further configured to:
[0204] Calculate the image loss value based on the sample mask image and the predicted mask image;
[0205] Calculate the classification loss value based on the sample classification result and the predicted classification result;
[0206] The model loss value is determined based on the image loss value and the classification loss value.
[0207] Optionally, the obtaining module 902 is further configured to:
[0208] Acquire multiple initial sample images containing the target detection region, and input the multiple initial sample images into the region segmentation model to obtain the target sample images corresponding to the multiple initial sample images;
[0209] The training sample pairs are obtained based on the multiple sample target images and sample detection results.
[0210] Optionally, the obtaining module 902 is further configured to:
[0211] A sample detection object is determined, and multiple sample target images and multiple sample enhancement images corresponding to the sample detection object are obtained. The sample enhancement images and sample target images corresponding to the sample detection object correspond to the same target detection region of the sample detection object. The multiple sample enhancement images generate mask annotation regions based on the target detection region.
[0212] When the plurality of sample target images are image registered with the plurality of sample enhanced images, the sample target images are processed using the mask annotation regions in the sample enhanced images to obtain sample mask images;
[0213] Based on the sample mask image and the sample classification results corresponding to the multiple sample augmentation images, the sample detection results are obtained, and based on the multiple sample target images and the sample detection results, the training sample pairs are obtained.
[0214] The above is a schematic scheme of an image processing model training device according to this embodiment. It should be noted that the technical solution of this image processing model training device and the technical solution of the image processing model training method described above belong to the same concept. For details not described in detail in the technical solution of the image processing model training device, please refer to the description of the technical solution of the image processing model training method described above.
[0215] Corresponding to the above method embodiments, this disclosure also provides an image processing apparatus embodiment. FIG10 shows a schematic structural diagram of an image processing apparatus provided in one embodiment of this disclosure. As shown in FIG10, the apparatus includes:
[0216] The receiving module 1002 is configured to receive an image processing task, wherein the image processing task carries multiple target images corresponding to the target detection region, and the image processing task is used to detect whether the target detection region is abnormal.
[0217] The acquisition module 1004 is configured to input the plurality of target images into a target image processing model to obtain the detection result of the target detection region output by the target image processing model, wherein the target image processing model is trained by the above-described image processing model training method.
[0218] The above is an illustrative scheme of an image processing apparatus according to this embodiment. It should be noted that the technical solution of this image processing apparatus and the technical solution of the image processing method described above belong to the same concept. For details not described in detail in the technical solution of the image processing apparatus, please refer to the description of the technical solution of the image processing method described above.
[0219] Figure 11 shows a structural block diagram of a computing device 1100 according to an embodiment of the present disclosure. The components of the computing device 1100 include, but are not limited to, a memory 1110 and a processor 1120. The processor 1120 is connected to the memory 1110 via a bus 1130, and a database 1150 is used to store data.
[0220] The computing device 1100 also includes an access device 1140, which enables the computing device 1100 to communicate via one or more networks 1160. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1140 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.
[0221] In one embodiment of this disclosure, the aforementioned components of the computing device 1100, as well as other components not shown in FIG11, may be interconnected, for example, via a bus. It should be understood that the computing device block diagram shown in FIG11 is merely for illustrative purposes and is not intended to limit the scope of this disclosure. Those skilled in the art can add or replace other components as needed.
[0222] The computing device 1100 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1100 can also be a mobile or stationary server.
[0223] The processor 1120 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above method.
[0224] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computing device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0225] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.
[0226] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer-readable storage medium embodiments are relatively simple in description because they are substantially similar to the method embodiments; relevant parts can be referred to the descriptions of the method embodiments.
[0227] An embodiment of this disclosure also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.
[0228] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the computer program product can be referred to the description of the technical solution of the above method.
[0229] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0230] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.
[0231] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this disclosure are not limited to the described order of actions, because according to the embodiments of this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this disclosure.
[0232] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0233] The preferred embodiments disclosed above are merely illustrative of this disclosure. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments of this disclosure. These embodiments are selected and specifically described in this disclosure to better explain the principles and practical applications of the embodiments of this disclosure, thereby enabling those skilled in the art to better understand and utilize this disclosure. This disclosure is limited only by the claims and their full scope and equivalents.
Claims
1. A method for training an image processing model, comprising: Obtain training sample pairs, wherein the training sample pairs include multiple sample target images and sample detection results corresponding to the target detection region; The plurality of sample target images are input into an initial image processing model to obtain a predicted detection result corresponding to the target detection region. The initial image processing model includes at least two feature extraction networks. The predicted detection result is generated based on the features of at least two sample images. The sample image features are obtained by processing the plurality of sample target images through the feature extraction networks. Calculate the model loss value based on the predicted detection results and the sample detection results; The model parameters of the initial image processing model are adjusted based on the model loss value until the model training stops, thereby obtaining the target image processing model with adjusted model parameters.
2. The image processing model training method according to claim 1, wherein inputting the plurality of sample target images into an initial image processing model to obtain the predicted detection result corresponding to the target detection region includes: The plurality of sample target images are input into at least two feature extraction networks of the initial image processing model to obtain at least two sample image features output by the at least two feature extraction networks, wherein one feature extraction network outputs one sample image feature; Feature fusion is performed on the features of the at least two sample images to obtain fused image features, and the predicted detection result corresponding to the target detection region is obtained based on the fused image features.
3. The image processing model training method according to claim 2, wherein obtaining the predicted detection result corresponding to the target detection region based on the fused image features includes: The fused image features are input into the prediction network of the initial image processing model to obtain the prediction detection results output by the prediction network, wherein the prediction detection results include the prediction mask image and the prediction classification results.
4. The image processing model training method according to claim 2, wherein obtaining the predicted detection result corresponding to the target detection region based on the fused image features includes: The fused image features are respectively input into at least two prediction networks of the initial image processing model to obtain at least two initial detection results output by the at least two prediction networks; By averaging the at least two initial detection results, a predicted detection result corresponding to the target detection region is obtained.
5. The image processing model training method according to claim 4, wherein the at least two prediction networks are trained in the following manner: The training sample pairs are grouped to determine at least two training sample groups; The training sample group in the at least two training sample groups is determined as the validation sample group in turn, and at least two training data groups are obtained consisting of the validation sample group and the training sample group. The at least two prediction networks are obtained by training using the at least two sets of training data.
6. The image processing model training method according to any one of claims 2-5, wherein the initial image processing model comprises a gating selection network; The feature fusion of the features of the at least two sample images to obtain fused image features includes: The multiple sample target images are input into the gated selection network to obtain the feature weights corresponding to the at least two feature extraction networks; Based on the feature weights corresponding to the at least two feature extraction networks, the features of the at least two sample images are subjected to feature weighting processing to obtain the fused image features.
7. The image processing model training method according to any one of claims 1-6, wherein the sample detection result includes a sample mask image and a sample classification result, and the prediction detection result includes a prediction mask image and a prediction classification result; The step of calculating the model loss value based on the predicted detection result and the sample detection result includes: Calculate the image loss value based on the sample mask image and the predicted mask image; Calculate the classification loss value based on the sample classification result and the predicted classification result; The model loss value is determined based on the image loss value and the classification loss value.
8. The method according to claim 7, wherein determining the model loss value based on the image loss value and the classification loss value comprises: The image loss value and the classification loss value are weighted and fused to determine the model loss value.
9. The image processing model training method according to any one of claims 1-8, wherein obtaining training sample pairs comprises: Acquire multiple initial sample images containing the target detection region, and input the multiple initial sample images into the region segmentation model to obtain the target sample images corresponding to the multiple initial sample images; The training sample pairs are obtained based on the multiple sample target images and sample detection results.
10. The image processing model training method according to any one of claims 1-8, wherein obtaining training sample pairs comprises: A sample detection object is determined, and multiple sample target images and multiple sample enhancement images corresponding to the sample detection object are obtained. The sample enhancement images and sample target images corresponding to the sample detection object correspond to the same target detection region of the sample detection object. The multiple sample enhancement images generate mask annotation regions based on the target detection region. When the plurality of sample target images are image registered with the plurality of sample enhanced images, the sample target images are processed using the mask annotation regions in the sample enhanced images to obtain sample mask images; Based on the sample mask image and the sample classification results corresponding to the multiple sample augmentation images, the sample detection results are obtained, and based on the multiple sample target images and the sample detection results, the training sample pairs are obtained.
11. The image processing model training method according to any one of claims 1-10, wherein the sample detection result includes an image in the target detection region marked with abnormal location information and / or text indicating whether there is an anomaly in the target detection region.
12. The image processing model training method of any one of claims 1-11, the model training stop condition comprising: The model loss value is less than a preset threshold, and / or the number of training epochs reaches a preset number of training epochs.
13. An image processing method, comprising: Receive an image processing task, wherein the image processing task carries multiple target images corresponding to a target detection region, and the image processing task is used to detect whether the target detection region is abnormal; The plurality of target images are input into a target image processing model to obtain the detection results of the target detection region output by the target image processing model, wherein the target image processing model is trained by the image processing model training method of any one of claims 1-12.
14. A CT image processing method, comprising: Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area; The plurality of plain CT images are input into a CT image processing model to obtain the detection results of the target detection region output by the CT image processing model, wherein the CT image processing model is trained by the image processing model training method of any one of claims 1-12.
15. A computer-aided diagnostic method for cancer, comprising: Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is a tumor in the target detection area; The plurality of plain CT images are input into a CT image processing model to obtain the detection results of the target detection region output by the CT image processing model, wherein the CT image processing model is trained by the image processing model training method of any one of claims 1-12.
16. A computer-aided diagnostic method for gastric cancer, comprising: The system receives a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the stomach region, and the CT image processing task is used to detect whether there is a tumor in the stomach region; The plurality of plain CT images are input into a CT image processing model to obtain the detection results of the stomach region output by the CT image processing model, wherein the CT image processing model is trained by the image processing model training method of any one of claims 1-12.
17. A computer-aided diagnosis system for cancer, comprising a client and a server, wherein, The client is used to send a CT image processing task to the server, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is a tumor in the target detection area; The server is used to input the plurality of plain CT images into a CT image processing model, obtain the detection results of the target detection region output by the CT image processing model, and send the detection results to the client. The CT image processing model is trained by the image processing model training method of any one of claims 1-12.
18. A computing device, comprising: Memory and processor; The memory is configured to store computer programs / instructions, and the processor is configured to execute the computer programs / instructions, and the computer programs / instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 16. 19.A computer readable storage medium storing computer programs / instructions, and the computer programs / instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 16. 20.A computer program product comprising computer programs / instructions, and the computer programs / instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 16.
