Image processing method and image processing model training method

Abstract

Embodiments of the present specification provide an image processing method and an image processing model training method, wherein the image processing method comprises: determining a target image for a target detection region; inputting the target image into a target image processing model to obtain a target detection result for the target detection region output by the target image processing model; wherein the target detection result is obtained by the target image processing model performing feature enhancement processing on fused image features and an initial detection result, the fused image features are obtained by the target image processing model performing fusion processing on target region features and abnormal region features, the target region features are obtained by the target image processing model performing feature extraction on a target region mask image, the abnormal region features are obtained by the target image processing model performing feature extraction on an abnormal region mask image, and the target region mask image, the abnormal region mask image and the initial detection result are obtained by the target image processing model performing segmentation processing on the target image.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to image processing methods and image processing model training methods. Background Technology

[0002] In the medical field, liver disease is a common clinical problem, and accurate diagnosis is crucial for improving patient prognosis, especially the early detection of malignant lesions, which can significantly improve patient survival rates. In practical applications, contrast-enhanced CT and MRI offer high diagnostic accuracy, but they also carry risks such as contrast agent exposure, high examination costs, and long examination times. In comparison, non-contrast CT is an ideal screening tool due to its low cost, fast imaging speed, and lack of contrast agent risks. However, because non-contrast CT relies on tissue density differences, the image features of lesions obtained from non-contrast CT scans are often subtle, making it difficult to distinguish early or minute lesions, as well as benign and malignant lesions, resulting in lower accuracy of detection results from non-contrast CT images. Therefore, an effective technical solution is urgently needed to address these issues. Summary of the Invention

[0003] In view of this, embodiments of this specification provide two image processing methods. One or more embodiments of this specification also relate to two image processing apparatuses, an image processing model training method, an image processing model training apparatus, a CT image processing method, a CT image processing apparatus, a computer-aided diagnosis method for tumors, a computer-aided diagnosis system for tumors, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, an image processing method is provided, comprising:

[0005] Determine the target image for the target detection region;

[0006] The target image is input into the target image processing model to obtain the target detection result for the target detection region output by the target image processing model;

[0007] Wherein, the target detection result is obtained by the target image processing model through feature enhancement processing of fused image features and initial detection results; the fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features; the target region features are obtained by the target image processing model through feature extraction of the target region mask image; the abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image; and the target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the target image.

[0008] According to a second aspect of the embodiments of this specification, an image processing apparatus is provided, comprising:

[0009] The determination module is configured to determine the target image for the target detection region;

[0010] The input module is configured to input the target image into a target image processing model to obtain the target detection result for the target detection region output by the target image processing model;

[0011] Wherein, the target detection result is obtained by the target image processing model through feature enhancement processing of fused image features and initial detection results; the fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features; the target region features are obtained by the target image processing model through feature extraction of the target region mask image; the abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image; and the target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the target image.

[0012] According to a third aspect of the embodiments of this specification, an image processing model training method is provided, comprising:

[0013] Determine the sample image and label detection results for the target detection region;

[0014] The sample image is input into an initial image processing model to obtain the predicted detection result for the sample image output by the initial image processing model. The predicted detection result is obtained by the initial image processing model performing feature enhancement processing on the fused image features and the initial detection result. The fused image features are obtained by the initial image processing model fusing target region features and abnormal region features. The target region features are obtained by the initial image processing model extracting features from the target region mask image. The abnormal region features are obtained by the initial image processing model extracting features from the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the initial image processing model performing segmentation processing on the sample image.

[0015] Based on the predicted detection results and the label detection results, the initial image processing model is trained until the training stops, thereby obtaining the target image processing model.

[0016] According to a fourth aspect of the embodiments of this specification, an image processing model training apparatus is provided, comprising:

[0017] The determination module is configured to determine the sample image and label detection results for the target detection region;

[0018] An input module is configured to input the sample image into an initial image processing model to obtain a prediction detection result for the sample image output by the initial image processing model. The prediction detection result is obtained by the initial image processing model performing feature enhancement processing on fused image features and the initial detection result. The fused image features are obtained by the initial image processing model fusing target region features and abnormal region features. The target region features are obtained by the initial image processing model extracting features from a target region mask image. The abnormal region features are obtained by the initial image processing model extracting features from an abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the initial image processing model performing segmentation processing on the sample image.

[0019] The training module is configured to train the initial image processing model based on the predicted detection results and the label detection results until the training stops, thereby obtaining the target image processing model.

[0020] According to a fifth aspect of the embodiments of this specification, a CT image processing method is provided, comprising:

[0021] Receive a CT image processing task, wherein the CT image processing task carries a plain CT image corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area;

[0022] The plain CT image is input into the target image processing model to obtain the target detection result for the target detection area output by the target image processing model;

[0023] The target detection result is obtained by the target image processing model through feature enhancement processing of the fused image features and the initial detection result. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the plain CT image.

[0024] According to a sixth aspect of the embodiments of this specification, a CT image processing apparatus is provided, comprising:

[0025] The receiving module is configured to receive a CT image processing task, wherein the CT image processing task carries a plain CT image corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area;

[0026] The input module is configured to input the plain CT image into the target image processing model to obtain the target detection result for the target detection area output by the target image processing model;

[0027] The target detection result is obtained by the target image processing model through feature enhancement processing of the fused image features and the initial detection result. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the plain CT image.

[0028] According to a seventh aspect of the embodiments of this specification, another image processing method is provided, applied to a cloud-side device, comprising:

[0029] The receiving end device sends an image processing task, wherein the image processing task carries a target image corresponding to the target detection area, and the image processing task is used to detect whether there is an anomaly in the target detection area;

[0030] The target image is input into the target image processing model to obtain the target detection result for the target detection region output by the target image processing model, and the target detection result is sent by the end device.

[0031] Wherein, the target detection result is obtained by the target image processing model through feature enhancement processing of fused image features and initial detection results; the fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features; the target region features are obtained by the target image processing model through feature extraction of the target region mask image; the abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image; and the target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the target image.

[0032] According to an eighth aspect of the embodiments of this specification, another image processing apparatus is provided, applied to a cloud-side device, comprising:

[0033] The receiving module is configured to receive an image processing task sent by the end-side device, wherein the image processing task carries a target image corresponding to the target detection area, and the image processing task is used to detect whether there is an anomaly in the target detection area;

[0034] The input module is configured to input the target image into a target image processing model, obtain the target detection result for the target detection region output by the target image processing model, and send the target detection result to the end device;

[0035] Wherein, the target detection result is obtained by the target image processing model through feature enhancement processing of fused image features and initial detection results; the fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features; the target region features are obtained by the target image processing model through feature extraction of the target region mask image; the abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image; and the target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the target image.

[0036] According to a ninth aspect of the embodiments of this specification, a computer-aided diagnosis method for tumors is provided, comprising:

[0037] Receive a tumor screening task, wherein the tumor screening task carries a plain CT image corresponding to a target detection area, and the tumor screening task is used to detect whether there is a tumor in the target detection area;

[0038] The plain CT image is input into the target image processing model to obtain the target detection result output by the target image processing model regarding whether a tumor exists in the target detection area;

[0039] The target detection result is obtained by the target image processing model through feature enhancement processing of the fused image features and the initial detection result. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the plain CT image.

[0040] According to a tenth aspect of the embodiments of this specification, a computer-aided diagnostic system for tumors is provided, comprising a client and a server, wherein,

[0041] The client is used to send a CT image processing task to the server, wherein the CT image processing task carries a plain CT image corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area;

[0042] The server is used to input the plain CT image into the target image processing model, obtain the target detection result output by the target image processing model regarding whether there is an abnormality in the target detection area, and send the target detection result to the client.

[0043] The target detection result is obtained by the target image processing model through feature enhancement processing of the fused image features and the initial detection result. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the plain CT image.

[0044] According to an eleventh aspect of the embodiments of this specification, a computing device is provided, comprising:

[0045] Memory and processor;

[0046] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above method.

[0047] According to a twelfth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0048] According to a thirteenth aspect of an embodiment of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0049] This specification provides an image processing method according to one embodiment, comprising: determining a target image for a target detection region; inputting the target image into a target image processing model to obtain a target detection result for the target detection region output by the target image processing model; wherein the target detection result is obtained by the target image processing model performing feature enhancement processing on fused image features and initial detection results, the fused image features are obtained by the target image processing model performing fusion processing on target region features and abnormal region features, the target region features are obtained by the target image processing model performing feature extraction on a target region mask image, the abnormal region features are obtained by the target image processing model performing feature extraction on an abnormal region mask image, and the target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model performing segmentation processing on the target image.

[0050] In the above method, after determining the target image for the target detection region, the target image can be input into the target image processing model. In this model, the target image can be segmented to obtain a target region mask image, an anomaly region mask image, and an initial detection result for the target detection region. Feature extraction is performed on the target region mask image to obtain target region features, and feature extraction is performed on the anomaly region mask image to obtain anomaly region features. By fusing the target region features and the anomaly region features, a fused image feature is obtained, realizing the interaction between the global features (i.e., target region features) and local features (i.e., anomaly region features) of the target image. Furthermore, using the initial detection result for the target detection region as prior knowledge, feature enhancement processing is performed on the fused image feature and the initial detection result to obtain the target detection result for the target detection region, thus realizing anomaly detection in the target detection region. Through the interaction between global features and local features, and by combining the prediction of prior knowledge, the accuracy of feature representation is improved, further enhancing the accuracy of the target detection result. Attached Figure Description

[0051] Figure 1 This is a schematic diagram illustrating an application scenario of an image processing method provided in one embodiment of this specification;

[0052] Figure 2 This is a flowchart illustrating an image processing method provided in one embodiment of this specification;

[0053] Figure 3 This is a flowchart illustrating an image processing model training method provided in one embodiment of this specification;

[0054] Figure 4 This is a flowchart illustrating the processing procedure of an image processing model training method provided in one embodiment of this specification.

[0055] Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in one embodiment of this specification;

[0056] Figure 6 This is a schematic diagram of the structure of an image processing model training device provided in one embodiment of this specification;

[0057] Figure 7 This is a flowchart of a CT image processing method provided according to an embodiment of this specification;

[0058] Figure 8 This is a schematic diagram of the structure of a CT image processing device provided in one embodiment of this specification;

[0059] Figure 9This is a flowchart of another image processing method provided according to an embodiment of this specification;

[0060] Figure 10 This is a schematic diagram of the structure of another image processing apparatus provided in one embodiment of this specification;

[0061] Figure 11 This is a flowchart illustrating a computer-aided diagnostic method for tumors provided in one embodiment of this specification;

[0062] Figure 12 This is a schematic diagram of the structure of a computer-aided diagnosis system for tumors provided in one embodiment of this specification;

[0063] Figure 13 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0064] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0065] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0066] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0067] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0068] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0069] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0070] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0071] CT (Computed Tomography): Computed tomography is a scanning technology that uses a precisely collimated X-ray beam and a highly sensitive detector to scan a specific part of the body one section after another. It features fast scanning time and clear images and can be used to examine a variety of diseases.

[0072] Plain CT scan: Also known as conventional CT or non-contrast CT, this refers to a scan performed intravenously without the administration of an iodine-containing contrast agent. It has advantages such as being fast, non-invasive, and low-cost, but its ability to detect certain lesions is relatively poor.

[0073] Enhanced CT: This refers to a scanning method that involves injecting a contrast agent into a blood vessel before scanning. The purpose is to increase the density difference between diseased and normal tissue, thereby revealing lesions that are not visible or poorly visualized on a plain CT scan. The presence or absence of enhancement and the type of enhancement help in the characterization of the lesion. It can more clearly show the blood supply to liver lesions, helping to differentiate between benign and malignant lesions in the liver region.

[0074] The softmax function is a commonly used function in machine learning and statistics. It is primarily used to transform a K-dimensional vector containing arbitrary real numbers into another K-dimensional vector, where each element's value is in the range (0,1), and the sum of all elements equals 1. Therefore, the softmax function is often used as the output layer activation function in multi-class classification problems, allowing the output to be interpreted as a probability distribution belonging to each class.

[0075] GNN stands for Graph Neural Networks. A graph neural network is a deep learning model specifically designed to process graph-structured data. Graph-structured data consists of nodes (also called vertices) and edges; nodes represent entities, and edges represent relationships between these entities. GNNs enable machine learning models to operate directly on this non-Euclidean space data structure, thereby effectively capturing patterns and dependencies in the graph.

[0076] HDA (Hierarchical Dual Attention) is a technique for feature extraction and fusion. It's a mechanism used in deep learning models to enhance feature extraction and representation capabilities. It's particularly suitable for handling complex data structures. HDA captures rich information within the data by introducing two different levels of attention mechanisms. In the field of assistive medical care, this mechanism can be used to combine global liver features and local lesion features at different scales to enhance the model's ability to identify lesions.

[0077] GPR: Graph-based prior reasoning, is a technique that uses graph structures for reasoning. Graph-based prior reasoning utilizes graph neural networks to reason about and optimize the prior knowledge of a pre-trained 3D segmentation model, thereby improving the accuracy of lesion classification.

[0078] DCA: Dual cross-attention. Also known as a two-way cross-attention mechanism, it's a technique in deep learning used to enhance model processing capabilities. It captures the relationships between different inputs by introducing cross-attention in two directions, thereby improving model performance. In the field of assistive medical care, it can be used to establish bidirectional information flow between global liver features and local lesion features, enhancing feature interaction and fusion.

[0079] This specification provides two image processing methods, and also relates to two image processing devices, an image processing model training method, an image processing model training device, a CT image processing method, a CT image processing device, a computer-aided diagnosis method for tumors, a computer-aided diagnosis system for tumors, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0080] See Figure 1 , Figure 1 The illustration shows an application scenario of an image processing method according to an embodiment of this specification, which includes the following steps.

[0081] Identify the target image for the target detection region.

[0082] The target image is input into the target image processing model to obtain the target detection result for the target detection region output by the target image processing model.

[0083] Wherein, the target detection result is obtained by the target image processing model through feature enhancement processing of fused image features and initial detection results; the fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features; the target region features are obtained by the target image processing model through feature extraction of the target region mask image; the abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image; and the target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the target image.

[0084] Figure 1 It includes end-side device 102 and cloud-side device 104.

[0085] In specific implementation, the user can send the target image of the target detection area to the cloud-side device 104 through the edge device 102. After receiving the target image, the cloud-side device 104 can input the target image into the target image processing model. In the target image processing model, the target image is segmented to obtain the target area mask image, the abnormal area mask image and the initial detection result. The abnormal area mask image is used to extract features to obtain abnormal area features, the target area mask image is used to extract features to obtain target area features, the target area features and the abnormal area features are fused to obtain fused image features, the fused image features and the initial detection result are used to perform feature enhancement processing to obtain the target detection result of the target detection area, and the target detection result is sent to the edge device 102.

[0086] In the field of assistive medical care, doctors can send CT images of the target detection area to cloud-based devices 104 via edge device 102. After receiving the CT image, cloud device 104 inputs the CT image into a target image processing model to obtain the target detection result for the target detection area. The target detection result can be used to indicate whether there is a lesion in the target detection area and the classification of the lesion. Cloud device 104 sends the target detection result to edge device 102, and edge device 102 can display the target detection result to the doctor through a display interface to achieve diagnosis of the target detection area.

[0087] The edge device 102 may include a browser, an app (application), or a web application such as an H5 (Hypertext Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. The edge device can be developed based on a software development kit (SDK) provided by the server, such as a real-time communication (RTC) SDK. The edge device can be deployed in an electronic device and depends on the device's operation or certain apps within the device to run. The electronic device may have a display screen and support information browsing, such as a personal mobile terminal like a mobile phone, tablet, or personal computer. Various other types of applications can also be configured in the electronic device, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.

[0088] Cloud-side device 104 can be understood as a server providing various services, including physical servers and cloud servers. Examples include servers providing communication services to multiple clients, servers supporting backend training of models used on clients, and servers processing data sent by clients. It should be noted that cloud-side device 104 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. Cloud-side device 104 can also be a server for a distributed system, or a server integrated with blockchain. Cloud-side device 104 can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0089] It is worth noting that the image processing method provided in the embodiments of this specification can be executed by the cloud-side device 104. In other embodiments of this specification, the target image processing model can be deployed in the edge device 102, so that the edge device 102 can also have similar functions to the cloud-side device 104, thereby executing the image processing method provided in the embodiments of this specification. In other embodiments, the image processing method provided in the embodiments of this specification can also be jointly executed by the edge device 102 and the cloud-side device 104.

[0090] See Figure 2 , Figure 2 A flowchart of an image processing method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0091] Step 202: Determine the target image for the target detection region.

[0092] Specifically, the image processing method provided in the embodiments of this specification can be applied to the field of auxiliary medical care, and can be used to determine whether there are abnormalities in the target detection area.

[0093] In the field of assistive medical care, the target detection area can be understood as the organ or part to be detected, such as the liver, lungs, or stomach. This image processing method can then be used to determine whether a lesion exists in the target detection area, for example, whether a lesion exists in the liver. The target image can be understood as a CT image obtained by medical scanning the target detection area. Optionally, the target image can be a plain CT image obtained without injecting contrast agent into the target detection area (i.e., a non-enhanced CT image), or an enhanced CT image obtained after injecting contrast agent into the target detection area, or other types of CT images obtained by medical scanning the target detection area using other medical scanning methods. This specification does not limit the specific types of CT images described in the embodiments.

[0094] In practical applications, the image processing method provided in the embodiments of this specification can be applied to the server. The server can then receive target images of the target detection region sent by the client. Furthermore, the server can receive multiple target images of the target detection region sent by the client. Subsequently, similar processing can be performed on each of the multiple target images to further achieve anomaly detection in the target detection region.

[0095] Step 204: Input the target image into the target image processing model to obtain the target detection result for the target detection region output by the target image processing model.

[0096] Wherein, the target detection result is obtained by the target image processing model through feature enhancement processing of fused image features and initial detection results; the fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features; the target region features are obtained by the target image processing model through feature extraction of the target region mask image; the abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image; and the target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the target image.

[0097] The target detection result can be understood as the detection result of whether there are abnormalities in the target detection area and the classification of the existing abnormalities. The target detection result can include information on the location of abnormalities in the target detection area and / or information on whether there are abnormalities in the target detection area and the classification of abnormalities (such as the number of abnormality classifications). For example, when the target detection area is the liver area, the target detection result can be used to determine whether there are lesions in the liver area and the classification of the lesions. The lesion classification can include benign lesions and malignant lesions.

[0098] Specifically, after receiving the target image for the target detection region, the target image can be input into the target image processing model to obtain the target detection result output by the target image processing model, which determines whether there are anomalies in the target detection region and classifies the anomalies.

[0099] In practical applications, the target image processing model includes an image segmentation unit, a feature extraction unit, a feature fusion unit, and a feature enhancement unit;

[0100] The step of inputting the target image into the target image processing model and obtaining the target detection result for the target detection region output by the target image processing model includes:

[0101] The target image is input into the image segmentation unit to obtain the target region mask image, the abnormal region mask image, and the initial detection result corresponding to the target image;

[0102] The target region mask image and the abnormal region mask image are input into the feature extraction unit to obtain target region features and abnormal region features;

[0103] The target region features and the abnormal region features are input into the feature fusion unit to obtain fused image features;

[0104] The fused image features and the initial detection results are input into the feature enhancement unit to obtain the target detection result for the target detection region.

[0105] The target region mask image can be understood as a mask image obtained by identifying and segmenting the target detection region in the target image. The target region feature can be understood as the regional features of the target detection region. The anomaly region mask image can be understood as a mask image obtained by identifying and segmenting the anomaly region in the target image. The anomaly region feature can be understood as the regional features of the anomaly region. The initial detection result can be understood as the detection result regarding whether anomalies exist in the target detection region and the classification of any existing anomalies. The feature extraction unit can be a feature extractor, which may include a global feature extractor and a local feature extractor. The global feature extractor can be used to extract target region features, and the local feature extractor can be used to extract anomaly region features.

[0106] In the field of assisted medicine, when the target detection area is the liver region, the target area mask image can be understood as the liver region mask image, and the target area features can be understood as liver features. The abnormal area mask image can be understood as the lesion area mask image, and the abnormal area features can be understood as lesion features. Therefore, it can be understood that since the target area features are the overall features of the liver region, the target area features can be global features, and the abnormal area features are the lesion features of the lesion area in the liver region, so the abnormal area features can be local features.

[0107] Specifically, after the target image is input into the target image processing model, the target image can be input into the image segmentation unit. The image segmentation unit performs segmentation mask processing on the target image to obtain the target region mask image, the abnormal region mask image, and the initial detection result. The target region mask image and the abnormal region mask image are input into the feature extraction unit. The feature extraction unit performs feature extraction on the target region mask image to obtain target region features and on the abnormal region mask image to obtain abnormal region features. The target region features and the abnormal region features are input into the feature fusion unit. The feature fusion unit performs feature fusion processing on the target region features and the abnormal region features to obtain fused image features. The fused image features and the initial detection result are input into the feature enhancement unit. The feature enhancement unit performs feature enhancement processing on the fused image features and the initial detection result to obtain the target detection result for the target detection region.

[0108] In practical applications, the image segmentation unit can be a pre-trained 3D segmentation model. The target image processing model is based on this pre-trained 3D segmentation model, and extends it with feature extraction, feature fusion, and feature enhancement units to form the target image processing model. It can be seamlessly integrated with any pre-trained 3D segmentation model. This plug-and-play design eliminates the need for retraining or extensive adjustments to the base model, improving its flexibility and scalability. It allows users to select a suitable 3D segmentation model based on their actual needs, further enhancing the performance of lesion analysis in the target detection region. In other words, the image processing framework (plus) provided in this specification, comprising a feature extraction unit, a feature fusion unit, and a feature enhancement unit, can integrate any pre-trained segmentation model to form the target image processing model.

[0109] Further, the step of inputting the target region features and the abnormal region features into the feature fusion unit to obtain fused image features includes:

[0110] The target region features and the abnormal region features are input into the feature fusion unit, and the target region features and the abnormal region features are processed by an attention mechanism to obtain the fused image features.

[0111] The feature fusion unit can employ hierarchical dual attention mechanism (HDA) and dual cross attention mechanism (DCA) to achieve effective information interaction between target region features and abnormal region features. In the field of assisted medical care, it can simulate the process of doctors making comprehensive judgments on target images.

[0112] Specifically, after inputting the target region features and abnormal region features into the feature fusion unit, the feature fusion unit can be used to process the target region features and abnormal region features using an attention mechanism, thereby achieving the fusion of the target region features and abnormal region features and obtaining fused image features.

[0113] In summary, by applying attention mechanisms to both target region features and abnormal region features, effective information interaction is achieved between the abnormal region features (local lesion features) and the target and region features (global features). This further mimics the process of a doctor making a comprehensive judgment on the target image of the target detection area, thereby improving the accuracy of the target detection results.

[0114] Further, the step of performing attention mechanism processing on the target region features and the abnormal region features to obtain the fused image features includes:

[0115] The target region features are subjected to multi-scale pooling to obtain multiple sub-target region features;

[0116] Attention mechanisms are applied to the features of each sub-target region and the features of the abnormal regions to obtain the fused image features.

[0117] Specifically, in the feature fusion unit, since the target region features are the overall features of the target detection region, multi-scale pooling can be performed on the target region features to obtain multiple sub-target region features. Then, a dual cross-attention mechanism is applied to each sub-target region feature and the abnormal region feature to obtain fused image features.

[0118] In specific implementation, taking the dual cross-attention mechanism processing of one sub-target region feature and one abnormal region feature as an example, the sub-target region feature can be linearly transformed through a fully connected layer to obtain the key (k) and value (v). The abnormal region feature can be linearly transformed through a fully connected layer to obtain the query (Q). The key and value provided by the sub-target region feature and the query provided by the abnormal region feature are processed by a cross-attention mechanism to obtain the first feature. Specifically, when processing the key, value and query, the query and key can be used to calculate the attention score, and the attention score can be calculated using the softmax function to obtain the attention weight. The attention weight and value are multiplied and summed to obtain the first feature after cross-attention mechanism processing. This first feature is generated based on the attention allocation of the sub-target region feature and the abnormal region feature, and can capture the important interaction information between the target region feature and the abnormal region feature. Furthermore, a fully connected layer can be used to perform a linear transformation on the sub-target region features to obtain a query (Q). Linear transformations can then be performed on the first feature and the abnormal region features using both linear and fully connected layers to obtain a key (k) and a value (v). A cross-attention mechanism is then applied to the query provided by the sub-target region features and the key and value provided by the abnormal region features to obtain a second feature. The first linear feature obtained by inputting the first feature and the abnormal region features into the linear layer, and the second linear feature obtained by inputting the second feature and the sub-target region features into the linear layer, are then fused to obtain a fused image feature. Through these two cross-attention mechanisms, a dual cross-attention mechanism is achieved between the sub-target region features and the abnormal region features. It is understandable that similar processing is performed on each sub-target region feature and the abnormal region feature to obtain the fused image feature.

[0119] Furthermore, before applying the attention mechanism to the features of each sub-target region and the features of the anomalous region, a learnable linear transformation matrix (i.e., a fully connected layer) can be used to map the anomalous region features to a new feature space, obtaining the mapped anomalous region features. A dual cross-attention mechanism is then applied to the features of each sub-target region and the mapped anomalous region features to obtain fused image features. By mapping the anomalous region features to a new feature space, the representational power of the anomalous region features as original features is enhanced, providing a data foundation for subsequent multi-scale feature interaction and fusion.

[0120] Understandably, the process of hierarchical dual attention mechanism processing can include the above-mentioned process of multi-scale pooling of target region features, feature mapping of abnormal region features, and dual cross-attention mechanism processing of features of each sub-target region and abnormal region features.

[0121] In summary, a hierarchical dual attention mechanism promotes bidirectional interaction between global target region features and local abnormal region features. This mechanism, through multi-scale feature interaction and dual differential attention, effectively captures the features of abnormal lesions at different scales and combines them with the overall morphological information of the target detection region, thereby improving the accuracy of lesion detection and classification. This mechanism can simulate the comprehensive judgment of global and local information made by doctors during the treatment process.

[0122] Furthermore, the multi-scale pooling processing of the target region features to obtain multiple sub-target region features includes:

[0123] According to the spatial resolution scale, the target region features are subjected to multi-scale pooling processing to obtain multiple sub-target region features.

[0124] Among them, spatial resolution scale can be used to describe the multi-level representation of the target detection region in spatial dimensions.

[0125] Specifically, the target region features can be sampled at multiple resolution scales according to the spatial resolution scale to obtain multiple sub-target region features. For example, when the target detection region is the liver region and the target region features are liver features, the liver features can be sampled at multiple resolution scales according to the spatial resolution scale to obtain liver features at the original resolution, liver features downsampled by 2 times, and liver features downsampled by 4 times.

[0126] In summary, by sampling the features of the target region at multiple resolution scales based on spatial resolution, it is possible to capture the features of a lesion at different scales, thereby further improving the accuracy of target detection results.

[0127] In specific implementation, the step of inputting the target image into the image segmentation unit to obtain the target region mask image, the abnormal region mask image, and the initial detection result corresponding to the target image includes:

[0128] The target image is input into the image segmentation unit, and the target image is subjected to segmentation mask processing to obtain a segmentation mask image;

[0129] The segmentation mask image is cropped to obtain the target region mask image, the abnormal region mask image, and the initial detection result for the target detection region.

[0130] Specifically, after the target image is input into the image segmentation unit, segmentation mask processing can be performed on the target image to obtain a segmentation mask image. This segmentation mask image can include the identified target region and abnormal region, such as the identified liver region and lesion region. The segmentation mask image can be cropped to obtain the target region mask image, the abnormal region mask image, and the initial detection results for the target detection region, thereby realizing the extraction of detailed abnormal information and target region information.

[0131] In practical applications, in the field of assisted medicine, pre-trained 3D segmentation models can be used to identify and segment target regions (i.e., liver regions) and abnormal regions (lesion regions) in target images (i.e., CT images). Then, detailed lesion information (i.e., abnormal region mask images) and liver information (i.e., target region mask images) can be extracted through regions of interest, providing a data foundation for subsequent information interaction between features and prior prediction.

[0132] In specific implementation, the step of inputting the fused image features and the initial detection result into the feature enhancement unit to obtain the target detection result for the target detection region includes:

[0133] The fused image features and the initial detection result are input into the feature enhancement unit, and the initial detection result is enhanced by category prior using the fused image features to obtain the target detection result for the target detection region.

[0134] Among them, the feature enhancement unit can be a graph-based reasoning unit. The feature enhancement unit can use graph neural networks to optimize prior predictions. These prior predictions can be understood as prior knowledge, i.e., the initial detection results mentioned above.

[0135] Specifically, after inputting the fused image features and the initial detection results into the feature enhancement unit, the feature enhancement unit can use the fused image features to perform category prior enhancement processing on the initial detection results used as prior predictions, thereby optimizing the initial detection results and obtaining target detection results for the target detection region.

[0136] In summary, by utilizing fused image features to optimize the initial detection results, we can optimize the prior knowledge of lesion classification and further improve the accuracy of target detection results.

[0137] Further, the step of using the fused image features to perform category prior enhancement processing on the initial detection result to obtain the target detection result for the target detection region includes:

[0138] The abnormal features contained in the fused image features are used as nodes, the reference category features corresponding to the initial detection results are used as nodes, and the connection edges between nodes are determined according to the similarity between features to construct a feature relationship graph.

[0139] An attention mechanism is applied to the abnormal feature nodes and reference category feature nodes in the feature relationship graph to obtain the target detection result for the target detection region.

[0140] Among them, the reference category feature can be understood as the category prototype of the abnormal lesion, the abnormal feature can be understood as the lesion feature, and the similarity between features can include the feature similarity between lesion features, the feature similarity between lesion features and reference category features, and the feature similarity between reference category features.

[0141] Specifically, lesion features included in the fused image features can be used as nodes, and reference category features corresponding to the initial detection results can be used as nodes. Based on the feature similarity between lesion features, between lesion features and reference category features, and between reference category features, connection edges are constructed between nodes corresponding to two features whose feature similarity reaches a preset similarity threshold. A feature relationship graph is constructed, and attention mechanism processing is applied to abnormal feature nodes and reference category feature nodes in the feature relationship graph to obtain target detection results for the target detection region.

[0142] In practical applications, the feature enhancement unit can construct a feature relationship graph based on a graph neural network (GNN). This feature relationship graph includes lesion feature nodes (i.e., abnormal feature nodes) and category prototype nodes (i.e., reference category feature nodes). The information exchange between lesion feature nodes and category prototype nodes is realized through attention mechanism, and finally the target detection result after category prior enhancement is output.

[0143] In summary, by using graph-based prior reasoning units to optimize the prior knowledge of lesion classification and by constructing a feature relationship graph between lesion features and category prototypes through graph neural networks, relevant prior knowledge can be automatically enhanced and unreliable predictions can be suppressed. This not only improves the robustness of lesion classification but also overcomes the limitations in processing prior knowledge.

[0144] In practical applications, the training process of the target image processing model includes:

[0145] Determine the sample image and label detection results for the target detection region;

[0146] The sample image is input into an initial image processing model to obtain the predicted detection result for the sample image output by the initial image processing model. The predicted detection result is obtained by the initial image processing model performing feature enhancement processing on the fused image features and the initial detection result. The fused image features are obtained by the initial image processing model fusing target region features and abnormal region features. The target region features are obtained by the initial image processing model extracting features from the target region mask image. The abnormal region features are obtained by the initial image processing model extracting features from the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the initial image processing model performing segmentation processing on the sample image.

[0147] Based on the predicted detection results and the label detection results, the initial image processing model is trained until the training stops, thereby obtaining the target image processing model.

[0148] The training method for the target image processing model can be supervised training. The sample images and label detection results can be training sample pairs from the training data during supervised training. Understandably, in supervised training, the model can be trained using multiple training sample pairs. The label detection results can be accurate detection results for the target detection region. The training stopping condition can be understood as the model loss value reaching a preset loss threshold and / or the model training iterations reaching a preset iteration threshold. The initial image processing model can be understood as an untrained image processing model; therefore, the target image processing model can be understood as a trained image processing model.

[0149] Specifically, sample images and label detection results for the target detection region can be obtained. The sample images are input into the initial image processing model to obtain the predicted detection results for the sample images output by the initial image processing model. The model loss value is calculated based on the predicted detection results and label detection results. The initial image processing model is trained based on the model loss value. When the training stops, the target image processing model is obtained.

[0150] Furthermore, during the training of the initial image processing model, the parameters of the feature extraction unit, feature fusion unit, and feature enhancement unit in the initial image processing model can be adjusted.

[0151] In summary, by training the initial image processing model as described above, the trained target image processing model is able to accurately detect the target detection region, thereby improving the inference performance of the target image processing model.

[0152] Further, the step of training the initial image processing model based on the predicted detection results and the label detection results until the training stopping condition is met to obtain the target image processing model includes:

[0153] Based on the preset performance index, determine the predicted performance detection result and the label performance detection result corresponding to the preset performance index from the predicted detection result and the label detection result;

[0154] Based on the prediction performance test results and the label performance test results, calculate the model loss value corresponding to the preset performance index;

[0155] The initial image processing model is trained based on the model loss value until the training stops, thereby obtaining the target image processing model.

[0156] Among them, the preset performance index can be understood as the performance index that the target image processing model is expected to meet.

[0157] Specifically, based on the preset performance index, the prediction performance detection result corresponding to the prediction performance index can be determined from the prediction detection result, and the label performance detection result corresponding to the preset performance index can be determined from the label detection result. Based on the prediction performance detection result and the label performance detection result, the model loss value corresponding to the preset performance index can be calculated, and the initial image processing model can be trained based on the model loss value until the training stopping condition is reached to obtain the target image processing model.

[0158] Furthermore, the preset performance indicators include anomaly level indicators, target object indicators, and screening level indicators;

[0159] The step of training the initial image processing model based on the predicted detection results and the label detection results until the training stopping condition is met to obtain the target image processing model includes:

[0160] Based on the predicted detection results and the label detection results, calculate the first model loss value corresponding to the anomaly level index, the second model loss value corresponding to the target object index, and the third model loss value corresponding to the screening level index;

[0161] The initial image processing model is trained based on the first model loss value, the second model loss value, and the third model loss value until the training stops, thereby obtaining the target image processing model.

[0162] The anomaly level index, which can be understood as a lesion level index, focuses on the classification accuracy of each individual liver lesion in the medical field. It quantifies the model's ability to distinguish different types of liver lesions through precision, recall, and F1 score. This index reflects the model's ability to capture subtle pathological features, which is crucial for clinicians to develop precise treatment plans. For example, the ability to accurately distinguish between hepatocellular carcinoma and cholangiocarcinoma directly affects subsequent treatment pathways. This index particularly focuses on the model's ability to handle complex situations, such as a patient potentially having multiple types of lesions simultaneously.

[0163] The target indicator can be understood as a patient indicator, which is used to evaluate the model's performance in overall patient diagnosis, particularly its ability to differentiate between malignant and benign cases. This indicator employs a special aggregation function to integrate the predictions of individual lesions into a patient-level diagnostic conclusion, prioritizing the detection of malignancy. In clinical practice, the assessment of patient indicators is more important than the assessment of individual lesions because it directly determines whether further intervention is needed. This patient indicator can include indicators for identifying malignant and benign patients.

[0164] Screening level metrics can be used to evaluate the effectiveness of a model as a primary screening tool, focusing on the model's ability to distinguish between patients with and without lesions. This metric is particularly applicable to large-scale population screening scenarios, such as opportunistic screening in routine physical examinations. Through F1 scores and accuracy, the model's ability to reliably identify patients requiring further investigation is assessed, thereby improving the efficiency of medical resource utilization. Given the widespread use of non-contrast CT in physical examinations, the practical significance of this metric is especially prominent.

[0165] The target image processing model provided in this solution integrates these three levels of metrics into its multi-metric training paradigm, balancing the optimization objectives at different levels through a comprehensive loss function. This design simulates a real clinical workflow, from detecting abnormalities and determining the patient's condition to accurately diagnosing the lesion type. Experimental results show that this multi-level evaluation method enables the target image processing model to achieve nearly 90% accuracy in identifying malignant lesions at the patient level while maintaining good lesion recognition capabilities, truly reflecting the harmonious unity between technical indicators and clinical needs.

[0166] Specifically, based on the prediction detection results and label detection results, the first model loss value corresponding to the anomaly level index, the second model loss value corresponding to the target object index, and the third model loss value corresponding to the screening level index can be calculated. The first model loss value, the second model loss value, and the third model loss value are weighted and summed to obtain the target model loss value. Based on the target model loss value, the initial image processing model is trained until the training stopping condition is reached to obtain the target image processing model.

[0167] In practical applications, a comprehensive optimization strategy can be used to combine the first loss function corresponding to the anomaly level index, the second loss function corresponding to the target object index, and the third loss function corresponding to the screening level index. Based on the prediction detection results and the label detection results, the first model loss value corresponding to the anomaly level index, the second model loss value corresponding to the target object index, and the third model loss value corresponding to the screening level index can be calculated respectively. The initial image processing model can be trained based on the first model loss value, the second model loss value, and the third model loss value, thereby improving the diagnostic performance of the target image processing model.

[0168] The loss value of the first model is shown in the following formula.

[0169]

[0170] Wherein, the first model loss value is the loss value corresponding to the lesion level index (i.e., lesion level loss), N can be the total number of lesion samples in the training batch, i can represent the index of the i-th lesion sample, C can be the total number of lesion categories (such as benign, malignant, or more subdivided categories), c can represent the index of the c-th category, p i,c y can represent the probability that the model predicts the i-th lesion sample belongs to category c. i,c It can be a true label, with a value of 1 when the i-th lesion sample belongs to category c, and 0 otherwise, z c It can be a class balancing factor, used to address the problem of imbalanced sample sizes across different classes in a dataset. γ can be a modulation factor, used to reduce attention to easily classified samples and enhance learning of difficult-to-classify samples.

[0171] The loss value for the second model is shown in the following formula.

[0172]

[0173] Wherein, the second model loss value can be the loss value corresponding to the patient-level index (i.e., patient-level loss), M represents the total number of patients in the training batch, j represents the index of the j-th patient, and S j Let represent the set of all lesions belonging to patient j. Y can represent the probability that the model predicts the i-th lesion in patient j to be malignant. j This is the true label for patient j, with a value of 1 if the patient has any malignant lesions and a value of 0 if all lesions are benign. It is the maximum probability predicted as malignant among all lesions of patient j.

[0174] The loss value for the third model is shown in the following formula.

[0175]

[0176] The third model loss value is the loss value corresponding to the screening level index (i.e., screening level loss), which is a binary cross-entropy loss used to distinguish between patients with and without lesions. M represents the total number of patients in the training batch, j represents the index of the j-th patient, and q... j y represents the probability that the model predicts patient j has any lesion (whether benign or malignant). j This is the true label of patient j, which is 1 when the patient has any lesions and 0 when there are no lesions.

[0177] The target model loss value is shown in the following formula.

[0178]

[0179] Here, α is the weighting coefficient for the lesion-level loss, controlling its importance in the total loss; β is the weighting coefficient for the patient-level loss, also controlling its importance in the total loss; and (1-α-β) is the weighting coefficient for the screening-level loss, ensuring that the sum of the weights of the three losses is 1. This weighted combination allows the model to balance its focus on different levels of diagnostic tasks during training.

[0180] These four loss functions together constitute the multi-index training paradigm of the image processing framework of this scheme. By adjusting the weight parameters α and β, the attention allocation of the model at different levels of tasks can be adjusted according to clinical needs, thereby achieving the best balance between accurate classification at the lesion level, accurate diagnosis at the patient level, and efficient identification at the screening level.

[0181] In practical applications, the performance of the target image processing model provided in the embodiments of this specification and other diagnostic models in the field of auxiliary medical care is compared on the test set based on the above-mentioned abnormality level indicators, target object indicators, and screening level indicators. Specifically, the performance of other diagnostic models M1, other diagnostic models combined with the image processing framework M1+plus provided in this solution, other diagnostic models M2, other diagnostic models combined with the image processing framework M2+plus provided in this solution, other diagnostic models M3 that ablate the image processing framework provided in this solution, and the target image processing model MO provided in this solution are compared under multiple indicators. According to the indicator data provided in the table below, the target image processing model provided in the embodiments of this specification has good diagnostic performance under the above-mentioned abnormality level indicators, target object indicators, and screening level indicators. It has high efficiency and accuracy, and can achieve the technical effects of improving diagnostic efficiency, reducing misdiagnosis rate, and supporting large-scale screening.

[0182] As summarized in the table below, the target image processing model provided by this scheme achieves best performance on most metrics, with F1 scores of 65.11%, 90.12%, and 73.10% for lesion detection, assessment of malignant patients, and assessment of benign patients, respectively. The target image processing model of this scheme shows a significant improvement in F1 performance score of 5.66% for lesions, 6.26% for malignant patients, and 4.03% for benign patients.

[0183]

[0184] In summary, the above method, after determining the target image for the target detection region, can input the target image into a target image processing model. This model can segment the target image to obtain a target region mask image, an anomaly region mask image, and initial detection results for the target detection region. Feature extraction is then performed on the target region mask image to obtain target region features, and on the anomaly region mask image to obtain anomaly region features. By fusing the target region features and the anomaly region features, a fused image feature is obtained, achieving interaction between global features (i.e., target region features) and local features (i.e., anomaly region features) of the target image. Furthermore, using the initial detection results for the target detection region as prior knowledge, feature enhancement processing is performed on the fused image feature and the initial detection results to obtain the target detection result for the target detection region, achieving anomaly detection in the target detection region. Through the interaction between global and local features, and by combining prior knowledge prediction, the accuracy of feature representation is improved, further enhancing the accuracy of the target detection result.

[0185] Corresponding to the above method embodiments, this specification also provides an image processing model training method, see [link to documentation]. Figure 3 , Figure 3 A flowchart of an image processing model training method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0186] Step 302: Determine the sample image and label detection results for the target detection area.

[0187] Step 304: Input the sample image into the initial image processing model to obtain the prediction and detection results of the sample image output by the initial image processing model.

[0188] The predicted detection result is obtained by the initial image processing model performing feature enhancement processing on the fused image features and the initial detection result. The fused image features are obtained by the initial image processing model performing fusion processing on the target region features and the abnormal region features. The target region features are obtained by the initial image processing model performing feature extraction on the target region mask image. The abnormal region features are obtained by the initial image processing model performing feature extraction on the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the initial image processing model performing segmentation processing on the sample image.

[0189] Step 306: Based on the predicted detection results and the label detection results, train the initial image processing model until the training stops, and obtain the target image processing model.

[0190] As is understandable, the training method for this image processing model is similar to the training process of the target image processing model in the above image processing methods, and will not be repeated here.

[0191] The following is in conjunction with the appendix Figure 4 Taking the application of the image processing model training method provided in this specification in the field of assisted medical care as an example, the image processing model training method will be further explained. Among other things, Figure 4 The present specification illustrates a flowchart of an image processing model training method according to an embodiment, which includes the following steps.

[0192] Step 402: Determine the sample image and label detection results for the target detection area.

[0193] Step 404: Input the sample image into the image segmentation unit of the initial image processing model, perform segmentation mask processing on the sample image, and obtain the segmentation mask image.

[0194] Step 406: Crop the segmentation mask image through the region of interest to obtain the target region mask image, the abnormal region mask image, and the initial detection results for the target detection region.

[0195] Step 408: Input the target region mask image into the global feature extractor in the feature extraction unit to obtain the target region features.

[0196] Step 410: Input the mask image of the abnormal region into the local feature extractor in the feature extraction unit to obtain the features of the abnormal region.

[0197] Step 412: Input the target region features and abnormal region features into the feature fusion unit, and perform attention mechanism processing on the target region features and abnormal region features to obtain fused image features.

[0198] Specifically, in the feature fusion unit, the target region features can be sampled at multiple resolution scales according to spatial resolution to obtain multiple sub-target region features. Anomaly region features are mapped to a new feature space using a learnable linear transformation matrix (i.e., a fully connected layer), resulting in mapped anomaly region features. A dual cross-attention mechanism is then applied to each sub-target region feature and the mapped anomaly region features to obtain fused image features.

[0199] Furthermore, taking the dual cross-attention mechanism processing of one sub-target region feature and one abnormal region feature as an example, the sub-target region feature can be linearly transformed by a fully connected layer to obtain the key (k) and value (v). The abnormal region feature can be linearly transformed by a fully connected layer to obtain the query (Q). The key and value provided by the sub-target region feature and the query provided by the abnormal region feature are processed by a cross-attention mechanism to obtain the first feature. Specifically, when processing the key, value and query, the query and key can be used to calculate the attention score, and the attention score can be calculated by the softmax function to obtain the attention weight. The attention weight and value are multiplied and summed to obtain the first feature after cross-attention mechanism processing. This first feature is generated based on the attention allocation of the sub-target region feature and the abnormal region feature, and can capture the important interaction information between the target region feature and the abnormal region feature. Furthermore, a fully connected layer can be used to perform a linear transformation on the sub-target region features to obtain a query (Q). Linear transformations can then be performed on the first feature and the abnormal region features using both linear and fully connected layers to obtain a key (k) and a value (v). A cross-attention mechanism is then applied to the query provided by the sub-target region features and the key and value provided by the abnormal region features to obtain a second feature. The first linear feature obtained by inputting the first feature and the abnormal region features into the linear layer, and the second linear feature obtained by inputting the second feature and the sub-target region features into the linear layer, are then fused to obtain a fused image feature. Through these two cross-attention mechanisms, a dual cross-attention mechanism is achieved between the sub-target region features and the abnormal region features. It is understandable that similar processing is performed on each sub-target region feature and the abnormal region feature to obtain the fused image feature.

[0200] Step 414: Input the fused image features and the initial detection results into the feature enhancement unit to obtain the predicted detection results for the target detection region.

[0201] Specifically, in the feature enhancement unit, based on a graph neural network, lesion features included in the fused image features are used as nodes, and reference category features corresponding to the initial detection results are used as nodes. According to the feature similarity between lesion features, between lesion features and reference category features, and between reference category features, connection edges are constructed between nodes corresponding to two features whose feature similarity reaches a preset similarity threshold, thus constructing a feature relationship graph. Attention mechanisms are then applied to the abnormal feature nodes and reference category feature nodes in the feature relationship graph to obtain the predicted detection results for the target detection region.

[0202] Step 416: Based on the prediction detection results and label detection results, train the initial image processing model until the training stops, and obtain the target image processing model.

[0203] Specifically, the first loss function corresponding to the anomaly level index, the second loss function corresponding to the target object index, and the third loss function corresponding to the screening level index can be combined. Based on the prediction detection results and the label detection results, the first model loss value corresponding to the anomaly level index, the second model loss value corresponding to the target object index, and the third model loss value corresponding to the screening level index can be calculated respectively. The first model loss value, the second model loss value, and the third model loss value are weighted and summed to obtain the target model loss value. The initial image processing model is trained based on the target model loss value until the training stopping condition is reached to obtain the target image processing model.

[0204] The aforementioned methods address issues such as dependence on specific imaging modalities, high computational resource requirements, insufficient real-time performance, and lack of integration with clinical workflows through their innovative plug-and-play design. This target image processing model can seamlessly integrate with any existing 3D segmentation model, is not limited by specific imaging equipment or modalities, reduces dependence on computational resources, and enhances the extraction and classification capabilities of lesion features through hierarchical dual attention (HDA) and graph-based prior reasoning units (GPR), thereby significantly improving the accuracy of liver lesion detection and diagnosis in plain CT images. Furthermore, the target image processing model, through comprehensive optimization strategies, simulates the diagnostic process of clinicians, providing end-to-end optimization from lesion detection to patient diagnosis. This makes it not only technically efficient and accurate but also more flexible and scalable in practical clinical applications, achieving the technical effects of improving diagnostic efficiency, reducing misdiagnosis rates, and supporting large-scale screening.

[0205] Corresponding to the above method embodiments, this specification also provides embodiments of an image processing apparatus. Figure 5 A schematic diagram of the structure of an image processing apparatus provided in one embodiment of this specification is shown. Figure 5 As shown, the device includes:

[0206] The determination module 502 is configured to determine the target image for the target detection region;

[0207] The input module 504 is configured to input the target image into a target image processing model to obtain the target detection result for the target detection region output by the target image processing model;

[0208] Wherein, the target detection result is obtained by the target image processing model through feature enhancement processing of fused image features and initial detection results; the fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features; the target region features are obtained by the target image processing model through feature extraction of the target region mask image; the abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image; and the target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the target image.

[0209] In one optional embodiment, the target image processing model includes an image segmentation unit, a feature extraction unit, a feature fusion unit, and a feature enhancement unit;

[0210] The input module 504 is further configured as follows:

[0211] The target image is input into the image segmentation unit to obtain the target region mask image, the abnormal region mask image, and the initial detection result corresponding to the target image;

[0212] The target region mask image and the abnormal region mask image are input into the feature extraction unit to obtain target region features and abnormal region features;

[0213] The target region features and the abnormal region features are input into the feature fusion unit to obtain fused image features;

[0214] The fused image features and the initial detection results are input into the feature enhancement unit to obtain the target detection result for the target detection region.

[0215] In an optional embodiment, the input module 504 is further configured to:

[0216] The target region features and the abnormal region features are input into the feature fusion unit, and the target region features and the abnormal region features are processed by an attention mechanism to obtain the fused image features.

[0217] In an optional embodiment, the input module 504 is further configured to:

[0218] The target region features are subjected to multi-scale pooling to obtain multiple sub-target region features;

[0219] Attention mechanisms are applied to the features of each sub-target region and the features of the abnormal regions to obtain the fused image features.

[0220] In an optional embodiment, the input module 504 is further configured to:

[0221] According to the spatial resolution scale, the target region features are subjected to multi-scale pooling processing to obtain multiple sub-target region features.

[0222] In an optional embodiment, the input module 504 is further configured to:

[0223] The target image is input into the image segmentation unit, and the target image is subjected to segmentation mask processing to obtain a segmentation mask image;

[0224] The segmentation mask image is cropped to obtain the target region mask image, the abnormal region mask image, and the initial detection result for the target detection region.

[0225] In an optional embodiment, the input module 504 is further configured to:

[0226] The fused image features and the initial detection result are input into the feature enhancement unit, and the initial detection result is enhanced by category prior using the fused image features to obtain the target detection result for the target detection region.

[0227] In an optional embodiment, the input module 504 is further configured to:

[0228] The abnormal features contained in the fused image features are used as nodes, the reference category features corresponding to the initial detection results are used as nodes, and the connection edges between nodes are determined according to the similarity between features to construct a feature relationship graph.

[0229] An attention mechanism is applied to the abnormal feature nodes and reference category feature nodes in the feature relationship graph to obtain the target detection result for the target detection region.

[0230] In an optional embodiment, the device further includes a training module configured to:

[0231] Determine the sample image and label detection results for the target detection region;

[0232] The sample image is input into an initial image processing model to obtain the predicted detection result for the sample image output by the initial image processing model. The predicted detection result is obtained by the initial image processing model performing feature enhancement processing on the fused image features and the initial detection result. The fused image features are obtained by the initial image processing model fusing target region features and abnormal region features. The target region features are obtained by the initial image processing model extracting features from the target region mask image. The abnormal region features are obtained by the initial image processing model extracting features from the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the initial image processing model performing segmentation processing on the sample image.

[0233] Based on the predicted detection results and the label detection results, the initial image processing model is trained until the training stops, thereby obtaining the target image processing model.

[0234] In an optional embodiment, the training module is further configured to:

[0235] Based on the preset performance index, determine the predicted performance detection result and the label performance detection result corresponding to the preset performance index from the predicted detection result and the label detection result;

[0236] Based on the prediction performance test results and the label performance test results, calculate the model loss value corresponding to the preset performance index;

[0237] The initial image processing model is trained based on the model loss value until the training stops, thereby obtaining the target image processing model.

[0238] In one optional embodiment, the preset performance indicators include anomaly level indicators, target object indicators, and screening level indicators;

[0239] The training module is further configured as follows:

[0240] Based on the predicted detection results and the label detection results, calculate the first model loss value corresponding to the anomaly level index, the second model loss value corresponding to the target object index, and the third model loss value corresponding to the screening level index;

[0241] The initial image processing model is trained based on the first model loss value, the second model loss value, and the third model loss value until the training stops, thereby obtaining the target image processing model.

[0242] In the aforementioned apparatus, after determining the target image for the target detection region, the target image can be input into a target image processing model. This model can segment the target image to obtain a target region mask image, an anomaly region mask image, and initial detection results for the target detection region. Feature extraction is performed on the target region mask image to obtain target region features, and feature extraction is performed on the anomaly region mask image to obtain anomaly region features. By fusing the target region features and anomaly region features, a fused image feature is obtained, achieving interaction between global features (i.e., target region features) and local features (i.e., anomaly region features) of the target image. Furthermore, using the initial detection results for the target detection region as prior knowledge, feature enhancement processing is performed on the fused image features and the initial detection results to obtain the target detection result for the target detection region, thus achieving anomaly detection in the target detection region. Through the interaction between global and local features, and by combining prior knowledge prediction, the accuracy of feature representation is improved, further enhancing the accuracy of the target detection result.

[0243] The above is an illustrative scheme of an image processing apparatus according to this embodiment. It should be noted that the technical solution of this image processing apparatus and the technical solution of the image processing method described above belong to the same concept. For details not described in detail in the technical solution of the image processing apparatus, please refer to the description of the technical solution of the image processing method described above.

[0244] Corresponding to the above method embodiments, this specification also provides embodiments of an image processing model training device. Figure 6 A schematic diagram of an image processing model training apparatus according to one embodiment of this specification is shown. Figure 6 As shown, the device includes:

[0245] The determining module 602 is configured to determine the sample image and label detection results for the target detection region;

[0246] Input module 604 is configured to input the sample image into an initial image processing model to obtain a prediction detection result for the sample image output by the initial image processing model. The prediction detection result is obtained by the initial image processing model performing feature enhancement processing on fused image features and the initial detection result. The fused image features are obtained by the initial image processing model fusing target region features and abnormal region features. The target region features are obtained by the initial image processing model extracting features from a target region mask image. The abnormal region features are obtained by the initial image processing model extracting features from an abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the initial image processing model performing segmentation processing on the sample image.

[0247] The training module 606 is configured to train the initial image processing model based on the predicted detection results and the label detection results until the training stopping condition is met, thereby obtaining the target image processing model.

[0248] The above is a schematic scheme of an image processing model training device according to this embodiment. It should be noted that the technical solution of this image processing model training device and the technical solution of the image processing method described above belong to the same concept. For details not described in detail in the technical solution of the image processing model training device, please refer to the description of the technical solution of the image processing method described above.

[0249] Corresponding to the above method embodiments, this specification also provides a CT image processing method, see [link to documentation]. Figure 7 , Figure 7 A flowchart of a CT image processing method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0250] Step 702: Receive a CT image processing task, wherein the CT image processing task carries a plain CT image corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area;

[0251] Step 704: Input the plain CT image into the target image processing model to obtain the target detection result for the target detection area output by the target image processing model;

[0252] The target detection result is obtained by the target image processing model through feature enhancement processing of the fused image features and the initial detection result. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the plain CT image.

[0253] Specifically, after determining the plain CT image for the target detection region, the plain CT image can be input into the target image processing model. This model can segment the plain CT image to obtain a target region mask image, an anomaly region mask image, and initial detection results for the target detection region. Feature extraction is then performed on the target region mask image to obtain target region features, and on the anomaly region mask image to obtain anomaly region features. By fusing the target region features and the anomaly region features, a fused image feature is obtained, enabling interaction between global features (i.e., target region features) and local features (i.e., anomaly region features) of the plain CT image. Furthermore, using the initial detection results for the target detection region as prior knowledge, feature enhancement processing is performed on the fused image features and the initial detection results to obtain the target detection result for the target detection region, thus achieving anomaly detection within the target detection region. Through the interaction between global and local features, and by combining prior knowledge prediction, the accuracy of feature representation is improved, further enhancing the accuracy of the target detection result.

[0254] The above is an illustrative scheme of a CT image processing method according to this embodiment. It should be noted that the technical solution of this CT image processing method belongs to the same concept as the technical solution of the image processing method described above. For details not described in detail in the technical solution of the CT image processing method, please refer to the description of the technical solution of the image processing method described above.

[0255] Corresponding to the above method embodiments, this specification also provides embodiments of a CT image processing apparatus. Figure 8 A schematic diagram of a CT image processing apparatus according to one embodiment of this specification is shown. Figure 8 As shown, the device includes:

[0256] The receiving module 802 is configured to receive a CT image processing task, wherein the CT image processing task carries a plain CT image corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area;

[0257] The input module 804 is configured to input the plain CT image into the target image processing model to obtain the target detection result for the target detection area output by the target image processing model;

[0258] The target detection result is obtained by the target image processing model through feature enhancement processing of the fused image features and the initial detection result. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the plain CT image.

[0259] The above is a schematic scheme of a CT image processing device according to this embodiment. It should be noted that the technical solution of this CT image processing device and the technical solution of the image processing method described above belong to the same concept. For details not described in detail in the technical solution of the CT image processing device, please refer to the description of the technical solution of the image processing method described above.

[0260] Corresponding to the above method embodiments, this specification also provides another image processing method applied to cloud-side devices, see [link to documentation]. Figure 9 , Figure 9 A flowchart of another image processing method provided according to an embodiment of this specification is shown, which specifically includes the following steps.

[0261] Step 902: Receive an image processing task sent by the receiving end device, wherein the image processing task carries a target image corresponding to the target detection area, and the image processing task is used to detect whether there is an anomaly in the target detection area;

[0262] Step 904: Input the target image into the target image processing model to obtain the target detection result for the target detection region output by the target image processing model, and send the target detection result to the end device;

[0263] Wherein, the target detection result is obtained by the target image processing model through feature enhancement processing of fused image features and initial detection results; the fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features; the target region features are obtained by the target image processing model through feature extraction of the target region mask image; the abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image; and the target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the target image.

[0264] The above is an illustrative scheme of an image processing method according to this embodiment. It should be noted that the technical solution of this image processing method belongs to the same concept as the technical solution of the image processing method described above. For details not described in detail in the technical solution of the image processing method, please refer to the description of the technical solution of the image processing method described above.

[0265] Corresponding to the above method embodiments, this specification also provides embodiments of an image processing apparatus applied to cloud-side devices. Figure 10 A schematic diagram of another image processing apparatus provided in one embodiment of this specification is shown. Figure 10 As shown, the device includes:

[0266] The receiving module 1002 is configured to receive an image processing task sent by the receiving end device, wherein the image processing task carries a target image corresponding to the target detection area, and the image processing task is used to detect whether there is an anomaly in the target detection area;

[0267] Input module 1004 is configured to input the target image into a target image processing model, obtain the target detection result output by the target image processing model for the target detection region, and send the target detection result to the end device;

[0268] Wherein, the target detection result is obtained by the target image processing model through feature enhancement processing of fused image features and initial detection results; the fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features; the target region features are obtained by the target image processing model through feature extraction of the target region mask image; the abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image; and the target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the target image.

[0269] The above is an illustrative scheme of an image processing apparatus according to this embodiment. It should be noted that the technical solution of this image processing apparatus and the technical solution of the image processing method described above belong to the same concept. For details not described in detail in the technical solution of the image processing apparatus, please refer to the description of the technical solution of the image processing method described above.

[0270] Corresponding to the above method embodiments, this specification also provides a computer-aided diagnostic method for tumors, see [link to documentation]. Figure 11 , Figure 11 A flowchart of a computer-aided diagnostic method for tumors according to one embodiment of this specification is shown, specifically including the following steps.

[0271] Step 1102: Receive a tumor screening task, wherein the tumor screening task carries a plain CT image corresponding to the target detection area, and the tumor screening task is used to detect whether there is a tumor in the target detection area;

[0272] Step 1104: Input the plain CT image into the target image processing model to obtain the target detection result output by the target image processing model regarding whether a tumor exists in the target detection area;

[0273] The target detection result is obtained by the target image processing model through feature enhancement processing of the fused image features and the initial detection result. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the plain CT image.

[0274] Corresponding to the above-described method embodiments, this specification also provides a computer-aided diagnostic system for tumors. Figure 12 A schematic diagram of the structure of a computer-aided diagnostic system for tumors according to one embodiment of this specification is shown. Figure 12 As shown, the system includes a client 1202 and a server 1204, wherein,

[0275] The client 1202 is used to send a CT image processing task to the server 1204, wherein the CT image processing task carries a plain CT image corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area.

[0276] The server 1204 is used to input the plain CT image into the target image processing model, obtain the target detection result output by the target image processing model regarding whether there is an abnormality in the target detection area, and send the target detection result to the client 1202.

[0277] The target detection result is obtained by the target image processing model through feature enhancement processing of the fused image features and the initial detection result. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the plain CT image.

[0278] Figure 13 A structural block diagram of a computing device 1300 according to one embodiment of this specification is shown. The components of the computing device 1300 include, but are not limited to, a memory 1310 and a processor 1320. The processor 1320 is connected to the memory 1310 via a bus 1330, and a database 1350 is used to store data.

[0279] The computing device 1300 also includes an access device 1340, which enables the computing device 1300 to communicate via one or more networks 1360. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1340 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0280] In one embodiment of this application, the aforementioned components of the computing device 1300 and Figure 13 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 13 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0281] The computing device 1300 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1300 can also be a mobile or stationary server.

[0282] The processor 1320 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above method.

[0283] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computing device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0284] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0285] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer-readable storage medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0286] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0287] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the computer program product can be referred to the description of the technical solution of the above method.

[0288] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0289] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0290] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0291] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0292] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. An image processing method, comprising: Determine the target image for the target detection region; The target image is input into the target image processing model to obtain the target detection result for the target detection region output by the target image processing model; Wherein, the target detection result is obtained by the target image processing model through category prior enhancement processing of fused image features and initial detection results. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the target image. The category prior enhancement processing includes: taking the abnormal features contained in the fused image features as nodes, taking the reference category features corresponding to the initial detection result as nodes, determining the connection edges between nodes according to the similarity between features, constructing a feature relationship graph, and performing attention mechanism processing on the abnormal feature nodes and reference category feature nodes in the feature relationship graph to obtain the target detection result for the target detection region.

2. The method according to claim 1, wherein the target image processing model comprises an image segmentation unit, a feature extraction unit, a feature fusion unit, and a feature enhancement unit; The step of inputting the target image into the target image processing model and obtaining the target detection result for the target detection region output by the target image processing model includes: The target image is input into the image segmentation unit to obtain the target region mask image, the abnormal region mask image, and the initial detection result corresponding to the target image; The target region mask image and the abnormal region mask image are input into the feature extraction unit to obtain target region features and abnormal region features; The target region features and the abnormal region features are input into the feature fusion unit to obtain fused image features; The fused image features and the initial detection results are input into the feature enhancement unit to obtain the target detection result for the target detection region.

3. The method according to claim 2, wherein inputting the target region features and the abnormal region features into the feature fusion unit to obtain fused image features includes: The target region features and the abnormal region features are input into the feature fusion unit, and the target region features and the abnormal region features are processed by an attention mechanism to obtain the fused image features; The step of performing attention mechanism processing on the target region features and the abnormal region features to obtain the fused image features includes: The target region features are subjected to multi-scale pooling to obtain multiple sub-target region features; Attention mechanisms are applied to the features of each sub-target region and the features of the abnormal regions to obtain the fused image features.

4. The method according to claim 2, wherein inputting the target image into the image segmentation unit to obtain the target region mask image, the abnormal region mask image, and the initial detection result corresponding to the target image includes: The target image is input into the image segmentation unit, and the target image is subjected to segmentation mask processing to obtain a segmentation mask image; The segmentation mask image is cropped to obtain the target region mask image, the abnormal region mask image, and the initial detection result for the target detection region.

5. The method according to claim 2, wherein inputting the fused image features and the initial detection result into the feature enhancement unit to obtain a target detection result for the target detection region includes: The fused image features and the initial detection result are input into the feature enhancement unit, and the initial detection result is enhanced by category prior using the fused image features to obtain the target detection result for the target detection region.

6. The method according to claim 1, wherein the training process of the target image processing model includes: Determine the sample image and label detection results for the target detection region; The sample image is input into an initial image processing model to obtain the predicted detection result output by the initial image processing model for the sample image. The predicted detection result is obtained by the initial image processing model performing category prior enhancement processing on the fused image features and the initial detection result. The fused image features are obtained by the initial image processing model fusing target region features and abnormal region features. The target region features are obtained by the initial image processing model extracting features from the target region mask image. The abnormal region features are obtained by the initial image processing model extracting features from the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the initial image processing model performing segmentation processing on the sample image. Based on the predicted detection results and the label detection results, the initial image processing model is trained until the training stops, thereby obtaining the target image processing model.

7. The method according to claim 6, wherein training the initial image processing model based on the predicted detection result and the label detection result until the training stopping condition is met to obtain the target image processing model, comprises: Based on the preset performance index, determine the predicted performance detection result and the label performance detection result corresponding to the preset performance index from the predicted detection result and the label detection result; Based on the prediction performance test results and the label performance test results, calculate the model loss value corresponding to the preset performance index; The initial image processing model is trained based on the model loss value until the training stops, thereby obtaining the target image processing model.

8. The method according to claim 7, wherein the preset performance indicators include anomaly level indicators, target object indicators, and screening level indicators; The step of training the initial image processing model based on the predicted detection results and the label detection results until the training stopping condition is met to obtain the target image processing model includes: Based on the predicted detection results and the label detection results, calculate the first model loss value corresponding to the anomaly level index, the second model loss value corresponding to the target object index, and the third model loss value corresponding to the screening level index; The initial image processing model is trained based on the first model loss value, the second model loss value, and the third model loss value until the training stops, thereby obtaining the target image processing model.

9. A method for training an image processing model, comprising: Determine the sample images and label detection results for the target detection region; The sample image is input into an initial image processing model to obtain the predicted detection result for the sample image output by the initial image processing model. The predicted detection result is obtained by the initial image processing model performing category prior enhancement processing on the fused image features and the initial detection result. The fused image features are obtained by the initial image processing model fusing target region features and abnormal region features. The target region features are obtained by the initial image processing model extracting features from a target region mask image. The abnormal region features are obtained by the initial image processing model extracting features from an abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the initial image processing model segmenting the sample image. The category prior enhancement processing includes: using abnormal features contained in the fused image features as nodes, using reference category features corresponding to the initial detection result as nodes, determining the connection edges between nodes based on the similarity between features, constructing a feature relationship graph, and performing attention mechanism processing on the abnormal feature nodes and reference category feature nodes in the feature relationship graph to obtain the target detection result for the target detection region. Based on the predicted detection results and the label detection results, the initial image processing model is trained until the training stops, thereby obtaining the target image processing model.

10. A CT image processing method, comprising: Receive a CT image processing task, wherein the CT image processing task carries a plain CT image corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area; The plain CT image is input into the target image processing model to obtain the target detection result for the target detection area output by the target image processing model; The target detection result is obtained by the target image processing model through category prior enhancement processing of the fused image features and the initial detection result. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the plain CT image. The category prior enhancement processing includes: taking the abnormal features contained in the fused image features as nodes, taking the reference category features corresponding to the initial detection result as nodes, determining the connection edges between nodes according to the similarity between features, constructing a feature relationship graph, and performing attention mechanism processing on the abnormal feature nodes and reference category feature nodes in the feature relationship graph to obtain the target detection result for the target detection region.

11. An image processing method applied to a cloud-side device, comprising: The receiving end device sends an image processing task, wherein the image processing task carries a target image corresponding to the target detection area, and the image processing task is used to detect whether there is an anomaly in the target detection area; The target image is input into the target image processing model to obtain the target detection result for the target detection region output by the target image processing model, and the target detection result is sent by the end device. Wherein, the target detection result is obtained by the target image processing model through category prior enhancement processing of fused image features and initial detection results. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the target image. The category prior enhancement processing includes: taking the abnormal features contained in the fused image features as nodes, taking the reference category features corresponding to the initial detection result as nodes, determining the connection edges between nodes according to the similarity between features, constructing a feature relationship graph, and performing attention mechanism processing on the abnormal feature nodes and reference category feature nodes in the feature relationship graph to obtain the target detection result for the target detection region.

12. A computer-aided diagnostic method for tumors, comprising: Receive a tumor screening task, wherein the tumor screening task carries a plain CT image corresponding to a target detection area, and the tumor screening task is used to detect whether there is a tumor in the target detection area; The plain CT image is input into the target image processing model to obtain the target detection result output by the target image processing model regarding whether a tumor exists in the target detection area; The target detection result is obtained by the target image processing model through category prior enhancement processing of the fused image features and the initial detection result. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the plain CT image. The category prior enhancement processing includes: taking the abnormal features contained in the fused image features as nodes, taking the reference category features corresponding to the initial detection result as nodes, determining the connection edges between nodes according to the similarity between features, constructing a feature relationship graph, and performing attention mechanism processing on the abnormal feature nodes and reference category feature nodes in the feature relationship graph to obtain the target detection result for the target detection region.

13. A computer-aided diagnostic system for tumors, comprising a client and a server, wherein, The client is used to send a CT image processing task to the server, wherein the CT image processing task carries a plain CT image corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area; The server is used to input the plain CT image into the target image processing model, obtain the target detection result output by the target image processing model regarding whether there is an abnormality in the target detection area, and send the target detection result to the client. The target detection result is obtained by the target image processing model through category prior enhancement processing of the fused image features and the initial detection result. The fused image features are obtained by the target image processing model through fusion processing of target region features and abnormal region features. The target region features are obtained by the target image processing model through feature extraction of the target region mask image. The abnormal region features are obtained by the target image processing model through feature extraction of the abnormal region mask image. The target region mask image, the abnormal region mask image, and the initial detection result are obtained by the target image processing model through segmentation processing of the plain CT image. The category prior enhancement processing includes: taking the abnormal features contained in the fused image features as nodes, taking the reference category features corresponding to the initial detection result as nodes, determining the connection edges between nodes according to the similarity between features, constructing a feature relationship graph, and performing attention mechanism processing on the abnormal feature nodes and reference category feature nodes in the feature relationship graph to obtain the target detection result for the target detection region.

14. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 12.

15. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 12.

Images