Image processing model training method and image processing method

Abstract

Embodiments of the description provide an image processing model training method and an image processing method. The image processing model training method comprises: acquiring a plurality of sample enhanced images and a plurality of sample target images, wherein the sample images comprise sample annotation information; inputting the sample enhanced images and the sample target images into an initial image processing model to obtain prediction annotations outputted by the initial image processing model, wherein the initial image processing model comprises an enhanced image processing sub-model and a target image processing sub-model, the enhanced image processing sub-model outputs first prediction annotation information on the basis of the sample enhanced images, and the target image processing sub-model outputs second prediction annotation information on the basis of the sample target images; calculating a model loss value on the basis of the sample annotation information and the prediction annotation information; adjusting model parameters on the basis of the model loss value and obtaining a reference image processing model; and generating an image processing model on the basis of the target image processing sub-model. The method improves the performance of image processing models in target image processing.

Description

Image processing model training methods, image processing methods

[0001] This application claims priority to Chinese Patent Application No. 202410676834.8, filed on May 28, 2024, entitled “Training Method for Image Processing Model, Image Processing Method”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The embodiments in this specification relate to the field of computer technology, and in particular to a training method for an image processing model and an image processing method. Background Technology

[0003] As people's living standards improve, more and more people are paying attention to their health. Vascular embolism is one of the major factors affecting human health. Identifying vascular embolism in medical images requires professional doctors to make identification based on their experience. However, due to the limitations of doctors' experience, image recognition and analysis using medical images has become an important topic.

[0004] In practical applications, vascular embolism presents with nonspecific symptoms. Current embolism detection methods rely on enhanced CT scans. However, enhanced CT exposes patients to contrast agents, which can cause allergic reactions or organ failure. Furthermore, enhanced CT scans are limited by technology and equipment, making them unavailable for 24-hour use in all regions, thus impacting their convenience. Therefore, how to utilize plain CT scans for vascular embolism detection and improve diagnostic accuracy using these images as supplementary diagnostic tools has become a pressing issue for technical personnel. Summary of the Invention

[0005] In view of this, embodiments of this specification provide a method for training an image processing model. One or more embodiments of this specification also relate to image processing methods, CT image processing apparatuses, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of the embodiments of this specification, a method for training an image processing model is provided, comprising:

[0007] Multiple training sample pairs are obtained, wherein each training sample pair includes a sample augmentation image and a sample target image, the sample augmentation image includes first sample annotation information for the target detection region, and the sample target image includes second sample annotation information for the target detection region;

[0008] The sample augmented image and the sample target image are input into the initial image processing model to obtain the first prediction annotation information and the second prediction annotation information output by the initial image processing model. The initial image processing model includes an augmented image processing sub-model and a target image processing sub-model. The augmented image processing sub-model outputs the first prediction annotation information based on the sample augmented image, and the target image processing sub-model outputs the second prediction annotation information based on the sample target image.

[0009] The model loss value is calculated based on the first sample annotation information, the second sample annotation information, the first prediction annotation information, and the second prediction annotation information.

[0010] Adjust the model parameters of the initial image processing model based on the model loss value, and continue training the initial image processing model until the model training stop condition is met to obtain a reference image processing model;

[0011] An image processing model is generated based on the target image processing sub-model in the reference image processing model.

[0012] According to a second aspect of the embodiments of this specification, an image processing method is provided, comprising:

[0013] Receive an image processing task, wherein the image processing task carries multiple target images corresponding to a target detection region, and the image processing task is used to detect whether there is an anomaly in the target detection region;

[0014] The multiple target images are input into an image processing model to obtain the detection results of the target detection region output by the image processing model, wherein the image processing model is trained by the above-described training method.

[0015] According to a third aspect of the embodiments of this specification, a CT image processing method is provided, comprising:

[0016] Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to a target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area;

[0017] The multiple plain CT images are input into a CT image processing model to obtain the detection results of the target detection region output by the image processing model, wherein the image processing model is trained by the above-described training method.

[0018] According to a fourth aspect of the embodiments of this specification, a method for training an image processing model is provided, applied to a cloud-based device, comprising:

[0019] Multiple training sample pairs are obtained, wherein each training sample pair includes a sample augmentation image and a sample target image, the sample augmentation image includes first sample annotation information for the target detection region, and the sample target image includes second sample annotation information for the target detection region;

[0020] The sample augmented image and the sample target image are input into the initial image processing model to obtain the first prediction annotation information and the second prediction annotation information output by the initial image processing model. The initial image processing model includes an augmented image processing sub-model and a target image processing sub-model. The augmented image processing sub-model outputs the first prediction annotation information based on the sample augmented image, and the target image processing sub-model outputs the second prediction annotation information based on the sample target image.

[0021] The model loss value is calculated based on the first sample annotation information, the second sample annotation information, the first prediction annotation information, and the second prediction annotation information.

[0022] Adjust the model parameters of the initial image processing model based on the model loss value, and continue training the initial image processing model until the model training stop condition is met to obtain a reference image processing model;

[0023] Based on the target image processing sub-model in the reference image processing model, an image processing model is generated, the model parameters of the image processing model are obtained, and the model parameters of the image processing model are sent to the end device.

[0024] According to a fifth aspect of the embodiments of this specification, an image processing method is provided, applied to a cloud-side device, comprising:

[0025] The receiving end device sends an image processing task, wherein the image processing task carries multiple target images corresponding to the target detection area, and the image processing task is used to detect whether there is an anomaly in the target detection area;

[0026] The plurality of target images are input into an image processing model to obtain the detection results of the target detection region output by the image processing model, wherein the image processing model is trained by the above training method;

[0027] The detection result is sent to the end-side device.

[0028] According to a sixth aspect of the embodiments of this specification, a method for detecting vascular embolism is provided, comprising:

[0029] Receive a vascular embolism detection task, wherein the vascular embolism detection task carries multiple plain CT images corresponding to a vascular region, and the vascular embolism detection task is used to detect whether there is an embolism in the vascular region;

[0030] The multiple plain CT images are input into the image processing model to obtain the detection results of whether there is vascular embolism in the vascular region output by the image processing model. The image processing model is trained by the above-described training method.

[0031] According to a seventh aspect of the embodiments of this specification, a tumor screening method is provided, comprising:

[0032] Receive a tumor screening task, wherein the tumor screening task carries multiple plain CT images corresponding to a target detection area, and the tumor screening task is used to detect whether a tumor exists in the target detection area;

[0033] The multiple plain CT images are input into the image processing model to obtain the detection result of whether there is a tumor in the target detection area, which is output by the image processing model. The image processing model is trained by the above-described training method.

[0034] According to an eighth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0035] Memory and processor;

[0036] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above method.

[0037] According to a ninth aspect of an embodiment of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0038] According to a tenth aspect of an embodiment of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0039] This specification provides an embodiment of an image processing model training method, comprising: acquiring multiple training sample pairs, wherein each training sample pair includes a sample augmentation image and a sample target image; the sample augmentation image includes first sample annotation information for a target detection region, and the sample target image includes second sample annotation information for the target detection region; inputting the sample augmentation image and the sample target image into an initial image processing model to obtain first and second prediction annotation information output by the initial image processing model; wherein the initial image processing model includes an augmentation image processing sub-model and a target image processing sub-model; the augmentation image processing sub-model outputs the first prediction annotation information based on the sample augmentation image, and the target image processing sub-model outputs the second prediction annotation information based on the sample target image; calculating a model loss value based on the first and second sample annotation information and the first and second prediction annotation information; adjusting the model parameters of the initial image processing model based on the model loss value, and continuing to train the initial image processing model until the model training stopping condition is met to obtain a reference image processing model; and generating an image processing model based on the target image processing sub-model in the reference image processing model.

[0040] The method provided in the embodiments of this specification offers a framework based on mutual learning between two sub-models, unifying the classification and segmentation tasks for two types of sample images (sample augmented images and sample target images). It achieves knowledge transfer from the augmented image processing sub-model to the target image processing sub-model, improving the performance of the image processing model in target image processing. Simultaneously, the structuring of the segmentation and classification tasks increases the interpretability of image recognition. Attached Figure Description

[0041] Figure 1 is a flowchart of a training method for an image processing model provided in one embodiment of this specification;

[0042] Figure 2 is a schematic diagram of the model structure of an initial image processing model provided in one embodiment of this specification;

[0043] Figure 3 is a schematic diagram of the structure for training an image processing model for pulmonary embolism detection provided in one embodiment of this specification;

[0044] Figure 4 is a flowchart of an image processing method provided in one embodiment of this specification;

[0045] Figure 5 is a flowchart of a CT image processing method provided in one embodiment of this specification;

[0046] Figure 6 is a schematic diagram of the structure of a training device for an image processing model provided in one embodiment of this specification;

[0047] Figure 7 is a schematic diagram of a CT image processing device provided in one embodiment of this specification;

[0048] Figure 8 is a schematic diagram of a training method for an image processing model applied to a cloud-side device according to an embodiment of this specification;

[0049] Figure 9 is a flowchart illustrating an image processing method applied to a cloud-side device according to an embodiment of this specification;

[0050] Figure 10 is a schematic flowchart of a method for detecting vascular embolism provided in one embodiment of this specification;

[0051] Figure 11 is a schematic flowchart of a tumor screening method provided in one embodiment of this specification;

[0052] Figure 12 is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0053] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0054] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0055] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0056] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0057] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0058] CT (Computed Tomography): Computed tomography is a scanning technology that uses a precisely collimated X-ray beam and a highly sensitive detector to scan a specific part of the body one section after another. It features fast scanning time and clear images and can be used to examine a variety of diseases.

[0059] Plain CT scan: also known as conventional CT, refers to a scan performed intravenously without the administration of iodine-containing contrast agents.

[0060] Enhanced CT: This refers to a scanning method that involves injecting a contrast agent into a blood vessel before scanning. The purpose is to increase the density difference between the diseased tissue and normal tissue, so as to show lesions that are not shown or are not clearly shown on plain CT. The presence or absence of enhancement and the type of enhancement help to characterize the lesion.

[0061] CAD (computer-aided diagnosis): Computer-aided diagnosis refers to the use of imaging, medical image processing technology, and other possible physiological and biochemical methods, combined with computer analysis and calculation, to assist in the detection of lesions and improve the accuracy of diagnosis.

[0062] With the continuous development of computer technology, various learning models are gradually being applied to prediction in various application scenarios. Correspondingly, deep learning models have also achieved significant success in computer-aided diagnosis (CAD) tasks in medical imaging. Case analysis and classification from medical images is an important topic in computer-aided diagnosis.

[0063] Pulmonary embolism is a serious and life-threatening condition, second only to myocardial infarction and sudden cardiac death in severity. Early diagnosis and treatment are crucial. Because pulmonary embolism presents with nonspecific symptoms, current detection primarily relies on contrast-enhanced CT scans of the pulmonary arteries. However, contrast-enhanced CT exposes patients to contrast agents, potentially causing allergic reactions or other discomfort. Furthermore, in some regions, contrast-enhanced CT is not widely available due to technological and equipment limitations. In contrast, plain CT scans are more convenient and practical for examination.

[0064] However, in plain CT images, the contrast between the embolic area and the surrounding pulmonary vessels is almost indistinguishable. Doctors are prone to misdiagnosing pulmonary embolism using plain CT images, thus delaying treatment. Therefore, researching methods for automated pulmonary embolism identification based on plain CT scans and applying them to corresponding auxiliary diagnostic products can improve diagnostic efficiency to some extent and provide patients with a window of opportunity for treatment.

[0065] However, automated identification of vascular embolism based on plain CT scans faces two challenges. First, the Hu value of the embolic region is similar to that of surrounding vessels, resulting in low contrast and making accurate segmentation of the embolic region extremely difficult. Second, the diverse morphological characteristics of emboli and the non-fixed location of their onset make it difficult to detect vascular embolism using morphological features and location information.

[0066] Based on this, this specification provides a training method for an image processing model. This specification also relates to an image processing method, a CT image processing device, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0067] Referring to Figure 1, Figure 1 shows a flowchart of a training method for an image processing model according to an embodiment of this specification, which specifically includes the following steps.

[0068] Step 102: Obtain multiple training sample pairs, wherein each training sample pair includes a sample augmentation image and a sample target image. The sample augmentation image includes first sample annotation information for the target detection region, and the sample target image includes second sample annotation information for the target detection region.

[0069] Specifically, the image processing model provided in this specification uses supervised training, which includes multiple training sample pairs. Each training sample pair includes multiple enhanced sample images and target sample images for the target detection region. The enhanced sample images specifically refer to images with enhanced contrast for the target detection region, while the target sample images specifically refer to ordinary images for the target detection region. The enhanced sample images include first sample annotation information for the target detection region, and the target sample images include second sample annotation information for the target detection region.

[0070] For example, in assisted medical scenarios, training sample pairs include enhanced CT images and plain CT images targeting the target detection region. The enhanced CT image can be understood as the enhanced sample image, and the plain CT image as the target sample image. In practical applications, because the image processing model needs to be trained, the enhanced CT image includes first sample annotation information targeting the target detection region, and the plain CT image also includes second sample annotation information targeting the target detection region. Specifically, the first and second sample annotation information refer to anomaly annotation information targeting the target detection region. For example, annotating the location of anomalies and / or indicating whether anomalies exist in the target detection region, etc.

[0071] It is important to note that in practical applications, the target detection area may exhibit different characteristics at different time points. To ensure data consistency between the enhanced sample image and the target sample image, the enhanced sample image and the target sample image provided in the embodiments of this specification are images of the same target detection area taken at the same time point. For example, taking a patient's CT image as an example, where the target detection area is the patient's lungs, the training sample pair consists of the enhanced CT image and the plain CT image taken when the patient underwent an examination at a certain time point. Both the enhanced CT image and the plain CT image are images of the lung region, and both images must be taken at the same time the patient underwent the examination. This prevents situations where abnormal information in the enhanced CT image and the plain CT image does not match after the patient has undergone treatment.

[0072] In one specific embodiment provided in this specification, multiple training sample pairs are obtained, including:

[0073] Identify the sample detection object and obtain the sample enhancement image and sample target image corresponding to the sample detection object, wherein the sample enhancement image and sample target image corresponding to the sample detection object correspond to the same target detection region of the sample detection object;

[0074] First sample annotation information is generated on the enhanced sample image for the target detection region;

[0075] The second sample annotation information is annotated on the sample target image based on the first sample annotation information.

[0076] Here, the sample detection object specifically refers to the object corresponding to the target detection region. For example, when the target detection region is the stomach, the sample detection object is the organism corresponding to the target detection region, such as a human, horse, or cow. After determining the sample detection object, the sample enhancement image and sample target image corresponding to the sample detection object are further obtained. The sample enhancement image and sample target image are images targeting the same target detection region.

[0077] In practical applications, anomalous information in the augmented image is more obvious than in the target image. Therefore, first sample annotation information can be generated in the augmented image for the target detection region. Specifically, technicians can manually annotate the augmented image, or the augmented image can be input into an annotation model, which will then recognize the augmented image and generate the first sample annotation information.

[0078] After determining the first sample annotation information, annotations can be performed on the target image based on this information. In the enhanced sample image, the anomaly information for abnormal regions is more obvious and can be directly annotated. However, in the target image, the anomaly information for abnormal regions is not obvious and cannot be annotated using annotation models or manual annotation methods. In this case, since the enhanced sample image and the target image were captured at the same time for the same target detection area, the first sample annotation information from the enhanced sample image can be used to annotate the target image. For example, if the range of the anomaly region annotated in the first sample annotation information is range 1, then the target image can be annotated based on range 1, resulting in the range of the anomaly region in the second sample annotation information being range 2. Here, range 1 and range 2 are the same region.

[0079] In practical applications, registration can be used to annotate the target image with the annotation information of the second sample based on the annotation information of the first sample. Registration refers to the process of matching and overlaying two or more images. The purpose of registration is to find the spatial transformation relationship between two or more images, aligning them in space, thereby enabling further image analysis or processing.

[0080] Registration methods and techniques vary depending on the application field and specific needs. In medical image processing, registration techniques are used to achieve spatial consistency between corresponding points on a medical image. In the method provided in the embodiments of this specification, first sample annotation information is annotated in the sample enhancement image. Through image registration, the first sample annotation information in the sample enhancement image is transferred to the sample target image, and second sample annotation information is generated in the sample target image.

[0081] In another specific embodiment provided in this specification, first sample annotation information is generated on the sample enhancement image for the target detection region, including:

[0082] The enhanced image of the sample is input into the information annotation model to obtain the first sample segmentation information and the first sample classification information for the target detection region output by the information annotation model;

[0083] Accordingly, based on the first sample annotation information, second sample annotation information is annotated on the sample target image, including:

[0084] Based on the first sample segmentation information, second sample segmentation information is marked on the sample target image, and based on the first sample classification information, the second sample classification information of the sample target image is determined.

[0085] In one specific embodiment provided in this specification, a sample augmentation image can be input into an information annotation model, which then annotates the sample augmentation image. Specifically, the information annotation model annotates the sample augmentation image with first sample segmentation information and labels the sample augmentation image with first sample classification information. The first sample segmentation information specifically refers to the segmentation mask information for abnormal regions within the target detection region, and the first sample classification information specifically refers to the abnormality classification information for the target detection region.

[0086] Accordingly, after determining the first sample segmentation information and the first sample classification information of the sample augmentation image, the second sample segmentation information can be labeled on the sample target image based on the first sample segmentation information, and the second sample classification information of the sample target image can be determined based on the first sample classification information.

[0087] The first sample segmentation information and the first sample classification information can be understood as the first sample annotation information of the sample augmentation image; the second sample segmentation information and the second sample classification information can be understood as the second sample annotation information of the sample target image. The specific content of the first sample classification information and the second sample classification information is the same.

[0088] Step 104: Input the sample augmented image and the sample target image into the initial image processing model to obtain the first prediction annotation information and the second prediction annotation information output by the initial image processing model. The initial image processing model includes an augmented image processing sub-model and a target image processing sub-model. The augmented image processing sub-model outputs the first prediction annotation information based on the sample augmented image, and the target image processing sub-model outputs the second prediction annotation information based on the sample target image.

[0089] The initial image processing model is the one that has not yet been trained. In practical applications, the initial image processing model consists of two parts: an enhancement image processing sub-model and a target image processing sub-model. Both sub-models have the same structure and are used to process the sample enhancement image and the sample target image, respectively. Specifically, the enhancement image processing sub-model processes the sample enhancement image, and the target image processing sub-model processes the sample target image.

[0090] Referring to Figure 2, which illustrates the model structure of the initial image processing model provided in this embodiment, the initial image processing model includes two sub-models: an enhanced image processing sub-model and a target image processing sub-model. Each sub-model includes an encoder, a decoder, and a classifier. The encoder extracts image feature information, the decoder generates image segmentation information, and the classifier generates image classification information. The image segmentation information and the image classification information together form the annotation information. Furthermore, the decoder and classifier in the enhanced image processing sub-model output first predicted annotation information, and the decoder and classifier in the target image processing sub-model output second predicted annotation information.

[0091] In one specific embodiment provided in this specification, the sample augmentation image and the sample target image are input into an initial image processing model to obtain first prediction annotation information and second prediction annotation information output by the initial image processing model, including:

[0092] The sample augmented image and the sample target image are input into the initial image processing model to obtain the first predicted annotation information, the second predicted annotation information, the classification difference loss value, the segmentation difference loss value, and the feature information difference loss value output by the initial image processing model.

[0093] In the specific embodiments provided in this specification, since the enhanced image processing sub-model processes sample enhanced images, it can identify abnormal information in the target detection region through sample enhanced images. However, the target image processing sub-model processes sample target images, which cannot accurately identify abnormal information in the sample target images. Therefore, in the embodiments provided in this specification, it is desirable for the target image processing sub-model to learn the feature information of the enhanced image processing sub-model during the recognition process. Thus, in the process of recognizing sample enhanced images and sample target images, the initial image processing model, in addition to generating the first predicted annotation information corresponding to the sample enhanced images and the second predicted annotation information corresponding to the sample target images, will further generate classification difference loss values, segmentation difference loss values, and feature information difference loss values.

[0094] Among them, the classification difference loss value can be understood as the difference loss value between the classification information generated by the enhanced image processing sub-model and the classification information generated by the target image processing sub-model; the segmentation difference loss value can be understood as the difference loss value between the segmentation information generated by the enhanced image processing sub-model and the segmentation information generated by the target image processing sub-model; and the feature information difference loss value can be understood as the difference loss value between the image coding features generated by the enhanced image processing sub-model and the image coding features generated by the target image processing sub-model.

[0095] Specifically, the enhanced image and the target image are input into an initial image processing model to obtain the first predicted annotation information, the second predicted annotation information, the classification difference loss value, the segmentation difference loss value, and the feature information difference loss value output by the initial image processing model, including:

[0096] The enhanced image of the sample is input into the enhanced image processing sub-model of the initial image processing model to obtain the first predicted classification information, the first predicted segmentation information, and the first image coding feature information.

[0097] The sample target image is input into the target image processing sub-model of the initial image processing model to obtain second predicted classification information, second predicted segmentation information, and second image coding feature information;

[0098] Calculate the classification difference loss value based on the first predicted classification information and the second predicted classification information;

[0099] Calculate the segmentation difference loss value based on the first predicted segmentation information and the second predicted segmentation information;

[0100] The feature information difference loss value is calculated based on the first image coding feature information and the second image coding feature information.

[0101] In practical applications, the enhanced image of the sample is input into the enhanced image processing sub-model of the initial image processing model for processing, and the first predicted classification information, the first predicted segmentation information, and the first image coding feature information output by the enhanced image processing sub-model are obtained. Among them, the first predicted classification information and the first predicted segmentation information are the first predicted annotation information in the above embodiment.

[0102] Similarly, the sample target image is input into the target image processing sub-model of the initial image processing model for processing, to obtain the second predicted classification information, the second predicted segmentation information, and the second image coding feature information output by the target image processing sub-model. The second predicted classification information and the second predicted segmentation information are the second predicted annotation information described in the above embodiment.

[0103] After obtaining the output results of the two sub-models, the classification difference loss value can be calculated based on the first predicted classification information and the second predicted classification information; the segmentation difference loss value can be calculated based on the first predicted segmentation information and the second predicted segmentation information; and the feature information difference loss value can be calculated based on the first image coding feature information and the second image coding feature information.

[0104] By identifying the differences between the first and second predicted classification information, the first and second predicted segmentation information, and the first and second image coding feature information, the differences between the enhanced image processing sub-model and the target image processing sub-model in the image recognition process are determined. This facilitates better training of the initial image processing model, enabling the target image processing sub-model to better recognize the target image.

[0105] Specifically, the enhanced image processing sub-model includes a first encoder, a first decoder, and a first classifier;

[0106] The enhanced image of the sample is input into the enhanced image processing sub-model of the initial image processing model to obtain first predicted classification information, first predicted segmentation information, and first image coding feature information, including:

[0107] The sample augmented image is input to the first encoder to obtain at least one augmented image coding feature information, wherein the at least one augmented image coding feature information includes classification augmented coding feature information;

[0108] The classification enhancement coding feature information is input into the first classifier to obtain the first predicted classification information;

[0109] The encoded feature information of each enhanced image is input into the first decoder to obtain the first predicted segmentation information;

[0110] The first image coding feature is determined from at least one enhanced image coding feature.

[0111] Referring to Figure 2, as shown in Figure 2, the image enhancement processing sub-model includes a first encoder, a first decoder, and a first classifier. The first encoder is used to extract image feature information from the sample enhanced image, the first decoder is used to generate segmentation information of the sample enhanced image, and the first classifier is used to generate classification information of the sample enhanced image.

[0112] The sample augmented image is input into the first encoder. After processing by the first encoder, at least one augmented image coding feature information is obtained. In practical applications, the first encoder includes multiple sequentially connected coding layers. After receiving the input image feature information, each coding layer outputs the augmented image coding feature information corresponding to the current coding layer.

[0113] For example, taking a first encoder with five coding layers as an example, the first coding layer receives the sample augmented image, processes it, and outputs the first augmented image coded feature information. This first augmented image coded feature information is then input into the second coding layer, processed, and outputs the second augmented image coded feature information; and so on, with the fifth coding layer outputting the fifth augmented image coded feature information. Thus, five augmented image coded feature information pieces are obtained. The augmented image coded feature information output by the last coding layer is the classification augmented image coded feature information.

[0114] The classification-enhanced image encoding feature information is input into the first classifier. The first classifier classifies the data using the category feature information in the classification-enhanced image encoding feature information to obtain the first predicted classification information.

[0115] The encoded feature information of each enhanced image is input into the first decoder, and the encoded feature information of each enhanced image is decoded in the first decoder to obtain the first predicted segmentation information.

[0116] To reflect the coding differences between the enhanced image processing sub-model and the target image processing sub-model during the coding process, a first image coding feature is determined from at least one enhanced image coding feature. Furthermore, the first image coding feature can be determined from at least one enhanced image coding feature according to a preset ratio. For example, the first 70% of the enhanced image coding features can be selected as the first image coding feature. When there are five enhanced image coding features, the first three or four enhanced image coding features are selected as the first image coding feature based on their coding order.

[0117] Correspondingly, the target image processing sub-model includes a second encoder, a second decoder, and a second classifier;

[0118] The sample target image is input into the target image processing sub-model of the initial image processing model to obtain second predicted classification information, second predicted segmentation information, and second image coding feature information, including:

[0119] The sample target image is input to the second encoder to obtain at least one target image coding feature information, wherein the at least one target image coding feature information includes classification target coding feature information;

[0120] The classification target encoded feature information is input into the second classifier to obtain the second predicted classification information;

[0121] The encoded feature information of each target image is input into the second decoder to obtain the second predicted segmentation information;

[0122] Determine the second image coding feature information from at least one target image coding feature information.

[0123] Referring to Figure 2, the target image processing sub-model includes a second encoder, a second decoder, and a second classifier. The model structure of the target image processing sub-model is the same as that of the augmented image processing sub-model. Regarding the processing method of the target image by the target image processing sub-model, it is the same as the processing method of the augmented image by the augmented image sub-model, and will not be repeated here.

[0124] The first predicted classification information, the first predicted segmentation information, and the first image coding feature information were obtained in the enhanced image processing sub-model branch; the second predicted classification information, the second predicted segmentation information, and the second image coding feature information were obtained in the target image processing sub-model branch.

[0125] The classification difference loss value is calculated based on the first predicted classification information and the second predicted classification information. In the method provided in the embodiments of this specification, KL divergence is used as a measure of the difference between the two, as shown in Formula 1 below:

[0126] Among them, L KL (p2|p1) represents the classification difference loss value, where p1 represents the first predicted classification information and p2 represents the second predicted classification information. m represents the classification result, where m=0 indicates no anomaly and m=1 indicates anomaly. x i Let i represent the i-th sample pair.

[0127] The segmentation difference loss value is calculated based on the first and second predicted segmentation information. This loss value is used to improve the ability to distinguish between background and abnormal regions in a segmentation scenario. In the method provided in the embodiments of this specification, an intra-class feature differentiation strategy based on a dense center loss function is offered. In each training iteration, the center is calculated as the centroid feature of pixels belonging to the corresponding category in the segmentation mask. For each center loss function, see Formula 2 below:

[0128] Among them, L disc x represents the segmentation difference loss value. k It is a feature of pixels belonging to category k, c k The k-th class center represents the deep feature, which allows the network to learn more effectively the compact and independent clusters corresponding to each class in the feature space.

[0129] The feature information difference loss value is calculated based on the first image encoding feature information and the second image encoding feature information. After encoding each sub-model, it is necessary to focus on the similarity between the spatial features of the enhanced image processing sub-model and the target image processing sub-model. Based on this, in the method provided in the embodiments of this specification, an affinity graph is constructed to encapsulate the relationship between the two. This graph is parameterized by the connection range α and the granularity β, which optimizes the resolution of the affinity graph and the fidelity of the captured spatial relationships.

[0130] Affinity diagrams have Each node and Each connection can serve as a dynamic representation of spatial correlation, enhancing the mutual learning process between the image processing sub-model and the target image processing sub-model. To quantify knowledge transfer between the two and facilitate the mutual learning process, the implementation provided in this specification introduces pairwise similarity distillation loss, combining the squared difference of pairwise similarities with the similarity itself to measure the alignment between network feature maps, as shown in Equation 3 below:

[0131] Among them, L alig This represents the loss value due to the difference in feature information. and This represents the similarity between the i-th and j-th nodes in the enhanced image processing sub-model and the target image processing sub-model. The similarity between the two nodes is represented by the aggregated feature f. i ,f j The calculation shows that, W′ and H′ are the sizes of the convolution kernel. In the specific implementation, average pooling is used to aggregate the β×C size features in a node into 1×C size features. During training, it is desirable for the image features of the target image processing sub-model to be close to those of the enhanced image processing sub-model. Therefore, in subsequent parameter tuning, the feature information difference loss value is only used to adjust the model parameters of the target image processing sub-model.

[0132] In practical applications, the area containing anomalies in the target detection region is usually smaller than the target detection region. If both the enhanced sample image and the target sample image are input into the initial image processing model, inaccurate recognition may occur. Therefore, in a specific embodiment provided in this specification, the enhanced sample image and the target sample image are input into the initial image processing model, including:

[0133] The sample enhancement image and the sample target image are cropped based on a preset cropping size;

[0134] The cropped sample augmented image and the sample target image are input into the initial image processing model.

[0135] In this embodiment, before inputting the sample augmentation image and the sample target image into the initial image processing model, the sample augmentation image and the sample target image are first cropped according to a preset cropping size. The cropped sample augmentation image and the sample target image are then input into the initial image processing model for processing. This allows the initial image processing model to better identify abnormal information in the target detection area.

[0136] Step 106: Calculate the model loss value based on the first sample annotation information, the second sample annotation information, the first prediction annotation information, and the second prediction annotation information.

[0137] In the specific implementation provided in the embodiments of this specification, the model loss value is calculated based on the first sample annotation information, the second sample annotation information, the first prediction annotation information, and the second prediction annotation information. In this case, the initial image processing model is an untrained model; therefore, the initial image processing model is trained by adjusting the model parameters by calculating the model loss value.

[0138] Furthermore, the enhanced sub-model loss value is calculated based on the first sample annotation information and the first predicted annotation information, the target sub-model loss value is calculated based on the second sample annotation information and the second predicted annotation information, and the model loss value is determined based on the enhanced sub-model loss value and the target sub-model loss value.

[0139] In one specific embodiment provided in this specification, the first sample annotation information includes first sample segmentation information and first sample classification information, the first prediction annotation information includes first prediction segmentation information and first prediction classification information, the second sample annotation information includes second sample segmentation information and second sample classification information, and the second prediction annotation information includes second prediction segmentation information and second prediction classification information.

[0140] The model loss value is calculated based on the first sample annotation information, the second sample annotation information, the first prediction annotation information, and the second prediction annotation information, including:

[0141] Calculate the classification loss value of the enhanced sub-model based on the classification information of the first sample and the first predicted classification information;

[0142] Calculate the segmentation loss value of the enhanced sub-model based on the segmentation information of the first sample and the segmentation information of the first prediction.

[0143] The loss value of the augmented submodel is determined based on the classification loss value and the segmentation loss value of the augmented submodel.

[0144] Calculate the classification loss value of the target sub-model based on the classification information of the second sample and the second predicted classification information;

[0145] Calculate the target sub-model segmentation loss value based on the second sample segmentation information and the second predicted segmentation information;

[0146] The target sub-model loss value is determined based on the target sub-model classification loss value and the target sub-model segmentation loss value.

[0147] In practical applications, the first sample annotation information includes first sample segmentation information and first sample classification information, and the first prediction annotation information includes first prediction segmentation information and first prediction classification information. The segmentation loss value of the enhanced sub-model is calculated based on the first sample segmentation information and the first sample classification information; the classification loss value of the enhanced sub-model is calculated based on the first sample classification information and the first prediction classification information; and finally, the loss value of the enhanced sub-model is determined based on the segmentation loss value of the enhanced sub-model and the classification loss value of the enhanced sub-model.

[0148] Accordingly, the second sample annotation information includes second sample segmentation information and second sample classification information, and the second prediction annotation information includes second prediction segmentation information and second prediction classification information. The target sub-model classification loss value is calculated based on the second sample segmentation information and the second sample classification information; the target sub-model segmentation loss value is calculated based on the second sample classification information and the second prediction classification information; and finally, the target sub-model loss value is determined based on the target sub-model classification loss value and the target sub-model segmentation loss value.

[0149] The loss value of the enhanced sub-model and the loss value of the target sub-model together constitute the model loss value.

[0150] Step 108: Adjust the model parameters of the initial image processing model according to the model loss value, and continue training the initial image processing model until the model training stops, to obtain a reference image processing model.

[0151] At this point, the initial image processing model is an untrained model. The model parameters need to be adjusted using the model loss value. Training of the initial image processing model continues until the training stops, resulting in a reference image processing model.

[0152] The reference image processing model at this point is the image processing model obtained after the initial image processing model has reached the model training stopping condition. The model structure of the reference image processing model is the same as that of the initial image processing model.

[0153] In one specific embodiment provided in this specification, adjusting the model parameters of the initial image processing model based on the model loss value includes:

[0154] The model parameters of the initial image processing model are adjusted based on the model loss value, classification difference loss value, segmentation difference loss value, and feature information difference loss value.

[0155] In the embodiments provided in this specification, the initial image processing model, in addition to outputting the first predicted annotation information and the second predicted annotation information, will also simultaneously determine the classification difference loss value, the segmentation difference loss value and the feature information difference loss value.

[0156] Accordingly, during the training of the initial image processing model, in addition to adjusting the model parameters using the model loss value, the model parameters can also be adjusted using the model loss value, classification difference loss value, segmentation difference loss value, and feature information difference loss value together. This allows the initial image processing model to learn not only the difference between the predicted result and the label, but also the difference information between the image encoded features generated during model processing, including classification difference information and segmentation difference information.

[0157] In another specific embodiment provided in this specification, the model loss value includes the enhancement sub-model loss value and the target sub-model loss value;

[0158] Adjusting the model parameters of the initial image processing model based on the model loss value, classification difference loss value, segmentation difference loss value, and feature information difference loss value includes:

[0159] The model parameters of the enhanced image processing sub-model are adjusted based on the enhanced sub-model loss value and the segmentation difference loss value;

[0160] The model parameters of the target image processing sub-model are adjusted based on the target sub-model loss value, classification difference loss value, segmentation difference loss value, and feature information difference loss value.

[0161] In practical applications, the model loss values ​​include the enhancement sub-model loss value and the target sub-model loss value. During the process of adjusting the parameters of the initial image processing model based on each loss value, the model parameters of the enhancement image processing sub-model are adjusted according to the enhancement sub-model loss value and the segmentation difference loss value; the model parameters of the target image processing sub-model are adjusted according to the target sub-model loss value, the classification difference loss value, the segmentation difference loss value, and the feature information difference loss value.

[0162] In the steps of the above implementation method, it can be determined that the classification difference loss value is expressed as L. KL The segmentation difference loss value is represented as L. disc The feature information difference loss value is represented as L. alig In addition, the loss value of the augmented sub-model includes the classification loss value L of the augmented sub-model. clas1 And the segmentation loss value L of the enhanced sub-model seg1 The target sub-model loss value includes the target sub-model classification loss value L. clas2The segmentation loss value L of the target sub-model seg2 .

[0163] The loss value of the enhanced image processing sub-model is given by Formula 4 below, and the loss value of the target image processing sub-model is given by Formula 5 below: L Total1 =L clas1 +L seg1 +λ3L disc Formula 4 L Total2 =L clas2 +L seg2 +λ1L KL +λ2L alig +λ3L disc Formula 5

[0164] Among them, L Total1 L represents the loss value of the augmented submodel. Total2 λ1, λ2, and λ3 represent the target sub-model loss values, and λ1, λ2, and λ3 are the weights of each loss value. In a specific embodiment provided in this specification, λ1, λ2, and λ3 are set to 0.25, 10, and 0.1, respectively.

[0165] After adjusting the model parameters of the enhanced image processing sub-model and the target image processing sub-model, the above steps can be repeated to continue training the initial image processing model until the model training stopping condition is met. In practical applications, the model training stopping condition for the initial image processing model includes:

[0166] The total loss value is less than the preset threshold, and / or the number of training rounds reaches the preset number of training rounds.

[0167] Specifically, during the training of the initial image processing model, the training stopping condition can be set to the total loss value, i.e., L. Total1 and L Total2 Less than a preset threshold, in practical applications, when L Total1 If the value is less than the preset threshold, there is no need to adjust the model parameters of the enhanced image processing sub-model. Total2 If the value is less than the preset threshold, there is no need to adjust the model parameters of the target image processing sub-model.

[0168] Furthermore, the training stopping condition of the initial image processing model can be set to the number of training rounds reaching a preset number. For example, if the preset number of training rounds is 10, then the training stopping condition is reached when the model has reached 10 training rounds.

[0169] The method provided in this specification does not impose any restrictions on the stopping conditions for model training. When the initial image processing model reaches the stopping conditions, it indicates that the initial image processing model training is complete, and a reference image processing model is obtained.

[0170] Step 110: Generate an image processing model based on the target image processing sub-model in the reference image processing model.

[0171] The reference image processing model at this point is not the final image processing model to be trained. In the method provided in this specification, the ultimate goal is to obtain an image processing model capable of processing the target image. The model structure of the reference image processing model is the same as that of the initial image processing model, including both an enhanced image processing sub-model and a target image processing sub-model. To ensure that the final image processing model processes only the target image, the image processing model can be obtained from the target image processing sub-model in the reference image processing model. Specifically, the enhanced image processing sub-model in the reference image processing model is deleted, and the target image processing sub-model is retained to obtain the final image processing model.

[0172] The image processing model training method provided in the embodiments of this specification offers a framework based on mutual learning between two sub-models. This framework unifies the classification and segmentation tasks for two types of sample images (sample augmented images and sample target images). Knowledge is used to transfer knowledge from the augmented image processing sub-model to the target image processing sub-model, improving the performance of the image processing model in target image processing. Furthermore, the structuring of the segmentation and classification tasks increases the interpretability of image recognition.

[0173] Secondly, in the specific implementation provided in this specification, an inter-class feature alignment strategy is designed. By capturing the similarity of paired spatial features through affinity graphs, spatial continuity is enhanced under the guidance of connection range and granularity parameters. This promotes mutual learning and transfer from the enhanced image processing sub-model to the target image processing sub-model path network, effectively integrating visual features from different paths, and making the recognition of the target detection region more accurate.

[0174] Furthermore, in the specific implementation methods provided in the embodiments of this specification, an intra-class feature differentiation strategy is proposed. Through the design of dense center loss, a clearer division is generated in the feature space, and the abnormal regions of the target detection area are accurately segmented for each single-path network in complex backgrounds, thereby improving the segmentation accuracy.

[0175] Referring to Figure 3, Figure 3 shows a schematic diagram of the structure for training an image processing model for pulmonary embolism detection provided in an embodiment of this specification.

[0176] In this embodiment, the training of an image detection model for pulmonary embolism is used as an example for explanation. First, multiple training sample pairs meeting image quality requirements are collected from the hospital. These training sample pairs include enhanced CT images and plain CT images of the pulmonary artery. The enhanced CT images are input into the image labeling model for recognition, obtaining voxel-level pulmonary embolism segmentation masks and classification labels. Based on the pulmonary embolism segmentation masks and classification labels from the enhanced CT images, the plain CT images are labeled, obtaining voxel-level pulmonary embolism segmentation masks and classification labels on the plain CT images.

[0177] Voxel level refers to the level of detail or refinement of the basic unit considered or manipulated in 3D data processing, analysis, or imaging. A voxel is short for volume element and is the smallest unit of division in 3D space. A solid containing voxels can be represented through stereo rendering or by extracting polygonal isosurfaces from a given threshold contour. Voxels are used in fields such as 3D imaging, scientific data, and medical imaging. Conceptually, they are similar to pixels, the smallest unit in 2D space.

[0178] The lungmask Python library was used to extract the maximum external 3D matrix of lung regions contained in enhanced and plain CT images. Training samples were divided into training and test sets in an 8:2 ratio to ensure a consistent ratio of positive to negative cases in both sets. lungmask Python is a Python library primarily used in medical image processing, particularly for the automatic segmentation of lung regions in chest CT images.

[0179] A multi-task learning framework for pulmonary embolism CT image classification and segmentation is constructed using an encoder based on EfficientNet as the backbone network and a decoder based on U-net, as shown in Figure 3. This framework includes an enhanced CT pathway network and a plain CT pathway network. In the method provided in the embodiments of this specification, the proposed inter-period mutual learning framework aims to simultaneously train two convolutional neural networks with different tasks (classification and segmentation). The enhanced CT pathway network is responsible for classifying and segmenting enhanced CT images, while the plain CT pathway network is responsible for classifying and segmenting plain CT images.

[0180] EfficientNet is a deep learning model that uses a convolutional neural network (CNN) architecture. It systematically expands the model's depth, width, and input resolution through a compound scaling method to achieve higher prediction accuracy while maintaining high computational efficiency.

[0181] In the framework provided in this manual, both the enhanced CT pathway network and the plain CT pathway network use the 3D version of EfficientNet as the backbone network, and the U-net 3D version decoder is selected as the decoder for the segmentation task. A classifier is added after the backbone network for classification. The architecture of the enhanced CT pathway network and the plain CT pathway network is the same, and subsequent processing of the pathway networks does not distinguish between the enhanced CT pathway network and the plain CT pathway network.

[0182] In the application, the input CT 3D image is randomly cropped into 224*224*96 dimensional segmented 3D images. Choosing the 224*224*96 dimensional size ensures that each 3D image includes the pulmonary embolism mask region, which guarantees that the model can identify the pulmonary embolism region. At the same time, it can reduce the volume of the image input into the model and reduce the amount of data processing required by the image processing model.

[0183] The segmented 3D image is input into the encoder of the pathway network (enhanced CT image is input into the first encoder, and plain CT image is input into the second encoder) to obtain at least one image coding feature information E = {E1, E2, E3, E4, E5} corresponding to the segmented 3D image, where E1-E5 are the image coding feature information output by each coding layer in the encoder. E5 is the image coding feature information output by the last coding layer. E5 can also be understood as classification coding feature information.

[0184] The E5 value is input into a classifier for processing. The logits obtained by the classifier represent the probability that the segmented 3D image is an embolic case or a normal case, i.e., the predicted classification information. The logits obtained from the enhanced CT pathway network and the plain CT pathway network are aligned and predicted to form a distribution. The KL divergence is used as a measure of the difference between logits, and the classification difference loss value is calculated using the above formula 1.

[0185] The encoded feature information E of each image is input into the decoder for decoding processing to obtain the segmented output mask. This means the probability that each pixel in the image represents a pulmonary embolism. To improve the ability of the segmentation network to distinguish between background and pulmonary embolism features, the method provided in this specification proposes an intra-class feature differentiation strategy based on a dense center loss function. In each training iteration, the center is calculated as the centroid feature of the pixels belonging to the corresponding category in the segmentation mask, and the segmentation difference loss value is calculated using the above formula 2.

[0186] In addition, the method provided in this specification also focuses on the similarity between the spatial features of the enhanced CT pathway network and the plain CT pathway network. Therefore, image-coded feature information used to calculate the feature information difference loss value is determined from at least one image-coded feature information. For example, {E1, E2, E3} is selected from at least one image-coded feature information E = {E1, E2, E3, E4, E5} for calculation. In the method provided in this specification, during the calculation of the feature information difference loss value, an affinity map is constructed to encapsulate the similarity between the spatial features of the enhanced CT pathway network and the plain CT pathway network, and the feature information difference loss value is calculated using Formula 3 above. During training, it is only desired that the features of the plain CT pathway network are close to those of the enhanced CT pathway network; therefore, the feature information difference loss value is used to update the model parameters of the plain CT pathway network.

[0187] Based on this, each predicted segmentation and predicted classification information is compared with the annotation information on the sample image to calculate the model loss value of the enhanced CT pathway network and the model loss value of the plain CT pathway network.

[0188] Using formulas 4 and 5 above, the model parameters of the enhanced CT pathway network and the plain CT pathway network are adjusted respectively, and training continues until the model training stops, thus obtaining the reference image processing model corresponding to the initial image processing model.

[0189] Based on the obtained reference image processing model, the enhanced CT pathway network was removed from the reference image processing model, retaining only the plain CT pathway network. This yielded an image processing model for auxiliary diagnosis and treatment of pulmonary embolism using plain CT images.

[0190] In the method provided in this manual, after obtaining the image processing model, three doctors with experience in cardiopulmonary imaging were invited to compare it with the model. 289 plain CT images from the test set were given to each of the three doctors, who were asked to determine the presence of pulmonary embolism based on the CT images. These 289 plain CT images were then input into the image processing model for pulmonary embolism annotation and identification. After testing, the image processing model's identification results for the plain CT images in the test set were higher than the conclusions given by the three doctors. Thus, the image processing model capable of identifying pulmonary embolism in plain CT images and obtaining diagnostic and treatment results was successfully trained.

[0191] The model training method for image processing models in pulmonary embolism scenarios provided in the embodiments of this specification proposes a novel cross-period mutual learning framework, unifying the classification and segmentation tasks of pulmonary embolism across two phases (enhanced CT and plain CT). By transferring knowledge from enhanced CT to plain CT, the model's performance on plain CT scans is improved. Simultaneously, the classification and segmentation tasks are decoupled, increasing the interpretability of image recognition results.

[0192] Secondly, the embodiments in this specification involve an inter-class feature alignment (IFA) strategy, which captures the similarity of paired spatial features through affinity graphs, enhances spatial continuity under the guidance of connection range and granularity parameters, promotes mutual learning and transfer of networks from enhanced CT to plain CT, and effectively integrates visual features from different paths, making pulmonary embolism identification more accurate.

[0193] In addition, the architecture provided in the embodiments of this specification also proposes an intra-class feature differentiation (IFD) strategy, which generates a clearer partition in the feature space through a designed dense center loss, and accurately segments pulmonary embolism for each single-path network in complex backgrounds, thereby improving the segmentation accuracy.

[0194] Referring to Figure 4, which shows a flowchart of an image processing method provided in an embodiment of this specification, the method specifically includes the following steps:

[0195] Step 402: Receive an image processing task, wherein the image processing task carries multiple target images corresponding to the target detection region, and the image processing task is used to detect whether there are any anomalies in the target detection region.

[0196] In this context, the image processing task can be understood as a task to detect whether anomalies exist within a target detection region. The image processing task carries multiple target images corresponding to the target detection region. Further, the target detection region can be understood as a partition used to predict whether anomalies exist. For example, the target detection region can be any organ, such as the liver, spleen, lungs, stomach, etc. By predicting whether anomalies exist in the target detection region, the prediction results can further assist in judging the state of the object to be detected, thus helping to determine the state of the object. Detecting whether anomalies exist in the target detection region can be understood as detecting whether a tumor exists in an organ, or whether an embolism exists in a blood vessel, etc.

[0197] It should be noted that, in one or more embodiments of this specification, the image processing task can be used to identify plain CT images and determine whether there are abnormalities in the target detection area in the plain CT image based on image features. For example, in the scenario of pulmonary embolism diagnosis, the plain CT image of the lungs can be segmented and classified to predict whether pulmonary embolism has occurred, thereby assisting doctors in judging whether there are abnormalities in the target detection area, which facilitates subsequent treatment.

[0198] In one specific embodiment provided in this specification, the detection of pulmonary embolism is used as an example for explanation. Plain CT images of the lungs are acquired; that is, multiple target images are plain CT images including the lung region. These multiple plain CT images form a 3D image of the lungs. Image detection is performed on these multiple plain CT images to detect whether pulmonary embolism has occurred.

[0199] Step 404: Input the plurality of target images into the image processing model to obtain the detection results of the target detection region output by the image processing model, wherein the image processing model is trained by the above-described image processing model training method.

[0200] In practical applications, after receiving an image processing task, multiple target images carried by the task are obtained. These target images are then input into an image processing model for processing. The model outputs the detection results corresponding to the target detection regions. The detection results specifically include information such as whether anomalies occur in the target detection region and the probability of such anomalies.

[0201] It should be noted that the image processing model in the image processing method of the embodiments in this specification is trained using the training method of the image processing model in the above embodiments. This image processing model can detect not only target images containing all target detection regions, but also target images containing only some target detection regions.

[0202] In one specific embodiment provided in this specification, receiving an image processing task includes:

[0203] Receive an image processing request sent by a user, wherein the image processing request carries an image processing task;

[0204] Accordingly, the method further includes:

[0205] The detection results are sent to the user.

[0206] Specifically, image processing tasks can be triggered locally on the terminal. This means multiple target images are stored locally on the terminal, and the terminal triggers the image processing task to initiate subsequent image processing steps. Alternatively, image processing tasks can be sent by the user. The terminal receives an image processing request from the user, which includes the image processing task. After processing the image using the image processing method mentioned in the above embodiments and obtaining the detection result, the terminal returns the detection result to the user so that the user can perform appropriate subsequent processing based on the result.

[0207] The image processing method provided by the embodiments of this specification offers a scanning and screening scheme for target images corresponding to target detection areas. This enables better identification of abnormal information in the target detection area during the target image detection process, achieving high sensitivity in target image processing and improving the accuracy of detection results.

[0208] Referring to Figure 5, Figure 5 shows a flowchart of a CT image processing method provided in an embodiment of this specification, which specifically includes the following steps:

[0209] Step 502: Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to the target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area.

[0210] Step 504: Input the plurality of plain CT images into the CT image processing model to obtain the detection results of the target detection region output by the image processing model, wherein the CT image processing model is trained by the above-mentioned image processing model training method.

[0211] It should be noted that the implementation methods of steps 502 to 504 are the same as those of steps 402 to 404 above, and will not be repeated in the embodiments of this specification.

[0212] Specifically, in the method provided in this embodiment, the detection of pulmonary embolism is taken as an example of CT image processing task for further explanation. Based on this, the CT image processing task includes plain CT images of the lung region. The plain CT images are input into the CT image processing model for recognition. The CT image processing model can identify whether there is embolism in the pulmonary artery in the plain CT image, which solves the problem that the current algorithm cannot detect pulmonary embolism based on plain CT images and lowers the threshold for pulmonary embolism detection.

[0213] Corresponding to the above method embodiments, this specification also provides an embodiment of an image processing model training device. Figure 6 shows a schematic diagram of the structure of an image processing model training device provided in one embodiment of this specification. As shown in Figure 6, the device includes:

[0214] The acquisition module 602 is configured to acquire multiple training sample pairs, wherein the training sample pairs include sample augmentation images and sample target images, the sample augmentation images include first sample annotation information for the target detection region, and the sample target images include second sample annotation information for the target detection region;

[0215] The prediction module 604 is configured to input the sample augmented image and the sample target image into an initial image processing model to obtain first prediction annotation information and second prediction annotation information output by the initial image processing model. The initial image processing model includes an augmented image processing sub-model and a target image processing sub-model. The augmented image processing sub-model outputs the first prediction annotation information based on the sample augmented image, and the target image processing sub-model outputs the second prediction annotation information based on the sample target image.

[0216] The calculation module 606 is configured to calculate the model loss value based on the first sample annotation information, the second sample annotation information, the first prediction annotation information, and the second prediction annotation information;

[0217] The parameter tuning module 608 is configured to adjust the model parameters of the initial image processing model according to the model loss value, and continue training the initial image processing model until the model training stop condition is met, thereby obtaining a reference image processing model.

[0218] The generation module 610 is configured to generate an image processing model based on the target image processing sub-model in the reference image processing model.

[0219] Optionally, the acquisition module 602 is further configured to:

[0220] Identify the sample detection object and obtain the sample enhancement image and sample target image corresponding to the sample detection object, wherein the sample enhancement image and sample target image corresponding to the sample detection object correspond to the same target detection region of the sample detection object;

[0221] First sample annotation information is generated on the enhanced sample image for the target detection region;

[0222] The second sample annotation information is annotated on the sample target image based on the first sample annotation information.

[0223] Optionally, the acquisition module 602 is further configured to:

[0224] The enhanced image of the sample is input into the information annotation model to obtain the first sample segmentation information and the first sample classification information for the target detection region output by the information annotation model;

[0225] Accordingly, based on the first sample annotation information, second sample annotation information is annotated on the sample target image, including:

[0226] Based on the first sample segmentation information, second sample segmentation information is marked on the sample target image, and based on the first sample classification information, the second sample classification information of the sample target image is determined.

[0227] Optionally, the prediction module 604 is further configured to:

[0228] The sample augmentation image and the sample target image are input into the initial image processing model to obtain the first prediction annotation information, the second prediction annotation information, the classification difference loss value, the segmentation difference loss value and the feature information difference loss value output by the initial image processing model.

[0229] Accordingly, the parameter tuning module 608 is further configured as follows:

[0230] The model parameters of the initial image processing model are adjusted based on the model loss value, classification difference loss value, segmentation difference loss value, and feature information difference loss value.

[0231] Optionally, the prediction module 604 is further configured to:

[0232] The enhanced image of the sample is input into the enhanced image processing sub-model of the initial image processing model to obtain the first predicted classification information, the first predicted segmentation information, and the first image coding feature information.

[0233] The sample target image is input into the target image processing sub-model of the initial image processing model to obtain second predicted classification information, second predicted segmentation information, and second image coding feature information;

[0234] Calculate the classification difference loss value based on the first predicted classification information and the second predicted classification information;

[0235] Calculate the segmentation difference loss value based on the first predicted segmentation information and the second predicted segmentation information;

[0236] The feature information difference loss value is calculated based on the first image coding feature information and the second image coding feature information.

[0237] Optionally, the enhanced image processing sub-model includes a first encoder, a first decoder, and a first classifier;

[0238] The prediction module 604 is further configured as follows:

[0239] The sample augmented image is input to the first encoder to obtain at least one augmented image coding feature information, wherein the at least one augmented image coding feature information includes classification augmented coding feature information;

[0240] The classification enhancement coding feature information is input into the first classifier to obtain the first predicted classification information;

[0241] The encoded feature information of each enhanced image is input into the first decoder to obtain the first predicted segmentation information;

[0242] The first image coding feature is determined from at least one enhanced image coding feature.

[0243] Optionally, the target image processing sub-model includes a second encoder, a second decoder, and a second classifier;

[0244] The prediction module 604 is further configured as follows:

[0245] The sample target image is input to the second encoder to obtain at least one target image coding feature information, wherein the at least one target image coding feature information includes classification target coding feature information;

[0246] The classification target encoded feature information is input into the second classifier to obtain the second predicted classification information;

[0247] The encoded feature information of each target image is input into the second decoder to obtain the second predicted segmentation information;

[0248] Determine the second image coding feature information from at least one target image coding feature information.

[0249] Optionally, the model loss value includes the enhancement sub-model loss value and the target sub-model loss value;

[0250] The parameter tuning module 608 is further configured as follows:

[0251] The model parameters of the enhanced image processing sub-model are adjusted based on the enhanced sub-model loss value and the segmentation difference loss value;

[0252] The model parameters of the target image processing sub-model are adjusted based on the target sub-model loss value, classification difference loss value, segmentation difference loss value, and feature information difference loss value.

[0253] Optionally, the first sample annotation information includes first sample segmentation information and first sample classification information, the first prediction annotation information includes first prediction segmentation information and first prediction classification information, the second sample annotation information includes second sample segmentation information and second sample classification information, and the second prediction annotation information includes second prediction segmentation information and second prediction classification information.

[0254] The computing module 606 is further configured as follows:

[0255] Calculate the classification loss value of the enhanced sub-model based on the classification information of the first sample and the first predicted classification information;

[0256] Calculate the segmentation loss value of the enhanced sub-model based on the segmentation information of the first sample and the segmentation information of the first prediction.

[0257] The loss value of the augmented submodel is determined based on the classification loss value and the segmentation loss value of the augmented submodel.

[0258] Calculate the classification loss value of the target sub-model based on the classification information of the second sample and the second predicted classification information;

[0259] Calculate the target sub-model segmentation loss value based on the second sample segmentation information and the second predicted segmentation information;

[0260] The target sub-model loss value is determined based on the target sub-model classification loss value and the target sub-model segmentation loss value.

[0261] Optionally, the prediction module 604 is further configured to:

[0262] The sample enhancement image and the sample target image are cropped based on a preset cropping size;

[0263] The cropped sample augmented image and the sample target image are input into the initial image processing model.

[0264] The image processing model training apparatus provided in the embodiments of this specification offers a framework based on mutual learning between two sub-models. This framework unifies the classification and segmentation tasks for two types of sample images (sample augmented images and sample target images). Knowledge is used to transfer knowledge from the augmented image processing sub-model to the target image processing sub-model, improving the performance of the image processing model in target image processing. Furthermore, the structuring of the segmentation and classification tasks increases the interpretability of image recognition.

[0265] Secondly, in the specific implementation provided in this specification, an inter-class feature alignment strategy is designed. By capturing the similarity of paired spatial features through affinity graphs, spatial continuity is enhanced under the guidance of connection range and granularity parameters. This promotes mutual learning and transfer from the enhanced image processing sub-model to the target image processing sub-model path network, effectively integrating visual features from different paths, and making the recognition of the target detection region more accurate.

[0266] Furthermore, in the specific implementation methods provided in the embodiments of this specification, an intra-class feature differentiation strategy is proposed. Through the design of dense center loss, a clearer division is generated in the feature space, and the abnormal regions of the target detection area are accurately segmented for each single-path network in complex backgrounds, thereby improving the segmentation accuracy.

[0267] The above is a schematic scheme of an image processing model training device according to this embodiment. It should be noted that the technical solution of this image processing model training device and the technical solution of the image processing model training method described above belong to the same concept. For details not described in detail in the technical solution of the image processing model training device, please refer to the description of the technical solution of the image processing model training method described above.

[0268] Corresponding to the above method embodiments, this specification also provides embodiments of a CT image processing apparatus. Figure 7 shows a schematic diagram of the structure of a CT image processing apparatus provided in one embodiment of this specification. As shown in Figure 7, the apparatus includes:

[0269] The receiving module 702 is configured to receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to a target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area.

[0270] The prediction module 704 is configured to input the plurality of plain CT images into a CT image processing model to obtain the detection result of the target detection region output by the image processing model, wherein the CT image processing model is trained by the above-mentioned image processing model training method.

[0271] The above is a schematic scheme of a CT image processing device according to this embodiment. It should be noted that the technical solution of this CT image processing device and the technical solution of the CT image processing method described above belong to the same concept. For details not described in detail in the technical solution of the CT image processing device, please refer to the description of the technical solution of the CT image processing method described above.

[0272] Figure 8 is a schematic diagram of a training method for an image processing model applied to a cloud-side device according to an embodiment of this application. As shown in Figure 8, the method includes:

[0273] Step 802: Obtain multiple training sample pairs, wherein each training sample pair includes a sample augmentation image and a sample target image. The sample augmentation image includes first sample annotation information for the target detection region, and the sample target image includes second sample annotation information for the target detection region.

[0274] Step 804: Input the sample augmented image and the sample target image into the initial image processing model to obtain the first prediction annotation information and the second prediction annotation information output by the initial image processing model. The initial image processing model includes an augmented image processing sub-model and a target image processing sub-model. The augmented image processing sub-model outputs the first prediction annotation information based on the sample augmented image, and the target image processing sub-model outputs the second prediction annotation information based on the sample target image.

[0275] Step 806: Calculate the model loss value based on the first sample annotation information, the second sample annotation information, the first prediction annotation information, and the second prediction annotation information.

[0276] Step 808: Adjust the model parameters of the initial image processing model according to the model loss value, and continue training the initial image processing model until the model training stop condition is met to obtain the reference image processing model.

[0277] Step 810: Based on the target image processing sub-model in the reference image processing model, generate an image processing model, obtain the model parameters of the image processing model, and send the model parameters of the image processing model to the end device.

[0278] It should be noted that the implementation methods of steps 802 to 810 are the same as those of steps 102 to 110 above, and will not be repeated in the embodiments of this specification.

[0279] In practical applications, training a model requires a large amount of data and significant computing resources, which edge devices may lack. Therefore, the model training process can be implemented on cloud devices. After obtaining the model parameters of the image processing model, the cloud device can send these parameters to the edge device. The edge device can then build the image processing model locally based on these parameters and further utilize the model for image processing.

[0280] The image processing model training method provided in the embodiments of this specification offers a framework based on mutual learning between two sub-models. This framework unifies the classification and segmentation tasks for two types of sample images (sample augmented images and sample target images). Knowledge is used to transfer knowledge from the augmented image processing sub-model to the target image processing sub-model, improving the performance of the image processing model in target image processing. Furthermore, the structuring of the segmentation and classification tasks increases the interpretability of image recognition.

[0281] Secondly, in the specific implementation provided in this specification, an inter-class feature alignment strategy is designed. By capturing the similarity of paired spatial features through affinity graphs, spatial continuity is enhanced under the guidance of connection range and granularity parameters. This promotes mutual learning and transfer from the enhanced image processing sub-model to the target image processing sub-model path network, effectively integrating visual features from different paths, and making the recognition of the target detection region more accurate.

[0282] Furthermore, in the specific implementation methods provided in the embodiments of this specification, an intra-class feature differentiation strategy is proposed. Through the design of dense center loss, a clearer division is generated in the feature space, and the abnormal regions of the target detection area are accurately segmented for each single-path network in complex backgrounds, thereby improving the segmentation accuracy.

[0283] Figure 9 is a flowchart illustrating an image processing method applied to a cloud-side device according to an embodiment of this application, specifically including:

[0284] Step 902: Receive an image processing task sent by the receiving end device, wherein the image processing task carries multiple target images corresponding to the target detection area, and the image processing task is used to detect whether there is an anomaly in the target detection area.

[0285] Step 904: Input the plurality of target images into the image processing model to obtain the detection results of the target detection region output by the image processing model, wherein the image processing model is trained by the above-described image processing model training method.

[0286] Step 906: Send the detection result to the end-side device.

[0287] Figure 10 is a flowchart illustrating a method for detecting vascular embolism according to an embodiment of this application, specifically including:

[0288] Step 1002: Receive a vascular embolism detection task, wherein the vascular embolism detection task carries multiple plain CT images corresponding to a vascular region, and the vascular embolism detection task is used to detect whether there is an embolism in the vascular region.

[0289] Step 1004: Input the multiple plain CT images into the image processing model to obtain the detection result of whether there is vascular embolism in the vascular region output by the image processing model, wherein the image processing model is trained by the above-mentioned image processing model training method.

[0290] In this embodiment, the image processing model is trained to detect the classification results of vascular embolism.

[0291] The vascular embolism detection method provided in this embodiment can be applied to screening for vascular embolism in arteries and veins. It provides guidance for doctors and helps improve their diagnostic accuracy.

[0292] Figure 11 is a schematic flowchart of a tumor screening method provided in an embodiment of this application, specifically including:

[0293] Step 1102: Receive a tumor screening task, wherein the tumor screening task carries multiple plain CT images corresponding to the target detection area, and the tumor screening task is used to detect whether there is a tumor in the target detection area.

[0294] Step 1104: Input the multiple plain CT images into the image processing model to obtain the detection result of whether there is a tumor in the target detection area output by the image processing model, wherein the image processing model is trained by the above-mentioned image processing model training method.

[0295] The tumor screening method provided in this embodiment can be applied to the screening of pancreatic cancer, esophageal cancer, liver cancer, lung cancer, breast cancer, colorectal cancer, stomach cancer, lymphoma, and other tumors. It can screen for the presence of tumors in various organs throughout the body, including the pancreas, esophagus, liver, lungs, breast, intestines, stomach, and lymph nodes. It provides guidance for doctors and helps improve their diagnostic accuracy.

[0296] Figure 12 shows a structural block diagram of a computing device 1200 according to an embodiment of this specification. The components of the computing device 1200 include, but are not limited to, a memory 1210 and a processor 1220. The processor 1220 is connected to the memory 1210 via a bus 1230, and a database 1250 is used to store data.

[0297] The computing device 1200 also includes an access device 1240, which enables the computing device 1200 to communicate via one or more networks 1260. Examples of such networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1240 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0298] In one embodiment of this specification, the aforementioned components of the computing device 1200, as well as other components not shown in FIG. 12, may be interconnected, for example, via a bus. It should be understood that the block diagram of the computing device shown in FIG. 12 is merely illustrative and not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0299] The computing device 1200 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1200 can also be a mobile or stationary server.

[0300] The processor 1220 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-mentioned image processing model training method or image processing method.

[0301] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the computing device embodiments are basically similar to the training method or image processing method embodiments of the image processing model described above, so the description is relatively simple. Relevant parts can be referred to the descriptions of the training method or image processing method embodiments of the image processing model described above.

[0302] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described image processing model training method or image processing method.

[0303] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer-readable storage medium embodiments are relatively simple in description because they are substantially similar to the above-described image processing model training method or image processing method embodiments. Relevant parts can be referred to the descriptions of the above-described image processing model training method or image processing method embodiments.

[0304] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described image processing model training method or image processing method.

[0305] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solution of the image processing model training method or image processing method described above. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the image processing model training method or image processing method described above.

[0306] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0307] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0308] It should be noted that the above description describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification.

[0309] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0310] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A training method for an image processing model, comprising: Multiple training sample pairs are obtained, wherein each training sample pair includes a sample augmentation image and a sample target image, the sample augmentation image includes first sample annotation information for the target detection region, and the sample target image includes second sample annotation information for the target detection region; The sample augmented image and the sample target image are input into the initial image processing model to obtain the first prediction annotation information and the second prediction annotation information output by the initial image processing model. The initial image processing model includes an augmented image processing sub-model and a target image processing sub-model. The augmented image processing sub-model outputs the first prediction annotation information based on the sample augmented image, and the target image processing sub-model outputs the second prediction annotation information based on the sample target image. The model loss value is calculated based on the first sample annotation information, the second sample annotation information, the first prediction annotation information, and the second prediction annotation information. Adjust the model parameters of the initial image processing model based on the model loss value, and continue training the initial image processing model until the model training stop condition is met to obtain a reference image processing model; An image processing model is generated based on the target image processing sub-model in the reference image processing model.

2. The method of claim 1, wherein obtaining multiple training sample pairs includes: Identify the sample detection object and obtain the sample enhancement image and sample target image corresponding to the sample detection object, wherein the sample enhancement image and sample target image corresponding to the sample detection object correspond to the same target detection region of the sample detection object; First sample annotation information is generated on the enhanced sample image for the target detection region; The second sample annotation information is annotated on the sample target image based on the first sample annotation information.

3. The method of claim 2, wherein generating first sample annotation information for the target detection region on the sample augmentation image includes: The enhanced image of the sample is input into the information annotation model to obtain the first sample segmentation information and the first sample classification information for the target detection region output by the information annotation model; Accordingly, based on the first sample annotation information, second sample annotation information is annotated on the sample target image, including: Based on the first sample segmentation information, second sample segmentation information is marked on the sample target image, and based on the first sample classification information, the second sample classification information of the sample target image is determined.

4. The method as described in claim 1, wherein the sample augmentation image and the sample target image are input into an initial image processing model to obtain first prediction annotation information and second prediction annotation information output by the initial image processing model, comprising: The sample augmentation image and the sample target image are input into the initial image processing model to obtain the first prediction annotation information, the second prediction annotation information, the classification difference loss value, the segmentation difference loss value and the feature information difference loss value output by the initial image processing model. Accordingly, the model parameters of the initial image processing model are adjusted based on the model loss value, including: The model parameters of the initial image processing model are adjusted based on the model loss value, classification difference loss value, segmentation difference loss value, and feature information difference loss value.

5. The method as described in claim 4, wherein the sample augmentation image and the sample target image are input into an initial image processing model to obtain first prediction annotation information, second prediction annotation information, classification difference loss value, segmentation difference loss value, and feature information difference loss value output by the initial image processing model, comprising: The enhanced image of the sample is input into the enhanced image processing sub-model of the initial image processing model to obtain the first predicted classification information, the first predicted segmentation information, and the first image coding feature information. The sample target image is input into the target image processing sub-model of the initial image processing model to obtain second predicted classification information, second predicted segmentation information, and second image coding feature information; Calculate the classification difference loss value based on the first predicted classification information and the second predicted classification information; Calculate the segmentation difference loss value based on the first predicted segmentation information and the second predicted segmentation information; The feature information difference loss value is calculated based on the first image coding feature information and the second image coding feature information.

6. The method of claim 5, wherein the enhanced image processing sub-model comprises a first encoder, a first decoder, and a first classifier; The enhanced image of the sample is input into the enhanced image processing sub-model of the initial image processing model to obtain first predicted classification information, first predicted segmentation information, and first image coding feature information, including: The sample augmented image is input to the first encoder to obtain at least one augmented image coding feature information, wherein the at least one augmented image coding feature information includes classification augmented coding feature information; The classification enhancement coding feature information is input into the first classifier to obtain the first predicted classification information; The encoded feature information of each enhanced image is input into the first decoder to obtain the first predicted segmentation information; The first image coding feature is determined from at least one enhanced image coding feature.

7. The method as described in claim 5, wherein the target image processing sub-model comprises a second encoder, a second decoder, and a second classifier; The sample target image is input into the target image processing sub-model of the initial image processing model to obtain second predicted classification information, second predicted segmentation information, and second image coding feature information, including: The sample target image is input to the second encoder to obtain at least one target image coding feature information, wherein the at least one target image coding feature information includes classification target coding feature information; The classification target encoded feature information is input into the second classifier to obtain the second predicted classification information; The encoded feature information of each target image is input into the second decoder to obtain the second predicted segmentation information; Determine the second image coding feature information from at least one target image coding feature information.

8. The method of claim 4, wherein the model loss value includes the enhancement sub-model loss value and the target sub-model loss value; Adjusting the model parameters of the initial image processing model based on the model loss value, classification difference loss value, segmentation difference loss value, and feature information difference loss value includes: The model parameters of the enhanced image processing sub-model are adjusted based on the enhanced sub-model loss value and the segmentation difference loss value; The model parameters of the target image processing sub-model are adjusted based on the target sub-model loss value, classification difference loss value, segmentation difference loss value, and feature information difference loss value.

9. The method as described in claim 1, wherein the first sample annotation information includes first sample segmentation information and first sample classification information, the first prediction annotation information includes first prediction segmentation information and first prediction classification information, the second sample annotation information includes second sample segmentation information and second sample classification information, and the second prediction annotation information includes second prediction segmentation information and second prediction classification information; The model loss value is calculated based on the first sample annotation information, the second sample annotation information, the first prediction annotation information, and the second prediction annotation information, including: Calculate the classification loss value of the enhanced sub-model based on the classification information of the first sample and the first predicted classification information; Calculate the segmentation loss value of the enhanced sub-model based on the segmentation information of the first sample and the segmentation information of the first prediction. The loss value of the augmented submodel is determined based on the classification loss value and the segmentation loss value of the augmented submodel. Calculate the classification loss value of the target sub-model based on the classification information of the second sample and the second predicted classification information; Calculate the target sub-model segmentation loss value based on the second sample segmentation information and the second predicted segmentation information; The target sub-model loss value is determined based on the target sub-model classification loss value and the target sub-model segmentation loss value.

10. The method of claim 1, wherein the sample augmentation image and the sample target image are input into an initial image processing model, comprising: The sample enhancement image and the sample target image are cropped based on a preset cropping size; The cropped sample augmented image and the sample target image are input into the initial image processing model.

11. An image processing method, comprising: Receive an image processing task, wherein the image processing task carries multiple target images corresponding to a target detection region, and the image processing task is used to detect whether there is an anomaly in the target detection region; The plurality of target images are input into an image processing model to obtain the detection results of the target detection region output by the image processing model, wherein the image processing model is trained by the training method of any one of claims 1-10.

12. The method of claim 11, wherein receiving an image processing task includes: Receive an image processing request sent by a user, wherein the image processing request carries an image processing task; Accordingly, the method further includes: The detection results are sent to the user.

13. A CT image processing method, comprising: Receive a CT image processing task, wherein the CT image processing task carries multiple plain CT images corresponding to a target detection area, and the CT image processing task is used to detect whether there is an abnormality in the target detection area; The plurality of plain CT images are input into a CT image processing model to obtain the detection results of the target detection region output by the image processing model, wherein the CT image processing model is trained by the training method described in any one of claims 1-10.

14. A training method for an image processing model, applied to a cloud-based device, comprising: Multiple training sample pairs are obtained, wherein each training sample pair includes a sample augmentation image and a sample target image, the sample augmentation image includes first sample annotation information for the target detection region, and the sample target image includes second sample annotation information for the target detection region; The sample augmented image and the sample target image are input into the initial image processing model to obtain the first prediction annotation information and the second prediction annotation information output by the initial image processing model. The initial image processing model includes an augmented image processing sub-model and a target image processing sub-model. The augmented image processing sub-model outputs the first prediction annotation information based on the sample augmented image, and the target image processing sub-model outputs the second prediction annotation information based on the sample target image. The model loss value is calculated based on the first sample annotation information, the second sample annotation information, the first prediction annotation information, and the second prediction annotation information. Adjust the model parameters of the initial image processing model based on the model loss value, and continue training the initial image processing model until the model training stop condition is met to obtain a reference image processing model; Based on the target image processing sub-model in the reference image processing model, an image processing model is generated, the model parameters of the image processing model are obtained, and the model parameters of the image processing model are sent to the end device.

15. An image processing method applied to a cloud-side device, comprising: The receiving end device sends an image processing task, wherein the image processing task carries multiple target images corresponding to the target detection area, and the image processing task is used to detect whether there is an anomaly in the target detection area; The plurality of target images are input into an image processing model to obtain the detection results of the target detection region output by the image processing model, wherein the image processing model is trained by the training method of any one of claims 1-10; The detection result is sent to the end-side device.

16. A method for detecting vascular embolism, comprising: Receive a vascular embolism detection task, wherein the vascular embolism detection task carries multiple plain CT images corresponding to a vascular region, and the vascular embolism detection task is used to detect whether there is an embolism in the vascular region; The plurality of plain CT images are input into an image processing model to obtain the detection result of whether there is vascular embolism in the vascular region output by the image processing model, wherein the image processing model is trained by the training method described in any one of claims 1-10.

17. A tumor screening method, comprising: Receive a tumor screening task, wherein the tumor screening task carries multiple plain CT images corresponding to a target detection area, and the tumor screening task is used to detect whether a tumor exists in the target detection area; The plurality of plain CT images are input into an image processing model to obtain a detection result of whether a tumor exists in the target detection area, as output by the image processing model, wherein the image processing model is trained by the training method described in any one of claims 1-10.

18. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 17.

19. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 17.

20. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 17.

Images