A method and system for lesion detection

Abstract

The embodiment of the present specification provides a lesion detection method, which comprises: acquiring a first type of image and a second type of image of a target object; processing the first type of image and the second type of image by using a multi-branch model to obtain lesion detection information, wherein: the multi-branch model comprises a first branch for processing the first type of image and a second branch for processing the second type of image; at a shallow part of the multi-branch model, the first branch and the second branch are independent of each other; at a deep part of the multi-branch model, the first branch is connected with the second branch.

Description

Technical Field

[0001] This manual relates to the medical field, and in particular to a method and system for detecting lesions. Background Technology

[0002] In the field of lesion detection, especially in the field of prostate cancer detection, the application of mpMRI (multi-parameter magnetic resonance imaging) is becoming more and more common. However, based on the existing mpMRI sequence analysis methods, it is difficult to simultaneously ensure a high lesion detection rate and the accuracy of the detected lesion contours.

[0003] Therefore, it is desirable to propose a lesion detection method and system that can improve the sensitivity and detection rate of lesion detection while ensuring the accuracy of the detected lesion outline. Summary of the Invention

[0004] This specification provides one or more embodiments of a lesion detection method, the lesion detection method comprising: acquiring a first type of image and a second type of image of a target object; processing the first type of image and the second type of image using a multi-branch model to obtain lesion detection information, wherein: the multi-branch model includes a first branch for processing the first type of image and a second branch for processing the second type of image; in the shallow part of the multi-branch model, the first branch and the second branch are independent of each other; in the deep part of the multi-branch model, the first branch is connected to the second branch through a skip connection.

[0005] This specification provides one or more embodiments of a lesion detection system, the lesion detection system comprising: a preprocessing module for acquiring a first type of image and a second type of image of a target object; a processing module for processing the first type of image and the second type of image using a multi-branch model to obtain lesion detection information; an auxiliary processing module for processing at least one of the first type of image and the second type of image based on the detection model to obtain optimized information; and a post-processing module for processing the lesion detection information based on the optimized information to generate lesion detection results.

[0006] This specification provides one or more embodiments of a lesion detection device, including a processor for executing a lesion detection method.

[0007] This specification provides one or more embodiments of a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions from the storage medium, the computer executes a lesion detection method. Attached Figure Description

[0008] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0009] Figure 1 These are schematic diagrams illustrating application scenarios of the lesion detection system according to some embodiments of this specification;

[0010] Figure 2 This is an exemplary flowchart of a lesion detection method according to some embodiments of this specification;

[0011] Figure 3 This is an exemplary flowchart of a method for obtaining a second type of image according to some embodiments of this specification;

[0012] Figure 4 This is an exemplary flowchart of a method for obtaining a first type of image and a second type of image based on registration processing, according to some embodiments of this specification;

[0013] Figure 5 This is a schematic diagram illustrating the process of acquiring a first type of image and a second type of image according to some embodiments of this specification;

[0014] Figure 6 This is a model structure diagram of the detection model shown in some embodiments of this specification;

[0015] Figure 7 This is a model structure diagram of a multi-branch model shown in some embodiments of this specification;

[0016] Figure 8 This is a schematic diagram of the joint training of the first branch and the regression branch according to some embodiments of this specification;

[0017] Figure 9 This is a block diagram of a lesion detection system according to some embodiments of this specification. Detailed Implementation

[0018] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0019] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0020] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0021] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0022] Figure 1 These are schematic diagrams illustrating application scenarios of the lesion detection system according to some embodiments of this specification. For example... Figure 1 As shown, application scenario 100 may include scanning device 110, processing device 120, network 130, storage device 140 and client 150.

[0023] The scanning device 110 can refer to a device used to acquire scanning data of a target object. For example, the scanning device 110 can be one or more of a magnetic resonance imaging (MRI) device, a positron emission tomography (PET) device, or a PET-MRI (Positron Emission Tomography-Magnetic Resonance Imaging) scanning device, and is not limited herein.

[0024] Corresponding to the scanning device 110, the scanning data can be one or more of MRI images, PET images, or PET-MRI images, etc., without limitation. Taking MRI images as an example, the MRI image can be a T1-weighted image (T1W), a T2-weighted image (T2W), an apparent diffusion coefficient (ADC) image, a diffusion-weighted imaging (DWI) image, etc.

[0025] In some embodiments, scan data can be categorized into a first type of image and a second type of image based on the data format or the content highlighted. These two types of images may exhibit different levels of representation in boundary and lesion information. For example, taking MRI images as the scan data, the boundary information contained in the first type of image may be superior to that contained in the second type of image, while the lesion information contained in the second type of image may be superior to that contained in the first type of image. For instance, a T2-weighted image (hereinafter referred to as T2W image) can be a first type of image, while an apparent diffusion coefficient image (hereinafter referred to as ADC image) or a diffusion-weighted image (hereinafter referred to as DWI image) can be a second type of image. For further explanation of the first and second types of images, see [link to documentation]. Figure 2 The corresponding content.

[0026] The target object is the object undergoing lesion detection. The target object can be a patient, a test subject, etc. As another example, the target object can include a specific part, organ, and / or tissue of a patient. For example, the target object can include the prostate, chest, heart, lungs, blood vessels, soft tissue, tumor, nodule, etc., or any combination thereof, without limitation. In some embodiments, the target object can be a specific part, organ, and / or tissue of the patient and other organs and / or tissues within a certain radius around it.

[0027] Processing device 120 can refer to a device used to generate lesion detection results. In some embodiments, processing device 120 may include a processor. Processing device 120 can process data and / or information obtained from scanning device 110, storage device 140, and / or client 150. For example, processing device 120 can acquire a first type of image and a second type of image of a target object, and process the first type of image and the second type of image using a multi-branch model to obtain lesion detection information. Alternatively, processing device 120 can process at least one of the first type of image and the second type of image based on a detection model to obtain optimization information, and process the lesion detection information based on the optimization information to generate lesion detection results. In some embodiments, processing device 120 may be a single server or a group of servers.

[0028] Network 130 can be used to connect the various components of application scenario 100 and / or connect the system with external resources. Network 130 enables communication between the components and with other parts outside the system, facilitating the exchange of data and / or information. For example, network 130 can transmit scan data acquired by scanning device 110 to processing device 120.

[0029] Storage device 140 can be used to store data and / or instructions. In some embodiments, storage device 140 can be used to store scan data of a target object acquired by scanning device 110. Storage device 140 can be implemented in a single central server, multiple servers connected via communication links, or multiple personal devices.

[0030] User terminal 150 refers to one or more terminal devices or software used by a user to initiate detection commands or receive lesion detection results generated by processing device 120. User refers to a person who analyzes and uses the lesion detection results, such as a doctor or medical researcher. In some embodiments, user terminal 150 may be one or any combination of devices with input and / or output functions, such as mobile device 151, tablet computer 152, laptop computer 153, and desktop computer 154.

[0031] It should be noted that application scenario 100 is provided for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can make various modifications or variations based on the description in this specification. For example, user terminal 150 may also include other types of terminals. However, these changes and modifications will not depart from the scope of this application.

[0032] Figure 2 This is an exemplary flowchart of a lesion detection method according to some embodiments of this specification. Figure 2 As shown, process 200 includes the following steps. In some embodiments, process 200 may be executed by lesion detection system 900 (such as processing module 920 and auxiliary processing module 930). For a description of lesion detection system 900, please refer to... Figure 9 Related descriptions.

[0033] Step 210: Obtain the first type of image and the second type of image of the target object.

[0034] For details regarding the target object, please refer to [link / reference]. Figure 1 The corresponding content. The first and second categories of images can be obtained by classifying scanned data based on preset classification rules. These preset classification rules can include various methods, such as classification based on imaging characteristics, classification based on data format (e.g., resolution), or classification based on imaging modality.

[0035] In some embodiments, the processing module 920 may classify the scanned data into a first type of image and a second type of image based on the imaging modality.

[0036] The first category of images can include images obtained by structural imaging of a target object. Structural imaging refers to imaging techniques that can reflect the anatomical structure and lesion conditions of tissues. Examples include T1W images, T2W images, and computed tomography (CT) images.

[0037] The second type of image may include images obtained by functional imaging of the target object. Functional imaging can refer to imaging techniques that can reflect the function of tissues (such as the presence of lesions and the degree of danger of lesions). Examples include DWI images, ADC images, and PET images. In some embodiments, the first and second types of images can be acquired using the same scanning device 110, for example, based on a magnetic resonance imaging device. Alternatively, the first and second types of images can be acquired using different scanning devices 110. For example, the first type of image can be acquired using a computed tomography (CT) device, and the second type of image can be acquired using a positron emission tomography (PET) device.

[0038] In some embodiments, the first and second type images can also be acquired in other ways. For example, the processing device 120 can perform image reconstruction on the data of the patient's examination site acquired by the scanning device to obtain the first and second type images of the patient. Alternatively, the first and second type images can be pre-reconstructed and stored in the storage device 140, and when lesion detection is needed, the first and second type images can be directly read from the storage device 140. Another example is that the processing module 920 can acquire the first type image and an initial second type image of the target object; register the initial second type image to the first type image to obtain the second type image, thereby obtaining both the first and second type images. See further details. Figure 4 Related descriptions.

[0039] In some embodiments, the boundary information contained in the first type of image is superior to the boundary information contained in the second type of image, and the lesion information contained in the second type of image is superior to the lesion information contained in the first type of image.

[0040] Boundary information can include anatomical information reflecting the outer boundaries of the lesion. For example, boundary information can include the outline of the lesion.

[0041] Lesion information can refer to functional information that reflects the presence and risk level of a lesion. For example, lesion information may include the probability of a lesion being present in an image.

[0042] In this article, "better" can be understood as "more obvious," "easier to detect," and "more conducive to improving detection accuracy." The first type of image, with its superior boundary information, allows for better identification of the boundary contours of lesions in medical images, thus more accurately determining the morphological information of the lesions. The second type of image, with its superior lesion information, enables better lesion detection, improving the detection rate of lesions.

[0043] In some embodiments, the first type of image may include a T2W image. A T2W image may refer to a magnetic resonance image that highlights differences in tissue T2 relaxation (lateral relaxation).

[0044] The second type of image may include at least one of ADC images and DWI images. An ADC image may refer to an image characterizing the rate of water molecule diffusion in tissue. A DWI image may refer to an image characterizing the degree of water molecule diffusion in tissue. In some embodiments, the second type of image may include a target diffusion-weighted image (i.e., a target DWI image). A target DWI image may refer to a DWI image corresponding to a target b-value. The b-value may refer to a diffusion sensitivity factor used to characterize the diffusion effect of the magnetic resonance signal. The target b-value may refer to the b-value corresponding to the target DWI image, and its value may be preset. In some embodiments, the target b-value may be 1500 g / mm². 2 This allows the target DWI image to not only effectively characterize the diffusion degree of water molecules in the target object, but also to have a high signal-to-noise ratio.

[0045] In some embodiments, the target DWI image may be generated based on an initial DWI image. For details regarding the initial DWI image and the generation of the target DWI image based on the initial DWI image, please refer to [link to relevant documentation]. Figure 3 .

[0046] In some embodiments, the first type of image and / or the second type of image (such as T2W image and ADC image) can be acquired based on the scan data of the target object acquired by the scanning device 110 and / or the scan data of the target object stored in the storage device 140. In some embodiments, the first type of image and / or the second type of image (such as T2W image and ADC image) can also be acquired by the processing device 120 processing the scan data of the target object.

[0047] In some embodiments of this specification, by using images with better boundary information representation capabilities, such as T2W images, as the first type of image, and images with better lesion information representation capabilities, such as ADC images and DWI images, as the second type of image, the accuracy of detected lesion contours can be improved while the sensitivity of lesion detection can be increased.

[0048] Step 220: Use a multi-branch model to process the first type of image and the second type of image to obtain lesion detection information.

[0049] A multi-branch model is a network model used to process first-class and second-class images to obtain lesion detection information.

[0050] In some embodiments, a multi-branch model may include multiple branches, such as a first branch for processing a first type of image and a second branch for processing a second type of image. The first branch includes multiple first processing blocks connected in sequence. The first branch processes the first type of image to obtain the outputs of the multiple first processing blocks. Some or all of the outputs corresponding to the multiple first processing blocks can be used as the first output of the first branch. For example, the output of the last first processing block in the first branch can be used as the first output. The second branch includes multiple second processing blocks connected in sequence. The second branch processes the second type of image to obtain the outputs of the multiple second processing blocks.

[0051] In the shallow part of a multi-branch model, the first branch and the second branch are independent of each other; in the deep part of the multi-branch model, the first branch and the second branch are connected. The shallow part of a multi-branch model can include the shallow parts of the first branch and the shallow parts of the second branch. The shallow parts of the first branch and the shallow parts of the second branch can refer to the first n processing blocks (such as the first n first processing blocks, the first n second processing blocks) that are processed first after the corresponding branches receive their corresponding inputs. The value of n can be preset, for example, 2.

[0052] The deep portion of a multi-branch model can include the deep portion of the first branch and the deep portion of the second branch. The deep portion of the first branch can refer to the remaining first processing block of the first branch excluding its shallow portion. The deep portion of the second branch can refer to the remaining second processing block of the second branch excluding its shallow portion.

[0053] In the shallow part of a multi-branch model, the independence of the first and second branches means that there is no data transfer between the first processing block contained in the shallow part of the first branch and the second processing block contained in the shallow part of the second branch. In the deep part of the multi-branch model, the connection between the first and second branches can mean that the first processing block contained in the deep part of the first branch and the second processing block contained in the deep part of the second branch are connected to each other through appropriate connection methods to facilitate data transfer. In some embodiments, the connection between the first and second branches can take various forms, such as skip connection, concatenation connection, add connection, or by adding connection processing blocks using an attention mechanism in the deep part of the multi-branch model to achieve the connection between the first and second branches.

[0054] For more information on multi-branch models, please refer to [link / reference]. Figure 7 Related descriptions.

[0055] Lesion detection information can refer to information that reflects the characteristics of lesions. For example, lesion detection information may include the region of interest of the target object, the area where lesions may occur, the probability of lesions occurring in the corresponding area, and the characteristics of the lesion area (such as shape, location, etc.).

[0056] In some embodiments, lesion detection information may include at least one of a predicted lesion probability map and a predicted lesion edge map. Further explanation of the predicted lesion probability map and the predicted lesion edge map can be found in [link to relevant documentation]. Figure 7 Related descriptions.

[0057] In some embodiments, the multi-branch model can be generated using labeled first training samples. The first training samples may include sample scan data. The sample scan data may include first-class and second-class images of the samples, and the labels of the first training samples may include gold-standard lesion detection information from the sample scan data. The gold-standard lesion detection information from the sample scan data may be determined by the user or the processing device 120. For detailed descriptions of the multi-branch model and its training, please refer to other parts of this specification, such as... Figure 7 and Figure 8 The corresponding content.

[0058] In some embodiments of this specification, a multi-branch model is used to process lesion-related image information, effectively improving the accuracy and efficiency of lesion detection results. Specifically, for the first and second branches, shallower network blocks output images with richer details and stronger spatial location, while deeper network blocks output semantic information with richer representation. Therefore, by connecting the first and second branches in the deeper parts of the multi-branch model, the powerful semantic information of lesions in the second type of image can be fully utilized, improving the sensitivity and specificity of lesion detection; simultaneously, the precise spatial location information of the first type of image can be fully utilized in the shallower network layers, improving the accuracy of lesion contours.

[0059] In some embodiments, such as Figure 2 As shown, process 200 may also include the following steps. The following steps may be performed by lesion detection system 900 (such as auxiliary processing module 930 and / or post-processing module 940).

[0060] Step 230: Process at least one of the first type of image and the second type of image based on the detection model to obtain optimization information.

[0061] The detection model can refer to a network model used to process the first type of image and / or the second type of image to obtain optimization information. Here, optimization information refers to information used to optimize (e.g., correct) the lesion detection information obtained in step 220.

[0062] In some embodiments, the detection model may include a first detection model and a second detection model. The first detection model can be used to process a first type of image to obtain a first reference result; the second detection model can be used to process a second type of image to obtain a second reference result. Optimization information may include the first reference result and the second reference result. For more information on the detection model, please refer to [link to relevant documentation]. Figure 6 Related descriptions.

[0063] The first reference result can be regarded as lesion detection information obtained solely based on the first type of image.

[0064] The second reference result can be regarded as lesion detection information obtained solely based on the second type of image.

[0065] In some embodiments, since the boundary information contained in the first type of image is superior to that contained in the second type of image, and the lesion information contained in the second type of image is superior to that contained in the first type of image, the first reference result and the second reference result can be used to assist in determining the lesion information in different partitions of the region of interest of the target object. A partition can refer to at least a portion of the region of interest of the target object.

[0066] Taking the region of interest of the target object as the prostate, the first type of image as T2W image, and the second type of image as DWI image and / or ADC image as an example, the first reference result can be used to help determine the lesion information of the prostate transition zone, and the second reference result can be used to help determine the lesion information of the prostate peripheral zone.

[0067] In some embodiments, in order to obtain the partitioning information of the region of interest of the target object more quickly, the detection model may also include a third detection model. The third detection model can be used to process the first type of image with better boundary information to obtain a third reference result for assisting in determining the partitioning information of the region of interest.

[0068] The third reference result can refer to the region-of-interest (ROI) mask of the target object obtained by processing the first type of image based on the third detection model. It reflects the ROI partitioning result of the target object. Taking the prostate gland as an example, the third reference result could be a glandular partitioning mask of the prostate gland, which could include masks for the central zone, peripheral zone, and transition zone. For more information on the third detection model, see [link to documentation]. Figure 6 Related descriptions.

[0069] Step 240: Process the lesion detection information based on the optimization information to generate lesion detection results.

[0070] The lesion detection result refers to the determination of whether a lesion exists within the region of interest of the target object. If a lesion is present, the lesion detection result can be considered positive.

[0071] In some embodiments, lesion detection results may also include other information about the lesion detection, such as the location and quantitative information of the lesion. Quantitative information about the lesion may include its maximum transverse diameter, longest diameter, volume, and volume percentage. Volume percentage refers to the ratio of the lesion volume to the volume of the region of interest of the target object. For example, lesion detection information could be "the lesion's maximum transverse diameter is 12.3 mm, its longest diameter is 17.9 mm, and its volume is 2.51 cm³". 3 "The volume percentage is 20%".

[0072] In some embodiments, the method for generating lesion detection results by combining optimization information and lesion detection information may include the following steps:

[0073] Step S1: Compare the lesion detection information with the first reference result in the optimization information to determine whether there are false positive lesions in the lesion detection information, and delete the false positive lesions.

[0074] False positive lesions refer to pseudo-lesions caused by detection errors. In some embodiments, if the first reference result has higher confidence in the detection result of a certain region, and the first reference result shows that there are no lesions in that region, while the lesion detection information shows that there are lesions in that region, then the lesions in that region in the lesion detection information can be considered false positive lesions and need to be removed.

[0075] Taking the prostate as the target object, T2W images as the first type of image, and DWI and / or ADC images as the second type of image as an example, the first reference result can be used to help determine lesion information in the prostatic transition zone. If no lesions are detected in the prostatic transition zone in the first reference result, but the lesion detection information indicates the presence of lesions in the prostatic transition zone, then the lesions in the prostatic transition zone in the lesion detection information can be considered false positives.

[0076] Step S2: Compare the lesion detection information with the second reference result in the optimization information to determine whether there are false positive lesions in the lesion detection information, and delete the false positive lesions.

[0077] Similar to step S1 above, by comparing the lesion detection information with the second reference result, it can be determined whether there are false positive lesions in the partition corresponding to the second reference result, so as to screen out false positive lesions. Continuing the previous example, the second reference result can be used to help determine the lesion information in the peripheral zone of the prostate. Similarly, if no lesions are detected in the peripheral zone in the second reference result, but the lesion detection information indicates that there are lesions in the peripheral zone, then the lesions in the peripheral zone of the prostate in the lesion detection information can be considered false positive lesions.

[0078] In some embodiments, the lesion detection information after filtering out false positive lesions in the above manner can be used as the lesion detection result.

[0079] In some embodiments, if the optimization information includes a third reference result, the corresponding mask in the third reference result can be used in both steps S1 and S2 to quickly obtain the corresponding partition in the lesion detection information and the first and second reference results, so as to quickly determine the lesion information of the partition.

[0080] In some embodiments, the post-processing module 940 may also filter out false positive lesions in the lesion detection information in other ways. For example, if a study finds that a certain lesion will not occur in patients who meet a certain preset condition (such as patients whose age is below a certain age threshold), then when the target object meets the preset condition, the lesions detected in the lesion detection information can be considered to be false positive lesions.

[0081] In some embodiments, if lesions still exist in the lesion detection information after screening out false positive lesions, the post-processing module 940 can further obtain the quantitative information of the lesions and use the quantitative information of the lesions and the lesion detection information after screening out false positive lesions as the final lesion detection result.

[0082] In some embodiments of this specification, the method for obtaining lesion detection results described above effectively utilizes the respective advantages of the two types of images in terms of boundary information and lesion information to make a comprehensive judgment on lesions, avoiding the influence of "false positive lesions" on lesion detection results, thereby improving the accuracy of lesion detection results.

[0083] It should be noted that the above description of process 200 is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to process 200 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification. For example, using manual annotation methods to obtain optimization information.

[0084] In some embodiments, in order to improve the prediction accuracy and model processing efficiency of the multi-branch model, it is necessary to process the data acquired by the scanning device 110 to generate a second type of image that is input to the second branch.

[0085] Figure 3 This is an exemplary flowchart of a method for acquiring a second type of image according to some embodiments of this specification. In some embodiments, process 300 may be performed by a lesion detection system 900 (such as a preprocessing module 910).

[0086] like Figure 3 As shown, process 300 includes the following steps:

[0087] Step 310: Obtain at least one initial DWI image of the target object.

[0088] An initial DWI image can refer to a raw, unprocessed DWI image. One initial DWI image can correspond to one initial b-value. Each initial DWI image may include one or more DWI images with the same b-value. The initial b-value corresponding to the initial DWI image can be the same as or different from the target b-value; for example, the initial b-value could be 0 or 500 g / mm². 2 1000g / mm 2 1500g / mm 2 2000g / mm 2 For an explanation of the target b value, please refer to [link / reference]. Figure 2 .

[0089] Step 320: Generate a target DWI image based on at least one initial DWI image.

[0090] In some embodiments, if the initial b-value corresponding to a certain initial DWI image is equal to the target b-value, then this initial DWI image can be used as the target DWI image.

[0091] In some embodiments, the preprocessing module 910 can process the initial DWI image according to different conditions met by the initial DWI image to generate the target DWI image.

[0092] For example: in response to determining that at least one initial DWI image meets a first preset condition, the preprocessing module 910 can process at least one initial DWI image based on a first preset method to generate a target DWI image; in response to determining that at least one initial DWI image meets a second preset condition, the preprocessing module 910 can process at least one initial DWI image based on a second preset method to generate a target DWI image.

[0093] The first preset condition can be: the initial DWI image includes only one type of DWI image sequence, and the initial b value corresponding to this type of DWI image sequence is not equal to the target b value.

[0094] The first preset method can refer to correcting the pixel values ​​of each pixel in the initial DWI image using a first preset formula to generate the target DWI image. The first preset formula is shown in formula (1): y = e (b′-B)×adc ×dwi (1).

[0095] Where y is the pixel value of any pixel in the corrected initial DWI image (i.e., the target DWI image), e is a natural constant, adc is the pixel value of the pixel corresponding to y in the ADC image, dwi is the pixel value of the pixel corresponding to y in the initial DWI image, b′ is the initial b value corresponding to the initial DWI image, and B is the target b value (e.g., 1500 g / mm²). 2 ).

[0096] The second preset condition can be: the initial DWI image includes two DWI image sequences, denoted as image sequence 1 and image sequence 2 respectively. The initial b values ​​corresponding to image sequence 1 and image sequence 2 are denoted as b1 and b2 respectively, where b1 ≠ b2 and neither is equal to the target b value.

[0097] The second preset method can refer to correcting the pixel values ​​of each pixel in the initial DWI image using a second preset formula to generate the target DWI image. The second preset formula is shown in formula (2):

[0098] Where y is the pixel value of any pixel in the corrected initial DWI image (i.e., the target DWI image), dwi2 is the pixel value of the pixel corresponding to y in image sequence 2, b2 is the initial b value corresponding to image sequence 2, and t can be calculated by formula (3): t=(ln(dwi1)-ln(dwi2)) / (b2-b1) (3).

[0099] Where dwi1 is the pixel value of the pixel corresponding to y in image sequence 1, b1 is the initial b value corresponding to image sequence 1, and the meanings of other variables are the same as in formula (2).

[0100] In some embodiments of this specification, even if the scanned DWI image does not have a target DWI image corresponding to the target b-value, an accurate target DWI image can still be obtained through the above-described correction method. By converting DWI image sequences corresponding to various b-values ​​into target DWI images with the target b-value, the sensitivity of lesion detection can be ensured.

[0101] It should be noted that the above description of process 300 is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to process 300 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification. For example, replacing the natural logarithm in formula (2) with other values.

[0102] In some embodiments, in order to improve the prediction accuracy and model processing efficiency of the multi-branch model, the data acquired by the scanning device 110 needs to be processed (such as registration processing) before being used as the input of the second type of image of the second branch.

[0103] Figure 4 This is an exemplary flowchart illustrating a method for obtaining a first type of image and a second type of image based on registration processing, according to some embodiments of this specification. Figure 4 As shown, process 400 includes the following steps. In some embodiments, process 400 may be performed by lesion detection system 900 (such as preprocessing module 910).

[0104] Step 410: Obtain the first type image and the initial second type image of the target object.

[0105] In some embodiments, the first type of image includes T2W images. In some embodiments, the T2W image can be an initial T2W image determined based on scan data, or a T2W image obtained by performing a preset processing on the initial T2W image. For example, the preset processing can be cropping, etc. See [link to documentation] for a detailed explanation of the preset processing. Figure 5 .

[0106] In some embodiments, the initial second-class image can be divided into two cases based on whether the initial second-class image contains the first initial DWI image. For example, the first case is that the initial second-class image includes the first initial DWI image, the second initial DWI image, and the initial ADC image, and the second case is that the initial second-class image only includes the second initial DWI image and the initial ADC image.

[0107] The first initial DWI image can refer to the DWI image corresponding to the reference b value.

[0108] The reference b-value is not equal to the target b-value. The reference b-value can be preset. In some embodiments, the reference b-value can be 0. For example, the first initial DWI image is a DWI image with a corresponding b-value of 0.

[0109] The first initial DWI image can be a DWI image with a corresponding reference b value determined based on the scan data, or it can be a DWI image obtained after the initial DWI image has undergone preset processing.

[0110] The second initial DWI image may refer to the DWI image corresponding to the target b-value. In some embodiments, the second initial DWI image may be the aforementioned target DWI image. Further explanation of the target DWI image and its acquisition method can be found in [link to relevant documentation]. Figure 3Related description. In some embodiments, the second initial DWI image may be a DWI image corresponding to a target b value determined based on the scan data, or it may be a DWI image obtained by pre-processing the initial DWI image.

[0111] In some embodiments, the initial ADC image may be the original ADC image determined based on the scan data, or it may be an ADC image obtained by pre-processing the original ADC image.

[0112] Step 420: Register the initial second-class image to the first-class image to obtain the second-class image.

[0113] Registering an initial second-class image to a first-class image can mean registering at least a portion of the initial second-class images to the first-class images to obtain a registration matrix; and applying the registration matrix to the remaining images in the initial second-class images to obtain the registered second-class images.

[0114] In some embodiments, the registration matrix may include a first registration matrix and a second registration matrix.

[0115] The first registration matrix can refer to the matrix form of data obtained by registering the first initial DWI image (such as a DWI image with a b value of 0) to the T2W image for the first case.

[0116] The application of the first registration matrix may include registering the second initial DWI image and the initial ADC image based on the first registration matrix, so that the registered DWI image and the registered ADC image can be aligned with the T2W image.

[0117] The second registration matrix refers to the matrix data obtained by registering the initial ADC image to the T2W image in the second case. The image obtained after registering the initial ADC image to the T2W image is the registered ADC image.

[0118] The application of the second registration matrix may include registering the second initial DWI image based on the second registration matrix so that the registered DWI image can be aligned with the T2W image.

[0119] The second type of image can include the registered DWI image and the registered ADC image mentioned above. The second type of image can be obtained based on the aforementioned registration process, and the obtained second type of image is used as the input of the second branch.

[0120] In some embodiments, the T2W image, the first initial DWI image, the second initial DWI image, and the initial ADC image used for registration can also be images obtained after corresponding preset processing. For example, the T2W image and the initial ADC image can be cropped first, and then the cropped ADC image can be registered to the cropped T2W image. Further explanation of other preset processing of the image before registration can be found in [link to documentation]. Figure 5 The corresponding content.

[0121] In some embodiments of this specification, registration of the initial second-type image is achieved in different ways depending on whether the initial second-type image contains the first initial DWI image, thus enabling accurate and efficient acquisition of the second-type image. For example, when the initial second-type image includes the first initial DWI image (i.e., the initial DWI image corresponding to the reference b-value), registration is easier because the deformation between the first initial DWI image and the T2W image is relatively small. Therefore, when the first initial DWI image is present, the first initial DWI image is registered to the T2W image, and then the resulting registration matrix is ​​applied to the second initial DWI image and the initial ADC image, ensuring more accurate alignment between the T2W image and the initial ADC image, as well as the second initial DWI image.

[0122] In some embodiments of this specification, since the first type of image and the initial second type of image sequence are often misaligned, registration between the sequences is required before use. The above registration method ensures that the registration process can be performed efficiently and accurately under different circumstances to obtain registered T2W, DWI, and ADC images.

[0123] It should be noted that the above description of process 400 is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to process 400 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification. For example, different registration cases can be divided based on non-zero reference b values, and different registration processes can be applied to different registration cases.

[0124] For illustrative purposes, Figure 5 Using T2W images, DWI images, and ADC images as examples, this section explains how to obtain the first and second types of images through preset processing. Figure 5 As shown, the preset processing can include segmentation processing, cropping processing, registration processing, etc.

[0125] In some embodiments, such as Figure 5As shown, the initial first type of image includes T2W image 511, and the initial second type of image 516 may include initial DWI image 515 and initial ADC image 513; wherein, the initial DWI image 515 may include a first initial DWI image 512 and a second initial DWI image 514. The first initial DWI image 512 corresponds to a reference b value (e.g., 0), and the second initial DWI image 514 corresponds to a target b value (e.g., 1500 g / mm²). 2 That is, the second initial DWI image 514 can be the aforementioned target DWI image. In some embodiments, the preprocessing module 910 can preprocess the first initial DWI image 512 and / or other DWI images that do not correspond to the target b value to generate the second initial DWI image 514. For the specific processing procedure of processing the DWI image into the second initial DWI image 514 corresponding to the target b value, please refer to [example missing]. Figure 3 The description of generating the target DWI image shown in step 320.

[0126] In some embodiments, preprocessing may also include other processing procedures, such as adjusting the resolution of the DWI image to a preset value.

[0127] Segmentation processing can be the segmentation of the T2W image by the preprocessing module 910. In some embodiments, such as... Figure 5 As shown, the preprocessing module 910 can process the T2W image 511 based on the segmentation model 510 to obtain the segmentation result 510-1 of the T2W image. The segmentation result 510-1 can represent the boundary information of the region of interest of the target object.

[0128] In some embodiments, the segmentation model can be a machine learning model. For example, the segmentation model can be a U-Net, DeepMedic, V-Net, PSPNet (Pyramid Scene Parseing Network), or DeepLab model.

[0129] In some embodiments, the segmentation model can be trained using historical T2W images as samples and the corresponding gold standard segmentation results as labels. As an example, this could involve obtaining the corresponding biopsy pathology results for each case based on historical T2W images, and using annotations from at least two experienced physicians to obtain the mask and edge map of the lesion region as the gold standard segmentation result. In some embodiments, to make the final predicted lesion contour more accurate, the lesion mask is annotated on the T2W image sequence, and the image features of both the case's DWI and ADC images can be referenced simultaneously during annotation.

[0130] In some embodiments, the segmentation result 510-1 obtained based on the segmentation model can be further used for cropping the corresponding images. Since T2W images have a strong ability to represent the boundary information of regions of interest such as glands, the T2W images, DWI images, and ADC images can be cropped based on the segmentation result 510-1 obtained from processing the T2W images using the segmentation model, so as to remove images that do not belong to the regions of interest in each type of image and improve the efficiency of subsequent image processing.

[0131] In some embodiments, such as Figure 5 As shown, the preprocessing module 910 can crop the T2W image 511 based on the segmentation result 510-1 to obtain the T2W cropped image 511-1; crop the first initial DWI image 512 based on the segmentation result 510-1 to obtain the first DWI cropped image 512-1; crop the second initial DWI image 514 based on the segmentation result 510-1 to obtain the second DWI cropped image 514-1; and crop the initial ADC image 513 based on the segmentation result 510-1 to obtain the ADC cropped image 513-1.

[0132] In some embodiments, when the preprocessing module 910 crops the corresponding image based on the segmentation result 510-1, it can expand the image to a certain extent based on the boundary information displayed in the segmentation result 510-1, and crop the corresponding image based on the expanded range, so as to avoid erroneous cropping of the region of interest due to inaccurate segmentation results. The size of the expanded range can be preset.

[0133] In some embodiments, the T2W cropped image 511-1 obtained after cropping the T2W image 511 can be used as input to the third detection model to obtain the partition mask of the region of interest of the target object. For more details on the third detection model, see [link to documentation]. Figure 6 The corresponding content.

[0134] In some embodiments, such as Figure 5 As shown, the preprocessing module 910 can register the first DWI cropped image 512-1 to the T2W cropped image 511-1 to generate a first registration matrix 531; and apply the first registration matrix 531 to the second DWI cropped image 514-1 and the ADC cropped image 513-1 to generate a registered DWI image 514-2 and a registered ADC image 513-2.

[0135] In some embodiments, the preprocessing module 910 may further register the ADC cropped image 513-1 to the T2W cropped image 511-1 to generate a second registration matrix 532 and a registered ADC image 513-2; and apply the second registration matrix 532 to the second DWI cropped image 514-1 to generate a registered DWI image 514-2.

[0136] In some embodiments, the T2W cropped image 511-1 can be used as a first type of image, and the registered ADC image 513-2 and the registered DWI image 514-2 can be used as a second type of image.

[0137] In some embodiments of this specification, a segmentation result is first obtained based on a segmentation model. Then, the T2W image, the first initial DWI image, the second initial DWI image, and the initial ADC image are cropped based on the segmentation result. Registration is then performed based on the cropped images, further improving the accuracy of alignment and registration efficiency between the T2W image and the initial ADC image and the second initial DWI image. This can further improve the accuracy of subsequently determined lesion detection information.

[0138] Figure 6 This is a model structure diagram of the detection model shown in some embodiments of this specification.

[0139] like Figure 6 As shown, the detection model 620 may include a first detection model 621 and a second detection model 622.

[0140] In some embodiments, the auxiliary processing module 930 can generate a first reference result 631 based on the processing of the first type of image 611 by the first detection model 621.

[0141] In some embodiments, the first detection model may be a trained U-Net model.

[0142] In some embodiments, the first type of image 611 may be a T2W image, and the first reference result may be information reflecting the presence of lesions in at least a portion of the region of interest of the target object. For example, the first reference result may be used to assist in determining lesion information in the prostatic transition zone. Further explanation of the first reference result can be found in [link to relevant documentation]. Figure 2 .

[0143] In some embodiments, the first detection model 621 can be trained using multiple second training samples labeled with second tags. For example, multiple second training samples with second tags can be input into the initial first detection model. The value of the loss function is determined by the second tags and the results of the initial first detection model. The parameters of the initial first detection model are iteratively updated based on the value of the loss function. The model training is completed when the loss function of the initial first detection model meets the preset conditions for training termination, resulting in a trained first detection model. The second training samples may include first-class images, such as T2W images, which can be obtained based on historical scans. The second tags may refer to the gold standard lesion detection information corresponding to the first-class images, which can be obtained based on manual annotation. The preset conditions for training termination may include loss function convergence, the number of iterations reaching a threshold, etc.

[0144] In some embodiments, the gold standard lesion detection information corresponding to the first type of sample image can refer to the actual lesion detection information corresponding to the first type of sample image, which can reflect whether there are lesions in the region of interest of the sample object. For instructions on obtaining the gold standard lesion detection information, please refer to [link to documentation]. Figure 8 The corresponding content in [the document / document].

[0145] In some embodiments, the auxiliary processing module 930 can generate a second reference result 632 based on the processing of the second type of image 612 by the second detection model 622.

[0146] In some embodiments, the second detection model may be a trained U-Net model.

[0147] In some embodiments, the second type of image 612 may be a DWI image and / or an ADC image, and the second reference result may be information reflecting the presence of lesions in at least a portion of the region of interest of the target object. For example, the second reference result can be used to assist in determining lesion information in the peripheral zone of the prostate. Further explanation of the second reference result can be found in [link to relevant documentation]. Figure 2 Related descriptions.

[0148] In some embodiments, the second detection model 622 can be trained using multiple third training samples labeled with a third tag. The third training samples may include sample second-class images, such as sample DWI images and / or sample ADC images, which can be obtained based on historical scanning processes; the third tag may refer to the gold standard lesion detection information corresponding to the sample second-class images, which can be obtained based on manual annotation.

[0149] In some embodiments, the gold standard lesion detection information corresponding to the second type of sample image can refer to the actual lesion detection information corresponding to the second type of sample image. For specific acquisition methods, please refer to [link / reference needed]. Figure 8 Methods for obtaining lesion detection information according to the gold standard.

[0150] In some embodiments, the first type of image and the second type of image input to the first detection model and the second detection model can be initial images determined based on scan data, or images obtained by partially or completely pre-processing the initial images. For a description of the pre-processing, see [link to documentation]. Figure 5 The corresponding content.

[0151] In some embodiments, the detection model 620 may further include a third detection model 623, and based on this, the optimization information 630 may further include a third reference result 633.

[0152] In some embodiments, the auxiliary processing module 930 can generate a third reference result 633 that reflects the partitioning results of the region of interest of the target object based on the processing of the first type of image 611 by the third detection model 623.

[0153] In some embodiments, the third detection model may be a trained V-Net model.

[0154] In some embodiments, the first type of image 611 can be a T2W image, or it can be a segmentation result obtained by segmenting a T2W image based on a segmentation model. Further explanation of the segmentation result can be found in [link to documentation]. Figure 5 The corresponding content.

[0155] In some embodiments, the third detection model 623 can be trained using multiple fourth training samples labeled with a fourth label. The fourth training samples may include first-class images, such as T2W images. The fourth label may include the actual partition mask of the region of interest in the first-class images, which can be obtained based on manual annotation.

[0156] In some embodiments, the post-processing module 940 can compare the lesion detection information with the first reference result and / or the second reference result in the optimized information to determine whether there are false positive lesions in the lesion detection information, and delete the false positive lesions to obtain accurate lesion detection results. More information about lesion detection information, lesion detection results, and their acquisition methods can be found in [link to documentation]. Figure 2 Related descriptions.

[0157] In some embodiments, the post-processing module 940 can use the corresponding mask in the third reference result to determine the boundary information of the region of interest of the target object. For example, the post-processing module 940 can use the corresponding mask in the third reference result to quickly obtain the corresponding partitions in the lesion detection information and the first and second reference results, so as to quickly determine the lesion information of the corresponding partitions.

[0158] In some embodiments of this specification, by using the above-described detection model, the efficiency of determining optimization information is improved while ensuring the accuracy of the optimization information.

[0159] Figure 7 This is a model structure diagram of a multi-branch model shown in some embodiments of this specification.

[0160] In some embodiments, processing the first type of image and the second type of image using a multi-branch model may further include: generating an edge image corresponding to the first type of image; processing the first type of image, the second type of image and the edge image using a multi-branch model to obtain lesion detection information, wherein the multi-branch model further includes an edge branch for lesion edge detection of the edge image.

[0161] The edge image can include the edge contour information of the tissue in the first type of image. For example, the first type of image can be processed using an edge extraction operator (such as the Sobel operator) to obtain the corresponding edge image. Alternatively, the boundary information of the tissue in the first type of image can be obtained based on the segmentation results, and then the edge image can be obtained.

[0162] In some embodiments, the multi-branch model can generate a predicted lesion probability map and a predicted lesion edge map based on the processing of a first-class image, a second-class image, and an edge image. For example... Figure 7 As shown, the input to the multi-branch model 700 may include a first-class image 711, a second-class image 721, and an edge image 731, and the output may include a predicted lesion probability map and a predicted lesion edge map. The multi-branch model can be a machine learning model.

[0163] A lesion probability map can be an image that reflects the probability of lesions (i.e., the occurrence of lesions) in each region within the region of interest of a target object.

[0164] Predictive lesion edge map refers to an image that reflects the lesion edge contour information of the lesion area within the region of interest of the target object.

[0165] In some embodiments, such as Figure 7 As shown, the multi-branch model 700 may include a first branch 710, a second branch 720, and an edge branch 730.

[0166] The first branch can be a machine learning model, such as a convolutional neural network (CNN) model.

[0167] like Figure 7As shown, the first branch 710 may include multiple first processing blocks. Taking a first branch 710 comprising five first processing blocks as an example, the five first processing blocks are stage1-0, stage1-1, stage1-2, stage1-3, and stage1-4; wherein stage1-0 and stage1-1 are shallow first processing blocks, and stage1-2, stage1-3, and stage1-4 are deep first processing blocks. The processing blocks in this specification can perform feature extraction. For example, a processing block may include one or more convolutional layers. In some embodiments, a processing block may be a residual block. The structures of different processing blocks may be the same or different.

[0168] The first branch 710 can obtain the output of multiple first processing blocks based on the processing of the first type of image 711, and the output of the first processing blocks stage1-4 can be used as the first output corresponding to the first branch 710. The first output can reflect the spatial location information of the lesion.

[0169] In some embodiments, the first branch 710 may perform a downsampling operation on the output of the first processing block of the previous layer (as shown by the striped arrows in the figure), and use the obtained downsampling result as the input of the first processing block of the next layer.

[0170] In some embodiments, the first processing block stage1-0 can be used as an input block to input a first type of image; the first processing block stage1-4 can be used as an output block to output a first output.

[0171] In some embodiments, the first branch 710 may be obtained based on joint training with the regression branch. For a description of the regression branch and the joint training of the first branch and the regression branch, please refer to [link to relevant documentation]. Figure 8 Related descriptions.

[0172] The second branch can be machine learning models, such as Convolutional Neural Network (CNN) models.

[0173] like Figure 7As shown, the second branch 720 may include multiple second processing blocks. Taking a second branch 720 comprising five second processing blocks as an example, the five second processing blocks are stage2-0, stage2-1, stage2-2, stage2-3, and stage2-4, where stage2-0 and stage2-1 are shallow second processing blocks, and stage2-2, stage2-3, and stage2-4 are deep second processing blocks. The second branch 720 can obtain the outputs of multiple second processing blocks based on the processing of the second type image 721. The outputs of multiple second processing blocks can fully utilize the strong lesion representation characteristics of the second type image.

[0174] In some embodiments, the second branch 720 may downsample the output of the second processing block of the previous layer, and use the downsampled result as the input of the second processing block of the next layer. In some embodiments, the second processing block stage2-0 may be used as an input block for inputting a second type of image.

[0175] In some embodiments, such as Figure 7 As shown, the deep first processing blocks stage1-2, stage1-3, and stage1-4 can be connected to the deep second processing blocks stage2-2, stage2-3, and stage2-4 respectively via skip connections. In this case, the input to each deep first processing block includes the downsampled output of the previous layer's first processing block and the output of the deep second processing block to which it is skipped.

[0176] Edge branches can be machine learning models, such as convolutional neural network (CNN) models.

[0177] like Figure 7 As shown, the edge branch 730 can include multiple edge processing blocks. Taking an edge branch 730 comprising four edge processing blocks as an example, the four edge processing blocks are block3-0, block3-2, block3-3, and block3-4. The edge branch 730 can obtain the outputs of multiple edge processing blocks based on the processing of the edge image 731 and the output of the deep first processing block, and the output of edge processing block block3-4 can be used as the second output of the edge branch 730. The edge branch can learn the boundary information of the lesion through the edge image, suppress other irrelevant information, and make the boundary of the finally detected lesion region clearer and the contour more accurate.

[0178] In some embodiments, the edge processing block can be a residual block.

[0179] In some embodiments, the edge branch 730 may perform an upsampling operation on the output of the deep first processing block corresponding to the edge processing block (as shown by the gray arrow in the figure), and the obtained upsampling result and the output of the edge processing block of the previous layer are used as the input of the edge processing block.

[0180] In some embodiments, edge processing block 3-0 can be used as an input block for inputting an edge image; edge processing block 3-4 can be used as an output block for outputting a second output.

[0181] In some embodiments, such as Figure 7 As shown, the multi-branch model 700 may further include a prediction branch 740. The prediction branch receives the first output of the first branch and the second output of the edge branches, and generates lesion detection information based on the processing of the first and second outputs. For an explanation of the lesion detection information, please refer to [link to relevant documentation]. Figure 2 Related descriptions.

[0182] The prediction branch can be a machine learning model, such as a convolutional neural network (CNN) model.

[0183] like Figure 7 As shown, prediction branch 740 may include multiple prediction processing blocks. Taking prediction branch 740 as an example, the four prediction processing blocks are stage4-0, stage4-1, stage4-2, and stage4-3. Prediction branch 740 can obtain the outputs of multiple prediction processing blocks based on the processing of the first output. Prediction branch 740 can obtain a predicted lesion probability map based on the processing of the first output. Prediction branch 740 can obtain a predicted lesion edge map based on the processing of the second output.

[0184] In some embodiments, the prediction branch 740 may upsample the first output or the output of the previous layer prediction processing block, and the resulting upsampled result may be used as the input to the first layer prediction processing block or the next layer prediction processing block. For example, Figure 7 As shown, the upsampled result of the first output is used as the input of stage4-3, and the upsampled results of the outputs of stage4-3, stage4-2, and stage4-1 are used as the inputs of stage4-2, stage4-1, and stage4-0, respectively. In some embodiments, the prediction processing block stage4-3 can be used as an input block to input the first output; the prediction processing block stage4-0 can be used as an output block.

[0185] In some embodiments, such as Figure 7As shown, the first processing blocks stage1-0, stage1-1, stage1-2, and stage1-3 can be connected to the prediction processing blocks stage4-0, stage4-1, stage4-2, and stage4-3 respectively via skip connections. In this case, the input to each prediction processing block includes the upsampled result of the first output or the upsampled result of the output of the previous prediction processing block, as well as the output of the first processing block to which it is skipped.

[0186] Lesion detection information includes at least one of a predicted lesion probability map and a predicted lesion edge map. For a description of lesion detection information, please refer to [link to relevant documentation]. Figure 2 Related descriptions. For training instructions on multi-branch models, please refer to... Figure 8 The corresponding content.

[0187] In some embodiments, the prediction branch may include a first prediction layer and a second prediction layer. For example... Figure 7 As shown, the first prediction layer 741 can be used to generate a predicted lesion probability map based on the processing of the output of the prediction processing block stage4-0. The second prediction layer 742 can be used to generate a predicted lesion edge map based on the processing of the second output. In some embodiments, the second prediction layer 742 can be connected to the prediction processing block stage4-0 via a skip connection. In this case, the input of the prediction processing block stage4-0 includes the upsampled result of the output of the prediction processing block stage4-1, the output of the connected first processing block stage1-0, and the output of the connected second prediction layer 742.

[0188] In some embodiments of this specification, the accuracy of the output results of the multi-branch model is effectively improved by introducing a first prediction layer and a second prediction layer in the prediction branch.

[0189] In some embodiments of this specification, by introducing a prediction branch into the multi-branch model, accurate processing of the first and second outputs is achieved to obtain accurate lesion detection information.

[0190] In some embodiments of this specification, the output of the first processing block of the first branch is obtained by connecting the first branch and the second branch in the deeper part of the multi-branch model. Furthermore, by introducing edge branches and prediction branches into the multi-branch model, and incorporating the output of the first processing block into the input of the edge processing block of the edge branch, and incorporating the output of the edge branch into the input of the prediction branch, the prediction branch can learn the boundary information of the lesion region based on the output of the edge branch, while suppressing other irrelevant information, resulting in a clearer boundary and more accurate contour of the finally detected lesion region.

[0191] In some embodiments, the multi-branch model can be obtained based on joint training. Figure 8 This is a schematic diagram illustrating the joint training of a multi-branch model according to some embodiments of this specification.

[0192] In some embodiments, the training process of the multi-branch model may include: acquiring multiple training samples, each training sample including a first-class image, a second-class image, and gold standard lesion detection information; iteratively updating the initial multi-branch model using the multiple training samples to obtain a multi-branch model, wherein the initial multi-branch model includes an initial first branch, an initial second branch, and an initial prediction branch, the initial second branch is used to process the second-class image to generate intermediate detection information, the initial first branch is used to process the first-class image and the intermediate detection information, the initial prediction branch is used to generate lesion prediction results based on the output of the initial first branch, and the first difference between the lesion prediction results and the gold standard lesion detection information is used to update the initial first branch and the initial second branch.

[0193] In some embodiments, the multi-branch model may further include a regression branch, and the training of the multi-branch model may be the joint training of the regression branch, the first branch, the second branch, the marginal branches, and the prediction branch. In some embodiments, the regression branch may be jointly trained with the first branch to obtain a trained first branch. See later sections for an explanation of the joint training of the regression branch and the first branch.

[0194] The regression branch can be a machine learning model. In some embodiments, the regression branch can be a fully convolutional neural network model, which can adopt an AE (Auto-Encoder) or VAE (Variational Auto-Encoder) structure.

[0195] The method for obtaining the first type of sample image may be: based on the first type of image of the sample object acquired by the scanning device 110 (e.g., a T2W image containing the sample object), wherein the sample object is a case with lesions. In some embodiments, the first type of sample image may also be a processed first type image obtained by performing a preset processing on the aforementioned first type of sample image. For an explanation of the preset processing, please refer to... Figure 5 Related descriptions.

[0196] The method for obtaining the second type of sample image may be: acquiring a second type of image of the sample object based on the scanning device 110 (e.g., including the ADC image and DWI image of the sample object, wherein the DWI image can be multiple DWI images with different b values), wherein the sample object is a case with lesions. In some embodiments, the second type of sample image may also be a processed second type image obtained by performing a preset processing on the second type of sample image described above.

[0197] The gold standard lesion detection information can include the actual lesion probability map and / or actual lesion edge map corresponding to the first type image and the second type image of the sample. The actual lesion probability map can refer to an image that reflects whether there are lesions in the sample object. The actual lesion edge map can refer to an image that reflects the boundary of the lesions in the sample object.

[0198] In some embodiments, the method for obtaining gold standard lesion detection information may be: obtaining the corresponding puncture pathology results of each sample object, and obtaining the mask and edge map of the lesion area of ​​the sample object through annotation by at least two senior doctors, which serves as the gold standard lesion detection information of the sample object.

[0199] In some embodiments, in order to make the final predicted lesion contour more accurate, the mask annotation of the lesion region can be performed on the T2W image, while also referencing the image features of the DWI and ADC images.

[0200] Intermediate detection information can refer to the output of multiple initial second processing blocks obtained by processing the sample second-class image based on the initial second branch.

[0201] The lesion prediction result can refer to the predicted lesion detection information. In some embodiments, the lesion prediction result may include a sample predicted lesion probability map and a sample predicted lesion edge map.

[0202] The first difference can refer to the discrepancy between the sample predicted lesion probability map in the lesion prediction results and the actual lesion probability map in the gold standard lesion detection information.

[0203] In some embodiments, the training samples may further include edge images corresponding to the first type of sample images, and the initial multi-branch model may further include an initial edge branch for processing the edge images, and a second difference between the lesion prediction result and the gold standard lesion detection information is used to update the initial edge branch.

[0204] The second difference can refer to the discrepancy between the predicted lesion edge map in the lesion prediction results and the actual lesion edge map in the gold standard lesion detection information.

[0205] In some embodiments, the first branch, the regression branch, and other branches of the multi-branch model can be obtained through joint training, which may include the following process:

[0206] The Sobel operator is used to extract the edges of the regions of interest of the target object in the first type of sample images (at least including T2W images) to obtain the sample edge image;

[0207] Input the first type of sample images (including at least T2W images) into the initial first branch, input the second type of sample images (including at least ADC images and DWI images) into the initial second branch, and input the sample edge images into the initial edge branch to obtain the predicted lesion probability map, the predicted lesion edge map, and the first type of predicted image (generated by the initial regression branch) predicted during the training phase.

[0208] The value of the first loss function is determined based on the first difference between the predicted lesion probability map and the gold standard lesion detection information; the value of the second loss function is determined based on the second difference between the predicted lesion edge map and the gold standard lesion detection information; and the value of the third loss function is determined based on the difference between the first type of predicted image and the sample first type image. The first loss function is used to update the initial first branch and the initial second branch; the second loss function is used to update the initial edge branch; and the third loss function is used to update the initial regression branch and the initial first branch.

[0209] The parameters of the initial first branch and the initial second branch are updated based on the first loss function, the parameters of the initial marginal branch are updated based on the second loss function, and the parameters of the initial regression branch and the initial first branch are updated based on the third loss function, until the preset conditions are met, the training phase is completed, and the branches of the trained multi-branch model are obtained. The preset conditions may be that the above loss functions converge, the number of iterations reaches a threshold, etc.

[0210] In some embodiments of this specification, the training process of the multi-branch model described above can efficiently complete the training of the multi-branch model, and enable the trained multi-branch model to have excellent lesion detection information prediction capabilities.

[0211] In some embodiments, the first branch can be obtained based on joint training with the regression branch. For example... Figure 8 As shown, the joint training of the first branch and the regression branch includes: inputting the sample first-class image 811 into the initial first branch 810 for processing to generate lesion detection prediction information 821; inputting the lesion detection prediction information 821 into the initial regression branch 820 for processing to generate a first-class prediction image 822; determining the value of the third loss function 823 based on the first-class prediction image 822 and the sample first-class image 811, and iteratively updating the initial first branch 810 and the initial regression branch 820 based on the value of the third loss function 823 until the termination condition is met, so as to obtain the trained first branch.

[0212] Lesion detection prediction information 821 can refer to an image generated based on the first type of sample image, which can reflect the probability that a lesion exists in the region of interest of the target object.

[0213] The first type of predicted image can refer to T2W images predicted based on lesion detection prediction information.

[0214] Termination conditions can include the convergence of the third loss function 823, the number of iterations reaching a threshold, etc.

[0215] In some embodiments of this specification, the training method based on the above-mentioned multi-branch model can effectively enhance the boundary information of T2W images while reducing the information loss of T2W images during processing. It also makes reasonable use of semantic information that contributes to the sensitivity of DWI / ADC in lesion detection, thereby improving the accuracy of the detected lesion location and contour information while ensuring the sensitivity and specificity of lesion detection.

[0216] Figure 9 This is a block diagram of a lesion detection system according to some embodiments of this specification. For example... Figure 9 As shown, the lesion detection system 900 may include a preprocessing module 910, a processing module 920, an auxiliary processing module 930, and a postprocessing module 940.

[0217] The preprocessing module 910 can be used to perform preset processing on the initial first-type image and the initial second-type image. Preset processing may include one or more of the following: image cropping, image registration, b-value adjustment, and image segmentation. For detailed descriptions of the above preset processing, please refer to [link to relevant documentation]. Figure 5 Related descriptions.

[0218] The processing module 920 can be used to acquire first-type and second-type images of the target object; and to process the first-type and second-type images using a multi-branch model to obtain lesion detection information. For detailed descriptions of the above functions, please refer to [link to documentation]. Figure 2 Related descriptions.

[0219] In some embodiments, the processing module 920 may further be used to generate an edge image corresponding to the first type of image; and to process the first type of image, the second type of image, and the edge image using a multi-branch model to obtain lesion detection information. For a detailed description of the above functions, please refer to [link to relevant documentation]. Figure 6 Related descriptions.

[0220] The auxiliary processing module 930 can be used to process at least one of the first type of image and the second type of image based on the detection model to obtain optimization information. For detailed descriptions of the above functions, please refer to [link to relevant documentation]. Figure 2 Related descriptions.

[0221] The post-processing module 940 can be used to process lesion detection information based on optimization information to generate lesion detection results. For detailed explanations of the above functions, please refer to [link to documentation]. Figure 2 Related descriptions.

[0222] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0223] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0224] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0225] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0226] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0227] For each patent, patent application, patent application publication, and other material, such as articles, books, specifications, publications, and documents, referenced in this specification, the entire contents of which are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification, as well as documents that limit the broadest scope of the claims in this specification (currently or subsequently appended to this specification). It should be noted that in the event of any inconsistency or conflict between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0228] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A method for detecting lesions, characterized in that, The method includes: Obtain the first-class and second-class images of the target object; The first and second types of images are processed using a multi-branch model to obtain lesion detection information, wherein: The multi-branch model includes a first branch for processing a first type of image and a second branch for processing a second type of image; in the shallow part of the multi-branch model, the first branch and the second branch are independent of each other; in the deep part of the multi-branch model, the first branch and the second branch are connected. The detection model processes at least one of the first type of images and the second type of images to obtain optimized information. The optimized information refers to information used to correct the lesion detection information. Specifically, the boundary information contained in the first type of images is superior to the boundary information contained in the second type of images, and the lesion information contained in the second type of images is superior to the lesion information contained in the first type of images. The optimized information includes a first reference result and a second reference result. The detection model includes a first detection model and a second detection model. The first detection model processes the first type of images to generate the first reference result, which is the lesion detection information obtained solely based on the first type of images. The second detection model processes the second type of images to generate the second reference result, which is the lesion detection information obtained solely based on the second type of images. The lesion detection information is processed based on the optimization information to generate lesion detection results.

2. The method according to claim 1, characterized in that, The second type of image includes target diffusion-weighted images; The acquisition of the second type of image of the target object includes: Obtain at least one initial diffusion-weighted image of the target object; The target diffusion-weighted image is generated based on the at least one initial diffusion-weighted image.

3. The method according to claim 1, characterized in that, The first type of image includes T2-weighted images, and the second type of image includes at least one of apparent diffusion coefficient images and target diffusion-weighted images.

4. The method according to claim 1, characterized in that, The acquisition of the first type of image and the second type of image of the target object includes: Obtain the first type of image and the initial second type of image of the target object; The initial second-class image is registered to the first-class image to obtain the second-class image.

5. The method according to claim 1, characterized in that, The process of using a multi-branch model to process the first type of image and the second type of image further includes: Generate the edge image corresponding to the first type of image; The first type of image, the second type of image, and the edge image are processed using the multi-branch model to obtain the lesion detection information, wherein the multi-branch model further includes an edge branch for lesion edge detection of the edge image.

6. The method according to claim 5, characterized in that, The multi-branch model further includes a prediction branch; The prediction branch is used to receive the first output of the first branch and the second output of the edge branch, and to generate the lesion detection information based on the processing of the first output and the second output. The lesion detection information includes at least one of a predicted lesion probability map and a predicted lesion edge map.

7. The method according to claim 1, characterized in that, The first branch is obtained based on joint training with the regression branch; the joint training includes: The first type of sample image is input into the initial first branch for processing to generate lesion prediction information; The lesion prediction information is input into the initial regression branch for processing to generate the first type of prediction image; Based on the first type of predicted image and the sample first type image, the initial first branch and the initial regression branch are iteratively updated until the termination condition is met, so as to obtain the trained first branch.

8. The method according to claim 1, characterized in that, The training of the multi-branch model includes: Multiple training samples are acquired, each training sample including a first-class image, a second-class image, and gold standard lesion detection information; The initial multi-branch model is iteratively updated using the multiple training samples to obtain the multi-branch model, wherein the initial multi-branch model includes an initial first branch, an initial second branch, and an initial prediction branch; The initial second branch is used to process the sample second-class images to generate intermediate detection information; The initial first branch is used to process the sample first-class image and the intermediate detection information; The prediction branch is used to generate lesion prediction results based on the output of the initial first branch; The first difference between the lesion prediction result and the gold standard lesion detection information is used to update the initial first branch and the initial second branch.

9. The method according to claim 1, characterized in that, The optimization information also includes a third reference result, and the detection model also includes a third detection model. The third detection model is used to process the first type of image to obtain the third reference result used to assist in determining the partition information of the region of interest.

10. A lesion detection system, characterized in that, The system includes: The preprocessing module is used to obtain the first and second type images of the target object; Processing module, used for: A multi-branch model is used to process the first and second types of images to obtain lesion detection information. The detection model processes at least one of the first type of images and the second type of images to obtain optimized information. The optimized information refers to information used to correct the lesion detection information. Specifically, the boundary information contained in the first type of images is superior to the boundary information contained in the second type of images, and the lesion information contained in the second type of images is superior to the lesion information contained in the first type of images. The optimized information includes a first reference result and a second reference result. The detection model includes a first detection model and a second detection model. The first detection model processes the first type of images to generate the first reference result, which is the lesion detection information obtained solely based on the first type of images. The second detection model processes the second type of images to generate the second reference result, which is the lesion detection information obtained solely based on the second type of images. The lesion detection information is processed based on the optimized information to generate lesion detection results; wherein: The multi-branch model includes a first branch for processing a first type of image and a second branch for processing a second type of image; in the shallow part of the multi-branch model, the first branch and the second branch are independent of each other; in the deep part of the multi-branch model, the first branch and the second branch are connected.

11. A lesion detection device, comprising a processor, the processor being configured to perform the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium storing computer instructions, wherein when a computer reads the computer instructions in the storage medium, the computer performs the method as described in any one of claims 1 to 9.

Images