Image detection method and related device and equipment

Abstract

The present application discloses an image detection method and related devices and equipment, wherein the image detection method comprises: obtaining a medical image to be tested; performing feature extraction on the medical image to be tested to obtain a first feature map of several dimensions; using the first feature map of preset dimensions as a reference feature map, and using the reference feature map to generate a lesion probability map, wherein the lesion probability map is used to indicate the probability that different areas in the medical image to be tested belong to lesions; fusing the lesion probability map with the first feature maps of several dimensions to obtain a final fused feature map; performing detection processing on the final fused feature map to obtain a detection result about lesions in the medical image to be tested. The above scheme can improve the accuracy of image detection.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image detection method and related devices and equipment. Background Art

[0002] Medical images such as CT (Computed Tomography) are of great clinical significance. For example, doctors can find organ lesions such as pneumonia through medical images. However, the accuracy of manual organ lesion examinations is often extremely dependent on the doctor's experience. In addition, when infectious diseases occur and the number of cases to be examined surges, it is inevitable that the workload will increase and the doctor's energy will decrease, thus affecting the accuracy of the examination.

[0003] With the development of information technology, electronic devices with processing capabilities such as computers are gradually replacing manual tasks in various industries. In the field of clinical applications, medical images are detected by electronic devices to obtain detection results about lesions in medical images to assist doctors in clinical practice. However, existing image detection technologies often have the problem of low accuracy. In view of this, how to improve the accuracy of image detection has become an urgent problem to be solved. Summary of the invention

[0004] The present application provides an image detection method and related devices and equipment.

[0005] In a first aspect, the present application provides an image detection method, comprising: acquiring a medical image to be tested; performing feature extraction on the medical image to be tested to obtain a first feature map of several dimensions; using the first feature map of preset dimensions as a reference feature map, and using the reference feature map to generate a lesion probability map, wherein the lesion probability map is used to represent the probability that different areas in the medical image to be tested belong to lesions; fusing the lesion probability map with the first feature maps of several dimensions to obtain a final fused feature map; performing detection processing on the final fused feature map to obtain a detection result about the lesion in the medical image to be tested.

[0006] Therefore, by performing feature extraction on the acquired medical image to be tested, a first feature map of several dimensions is obtained, and the first feature map of preset dimensions is used as a reference feature map, so as to generate a lesion probability map using the reference feature map, and the lesion probability map is used to represent the probability that different areas in the medical image to be tested belong to lesions, and then the lesion probability map is fused with the first feature maps of several dimensions to obtain a final fused feature map, so that the lesion probability map can be fused with the first feature map as a global feature, so that the final fused feature map can enhance the specificity of the lesion, and then when the final fused feature map is used for detection processing to obtain the detection result of the lesion in the medical image to be tested, the accuracy of image detection can be improved.

[0007] Before using the reference feature map to generate a lesion probability map, the method also includes: using the reference feature map to perform predictive processing to obtain a first probability value of the medical image to be tested containing a lesion; based on the first probability value, determining whether to execute the step of generating a lesion probability map using the reference feature map and subsequent steps.

[0008] Therefore, by using the reference feature map for predictive processing, a first probability value of whether the medical image to be tested contains a lesion is obtained, and based on the first probability value, it is determined whether to execute the step of generating a lesion probability map using the reference feature map and subsequent steps, thereby avoiding false positive detection results when the medical image to be tested does not contain a lesion, which can help to further improve the accuracy of image detection. Moreover, since negative data can be pre-screened before detection, the efficiency of image detection can be improved.

[0009] Among them, based on the first probability value, determining whether to execute the step of generating a lesion probability map using a reference feature map and subsequent steps, including: if the first probability value satisfies a first preset condition, executing the step of generating a lesion probability map using a reference feature map and subsequent steps; or, in the case where the medical image to be tested is a two-dimensional medical image contained in a three-dimensional medical image; based on the first probability value, determining whether to execute the step of generating a lesion probability map using a reference feature map and subsequent steps, including: sorting the first probability values ​​corresponding to the two-dimensional medical image in descending order, and selecting a preset number of first probability values; performing preset processing on the preset number of first probability values ​​to obtain a second probability value; if the second probability value satisfies a second preset condition, executing the step of generating a lesion probability map using a reference feature map and subsequent steps.

[0010] Therefore, when the first probability value meets the first preset condition, the step of generating a lesion probability map using the reference feature map and the subsequent steps are executed, or, when the medical image to be tested is a two-dimensional medical image contained in a three-dimensional medical image, the first probability values ​​of the two-dimensional medical image are sorted in order from large to small, and a preset number of first probability values ​​are selected, and the preset number of first probability values ​​are preset processed to obtain a second probability value, so that when the second probability value meets the second preset condition, the step of generating a lesion probability map using the reference feature map and the subsequent steps are executed, which can help to pre-screen out negative data before detection, thereby improving the accuracy and efficiency of image detection.

[0011] Among them, the first preset condition includes: the first probability value is greater than or equal to the first probability threshold; the second preset condition includes: the second probability value is greater than or equal to the second probability threshold; the preset processing is an average operation.

[0012] Therefore, by setting the first preset condition to that the first probability value is greater than or equal to the first probability threshold, by setting the second preset condition to that the second probability value is greater than or equal to the second probability threshold, and setting the preset processing to the average operation, the calculation amount of the second probability value can be reduced, and the second probability value can accurately reflect the possibility that the three-dimensional medical image contains a lesion. Therefore, when the first probability value is greater than or equal to the first probability threshold, the step of generating a case probability map using the reference feature map and the subsequent steps can be executed, and when the second probability value is greater than or equal to the second probability threshold, the step of generating a case probability map using the reference feature map and the subsequent steps can be executed. Therefore, it is beneficial to pre-screen out negative data before detection, thereby improving the accuracy and efficiency of image detection.

[0013] If the first probability value does not satisfy the first preset condition or the second probability value does not satisfy the second preset condition, it is determined that the medical image to be tested does not contain a lesion.

[0014] Therefore, by determining that the medical image to be tested does not contain a lesion when the first probability value does not satisfy the first preset condition or the second probability value does not satisfy the second preset condition, the user can promptly perceive the negative detection result of the medical image to be tested, which can help improve the user experience.

[0015] The reference feature map is used to generate a lesion probability map, including: counting the gradient value of each pixel point in the reference feature map with respect to the lesion, and generating a class activation map as the lesion probability map.

[0016] Therefore, by statistically analyzing the gradient values ​​of each pixel in the reference feature map with respect to the lesion, a class activation map is generated as a lesion probability map, which can improve the accuracy of the lesion probability map and thus help improve the accuracy of subsequent image detection.

[0017] The method comprises: fusing the lesion probability map with the first feature maps of several dimensions to obtain a final fused feature map, including: encoding the reference feature map using the lesion probability map to obtain a second feature map; and fusing the second feature map with the first feature maps of several dimensions to obtain a final fused feature map.

[0018] Therefore, by using the lesion probability map to encode the reference feature map, a second feature map is obtained, and the second feature map is fused with the first feature maps of several dimensions to obtain the final fused feature map. Therefore, the lesion probability map can be used as a global feature to participate in the feature map fusion, so that the final fused feature map can enhance the specificity of the lesion, which can help improve the accuracy of subsequent image detection.

[0019] Among them, the reference feature map is encoded using the lesion probability map to obtain the second feature map, including: multiplying the pixel value of the first pixel point in the lesion probability map with the pixel value of the second pixel point corresponding to the first pixel point in the reference feature map to obtain the pixel value of the corresponding pixel point in the second feature map.

[0020] Therefore, by multiplying the pixel value of the first pixel point in the lesion probability map with the pixel value of the second pixel point corresponding to the first pixel point in the reference feature map, the pixel value of the corresponding pixel point in the second feature map is obtained, thereby realizing the encoding processing of the lesion probability map to the reference feature map, which can help reduce the amount of calculation.

[0021] The second feature map is fused with the first feature maps of several dimensions to obtain a final fused feature map, including: fusing the second feature map with the first feature map of each dimension sorted in order from high to low dimensions to obtain the final fused feature map.

[0022] Therefore, by fusing the second feature map with the first feature map of each dimension in order from high to low dimensions to obtain the final fused feature map, it is possible to facilitate the fusion of feature maps dimension by dimension, thereby facilitating the full fusion of contextual information, improving the accuracy and feature richness of the final fused feature map, and further, improving the accuracy of subsequent image detection.

[0023] Among them, the reference feature map is the first feature map with the highest dimension; in order from high to low dimensions, the second feature map is fused with the first feature map of each dimension sorted in order to obtain a final fused feature map, including: fusing the reference feature map with the first low-dimensional feature map to obtain a first fused feature map with the same dimension as the first low-dimensional feature map, wherein the first low-dimensional feature map is a first feature map with one dimension lower than the reference feature map; fusing the second feature map with the first fused feature map to obtain a second fused feature map with the same dimension as the first fused feature map; repeatedly fusing the second fused feature map with the second low-dimensional feature map to obtain a new second fused feature map with the same dimension as the second low-dimensional feature map until the first feature maps of several dimensions are fused; wherein the second low-dimensional feature map is a first feature map with one dimension lower than the current second fused feature map; and the second fused feature map finally obtained by fusion is used as the final fused feature map.

[0024] Therefore, by fusing the reference feature map with the first low-dimensional feature map, a first fused feature map with the same dimension as the first low-dimensional feature map is obtained, and the first low-dimensional feature map is a first feature map with one dimension lower than the reference feature map, and the second feature map is fused with the first fused feature map to obtain a second fused feature map with the same dimension as the first fused feature map, thereby repeatedly fusing the second fused feature map with the second low-dimensional feature map to obtain a new second fused feature map with the same dimension as the second low-dimensional feature map, until the first feature maps of several dimensions are fused, and the second low-dimensional feature map is a first feature map with one dimension lower than the current second fused feature map, and the second fused feature map finally fused is used as the final fused feature map, so that the lesion probability map can be coupled with the decoding process of image detection as a global feature, so that the final fused feature map can enhance the specificity of the lesion, and can fully fuse the feature map context information, improve the accuracy and feature richness of the final fused feature map, and thus can be beneficial to improve the accuracy of subsequent image detection.

[0025] Among them, the detection result includes the detection area of ​​the lesion in the medical image to be tested; the method also includes: performing organ detection on the medical image to be tested to obtain the organ area in the medical image to be tested; and obtaining the lesion ratio of the detection area of ​​the lesion in the organ area.

[0026] Therefore, by performing organ detection on the medical image to be tested, the organ area in the medical image to be tested is obtained, and the proportion of the lesion detection area in the organ area is obtained, which can help to use the detection results to further generate reference information that is beneficial to clinical practice, thereby improving the user experience.

[0027] Among them, before extracting features of the medical image to be tested and obtaining first feature maps of several dimensions, the method also includes: preprocessing the medical image to be tested, wherein the preprocessing operation at least includes: normalizing the pixel values ​​of the medical image to be tested within a preset range using a preset window value.

[0028] Therefore, before feature extraction is performed on the medical image to be tested, the medical image to be tested is preprocessed, and the preprocessing operation at least includes: using a preset window value to normalize the pixel value of the medical image to be tested to within a preset range, which can help enhance the contrast of the medical image to be tested, thereby helping to improve the accuracy of the first feature map extracted subsequently.

[0029] Among them, feature extraction is performed on the medical image to be tested to obtain first feature maps of several dimensions, including: using the feature extraction subnetwork of the image detection model to extract features of the medical image to be tested to obtain first feature maps of several dimensions; fusing the lesion probability map with the first feature maps of several dimensions to obtain a final fused feature map, and performing detection processing on the final fused feature map to obtain a detection result of the lesion in the medical image to be tested, including: using the fusion processing subnetwork of the image detection model to fuse the lesion probability map with the first feature maps of several dimensions to obtain a final fused feature map; using the fusion processing subnetwork of the image detection model to perform detection processing on the final fused feature map to obtain a detection result of the lesion in the medical image to be tested.

[0030] Therefore, by utilizing the feature extraction subnetwork of the image detection model to perform feature extraction on the medical image to be tested, the first feature maps of several dimensions are obtained, and the fusion processing subnetwork of the image detection model is used to fuse the lesion probability map with the first feature maps of several dimensions to obtain the final fused feature map, and the fusion processing subnetwork of the image detection model is used to perform detection processing on the final fused feature map to obtain the detection result of the lesion in the medical image to be tested, thereby performing feature extraction, fusion processing, and image detection tasks through the image detection model, which can help improve the efficiency of image detection.

[0031] Among them, before using the feature extraction subnetwork of the image detection model to extract features from the medical image to be tested and obtaining the first feature maps of several dimensions, the method also includes: obtaining a sample medical image, wherein the sample medical image contains an actual area of ​​the lesion; using the feature extraction subnetwork to extract features from the sample medical image to obtain the first sample feature maps of several dimensions; using the first sample feature map of preset dimensions as a reference sample feature map, and using the reference sample feature map to generate a lesion sample probability map, wherein the lesion sample probability map is used to represent the probability that different areas of the sample medical image belong to the lesion; using the fusion processing subnetwork to fuse the lesion sample probability map with the first sample feature maps of several dimensions to obtain a final fused sample feature map; using the fusion processing subnetwork to perform detection processing on the final fused sample feature map to obtain a detection area about the lesion in the sample medical image; and adjusting the network parameters of the image detection model using the difference between the actual area and the detection area.

[0032] Therefore, by obtaining a sample medical image, and the sample medical image contains an actual area of ​​the lesion, and using a feature extraction subnetwork to extract features from the sample medical image, a first sample feature map of several dimensions is obtained, so that the first sample feature map of the preset dimension is used as a reference sample feature map, and the reference sample feature map is used to generate a lesion sample probability map, and the lesion sample probability map is used to represent the probability that different areas of the sample medical image belong to the lesion, and then the lesion probability map is fused with the first sample feature maps of several dimensions to obtain a final fused sample feature map, so as to perform detection processing on the final fused sample feature map to obtain the detection area of ​​the lesion in the sample medical image, and use the difference between the actual area and the detection area to adjust the network parameters of the image detection model. Therefore, in the process of training the image detection model, the lesion sample probability map can be used as a global feature to couple with the decoding process of image detection, so that the final fused sample feature map can enhance the specificity of the lesion, thereby enhancing the sensitivity of the image detection model to the lesion, which can help to improve the training speed of the model.

[0033] Among them, using the difference between the actual area and the detection area to adjust the network parameters of the feature extraction subnetwork and the fusion processing subnetwork includes: using a set similarity loss function to process the actual area and the detection area to determine the loss value of the image detection model; using the loss value to adjust the network parameters of the image detection model at a preset learning rate.

[0034] Therefore, using the set similarity loss function to process the actual area and the detection area to determine the loss value of the image detection model can ensure the accuracy of the loss value, and then using the loss value to adjust the network parameters of the image detection model with a preset learning rate can reduce the difference between the detection area and the actual area during the training process, thereby improving the accuracy of the image detection model.

[0035] The second aspect of the present application provides an image detection device, including an image acquisition module, a feature extraction module, an image generation module, an image fusion module and an image detection model, wherein the image acquisition module is used to acquire a medical image to be tested; the feature extraction module is used to perform feature extraction on the medical image to be tested to obtain a first feature map of several dimensions; the image generation module is used to use the first feature map of preset dimensions as a reference feature map, and generate a lesion probability map using the reference feature map, wherein the lesion probability map is used to indicate the probability that different areas of the medical image to be tested belong to lesions; the image fusion module is used to fuse the lesion probability map with the first feature maps of several dimensions to obtain a final fused feature map; the detection processing module is used to perform detection processing on the final fused feature map to obtain a detection result about the lesion in the medical image to be tested.

[0036] A third aspect of the present application provides an electronic device, comprising a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the graphic detection method in the first aspect.

[0037] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, and when the program instructions are executed by a processor, the graphic detection method in the first aspect described above is implemented.

[0038] The above scheme extracts features from the acquired medical image to be tested, thereby obtaining first feature maps of several dimensions, and uses the first feature map of preset dimensions as a reference feature map, thereby generating a lesion probability map using the reference feature map, and the lesion probability map is used to represent the probability that different areas of the medical image to be tested belong to lesions, and then the lesion probability map is fused with the first feature maps of several dimensions to obtain a final fused feature map, so that the lesion probability map can be fused with the first feature map as a global feature, so that the final fused feature map can enhance the specificity of the lesion, and then when the final fused feature map is used for detection processing to obtain the detection result of the lesion in the medical image to be tested, the accuracy of image detection can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flowchart of an embodiment of the image detection method of the present application;

[0040] Figure 2 It is a schematic diagram of the framework of an embodiment of an image detection model;

[0041] Figure 3 is a flow chart of another embodiment of the image detection method of the present application;

[0042] Figure 4 is a flow chart of an embodiment of training an image detection model;

[0043] Figure 5 It is a schematic diagram of the framework of an embodiment of the image detection device of the present application;

[0044] Figure 6 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0045] Figure 7 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0046] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0047] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0048] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0049] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the image detection method of the present application. Specifically, it may include the following steps:

[0050] Step S11: Acquire a medical image to be tested.

[0051] The medical images to be tested may include CT images and MR (Magnetic Resonance) images, which are not limited here. In one implementation scenario, the medical images to be tested may be images obtained by scanning the lung area, liver area, heart area, etc., which are not limited here and can be set according to the actual application situation. For example, when the lungs need to be examined to screen for pneumonia, the lung area can be scanned; or, when the liver needs to be examined to screen for liver lesions, the liver area can be scanned, and so on. Other application scenarios can be deduced by analogy, and examples are not given one by one here.

[0052] In one implementation scenario, the medical image to be tested may be a two-dimensional medical image; in another implementation scenario, the medical image to be tested may also be a two-dimensional medical image contained in a three-dimensional medical image. For example, a CT scan is performed on the scanned object to obtain three-dimensional CT data, and the medical image to be tested may be a two-dimensional medical image contained in the three-dimensional CT data.

[0053] Step S12: extracting features from the medical image to be tested to obtain first feature maps of several dimensions.

[0054] In one implementation scenario, the medical image to be tested can also be preprocessed before feature extraction, for example, at least the pixel value of the medical image to be tested is normalized to a preset range using a preset window value, so that the contrast of the medical image to be tested can be enhanced and the accuracy of the first feature map extracted can be improved. Specifically, the preset window value can be set according to the scanning part. For example, when the medical image to be tested is a CT image obtained by scanning the lungs, the preset window value can be -1400 to 100 Hounsfield Units (HU), and other parts can be set according to actual conditions, and examples are not given one by one here. In addition, the preset range can be set to 0 to 1, so that when the medical image to be tested is a CT image obtained by scanning the lungs, and the preset window value is -1400 to 100 Hounsfield Units, pixels with pixel values ​​lower than -1400 can be set to -1400, and pixels with pixel values ​​higher than 100 can be set to 100, and finally the pixel values ​​in the range of -1400 to 100 are mapped to the range of 0 to 1. When the preset window value and the preset range are other values, the same can be applied and no further examples are given here.

[0055] In an implementation scenario, in order to improve the convenience of feature extraction, an image detection model can also be pre-trained. The image detection model may include a feature extraction subnetwork, so that the feature extraction subnetwork of the image detection model can be used to extract features from the medical image to be tested, and obtain first feature maps of several dimensions. Several dimensions can be one dimension or multiple dimensions, for example, two dimensions, three dimensions, etc., which are not limited here. The network depth of the feature extraction subnetwork is set according to the actual situation to obtain first feature maps of different dimensions. Among them, the higher the dimension, the larger the number of channels of the corresponding first feature map and the smaller the resolution.

[0056] Please refer to Figure 2 , Figure 2 Schematic diagram of the framework of an embodiment of an image detection model. Figure 2As shown, the feature extraction subnetwork may include multiple sequentially connected feature extraction submodules, and the feature extraction submodule may perform processing tasks such as convolution, regularization, activation and pooling. Specifically, the feature extraction submodule may include any one of a residual block, an inception block, and a dense block, which are not limited here. Specifically, the pooling process may include any one of maximum pooling, average pooling, and a convolution layer with a stride of 2, which are not limited here. In the process of performing feature extraction by the sequentially connected feature extraction submodules, the first feature map from low dimension to high dimension can be obtained sequentially, and when the feature extraction submodule is executed, the number of channels is doubled and the resolution is halved. For example, the resolution of the medical image to be tested is 256*256, Figure 2 The number of channels of the first feature map extracted by the first feature extraction submodule is 64, and the resolution is 128*128. For the convenience of description, the size of the first feature map is uniformly expressed as the number of channels*resolution, that is, 64*128*128. The size of the first feature map extracted by the second feature extraction submodule connected in sequence is 128*64*64, the size of the first feature map extracted by the third feature extraction submodule is 256*32*32, and the size of the first feature map extracted by the fourth feature extraction submodule is 512*16*16. When the feature extraction subnetwork has other structures, it can be deduced by analogy, and no examples are given here one by one.

[0057] Step S13: Using the first feature map of the preset dimension as a reference feature map, and using the reference feature map to generate a lesion probability map.

[0058] The lesion probability map is used to indicate the probability that different areas in the medical image to be tested belong to lesions. Still taking the resolution of the medical image to be tested as 256*256 as an example, the reference feature map may include the first feature map with the highest dimension, that is, the first feature map with a size of 512*16*16. Each pixel in the reference feature map can correspond to a 16*16 area in the medical image to be tested. Therefore, the pixel value of each pixel in the lesion probability map generated by the reference feature map can represent the probability that a 16*16 area in the medical image to be tested belongs to a lesion. In addition, the reference feature map may also include first feature maps of other dimensions. For example, the reference feature map may also include a first feature map of one dimension lower than the first feature map with the highest dimension, which is not limited here. In other implementation scenarios, depending on the specific application situation, the reference feature map may also select first feature maps of other dimensions, which is not limited here.

[0059] In one implementation scenario, in order to improve the accuracy of the lesion probability map, the gradient value of each pixel in the reference feature map with respect to the lesion can be statistically analyzed to generate a class activation map (CAM) as the lesion probability map. In a specific implementation scenario, the reference feature map can be used for prediction processing to obtain a first probability value y that the medical image to be tested contains a lesion. c , so that by calculating the first probability value y c For each pixel in the reference feature map The gradient value in, The k in the above formula represents the kth reference feature map in the reference feature maps of multiple channels, and ij represents the pixel in the i-th row and j-th column of the kth reference feature map. Specifically, an image detection model can be pre-trained, and the image detection model includes a prediction processing sub-network, so that the prediction processing sub-network can be used to perform prediction processing on the reference feature map to obtain a first probability value that the medical image to be tested contains a lesion. Please continue to refer to Figure 2 The prediction processing subnetwork may include a global average pooling and a fully connected layer. Taking the resolution of the medical image to be tested as 256*256 as an example, a global average pooling (GAP) process is performed on the reference feature map of size 512*16*16 to obtain a vector of size 512*1*1, and the vector is processed using a fully connected layer to obtain a first probability value of the medical image to be tested containing a lesion.

[0060] In one implementation scenario, in order to pre-screen negative data before detection and avoid false positive results in subsequent detection, that is, there is no lesion in the medical image to be tested, but a lesion is obtained by detecting the medical image to be tested, and the false positive result will interfere with clinical application, so a reference feature map can be used for prediction processing to obtain a first probability value of the lesion in the medical image to be tested, and based on the first probability value, it is determined whether to perform the step of generating a lesion probability map using the reference feature map and subsequent steps. In a specific implementation scenario, the method of using the reference feature map to perform prediction processing to obtain the first probability value can be specifically referred to the relevant description in the aforementioned implementation scenario, which will not be repeated here. In another specific implementation scenario, a probability threshold can be set in advance, and when the first probability value is lower than the probability threshold, it can be considered that the medical image to be tested does not contain a lesion, and the step of generating a lesion probability map using the reference feature map and subsequent steps can no longer be performed, thereby greatly reducing the possibility of obtaining a false positive result by performing image detection on the medical image to be tested in this case. The probability threshold can be set according to the actual application situation. For example, it can be set to a value lower than 20%, 30%, etc., so that the medical images to be tested that have a high probability of not containing lesions will not be subjected to subsequent testing, so as to improve the detection efficiency and reduce the possibility of false positive results.

[0061] Step S14: Fusing the lesion probability map with the first feature maps of several dimensions to obtain a final fused feature map.

[0062] Specifically, during the fusion process, the lesion probability map can be fused with the first feature maps of several dimensions in order from high to low dimensions to obtain a final fused feature map. In an implementation scenario, the lesion probability map can be used to encode the reference feature map to obtain a second feature map, and the second feature map can be fused with the first feature maps of several dimensions to obtain a final fused feature map, so that the lesion probability map is used as a global feature to participate in the feature map fusion, so that the final fused feature map can enhance the specificity of the lesion, which can be beneficial to improve the accuracy of subsequent image detection. Still taking the resolution of the medical image to be tested as 256*256 as an example, the size of the reference feature map is 512*16*16, and corresponding to each reference feature map of 512 channels, the corresponding lesion probability map is obtained, so the size of the lesion probability map is 512*16*16, and then the lesion probability map of size 512*16*16, the reference feature map of size 512*16*16, the first feature map of size 256*32*32, the first feature map of size 128*64*64, and the first feature map of size 64*128*128 can be fused to obtain the final fused feature map with a channel number of 1.

[0063] In a specific implementation scenario, in order to reduce the amount of calculation for encoding processing, the pixel value of the first pixel in the lesion probability map can be directly multiplied by the pixel value of the second pixel corresponding to the first pixel in the reference feature map to obtain the pixel value of the corresponding pixel of the second feature map. Still taking the resolution of the medical image to be tested as 256*256 as an example, through the above processing, a reference feature map of size 512*16*16 and a lesion probability map of size 512*16*16 corresponding thereto can be obtained. Therefore, for each channel, the pixel value of the first pixel in the lesion probability map can be multiplied by the pixel value of the second pixel corresponding to the first pixel in the reference feature map of the corresponding channel to obtain the pixel value of the corresponding pixel of the second feature map of the corresponding channel. By performing the same processing on 512 channels, a second feature map of size 512*16*16 can be obtained. When the medical image to be tested is an image of other resolutions, or when the reference feature map is an image of other sizes, the same can be applied, and examples are not given one by one here.

[0064] In another specific implementation scenario, in order to improve the accuracy and richness of the final fused image, the second feature map can be fused with the first feature map of each dimension sorted in the above order in order from high to low dimensions to obtain a final fused feature map, which can be conducive to fully integrating contextual information. Specifically, the reference feature map can be fused with the first low-dimensional feature map to obtain a first fused feature map with the same dimension as the first low-dimensional feature map, and the first low-dimensional feature map is a first feature map with a first dimension lower than the reference feature map, and the second feature map is fused with the first fused feature map to obtain a second fused feature map with the same dimension as the first fused feature map, and then the second fused feature map is repeatedly fused with the second low-dimensional feature map to obtain a new second fused feature map with the same dimension as the second low-dimensional feature map, until the first feature maps of several dimensions are fused, wherein the second low-dimensional feature map is a first feature map with a dimension lower than the current second fused feature map, and the second fused feature map finally obtained by fusion can be used as the final fused feature map. Still taking the example of a medical image to be tested with a resolution of 256*256, a reference feature map of size 512*16*16 can be fused with its corresponding first low-dimensional feature map, that is, a first feature map of size 256*32*32. During the fusion process, the number of channels of the reference feature map of size 512*16*16 can be halved and the resolution can be doubled, so as to adjust its size to the same as the corresponding first low-dimensional feature map, and then the adjusted reference feature map and the first low-dimensional feature map of size 256*32*32 can be merged into a feature map of size 512*32*32. In order to facilitate subsequent fusion, the number of channels can be halved to obtain a first fused feature map of the same size as the first low-dimensional feature map, that is, a first fused feature map of size 256*32*32. Then, the second feature map of size 512*16*16 is fused with the first fused feature map. Similarly, during the fusion process, the number of channels of the second feature map of size 512*16*16 can be halved and the resolution can be doubled, so as to adjust its size to the same as the first fused feature map. Then, the adjusted second feature map and the first fused feature map of size 256*32*32 are merged into a feature map of size 512*32*32. To facilitate subsequent fusion, the number of channels can be halved to obtain a second fused feature map of size 512*32*32, that is, a second fused feature map of size 256*32*32. Then, the step of fusing the second fused feature map of size 256*32*32 with the corresponding second low-dimensional feature map (that is, the second feature map of size 128*64*64) is performed. Similarly, during the fusion process, the number of channels of the second fused feature map of size 256*32*32 can be halved and the resolution can be doubled, so as to adjust its size to the same as the corresponding second low-dimensional feature map. Figure 1The adjusted second fused feature map and the corresponding second low-dimensional feature map (i.e., the second feature map of size 128*64*64) are then merged into a feature map of size 256*64*64. To facilitate subsequent fusion, the number of channels can be halved to obtain a new second fused feature map of the same size as the corresponding second low-dimensional feature map, i.e., a second fused feature map of size 128*64*64. Then, the step of fusing the second fused feature map of size 128*64*64 with the corresponding second low-dimensional feature map (i.e., the second feature map of size 64*128*128) is performed. Similarly, during the fusion process, the number of channels of the second fused feature map of size 128*64*64 can be halved and the resolution can be doubled to adjust its size to the same size as the corresponding second low-dimensional feature map. Figure 1 The adjusted second fused feature map and the corresponding second low-dimensional feature map (i.e., the second feature map of size 64*128*128) are then merged into a feature map of size 128*128*128. For the convenience of subsequent processing, the number of channels is also halved to obtain a new second fused feature map of the same size as the corresponding second low-dimensional feature map, i.e., a second fused feature map of size 64*128*128. Since all the first feature maps of various dimensions have been fused, the second fused feature map of size 64*128*128 obtained by the final fusion can be used as the final fused feature map. Other situations can be deduced by analogy, and no examples are given here one by one.

[0065] In another specific implementation scenario, in order to improve the efficiency of fusion processing, an image detection model can be pre-trained, and the image detection model includes a fusion processing sub-network, so that the lesion probability map is fused with the first feature maps of several dimensions by using the fusion processing sub-network of the image detection model to obtain the final fusion feature map. Figure 2 ,like Figure 2As shown, the fusion processing subnetwork includes a plurality of sequentially connected fusion processing submodules, which are used to perform the steps of fusing the lesion probability map with the first feature maps of several dimensions to obtain the final fused feature map. Each fusion processing submodule can perform processing operations such as upsampling, convolution, regularization, activation, merging, convolution, regularization, activation, etc. Taking the fusion process of the higher-dimensional feature map of size 512*16*16 and the corresponding low-dimensional feature map of size 256*32*32 in the aforementioned implementation scenario as an example, the upsampling process is used to double the resolution of the higher-dimensional feature map of size 512*16*16 during the fusion process to obtain a feature map of size 512*32*32; the first convolution process is used to halve the number of channels of the higher-dimensional feature map (i.e., the feature map of size 512*32*32) after the resolution is doubled during the fusion process to obtain a low-dimensional feature map of size and corresponding size. The same feature map (i.e., the size is adjusted to 256*32*32); the merging process is used to merge the adjusted higher-dimensional feature map with its corresponding low-dimensional feature map during the fusion process to double the number of channels, that is, the size of the merged feature map is 512*32*32, and the second convolution process is used to halve the feature map with doubled channels obtained after merging during the fusion process, so that the size of the fused feature map obtained by this fusion is the same as that of the corresponding low-dimensional feature map, that is, a fused feature map of size 256*32*32 is obtained. For details, please refer to the relevant steps in the aforementioned implementation scenario, which will not be repeated here.

[0066] Step S15: Perform detection processing on the final fused feature map to obtain the detection result of the lesion in the medical image to be tested.

[0067] In one implementation scenario, in order to facilitate the doctor's review, the test results may include the detection area of ​​the lesion in the image to be tested. Specifically, the detection area can be represented by a line of a preset color and a preset line type, which is not limited here. In another implementation scenario, the medical image to be tested is a two-dimensional medical image contained in a three-dimensional medical image, so the detection area detected in the two-dimensional medical image can be used to obtain the detection area of ​​the lesion in the three-dimensional medical image. For example, the detection area detected in the two-dimensional medical image can be fused in three-dimensional space such as stacking to obtain the detection area of ​​the lesion in the three-dimensional medical image. The specific setting can be made according to the actual application and is not limited here.

[0068] In another implementation scenario, in order to improve the efficiency of detection processing, an image detection model can be pre-trained, and the image detection model can include a fusion processing sub-network, so that the fusion processing sub-network of the image detection model can be used to detect and process the final fusion feature map to obtain the detection result of the lesion in the medical image to be tested. Figure 2In addition to the multiple fusion processing submodules connected in sequence, the fusion processing subnetwork may also include an activation processing submodule connected to the last one of the multiple fusion processing submodules connected in sequence. The activation processing submodule is used to obtain a feature map with a channel number of 1 after convolution and activation processing of the final fusion feature image, and normalize the feature map to obtain the detection result of the lesion. Specifically, the activation processing submodule may include a convolution layer and an activation layer connected in sequence. The activation layer may use a sigmoid activation function, which may be set according to the actual application situation and is not limited here.

[0069] In another implementation scenario, in order to provide clinical reference, organ detection can also be performed on the medical image to be tested to obtain the organ area in the medical image to be tested, so that the proportion of the lesion detection area in the organ area can be obtained, and then the doctor can be provided with reference information that is beneficial to clinical practice and improve the user experience. For example, lung detection can be performed on the medical image to be tested to obtain the lung lobe area in the medical image to be tested, so that the proportion of the lesion detection area in the lung lobe area can be obtained. Other application scenarios can be deduced by analogy and are not limited here. Specifically, in order to improve the efficiency of organ detection, an organ detection model can also be pre-trained, so that the organ detection model can be used to perform organ detection on the medical image to be tested to obtain the organ area in the medical image to be tested. Specifically, the organ detection model can be based on U-net, FCN (Fully Convolutional Networks), etc., which are not limited here.

[0070] The above scheme extracts features from the acquired medical image to be tested, thereby obtaining first feature maps of several dimensions, and uses the first feature map of preset dimensions as a reference feature map, thereby generating a lesion probability map using the reference feature map, and the lesion probability map is used to represent the probability that different areas of the medical image to be tested belong to lesions, and then the lesion probability map is fused with the first feature maps of several dimensions to obtain a final fused feature map, so that the lesion probability map can be fused with the first feature map as a global feature, so that the final fused feature map can enhance the specificity of the lesion, and then when the final fused feature map is used for detection processing to obtain the detection result of the lesion in the medical image to be tested, the accuracy of image detection can be improved.

[0071] See also Figure 3 , Figure 3 This is a flow chart of another embodiment of the image detection method of the present application. Specifically, it may include the following steps:

[0072] Step S31: Acquire a medical image to be tested.

[0073] For details, please refer to the relevant steps in the above embodiments.

[0074] Step S32: extracting features from the medical image to be tested, and obtaining first feature maps of several dimensions.

[0075] For details, please refer to the relevant steps in the above embodiments.

[0076] Step S33: Using the first feature map of the preset dimension as a reference feature map, performing prediction processing using the reference feature map to obtain a first probability value of the lesion being included in the medical image to be tested.

[0077] Specifically, an image detection model can be pre-trained, and the image detection model includes a prediction processing sub-network, so that the prediction processing sub-network can be used to perform prediction processing on the reference feature map to obtain a first probability value of the lesion in the medical image to be tested. For details, please refer to the relevant steps in the above-mentioned embodiment, which will not be repeated here.

[0078] Step S34: Based on the first probability value, determine whether to execute the step of generating a lesion probability map using the reference feature map and subsequent steps. If so, execute step S35; otherwise, execute step S38.

[0079] In an implementation scenario, when the first probability value satisfies the first preset condition, the step of generating a lesion probability map using the reference feature map and subsequent steps can be determined. Specifically, the first preset condition may include that the first probability value is greater than or equal to a first probability threshold, and the first probability threshold can be set according to actual applications, for example, it can be set to 15%, 20%, 25%, etc., which is not limited here.

[0080] In another implementation scenario, the medical image to be tested is a two-dimensional medical image contained in a three-dimensional medical image, so the first probability values ​​of the two-dimensional medical image can be sorted in order from large to small, and the first preset number of first probability values ​​can be selected. The preset number can be set according to the actual situation, for example, it can be set to 5, 6, etc., which is not limited here. And the preset number of first probability values ​​are processed by preset processing, for example, the preset number of first probability values ​​are averaged to obtain the second probability value. When the second probability value meets the second preset condition, it can be determined to execute the step of generating a lesion probability map using the reference feature map and the subsequent steps. Specifically, the second preset condition can include that the second probability value is greater than or equal to the second probability threshold value. The second probability threshold value can be set according to the actual application, for example, it can be set to 15%, 20%, 25%, etc., which is not limited here.

[0081] Step S35: Generate a lesion probability map using the reference feature map.

[0082] For details, please refer to the relevant steps in the above embodiments.

[0083] Step S36: Fusing the lesion probability map with the first feature maps of several dimensions to obtain a final fused feature map.

[0084] For details, please refer to the relevant steps in the above embodiments.

[0085] Step S37: Perform detection processing on the final fused feature map to obtain the detection result of the lesion in the medical image to be tested.

[0086] For details, please refer to the relevant steps in the above embodiments.

[0087] Step S38: prompting that the medical image to be tested does not contain a lesion.

[0088] When it is determined based on the first probability value that the step of generating a lesion probability map using the reference feature map and subsequent steps are not performed, it can be determined that the medical image to be tested does not contain a lesion. In addition, the medical staff can be prompted that the medical image to be tested does not contain a lesion through text, images, voice, etc., which are not limited here.

[0089] Different from the aforementioned embodiment, by using the reference feature map for predictive processing, a first probability value of the medical image to be tested containing a lesion is obtained, and based on the first probability value, it is determined whether to execute the step of generating a lesion probability map using the reference feature map and subsequent steps, thereby avoiding false positive detection results when the medical image to be tested does not contain a lesion, which can be beneficial to further improve the accuracy of image detection. Moreover, since negative data can be pre-screened before detection, the efficiency of image detection can be improved.

[0090] See also Figure 4 , Figure 4 1 is a flow chart of an embodiment of training an image detection model. Specifically, the image detection model can be trained before the feature extraction subnetwork of the image detection model is used to extract features from the medical image to be tested. The network structure of the image detection model can refer to the above embodiment and will not be described in detail here. Specifically, the training steps may include:

[0091] Step S41: Acquire a sample medical image, wherein the sample medical image contains an actual area of ​​the lesion.

[0092] The sample medical images may include CT images and MR (Magnetic Resonance) images, which are not limited here. Specifically, the sample medical image may be an image obtained by scanning the lung area, liver area, heart area, etc., which is not limited here and can be set according to the actual application situation. In an implementation scenario, the sample medical image may be a two-dimensional medical image contained in a three-dimensional medical image. For example, the three-dimensional CT data obtained by performing a CT scan on the scanned object, then the sample medical image may be a two-dimensional medical image contained in the three-dimensional CT data.

[0093] In one implementation scenario, in order to improve sample diversity, data enhancement may be performed on the sample medical image; in another implementation scenario, in order to improve the contrast of the sample medical image, the pixel value of the sample medical image may be normalized to a preset range using a preset window value. The specific setting method of the preset window value and the preset range can refer to the relevant steps in the aforementioned embodiment, which will not be repeated here.

[0094] Step S42: Use the feature extraction subnetwork to extract features from the sample medical image to obtain first sample feature maps of several dimensions.

[0095] For details, please refer to the relevant steps in the above embodiments.

[0096] Step S43: Using the first sample feature map of the preset dimension as a reference sample feature map, and using the reference sample feature map to generate a lesion sample probability map.

[0097] The lesion sample probability map is used to indicate the probability that different regions in the sample medical image belong to lesions. The specific method for obtaining the lesion sample probability map can refer to the steps for obtaining the lesion probability map in the above embodiment, which will not be repeated here.

[0098] Step S44: using the fusion processing sub-network to fuse the lesion sample probability map with the first sample feature maps of several dimensions to obtain a final fused sample feature map.

[0099] For details, please refer to the relevant steps in the aforementioned embodiments, which will not be described again here.

[0100] Step S45: Use the fusion processing sub-network to perform detection processing on the final fusion sample feature map to obtain the detection area of ​​the lesion in the sample medical image.

[0101] For details, please refer to the relevant steps in the aforementioned embodiments, which will not be described again here.

[0102] Step S46: Using the difference between the actual area and the detected area, adjust the network parameters of the image detection model.

[0103] In one implementation scenario, the sampled set similarity loss function (Dice loss) can be used to process the actual area and the detection area to determine the loss value of the image detection model, so that the loss value can be used to adjust the network parameters of the image detection model at a preset learning rate (for example, 3e-4). In another implementation scenario, the cross entropy loss function (CEloss) can also be used to process the actual area and the detection area to determine the loss value of the image detection model, so that the loss value can be used to adjust the network parameters of the image detection model at a preset learning rate (for example, 3e-4). This is not limited here. In another implementation scenario, please refer to Figure 2 The image detection model also includes a prediction processing subnetwork, which is used to perform prediction processing on the reference sample feature map to obtain the prediction probability that the reference sample feature map contains lesions. During the training process, the prediction probability can also be processed using the binary cross entropy loss function to determine the classification loss value of the image detection model, and the loss value and classification loss value of the image detection model determined by processing the actual area and the detection area are weighted to obtain the weighted loss value of the image detection model, and then the weighted loss value is used to adjust the network parameters of the image detection model.

[0104] In an implementation scenario, a training end condition may also be pre-set. When the preset training end condition is met, the training of the image detection model may be terminated. Specifically, the training end condition may include: the loss value is less than a preset loss threshold, and the number of training times reaches any one of the preset number thresholds, which are not limited here. Specifically, the preset loss threshold and the preset number threshold can be set according to actual conditions. For example, the preset number threshold can be set to 1000 times, 2000 times, etc. This is not limited here.

[0105] In an implementation scenario, the network parameters of the image detection model can be adjusted using loss values ​​by using methods such as Stochastic Gradient Descent (SGD), Batch Gradient Descent (BGD), and Mini-Batch Gradient Descent (MBGD). Batch gradient descent means that all samples are used to update the parameters at each iteration; Stochastic gradient descent means that one sample is used to update the parameters at each iteration; Mini-batch gradient descent means that a batch of samples is used to update the parameters at each iteration, which will not be described in detail here.

[0106] Different from the aforementioned embodiment, by acquiring a sample medical image, and the sample medical image contains an actual area of ​​the lesion, and using a feature extraction subnetwork to extract features from the sample medical image, a first sample feature map of several dimensions is obtained, so that the first sample feature map of the preset dimension is used as a reference sample feature map, and the reference sample feature map is used to generate a lesion sample probability map, and the lesion sample probability map is used to represent the probability that different areas in the sample medical image belong to the lesion, and then the lesion probability map is fused with the first sample feature maps of several dimensions to obtain a final fused sample feature map, so as to perform detection processing on the final fused sample feature map to obtain the detection area of ​​the lesion in the sample medical image, and use the difference between the actual area and the detection area to adjust the network parameters of the image detection model, so that in the process of training the image detection model, the lesion sample probability map can be used as a global feature to couple with the decoding process of image detection, so that the final fused sample feature map can enhance the specificity of the lesion, thereby enhancing the sensitivity of the image detection model to the lesion, and thus can be beneficial to improve the training speed of the model.

[0107] See also Figure 5 , Figure 5 It is a schematic diagram of the framework of an embodiment of an image detection device 50 of the present application. The image detection device 50 includes an image acquisition module 51, a feature extraction module 52, an image generation module 53, an image fusion module 54 and a detection processing module 55. The image acquisition module 51 is used to acquire a medical image to be tested; the feature extraction module 52 is used to perform feature extraction on the medical image to be tested to obtain a first feature map of several dimensions; the image generation module 53 is used to use the first feature map of preset dimensions as a reference feature map, and use the reference feature map to generate a lesion probability map, wherein the lesion probability map is used to indicate the probability that different areas of the medical image to be tested belong to lesions; the image fusion module 54 is used to fuse the lesion probability map with the first feature maps of several dimensions to obtain a final fused feature map; the detection processing module 55 is used to perform detection processing on the final fused feature map to obtain a detection result about the lesion in the medical image to be tested.

[0108] The above scheme extracts features from the acquired medical image to be tested, thereby obtaining first feature maps of several dimensions, and uses the first feature map of preset dimensions as a reference feature map, thereby generating a lesion probability map using the reference feature map, and the lesion probability map is used to represent the probability that different areas of the medical image to be tested belong to lesions, and then the lesion probability map is fused with the first feature maps of several dimensions to obtain a final fused feature map, so that the lesion probability map can be fused with the first feature map as a global feature, so that the final fused feature map can enhance the specificity of the lesion, and then when the final fused feature map is used for detection processing to obtain the detection result of the lesion in the medical image to be tested, the accuracy of image detection can be improved.

[0109] In some embodiments, the image detection device 50 also includes a prediction processing module for performing prediction processing using a reference feature map to obtain a first probability value of a lesion being included in the medical image to be tested. The image detection device 50 also includes an execution determination module for determining whether to execute the step of generating a lesion probability map using the reference feature map and subsequent steps based on the first probability value.

[0110] Different from the aforementioned embodiment, a first probability value of whether the medical image to be tested contains a lesion is obtained by using a reference feature map for predictive processing, and based on the first probability value, it is determined whether to execute the step of generating a lesion probability map using the reference feature map and subsequent steps, thereby avoiding false positive detection results when the medical image to be tested does not contain a lesion, which can help to further improve the accuracy of image detection. Moreover, since negative data can be pre-screened before detection, the efficiency of image detection can be improved.

[0111] In some embodiments, the execution determination module is specifically used to execute the step of generating a lesion probability map using a reference feature map and subsequent steps when the first probability value meets a first preset condition.

[0112] In some embodiments, the medical image to be tested is a two-dimensional medical image contained in a three-dimensional medical image. The execution and determination module also includes a probability selection submodule, which is used to sort the first probability values ​​of the two-dimensional medical image in order from large to small, and select a preset number of first probability values. The execution and determination module also includes a probability processing submodule, which is used to perform preset processing on a preset number of first probability values ​​to obtain a second probability value. The execution and determination module also includes a determination execution submodule, which is used to execute the step of generating a lesion probability map using a reference feature map and subsequent steps when the second probability value meets a second preset condition.

[0113] Different from the aforementioned embodiment, when the first probability value satisfies the first preset condition, the step of generating a lesion probability map using the reference feature map and the subsequent steps are executed, or, when the medical image to be tested is a two-dimensional medical image contained in a three-dimensional medical image, the first probability values ​​of the two-dimensional medical image are sorted in descending order, and a preset number of first probability values ​​are selected, and the preset number of first probability values ​​are processed by a preset process to obtain a second probability value, so that when the second probability value meets the second preset condition, the step of generating a lesion probability map using the reference feature map and the subsequent steps are executed, which can help to pre-screen out negative data before detection, thereby improving the accuracy and efficiency of image detection.

[0114] In some embodiments, the first preset condition includes: the first probability value is greater than or equal to the first probability threshold; the second preset condition includes: the second probability value is greater than or equal to the second probability threshold; and the preset processing is an average operation.

[0115] Different from the aforementioned embodiment, by setting the first preset condition as the first probability value being greater than or equal to the first probability threshold, by setting the second preset condition as the second probability value being greater than or equal to the second probability threshold, and by setting the preset processing as the average operation, the calculation amount of the second probability value can be reduced, and the second probability value can accurately reflect the possibility that the three-dimensional medical image contains a lesion. Therefore, when the first probability value is greater than or equal to the first probability threshold, the step of generating a case probability map using the reference feature map and the subsequent steps can be executed, and when the second probability value is greater than or equal to the second probability threshold, the step of generating a case probability map using the reference feature map and the subsequent steps can be executed. Therefore, it can be beneficial to pre-screen out negative data before detection, thereby improving the accuracy and efficiency of image detection.

[0116] In some embodiments, the execution determination module is further specifically configured to determine that the medical image to be tested does not contain a lesion when the first probability value does not satisfy a first preset condition or the second probability value does not satisfy a second preset condition.

[0117] Different from the aforementioned embodiment, by determining that the medical image to be tested does not contain a lesion when the first probability value does not satisfy the first preset condition or the second probability value does not satisfy the second preset condition, the user can promptly perceive the negative detection result of the medical image to be tested, which can help improve the user experience.

[0118] In some embodiments, the image generation module 53 is specifically used to count the gradient values ​​of each pixel point in the reference feature map with respect to the lesion, and generate a class activation map as a lesion probability map.

[0119] Different from the aforementioned embodiment, by statistically analyzing the gradient values ​​of each pixel point in the reference feature map with respect to the lesion, a class activation map is generated as a lesion probability map, which can improve the accuracy of the lesion probability map and thus help improve the accuracy of subsequent image detection.

[0120] In some embodiments, the image fusion module 54 includes an encoding processing submodule for encoding the reference feature map using the lesion probability map to obtain a second feature map. The image fusion module 54 includes a fusion processing submodule for fusing the second feature map with the first feature maps of several dimensions to obtain a final fused feature map.

[0121] Different from the aforementioned embodiment, the reference feature map is encoded by using the lesion probability map to obtain a second feature map, and the second feature map is fused with the first feature maps of several dimensions to obtain a final fused feature map. Therefore, the lesion probability map can be used as a global feature to participate in the feature map fusion, so that the final fused feature map can enhance the specificity of the lesion, which can help improve the accuracy of subsequent image detection.

[0122] In some embodiments, the encoding processing submodule is specifically used to multiply the pixel value of the first pixel point in the lesion probability map with the pixel value of the second pixel point corresponding to the first pixel point in the reference feature map to obtain the pixel value of the corresponding pixel point in the second feature map.

[0123] Different from the aforementioned embodiment, the pixel value of the first pixel point in the lesion probability map is multiplied by the pixel value of the second pixel point corresponding to the first pixel point in the reference feature map to obtain the pixel value of the corresponding pixel point in the second feature map, thereby realizing the encoding processing of the reference feature map by the lesion probability map, which can help reduce the amount of calculation.

[0124] In some embodiments, the fusion processing submodule is specifically used to fuse the second feature map with the first feature map of each dimension in order from high to low dimensions to obtain a final fused feature map.

[0125] Different from the aforementioned embodiment, by fusing the second feature map with the first feature map of each dimension in order from high to low dimensions to obtain the final fused feature map, it is possible to facilitate the fusion of feature maps dimension by dimension, thereby facilitating the full fusion of contextual information, improving the accuracy and feature richness of the final fused feature map, and further, improving the accuracy of subsequent image detection.

[0126] In some embodiments, the reference feature map is the first feature map with the highest dimension, and the fusion processing submodule includes a first fusion unit for fusing the reference feature map with the first low-dimensional feature map to obtain a first fused feature map with the same dimension as the first low-dimensional feature map, wherein the first low-dimensional feature map is a first feature map with a dimension lower than the reference feature map, and the fusion processing submodule includes a second fusion unit for fusing the second feature map with the first fused feature map to obtain a second fused feature map with the same dimension as the first fused feature map, and the fusion processing submodule includes a third fusion unit for repeatedly fusing the second fused feature map with the second low-dimensional feature map to obtain a new second fused feature map with the same dimension as the second low-dimensional feature map until the first feature maps of several dimensions are fused; wherein the second low-dimensional feature map is a first feature map with a dimension lower than the current second fused feature map, and the fusion processing submodule includes a final fusion unit for using the second fused feature map obtained by the final fusion as the final fused feature map.

[0127] Different from the aforementioned embodiment, by fusing the reference feature map with the first low-dimensional feature map, a first fused feature map with the same dimension as the first low-dimensional feature map is obtained, and the first low-dimensional feature map is a first feature map with one dimension lower than the reference feature map, and the second feature map is fused with the first fused feature map to obtain a second fused feature map with the same dimension as the first fused feature map, thereby repeatedly fusing the second fused feature map with the second low-dimensional feature map to obtain a new second fused feature map with the same dimension as the second low-dimensional feature map, until the first feature maps of several dimensions are fused, and the second low-dimensional feature map is a first feature map with one dimension lower than the current second fused feature map, and the second fused feature map finally obtained by fusion is used as the final fused feature map, so that the lesion probability map can be coupled with the decoding process of image detection as a global feature, so that the final fused feature map can enhance the specificity of the lesion, and can fully fuse the feature map context information, improve the accuracy and feature richness of the final fused feature map, and thus can be beneficial to improve the accuracy of subsequent image detection.

[0128] In some embodiments, the detection result includes a detection area of ​​a lesion in the medical image to be tested, and the image detection device 50 also includes an organ detection module for performing organ detection on the medical image to be tested to obtain the organ area in the medical image to be tested. The image detection device 50 also includes a ratio acquisition module for acquiring the ratio of the lesion detection area to the organ area.

[0129] Different from the aforementioned embodiments, by performing organ detection on the medical image to be tested, the organ area in the medical image to be tested is obtained, and the proportion of the lesion detection area in the organ area is obtained, which can be beneficial to further generate reference information that is beneficial to clinical practice using the detection results, thereby improving the user experience.

[0130] In some embodiments, the image detection device 50 further includes a preprocessing module for preprocessing the medical image to be detected, wherein the preprocessing operation at least includes: normalizing the pixel values ​​of the medical image to be detected within a preset range using a preset window value.

[0131] Different from the aforementioned embodiment, before feature extraction is performed on the medical image to be tested, the medical image to be tested is preprocessed, and the preprocessing operation at least includes: using a preset window value to normalize the pixel value of the medical image to be tested to within a preset range, which can help enhance the contrast of the medical image to be tested, thereby helping to improve the accuracy of the first feature map extracted subsequently.

[0132] In some embodiments, the feature extraction module 52 is specifically used to use the feature extraction subnetwork of the image detection model to perform feature extraction on the medical image to be tested, and obtain a first feature map of several dimensions. The image fusion module 54 is specifically used to use the fusion processing subnetwork of the image detection model to fuse the lesion probability map with the first feature maps of several dimensions to obtain a final fused feature map. The detection processing module 55 is specifically used to use the fusion processing subnetwork of the image detection model to perform detection processing on the final fused feature map to obtain a detection result about the lesion in the medical image to be tested.

[0133] Different from the aforementioned embodiments, the feature extraction subnetwork of the image detection model is used to perform feature extraction on the medical image to be tested, so as to obtain first feature maps of several dimensions. The fusion processing subnetwork of the image detection model is used to fuse the lesion probability map with the first feature maps of several dimensions to obtain a final fused feature map. The fusion processing subnetwork of the image detection model is used to perform detection processing on the final fused feature map to obtain the detection result of the lesion in the medical image to be tested, thereby performing feature extraction, fusion processing, and image detection tasks through the image detection model, which can help improve the efficiency of image detection.

[0134] In some embodiments, the image detection device 50 includes a sample image acquisition module for acquiring a sample medical image, wherein the sample medical image includes an actual area of ​​the lesion; the image detection device 50 includes a sample feature extraction module for performing feature extraction on the sample medical image using a feature extraction subnetwork to obtain a first sample feature map of several dimensions; the image detection device 50 includes a probability image generation module for using the first sample feature map of a preset dimension as a reference sample feature map, and generating a lesion sample probability map using the reference sample feature map, wherein the lesion sample probability map is used to represent the probability that different areas in the sample medical image belong to the lesion; the image detection device 50 includes a sample image fusion module for using a fusion processing subnetwork to fuse the lesion sample probability map with the first sample feature maps of several dimensions to obtain a final fused sample feature map; the image detection device 50 includes a sample detection processing module for performing detection processing on the final fused sample feature map using a fusion processing subnetwork to obtain a detection area about the lesion in the sample medical image; the image detection device 50 includes a training adjustment module for adjusting the network parameters of the image detection model using the difference between the actual area and the detection area.

[0135] Different from the aforementioned embodiment, by acquiring a sample medical image, and the sample medical image contains an actual area of ​​the lesion, and using a feature extraction subnetwork to extract features from the sample medical image, a first sample feature map of several dimensions is obtained, so that the first sample feature map of the preset dimension is used as a reference sample feature map, and the reference sample feature map is used to generate a lesion sample probability map, and the lesion sample probability map is used to represent the probability that different areas in the sample medical image belong to the lesion, and then the lesion probability map is fused with the first sample feature maps of several dimensions to obtain a final fused sample feature map, so as to perform detection processing on the final fused sample feature map to obtain the detection area of ​​the lesion in the sample medical image, and use the difference between the actual area and the detection area to adjust the network parameters of the image detection model, so that in the process of training the image detection model, the lesion sample probability map can be used as a global feature to couple with the decoding process of image detection, so that the final fused sample feature map can enhance the specificity of the lesion, thereby enhancing the sensitivity of the image detection model to the lesion, and thus can be beneficial to improve the training speed of the model.

[0136] In some embodiments, the training adjustment module includes a loss determination submodule, which is used to process the actual area and the detection area using a set similarity loss function to determine the loss value of the image detection model. The training adjustment module includes a parameter adjustment submodule, which is used to use the loss value to adjust the network parameters of the image detection model at a preset learning rate.

[0137] Different from the aforementioned embodiment, the actual area and the detection area are processed by using a set similarity loss function to determine the loss value of the image detection model, which can ensure the accuracy of the loss value. The loss value is then used to adjust the network parameters of the image detection model at a preset learning rate, which can reduce the difference between the detection area and the actual area during the training process and improve the accuracy of the image detection model.

[0138] See also Figure 6 , Figure 6 : is a schematic diagram of a framework of an embodiment of an electronic device 60 of the present application. The electronic device 60 includes a memory 61 and a processor 62 coupled to each other, and the processor 62 is used to execute program instructions stored in the memory 61 to implement the steps of any of the above-mentioned image detection method embodiments. In a specific implementation scenario, the electronic device 60 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 60 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0139] Specifically, the processor 62 is used to control itself and the memory 61 to implement the steps of any of the above-mentioned image detection method embodiments. The processor 62 can also be referred to as a CPU (Central Processing Unit). The processor 62 may be an integrated circuit chip with signal processing capabilities. The processor 62 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 62 may be implemented by an integrated circuit chip.

[0140] The above solution can improve the accuracy of image detection.

[0141] See also Figure 7 , Figure 7 The schematic diagram of the framework of an embodiment of a computer-readable storage medium 70 of the present application. The computer-readable storage medium 70 stores program instructions 701 that can be executed by a processor, and the program instructions 701 are used to implement the steps of any of the above-mentioned image detection method embodiments.

[0142] The above solution can improve the accuracy of image detection.

[0143] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0144] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0145] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0146] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. An image detection method, characterized in that: include: Acquire a medical image to be tested; Extracting features from the medical image to be tested to obtain first feature maps of multiple dimensions; Using the first feature map of the preset dimension as a reference feature map, and using the reference feature map to generate a lesion probability map, wherein the lesion probability map is used to represent the probability that different areas in the medical image to be tested belong to lesions; Multiplying a pixel value of a first pixel in the lesion probability map by a pixel value of a second pixel corresponding to the first pixel in the reference feature map to obtain a pixel value of a corresponding pixel in the second feature map; In order from high to low dimensions, the second feature map is fused with the first feature map of each dimension sorted in the order to obtain a final fused feature map; The final fused feature map is subjected to detection processing to obtain a detection result of the lesion in the medical image to be tested.

2. The method according to claim 1, characterized in that: Before generating a lesion probability map using the reference feature map, the method further includes: Performing prediction processing using the reference feature map to obtain a first probability value of the lesion being included in the medical image to be tested; Based on the first probability value, determine whether to execute the step of generating a lesion probability map using the reference feature map and subsequent steps.

3. The method according to claim 2, characterized in that The step of determining whether to perform the step of generating a lesion probability map using the reference feature map and subsequent steps based on the first probability value includes: If the first probability value satisfies a first preset condition, the step of generating a lesion probability map using the reference feature map and subsequent steps are performed; or In the case where the medical image to be tested is a two-dimensional medical image contained in a three-dimensional medical image, the step of determining whether to perform the step of generating a lesion probability map using the reference feature map and subsequent steps based on the first probability value includes: sorting the first probability values ​​corresponding to the two-dimensional medical images in descending order, and selecting a preset number of the first probability values; Performing preset processing on the preset number of the first probability values ​​to obtain a second probability value; If the second probability value meets the second preset condition, the step of generating a lesion probability map using the reference feature map and subsequent steps are performed.

4. The method according to claim 3, characterized in that The first preset condition includes: the first probability value is greater than or equal to a first probability threshold; the second preset condition includes: the second probability value is greater than or equal to a second probability threshold; the preset processing is an average operation; And / or, the method further comprises: If the first probability value does not satisfy a first preset condition or the second probability value does not satisfy a second preset condition, it is determined that the medical image to be tested does not contain the lesion.

5. The method according to claim 1, characterized in that The step of generating a lesion probability map by using the reference feature map comprises: The gradient value of each pixel point in the reference feature map with respect to the lesion is counted to generate a class activation map as the lesion probability map.

6. The method according to claim 1, characterized in that The reference feature map is the first feature map with the highest dimension; the second feature map is fused with the first feature map of each dimension sorted in the order from high to low dimensions to obtain a final fused feature map, including: Fusing the reference feature map with a first low-dimensional feature map to obtain a first fused feature map having the same dimension as the first low-dimensional feature map, wherein the first low-dimensional feature map is the first feature map having a dimension lower than that of the reference feature map; Fusing the second feature map with the first fused feature map to obtain a second fused feature map having the same dimension as the first fused feature map; Repeating the step of fusing the second fused feature map with the second low-dimensional feature map to obtain a new second fused feature map having the same dimension as the second low-dimensional feature map, until the fusion of the first feature maps of the plurality of dimensions is completed, wherein the second low-dimensional feature map is the first feature map having a dimension lower than that of the current second fused feature map; The second fused feature map finally obtained by fusion is used as the final fused feature map.

7. The method according to claim 1, characterized in that The detection result includes a detection area of ​​the lesion in the medical image to be detected, and the method further includes: Performing organ detection on the medical image to be tested to obtain an organ region in the medical image to be tested; Acquiring the proportion of the lesion detected area in the organ area; And / or, before extracting features from the medical image to be tested to obtain first feature maps of multiple dimensions, the method further includes: The medical image to be tested is preprocessed, wherein the preprocessing operation at least includes: using a preset window value to normalize the pixel value of the medical image to be tested to a preset range.

8. The method according to any one of claims 1 to 7, characterized in that: The feature extraction of the medical image to be tested is performed to obtain a first feature map of several dimensions, including: Using a feature extraction subnetwork of an image detection model to extract features from the medical image to be tested, to obtain the first feature graphs of the plurality of dimensions; The step of fusing the lesion probability map with the first feature maps of the plurality of dimensions to obtain a final fused feature map, and the step of performing detection processing on the final fused feature map to obtain a detection result of the lesion in the medical image to be tested, comprises: Using the fusion processing subnetwork of the image detection model to fuse the lesion probability map with the first feature maps of the multiple dimensions to obtain a final fused feature map; The final fusion feature map is detected and processed using the fusion processing subnetwork of the image detection model to obtain a detection result about the lesion in the medical image to be tested.

9. The method according to claim 8, characterized in that Before extracting features from the medical image to be tested using the feature extraction subnetwork of the image detection model to obtain the first feature graphs of the plurality of dimensions, the method further includes: Acquire a sample medical image, wherein the sample medical image contains an actual area of ​​the lesion; Using the feature extraction subnetwork to extract features from the sample medical image, to obtain first sample feature maps of several dimensions; Using the first sample feature map of the preset dimension as a reference sample feature map, and using the reference sample feature map to generate a lesion sample probability map, wherein the lesion sample probability map is used to represent the probability that different areas in the sample medical image belong to lesions; Using the fusion processing sub-network, the lesion sample probability map is fused with the first sample feature maps of the plurality of dimensions to obtain a final fused sample feature map; Using the fusion processing sub-network to perform detection processing on the final fusion sample feature map, to obtain a detection area of ​​the lesion in the sample medical image; The network parameters of the image detection model are adjusted by using the difference between the actual area and the detected area.

10. The method according to claim 9, characterized in that The adjusting the network parameters of the image detection model by utilizing the difference between the actual area and the detection area comprises: Using a set similarity loss function to process the actual area and the detection area to determine a loss value of the image detection model; The loss value is used to adjust the network parameters of the image detection model with a preset learning rate.

11. An image detection device, characterized in that: include: An image acquisition module, used for acquiring a medical image to be tested; A feature extraction module, used for extracting features from the medical image to be tested to obtain first feature maps of several dimensions; An image generation module, configured to use the first feature map of a preset dimension as a reference feature map, and to generate a lesion probability map using the reference feature map, wherein the lesion probability map is used to indicate the probability that different areas in the medical image to be tested belong to lesions; An encoding processing submodule, configured to multiply a pixel value of a first pixel point in the lesion probability map by a pixel value of a second pixel point corresponding to the first pixel point in the reference feature map to obtain a pixel value of a corresponding pixel point in the second feature map; A fusion processing submodule, used to fuse the second feature map with the first feature map of each dimension sorted according to the order from high to low dimensions to obtain a final fused feature map; The detection processing module is used to perform detection processing on the final fusion feature map to obtain the detection result of the lesion in the medical image to be tested.

12. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the image detection method according to any one of claims 1 to 10.

13. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the image detection method according to any one of claims 1 to 10 is implemented.

Images