Focus detection method, device and equipment and storage medium

Abstract

The embodiment of the invention provides a focus detection method and device, equipment and a storage medium. The method comprises the steps that an original feature map of a three-dimensional medical image is acquired; inputting the original feature map into a plurality of first feature extraction networks connected in sequence and a plurality of second feature extraction networks connected in sequence, and for each first feature extraction network, determining a first transition feature output by the original feature map in the current first feature extraction network; for each second feature extraction network, determining a second transition feature output by the first transition feature in the current second feature extraction network; fusing the first transition feature output by the last first feature extraction network with the second transition feature output by the last second feature extraction network to obtain a target fusion feature; and inputting the target fusion feature into a classification model for classification processing to obtain a focus detection result of the three-dimensional medical image. According to the embodiment of the invention, the accuracy of the focus detection result can be improved.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of medical image detection technology, and in particular to a lesion detection method, device, equipment and storage medium. Background Art

[0002] At present, convolutional neural networks (CNNs) have become the mainstream method in the field of medical image classification due to their powerful feature extraction capabilities. However, traditional convolutional neural networks have limitations in certain specific application scenarios. For example, in the detection of pelvic incomplete fractures (PIF), the image performance is relatively complex. Since two-dimensional medical images are prone to loss of key information, three-dimensional medical images are often required for more accurate analysis. However, traditional convolutional neural networks are mainly used for the recognition of two-dimensional medical images, and their recognition ability for three-dimensional medical images is limited, resulting in low accuracy of lesion detection. Summary of the invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The embodiments of the present application provide a lesion detection method, apparatus, device and storage medium, which can improve the accuracy of lesion detection results.

[0005] To achieve the above-mentioned purpose, the first aspect of an embodiment of the present application proposes a lesion detection method, comprising: acquiring a three-dimensional medical image and extracting an original feature map of the three-dimensional medical image; inputting the original feature map into a plurality of first feature extraction networks connected in sequence, and for each of the first feature extraction networks, determining a first transition feature of the original feature map at the output of the current first feature extraction network, wherein the output of the current first feature extraction network is the input of the next first feature extraction network; inputting the first transition feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in sequence, and for each of the second feature extraction networks, determining a second transition feature of the first transition feature at the output of the current second feature extraction network, wherein the output of the current second feature extraction network is the input of the next second feature extraction network; fusing the first transition feature output by the last first feature extraction network with the second transition feature output by the last second feature extraction network to obtain a target fused feature; inputting the target fused feature into a classification model for classification processing to obtain a lesion detection result of the three-dimensional medical image.

[0006] In some embodiments, the first feature extraction network includes a spatial attention layer and a channel attention layer, and determining the first transition feature of the original feature map in the current output of the first feature extraction network includes: mapping the input of the first feature extraction network based on the spatial attention layer in the current first feature extraction network to obtain a spatial weighted feature map; mapping the input of the first feature extraction network based on the channel attention layer in the current first feature extraction network to obtain a channel weighted feature map; fusing the spatial weighted feature map and the channel weighted feature map to obtain the first transition feature of the current output of the first feature extraction network.

[0007] In some embodiments, the spatial attention layer includes a first convolutional layer and a first fusion layer, and the spatial attention layer based on the current first feature extraction network maps the input of the first feature extraction network to obtain a spatial weighted feature map, including: convolving the input of the current first feature extraction network based on the first convolutional layer in the current first feature extraction network to obtain spatial attention; multiplying the spatial attention with the input of the current first feature extraction network pixel by pixel based on the first fusion layer to obtain a spatial weighted feature map.

[0008] In some embodiments, the channel attention layer includes a maximum pooling layer, an average pooling layer and a second fusion layer, and the channel attention layer based on the current first feature extraction network maps the input of the current first feature extraction network to obtain a channel weighted feature map, including: pooling the input of the current first feature extraction network based on the maximum pooling layer in the current first feature extraction network to obtain a first channel attention; pooling the input of the current first feature extraction network based on the average pooling layer in the current first feature extraction network to obtain a second channel attention; multiplying the first channel attention and the second channel attention with the input of the first feature extraction network channel by channel based on the second fusion layer to obtain a channel weighted feature map.

[0009] In some embodiments, the first feature extraction network also includes a projection convolution layer, and the fusing of the spatial weighted feature map and the channel weighted feature map to obtain the first transition feature currently output by the first feature extraction network includes: fusing the spatial weighted feature map and the channel weighted feature map to obtain a first intermediate feature; mapping the input of the current first feature extraction network based on the projection convolution layer to obtain a second intermediate feature; and performing a residual connection between the first intermediate feature and the second intermediate feature to obtain the first transition feature currently output by the first feature extraction network.

[0010] In some embodiments, the second feature extraction network includes a second convolutional layer and a third convolutional layer, the convolution kernel size of the second convolutional layer is smaller than the convolution kernel size of the third convolutional layer, and determining the second transition feature of the first transition feature in the current output of the second feature extraction network includes: convolving the input of the current second feature extraction network based on the second convolutional layer in the current second feature extraction network to obtain the third transition feature; convolving the third transition feature based on the third convolutional layer in the current second feature extraction network to obtain the second transition feature.

[0011] In some embodiments, the number of the second feature extraction networks is the same as the number of the first feature extraction networks, the first feature extraction network corresponds to the second feature extraction network one-to-one, and the third transition feature is convolved based on the third convolutional layer in the current second feature extraction network to obtain the second transition feature, including: if the current second feature extraction network is not the first second feature extraction network, the input of the current second feature extraction network is fused with the reference feature to obtain the fourth transition feature, wherein the reference feature is the first transition feature output by the first feature extraction network corresponding to the current second feature extraction network; the fourth transition feature is convolved based on the third convolutional layer in the current second feature extraction network to obtain the second transition feature.

[0012] To achieve the above-mentioned purpose, the second aspect of an embodiment of the present application proposes a lesion detection device, comprising: an acquisition module, used to acquire a three-dimensional medical image and extract an original feature map of the three-dimensional medical image; a first feature extraction module, used to input the original feature map into a plurality of first feature extraction networks connected in sequence, and for each of the first feature extraction networks, determine the first transition feature of the original feature map at the output of the current first feature extraction network, wherein the output of the current first feature extraction network is the input of the next first feature extraction network; a second feature extraction module, used to input the first transition feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in sequence, and for each of the second feature extraction networks, determine the second transition feature of the first transition feature at the output of the current second feature extraction network, wherein the output of the current second feature extraction network is the input of the next second feature extraction network; a feature fusion module, used to fuse the first transition feature output by the last first feature extraction network with the second transition feature output by the last second feature extraction network to obtain a target fusion feature; and a classification module, used to input the target fusion feature into a classification model for classification processing to obtain a lesion detection result of the three-dimensional medical image.

[0013] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the lesion detection method described in the first aspect when executing the computer program.

[0014] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, it implements the lesion detection method described in the first aspect above.

[0015] The embodiments of the present application include at least the following beneficial effects: by extracting the original feature map of the three-dimensional medical image, and then inputting the original feature map into a plurality of first feature extraction networks connected in sequence, the first transition features output by each first feature extraction network can be determined in sequence, which is equivalent to extracting feature information of different levels in the original feature map through the first feature extraction network, and can perform multi-angle feature extraction on small target lesions in the three-dimensional medical image, effectively enhancing the sensitivity of the network to small targets, thereby improving the detection capability of small target lesions; in addition, the first transition feature output by the first first feature extraction network is input into a plurality of second feature extraction networks connected in sequence, and the second transition features output by each second feature extraction network can be determined in sequence, which is equivalent to extracting feature information of different levels in the original feature map through the first feature extraction network. The feature extraction network extracts feature information of different levels in the first transition feature, which can further enhance the network's ability to capture feature information in three-dimensional medical images, thereby further improving the detection capability of small target lesions. Then, the first transition feature output by the last first feature extraction network is fused with the second transition feature output by the last second feature extraction network to obtain the target fusion feature, and then the target fusion feature is classified and processed through the classification model to obtain the lesion detection result of the three-dimensional medical image. By combining the extraction results of the first feature extraction network and the extraction results of the second feature extraction network for classification processing, the feature extraction advantages of the first feature extraction network and the second feature extraction network for small target lesions can be effectively integrated, thereby effectively improving the accuracy of lesion detection.

[0016] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0018] Figure 1 A schematic diagram of an optional flow chart of a lesion detection method provided in an embodiment of the present application;

[0019] Figure 2 An optional flow chart of feature extraction provided in an embodiment of the present application;

[0020] Figure 3 An optional flowchart of spatial attention processing provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of an optional flow chart of channel attention processing provided in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of an optional process for dual-dimensional fusion of space and channel provided in an embodiment of the present application;

[0023] Figure 6 Another optional flow chart of feature extraction provided in the embodiment of the present application;

[0024] Figure 7 A schematic diagram of an optional process of cascade feature extraction provided in an embodiment of the present application;

[0025] Figure 8 An optional flowchart of the reasoning process provided in the embodiment of the present application;

[0026] Fig. 9 An optional flowchart of spatial and channel attention processing provided in an embodiment of the present application;

[0027] Fig.10 An optional flowchart of the first feature extraction network inference provided in the embodiment of the present application;

[0028] Fig.11 An optional flowchart of convolutional layer reasoning provided in an embodiment of the present application;

[0029] Fig.12 An optional flow chart of the second feature extraction network inference provided in the embodiment of the present application;

[0030] Fig.13 Another optional flow chart of the second feature extraction network inference provided in the embodiment of the present application;

[0031] Fig.14A schematic diagram of an optional structure of a lesion detection device provided in an embodiment of the present application;

[0032] Fig.15 A schematic diagram of an optional hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0034] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the characteristics of the target object such as target object attribute information or attribute information set, the permission or consent of the target object will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. Among them, the target object can be a user. In addition, when the embodiment of the present application needs to obtain the target object attribute information, the target object's separate permission or separate consent will be obtained by means of a pop-up window or jumping to a confirmation page, etc., and after the separate permission or separate consent of the target object is clearly obtained, the necessary target object-related data for enabling the normal operation of the embodiment of the present application will be obtained.

[0035] In the description of this application, “several” means one or more, “more” means more than two, “greater than”, “less than”, “exceed”, etc. are understood to exclude the number, and “above”, “below”, “within”, etc. are understood to include the number.

[0036] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0037] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0038] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0039] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0040] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0041] At present, convolutional neural networks (CNNs) have become the mainstream method in the field of medical image classification due to their powerful feature extraction capabilities. However, traditional convolutional neural networks have limitations in certain specific application scenarios. For example, in the detection of pelvic incomplete fractures (PIF), the image performance is relatively complex. Since two-dimensional medical images are prone to loss of key information, three-dimensional medical images are often required for more accurate analysis. However, traditional convolutional neural networks are mainly used for the recognition of two-dimensional medical images, and their recognition ability for three-dimensional medical images is limited, resulting in low accuracy of lesion detection.

[0042] In response to the problem of low accuracy of lesion detection, the present application provides a lesion detection method, device, equipment and storage medium, the method comprising: acquiring a three-dimensional medical image and extracting an original feature map of the three-dimensional medical image; inputting the original feature map into a plurality of first feature extraction networks connected in sequence, and for each of the first feature extraction networks, determining a first transition feature of the original feature map at the output of the current first feature extraction network, wherein the output of the current first feature extraction network is the input of the next first feature extraction network; inputting the first transition feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in sequence, and for each of the second feature extraction networks, determining a second transition feature of the first transition feature at the output of the current second feature extraction network, wherein the output of the current second feature extraction network is the input of the next second feature extraction network; fusing the first transition feature output by the last first feature extraction network with the second transition feature output by the last second feature extraction network to obtain a target fused feature; inputting the target fused feature into a classification model for classification processing to obtain a lesion detection result of the three-dimensional medical image. According to the solution provided in the embodiment of the present application, by extracting the original feature map of the three-dimensional medical image, and then inputting the original feature map into a plurality of first feature extraction networks connected in sequence, the first transition features output by each first feature extraction network can be determined in sequence, which is equivalent to extracting feature information of different levels in the original feature map through the first feature extraction network, and can perform multi-angle feature extraction on small target lesions in the three-dimensional medical image, effectively enhancing the sensitivity of the network to small targets, thereby improving the detection capability of small target lesions; in addition, the first transition feature output by the first first feature extraction network is input into a plurality of second feature extraction networks connected in sequence, and the second transition features output by each second feature extraction network can be determined in sequence, which is equivalent to extracting feature information of different levels in the original feature map through the first feature extraction network. Taking the feature information of different levels in the first transition feature extracted by the network can further enhance the network's ability to capture feature information in three-dimensional medical images, thereby further improving the detection ability of small target lesions, and then fusing the first transition feature output by the last first feature extraction network with the second transition feature output by the last second feature extraction network to obtain the target fusion feature, and then classifying the target fusion feature through the classification model to obtain the lesion detection result of the three-dimensional medical image. By combining the extraction result of the first feature extraction network and the extraction result of the second feature extraction network for classification processing, the feature extraction advantages of the first feature extraction network and the second feature extraction network for small target lesions can be effectively integrated, thereby effectively improving the accuracy of lesion detection.

[0043] The lesion detection method, device, equipment and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the lesion detection method in the embodiments of the present application is described.

[0044] The lesion detection method provided in the embodiment of the present application relates to the field of computer technology. The lesion detection method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the lesion detection method, etc., but is not limited to the above forms.

[0045] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0046] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0047] like Figure 1 As shown, Figure 1 An optional flow chart of a lesion detection method provided in an embodiment of the present application, the lesion detection method can be executed by a server, or can also be executed by a terminal, or can also be executed by a server in cooperation with a terminal, the lesion detection method includes but is not limited to the following steps S110 to S150:

[0048] Step S110, acquiring a three-dimensional medical image, and extracting an original feature map of the three-dimensional medical image.

[0049] Step S120, inputting the original feature map into a plurality of first feature extraction networks connected in sequence, and for each first feature extraction network, determining a first transition feature of the original feature map output by the current first feature extraction network.

[0050] Step S130, inputting the first transition feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in sequence, and for each second feature extraction network, determining the second transition feature of the first transition feature output by the current second feature extraction network.

[0051] Step S140, fusing the first transition feature output by the last first feature extraction network with the second transition feature output by the last second feature extraction network to obtain a target fused feature.

[0052] Step S150, inputting the target fusion features into a classification model for classification processing to obtain a lesion detection result of the three-dimensional medical image.

[0053] The output of the current first feature extraction network is the input of the next first feature extraction network, and the output of the current second feature extraction network is the input of the next second feature extraction network.

[0054] Among them, three-dimensional medical images refer to three-dimensional stereoscopic image data obtained through medical imaging technology. These images can provide detailed information about the internal structure of the human body. Unlike traditional two-dimensional images, three-dimensional medical images contain depth information and can observe the internal structure of the human body from different angles and sections.

[0055] Specifically, the three-dimensional medical image can be obtained in many ways. For example, the three-dimensional medical image can be obtained by performing a tomographic scan on the object to be tested using a computer tomography machine.

[0056] It can be understood that extracting the original feature map of a three-dimensional medical image is equivalent to extracting more abstract and more informative features from the three-dimensional medical image. The original feature map can capture the rich spatial information in the three-dimensional medical image, which helps to more accurately identify and locate lesions. The original feature map can also capture more contextual information, which helps the model understand complex internal structures and lesion characteristics.

[0057] In one possible implementation, reference Figure 8, extract the original feature map of the three-dimensional medical image, specifically, the three-dimensional medical image can be first input into the two-channel 3×3×3 original convolution layer for convolution, and then the convolution result can be input into the two-channel 3×3×3 original maximum pooling layer for pooling to obtain the original feature map of the three-dimensional medical image, which can effectively improve the feature extraction effect, and then the original feature map is input into four first feature extraction networks connected in sequence, and the first transition feature output by the first first feature extraction network is input into four second feature extraction networks connected in sequence, and the first feature extraction network corresponds to the second feature extraction network one by one, and finally the first transition feature output by the last first feature extraction network is fused with the second transition feature output by the last second feature extraction network to obtain the target fusion feature, and the target fusion feature is averaged and pooled and then input into the fully connected layer for mapping to obtain the lesion detection result of the three-dimensional medical image, where the fully connected layer is equivalent to the classification model.

[0058] In one possible implementation, reference Fig.11 , Fig.11 An optional flow chart of convolutional layer inference provided for an embodiment of the present application, taking the original convolutional layer as an example, the original convolutional layer includes a three-dimensional convolutional layer, a batch normalization layer, and an activation layer. The three-dimensional medical image is input into the three-dimensional convolutional layer, the batch normalization layer, and the activation layer connected in sequence to obtain the corresponding convolution result. The architecture of the convolutional layer in other disclosed embodiments of the present application is consistent with the original convolutional layer.

[0059] It can be understood that the three-dimensional convolution layer can flexibly control the depth and size of the feature map by adjusting the number of convolution kernels and the step size; the batch normalization layer can standardize the convolution output, stabilize the training process, increase the convergence speed, and reduce the risk of overfitting of the model; the activation layer uses ReLU (Rectified Linear Unit) as the activation function, which can enhance the nonlinear representation ability of the network and help capture complex image features.

[0060] The first transition feature and the second transition feature are fused, specifically, the first transition feature and the second transition feature are added together, or the first transition feature and the second transition feature are weighted and then added together, which is not limited in the embodiments of the present disclosure.

[0061] Based on this, by extracting the original feature map of the three-dimensional medical image, and then inputting the original feature map into a plurality of first feature extraction networks connected in sequence, the first transition features output by each first feature extraction network can be determined in sequence, which is equivalent to extracting feature information of different levels in the original feature map through the first feature extraction network, and can perform multi-angle feature extraction on small target lesions in the three-dimensional medical image, effectively enhancing the sensitivity of the network to small targets, thereby improving the detection capability of small target lesions. In addition, the first transition feature output by the first first feature extraction network is input into a plurality of second feature extraction networks connected in sequence, and the second transition features output by each second feature extraction network can be determined in sequence, which is equivalent to extracting the first feature through the first feature extraction network. Feature information at different levels in the transition features can further enhance the network's ability to capture feature information in three-dimensional medical images, thereby further improving the ability to detect small target lesions. The first transition feature output by the last first feature extraction network is fused with the second transition feature output by the last second feature extraction network to obtain the target fusion feature, and then the target fusion feature is classified and processed through the classification model to obtain the lesion detection result of the three-dimensional medical image. By combining the extraction results of the first feature extraction network and the extraction results of the second feature extraction network for classification processing, the feature extraction advantages of the first feature extraction network and the second feature extraction network for small target lesions can be effectively integrated, thereby effectively improving the accuracy of lesion detection.

[0062] In a possible implementation, the first feature extraction network, the second feature extraction network and the classification model may be obtained by joint training, and the training process may be specifically as follows:

[0063] First, a three-dimensional training image and a category label corresponding to the three-dimensional training image are obtained, and a training feature map of the three-dimensional training image is extracted;

[0064] Then, the training feature graph is input into a plurality of first feature extraction networks connected in sequence, and for each first feature extraction network, a first training feature of the training feature graph output by the current first feature extraction network is determined, wherein the output of the current first feature extraction network is the input of the next first feature extraction network;

[0065] Then, the first training feature output by the first first feature extraction network is input to a plurality of second feature extraction networks connected in sequence, and for each second feature extraction network, a second training feature of the first training feature output by the current second feature extraction network is determined, wherein the output of the current second feature extraction network is the input of the next second feature extraction network;

[0066] Then, the first training feature output by the last first feature extraction network is fused with the second training feature output by the last second feature extraction network to obtain a training fusion feature;

[0067] Then, the training fusion features are input into the classification model for classification processing to obtain the predicted classification results of the three-dimensional training image;

[0068] Then, the model loss is determined based on the difference between the predicted classification result and the category label, and based on the model loss, the first feature extraction network, the second feature extraction network and the classification model are jointly trained.

[0069] Based on this, by jointly training the first feature extraction network, the second feature extraction network and the classification model, the feature extraction effects of the first feature extraction network and the second feature extraction network, as well as the prediction accuracy of the classification model can be effectively improved, so that accurate lesion detection results can be predicted in the reasoning stage.

[0070] Specifically, the model loss can be determined by the focal loss function, so that the model loss can effectively solve the class imbalance problem in medical image classification, especially in the detection of minority lesions such as PIF. By adjusting the weights of positive and negative samples, the model's attention to minority classes can be significantly improved, reducing the misclassification rate.

[0071] In addition, refer to Figure 2 In one embodiment, the first feature extraction network includes a spatial attention layer and a channel attention layer, and determining the first transition feature of the original feature map output by the current first feature extraction network includes but is not limited to the following steps:

[0072] Step S210, mapping the input of the first feature extraction network based on the spatial attention layer in the current first feature extraction network to obtain a spatial weighted feature map.

[0073] Step S220, mapping the input of the first feature extraction network based on the channel attention layer in the current first feature extraction network to obtain a channel weighted feature map.

[0074] Step S230, fusing the spatial weighted feature map and the channel weighted feature map to obtain the first transition feature output by the current first feature extraction network.

[0075] Among them, the input of the current first feature extraction network is named the first input feature. If the current first feature extraction network is the first first feature extraction network, the first input feature is the original feature map; if the current first feature extraction network is not the first first feature extraction network, the first input feature is the first transition feature output by the previous first feature extraction network.

[0076] Among them, reference Fig.10 The first feature extraction network also includes a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer. The convolution kernel sizes of the fourth convolutional layer and the sixth convolutional layer are smaller than the convolution kernel size of the fifth convolutional layer, and the convolution kernel sizes of the fourth convolutional layer and the sixth convolutional layer are the same. For example, the convolution kernel sizes of the fourth convolutional layer and the sixth convolutional layer are both 1×1×1, and the convolution kernel size of the fifth convolutional layer is both 3×3×3.

[0077] Among them, before the spatial attention layer and the channel attention layer map the first input feature, the first input feature needs to be mapped through each convolution layer of the first feature extraction network. Specifically, the first input feature is input into the fourth convolution layer for convolution. While maintaining the quality of the first input feature, the number of channels of the first input feature is reduced to obtain a first temporary feature. The first temporary feature is input into the fifth convolution layer for convolution, and feature extraction is performed on the first temporary feature to obtain a second temporary feature. The second temporary feature is input into the sixth convolution layer for convolution, and the number of channels of the second temporary feature is restored to the number of channels of the first input feature to obtain a third temporary feature.

[0078] It can be understood that after obtaining the third temporary feature, the third temporary feature is mapped in the spatial attention layer and the channel attention layer respectively, so as to obtain a spatial weighted feature map and a channel weighted feature map.

[0079] Based on this, the input of the first feature extraction network is mapped based on the spatial attention layer in the current first feature extraction network to obtain a spatial weighted feature map. The spatial weighted feature map can focus on the feature changes of the original feature map in the spatial dimension, and can enhance the network's response to local key information, such as lesion edges or texture features, so that the spatial weighted feature map can highlight the important pixels in the input of the first feature extraction network. At the same time, the input of the first feature extraction network is mapped based on the channel attention layer in the current first feature extraction network to obtain a channel weighted feature map. The channel weighted feature map can identify and strengthen the important feature channels in the original feature map, while suppressing useless or redundant information, so that the channel weighted feature map can highlight the important channels of the input of the first feature extraction network. The spatial weighted feature map and the channel weighted feature map are fused to obtain the first transition feature of the current first feature extraction network output, which can simultaneously highlight the important pixels and important channels in the input of the first feature extraction network, provide a reference for the network in two different dimensions, help the network focus resources on important data, reduce dependence on irrelevant information, improve the performance and robustness of the network, and improve the accuracy of lesion detection results.

[0080] In addition, refer to Figure 3In one embodiment, the spatial attention layer includes a first convolutional layer and a first fusion layer, and the input of the first feature extraction network is mapped based on the spatial attention layer in the current first feature extraction network to obtain a spatial weighted feature map, including but not limited to the following steps:

[0081] Step S310, convolving the input of the current first feature extraction network based on the first convolutional layer in the current first feature extraction network to obtain spatial attention.

[0082] Step S320: Based on the first fusion layer, the spatial attention is multiplied pixel by pixel with the input of the current first feature extraction network to obtain a spatial weighted feature map.

[0083] Among them, reference Fig. 9 In the first convolution layer, the first input feature is channel compressed by a 1×1×1 three-dimensional convolution kernel, all channels of the first input feature are merged into one, and the spatial attention of the first input feature is obtained. Then, the spatial attention and the first input feature are input into the first fusion layer for pixel-by-pixel multiplication to obtain a spatial weighted feature map. For example, the size of the first input feature is 1×W×H×C. After the first input feature is convolved with the 1×1×1 three-dimensional convolution kernel, a spatial attention of a size of 1×W×H×1 is obtained. The spatial attention and the first input feature are input into the first fusion layer for pixel-by-pixel multiplication to obtain a spatial weighted feature map of a size of 1×W×H×C.

[0084] It can be understood that the important pixels in the spatially weighted feature map are more prominent, and the unimportant pixels are relatively more suppressed.

[0085] Based on this, spatial attention is obtained by convolving the input of the current first feature extraction network based on the first convolutional layer in the current first feature extraction network. The spatial attention can focus on the feature changes of the image in the spatial dimension, and can enhance the network's response to local key information, such as lesion edges or texture features, reflecting the distribution of important pixels in the input of the first feature extraction network. Then, based on the first fusion layer, the spatial attention is multiplied pixel by pixel with the input of the current first feature extraction network to obtain a spatial weighted feature map, so that important pixels in the spatial weighted feature map are more prominent, and non-important pixels are relatively more suppressed, highlighting potential small target lesions, suppressing background or irrelevant information, improving the detection ability of small target lesions, and effectively improving the accuracy of lesion detection. Specifically, the spatial weighted feature map can identify the features of key positions in the image, can improve the performance of the network when processing complex backgrounds or small targets, and is an effective means to improve target detection and segmentation effects.

[0086] In addition, refer to Figure 4In one embodiment, the channel attention layer includes a maximum pooling layer, an average pooling layer, and a second fusion layer. Based on the channel attention layer in the current first feature extraction network, the input of the current first feature extraction network is mapped to obtain a channel weighted feature map, including but not limited to the following steps:

[0087] Step S410, pooling the input of the current first feature extraction network based on the maximum pooling layer in the current first feature extraction network to obtain the first channel attention.

[0088] Step S420, pooling the input of the current first feature extraction network based on the average pooling layer in the current first feature extraction network to obtain the second channel attention.

[0089] Step S430: Based on the second fusion layer, the first channel attention and the second channel attention are respectively multiplied channel by channel with the input of the first feature extraction network to obtain a channel weighted feature map.

[0090] Among them, reference Fig. 9 After the first input feature is subjected to maximum pooling, the feature map corresponding to each channel of the first input feature is compressed into a value to obtain the first channel feature. For each channel of the first channel feature, the current channel is locally interacted with the adjacent k channels across channels. Specifically, the first channel feature is expanded into a one-dimensional vector, and the first channel feature is convolved with a one-dimensional convolution kernel of length k. After passing through the fully connected layer, the first channel attention is obtained. For example, after the first input feature of size 1×W×H×C is subjected to maximum pooling, the maximum value of each channel of the first input feature is taken as the channel representative value to obtain the first channel feature. The first channel feature is expanded into a one-dimensional vector to obtain the first channel feature of size 1×1×C. If k is 5, the one-dimensional convolution kernel of length 5 is convolved with the first channel feature. After passing through the fully connected layer, the first channel attention of size 1×1×C is obtained.

[0091] In addition, similarly, after the first input feature is average pooled, the feature map corresponding to each channel of the first input feature is compressed into a value to obtain the second channel feature. For each channel of the second channel feature, the current channel is locally interacted with the adjacent k channels across channels. Specifically, the second channel feature is expanded into a one-dimensional vector, and the second channel feature is convolved with a one-dimensional convolution kernel of length k. After passing through the fully connected layer, the second channel attention is obtained. For example, after the first input feature of size 1×W×H×C is average pooled, the average value of each channel of the first input feature is used as the channel representative value to obtain the second channel feature. The second channel feature is expanded into a one-dimensional vector to obtain a second channel feature of size 1×1×C. If k is 5, a one-dimensional convolution kernel of length 5 is convolved with the second channel feature. After passing through the fully connected layer, the second channel attention of size 1×1×C is obtained.

[0092] Among them, based on the second fusion layer, the first channel attention and the second channel attention are respectively multiplied channel by channel with the input of the first feature extraction network. Specifically, the first channel attention, the second channel attention and the first input feature can be multiplied, or the first channel attention, the second channel attention and the first input feature can be weighted and then multiplied. The embodiments of the present disclosure are not limited here.

[0093] Based on this, the input of the current first feature extraction network is pooled based on the maximum pooling layer in the current first feature extraction network to obtain the first channel attention. The first channel attention can assign different weights to each feature channel, identify and strengthen important feature channels, and suppress useless or redundant information, thereby capturing the most significant features in the input of the first feature extraction network, which is conducive to detecting edges or mutation features, and can enhance the image edges corresponding to small target lesions, and improve the recognition of small target lesions. Then, the input of the current first feature extraction network is pooled based on the average pooling layer in the current first feature extraction network to obtain the second channel attention. Similarly, the second channel attention can also be Different weights are assigned to each feature channel to identify and strengthen important feature channels, while suppressing useless or redundant information, thereby capturing the most significant features in the input of the first feature extraction network, better reflecting the overall information of the feature map, and having a better ability to capture the global context. Finally, based on the second fusion layer, the first channel attention and the second channel attention are multiplied channel by channel with the input of the first feature extraction network to obtain a channel-weighted feature map, so that the channel-weighted feature map can comprehensively utilize the advantages of both and enhance the information of important channels, which is equivalent to enhancing the network's selectivity for image features, allowing the network to focus resources on important channels, improving the performance and robustness of the network, and improving the accuracy of lesion detection results.

[0094] In addition, refer to Figure 5 In one embodiment, the first feature extraction network further includes a projection convolution layer, which fuses the spatial weighted feature map and the channel weighted feature map to obtain the first transition feature output by the current first feature extraction network, including but not limited to the following steps:

[0095] Step S510, fusing the spatial weighted feature map and the channel weighted feature map to obtain a first intermediate feature.

[0096] Step S520, mapping the input of the current first feature extraction network based on the projection convolution layer to obtain a second intermediate feature.

[0097] Step S530, performing a residual connection on the first intermediate feature and the second intermediate feature to obtain a first transition feature output by the current first feature extraction network.

[0098] Among them, reference Fig. 9 , the spatial weighted feature map and the channel weighted feature map are fused. Specifically, the spatial weighted feature map and the channel weighted feature map can be added, or the spatial weighted feature map and the channel weighted feature map can be weighted and added to obtain the first intermediate feature, which is not limited in the embodiments of the present disclosure.

[0099] Among them, the convolution kernel size of the projection convolution layer is the same as the convolution kernel size of the fifth convolution layer. For example, the convolution kernel size of the projection convolution layer and the convolution kernel size of the fifth convolution layer can both be 3×3×3.

[0100] Among them, reference Fig.10 , convolve the first input feature with the convolution kernel of the projection convolution layer to obtain the second intermediate feature.

[0101] Among them, the first intermediate feature is residually connected with the second intermediate feature, which is equivalent to adding the first intermediate feature to the second intermediate feature to obtain the third intermediate feature, and then the third intermediate feature is input into the fully connected layer with the activation function of ReLU to obtain the first transition feature.

[0102] Based on this, the first intermediate feature is obtained by fusing the spatial weighted feature map and the channel weighted feature map, and the multi-dimensional features are fused to enhance the features. The detail features in the medical image are doubly enhanced to fully explore the synergy between the two. By comprehensively processing the channel and space in the same module, the network can simultaneously focus on the key information of the feature map in different dimensions, and map the input of the current first feature extraction network based on the projection convolution layer to obtain the second intermediate feature. Finally, the first intermediate feature is residually connected with the second intermediate feature to obtain the first transition feature output by the current first feature extraction network. Therefore, the second intermediate feature closer to the input layer is directly merged with the first intermediate feature closer to the output layer through the residual connection, and the information of the second intermediate feature can be directly transmitted to the later layer, which simplifies the transmission of information, reduces the training cost of the network, increases the feature richness, and thus improves the accuracy of the lesion detection results.

[0103] Specifically, the gradient vanishing problem that is prone to occur in deep neural networks during training is solved by introducing residual connections. Residual connections refer to adding short-circuit paths between network layers, directly introducing inputs to outputs, thereby simplifying information transfer. This design enables the network to train deep structures more efficiently and has achieved remarkable success in tasks such as large-scale image classification and detection.

[0104] In addition, refer to Figure 8 Specifically, there may be 4 first feature extraction networks connected in sequence and 4 second feature extraction networks connected in sequence. Among the 4 first feature extraction networks connected in sequence, the first first feature extraction network may include 3 network modules, the second first feature extraction network may include 4 network modules, the third first feature extraction network may include 6 network modules, and the fourth first feature extraction network may include 3 network modules. Each network module may include a projection convolution layer, a fourth convolution layer, a fifth convolution layer, a sixth convolution layer, a spatial attention layer, a channel attention layer, and a ReLU layer.

[0105] In addition, refer to Figure 6 In one embodiment, the second feature extraction network includes a second convolution layer and a third convolution layer, the convolution kernel size of the second convolution layer is smaller than the convolution kernel size of the third convolution layer, and determining the second transition feature of the first transition feature output by the current second feature extraction network includes but is not limited to the following steps:

[0106] Step S610, convolving the input of the current second feature extraction network based on the second convolutional layer in the current second feature extraction network to obtain a third transition feature.

[0107] Step S620, convolving the third transition feature based on the third convolutional layer in the current second feature extraction network to obtain a second transition feature.

[0108] Among them, the input of the current second feature extraction network is named the second input feature. If the current second feature extraction network is the first second feature extraction network, the second input feature is the first transition feature output by the first first feature extraction network; if the current second feature extraction network is not the first second feature extraction network, the second input feature is the second transition feature output by the previous second feature extraction network.

[0109] Among them, reference Fig.12 The convolution kernel of the second convolution layer can be a 1×1×1 three-dimensional convolution kernel. The convolution kernel of the second convolution layer is convolved with the second input feature to obtain the third transition feature, which reduces the number of channels and can reduce the amount of data to be processed by the second feature extraction network.

[0110] The convolution kernel of the third convolution layer may be a 3×3×3 three-dimensional convolution kernel, and the convolution kernel of the third convolution layer is convolved with the third transition feature, and local feature extraction is performed on the third transition feature to obtain the second transition feature.

[0111] It should be noted that since the convolution kernels of the second convolution layer and the third convolution layer can be three-dimensional convolution kernels, and the three-dimensional medical image corresponds to multiple stacked slices, it is equivalent to that the second convolution layer and the third convolution layer can process multiple slices at one time, which can increase the processing efficiency of the network and enhance the network's ability to capture the spatial information between the upper and lower slices.

[0112] Based on this, by convolving the first transition feature based on the second convolutional layer in the current second feature extraction network to obtain the third transition feature, the amount of data processed by the network can be initially reduced, making the network respond faster and the reasoning efficiency higher. Then, based on the third convolutional layer in the current second feature extraction network, the third transition feature is convolved to obtain the second transition feature, and local feature extraction can be performed on the third transition feature, so that the second transition feature can further aggregate deeper image information of the three-dimensional medical image, further improve the network's response to the target area, thereby improving the accuracy of the lesion detection results.

[0113] Specifically, by introducing the second feature extraction network, the network's perception of three-dimensional structures can be effectively enhanced, effectively avoiding the problem of inaccurate classification caused by the loss of spatial information in traditional methods. By capturing deep spatial features in images, the ability to detect complex lesions can be greatly improved, especially in the location and identification of small target lesions.

[0114] In addition, refer to Figure 7In one embodiment, the number of the second feature extraction networks is the same as the number of the first feature extraction networks, the first feature extraction network corresponds to the second feature extraction network one by one, and the third transition feature is convolved based on the third convolutional layer in the current second feature extraction network to obtain the second transition feature, including but not limited to the following steps:

[0115] Step S710: If the current second feature extraction network is not the first second feature extraction network, the input of the current second feature extraction network is fused with the reference feature to obtain a fourth transition feature.

[0116] Step S720, convolving the fourth transition feature based on the third convolutional layer in the current second feature extraction network to obtain the second transition feature.

[0117] The reference feature is a first transition feature output by the first feature extraction network corresponding to the current second feature extraction network.

[0118] For example, again refer to Figure 8 , and refer to Fig.13 , each first feature extraction network has a corresponding second feature extraction network, the reference feature and the second input feature need to be upsampled first, the sizes of the two need to be aligned, and then the reference feature and the second input feature are added or multiplied, or weighted summed, to obtain the fourth transition feature, which is not limited in the embodiments of the present disclosure.

[0119] It should be noted that fusing the third transition feature with the fourth transition feature and then performing local feature extraction through the third convolutional layer in the current second feature extraction network is equivalent to performing local feature extraction on the features extracted by all previous networks at the same time to obtain the second transition feature.

[0120] Based on this, if the current second feature extraction network is not the first second feature extraction network, the fourth transition feature is obtained by fusing the input of the current second feature extraction network with the reference feature. The fourth transition feature can contain the feature information extracted by all previous networks and the current corresponding first feature extraction network, and then the fourth transition feature is convolved based on the third convolutional layer in the current second feature extraction network to obtain the second transition feature. With the multi-level and multi-scale feature information in the fourth transition feature, the second transition feature can aggregate the features of three-dimensional medical images at multiple angles, which can improve the network's understanding of three-dimensional medical images and strengthen the detection of small target lesions, thereby improving the accuracy of lesion detection results.

[0121] In a possible implementation, the third transition feature is fused with the reference feature to obtain a fifth transition feature, and then the fifth transition feature is input into the third convolutional layer of the current second feature extraction network for convolution to obtain the second transition feature.

[0122] In addition, refer to Fig.14 The present application also provides a lesion detection device 1400, comprising:

[0123] An acquisition module 1410 is used to acquire a three-dimensional medical image and extract an original feature map of the three-dimensional medical image;

[0124] A first feature extraction module 1420 is used to input the original feature map into a plurality of first feature extraction networks connected in sequence, and for each first feature extraction network, determine a first transition feature of the original feature map output by a current first feature extraction network, wherein the output of the current first feature extraction network is the input of a next first feature extraction network;

[0125] The second feature extraction module 1430 is used to input the first transition feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in sequence, and for each second feature extraction network, determine the second transition feature of the first transition feature output by the current second feature extraction network, wherein the output of the current second feature extraction network is the input of the next second feature extraction network;

[0126] A feature fusion module 1440 is used to fuse the first transition feature output by the last first feature extraction network with the second transition feature output by the last second feature extraction network to obtain a target fusion feature;

[0127] The classification module 1450 is used to input the target fusion features into the classification model for classification processing to obtain the lesion detection result of the three-dimensional medical image.

[0128] It can be understood that the specific implementation of the lesion detection device 1400 is basically the same as the specific embodiment of the above-mentioned lesion detection method, and will not be repeated here.

[0129] In addition, refer to Fig.15 , Fig.15 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0130] The processor 1501 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0131] The memory 1502 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1502 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1502, and the processor 1501 is called to execute the lesion detection method of the embodiment of the present application.

[0132] Input / output interface 1503, used to implement information input and output;

[0133] Communication interface 1504, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0134] A bus 1505 that transmits information between the various components of the device (e.g., the processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504);

[0135] The processor 1501 , the memory 1502 , the input / output interface 1503 and the communication interface 1504 are connected to each other in communication within the device via the bus 1505 .

[0136] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned lesion detection method.

[0137] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0138] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0139] It can be understood by those skilled in the art that Figures 1 to 15 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0140] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0142] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0143] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0144] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0145] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0147] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0148] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A lesion detection method, characterized in that: include: Acquire a three-dimensional medical image, and extract an original feature map of the three-dimensional medical image; Input the original feature map to a plurality of first feature extraction networks connected in sequence, and for each of the first feature extraction networks, determine a first transition feature of the original feature map output by a current first feature extraction network, wherein the output of the current first feature extraction network is the input of a next first feature extraction network; Input the first transition feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in sequence, and for each of the second feature extraction networks, determine a second transition feature of the first transition feature output by the current second feature extraction network, wherein the output of the current second feature extraction network is the input of the next second feature extraction network; Fusing the first transition feature output by the last first feature extraction network with the second transition feature output by the last second feature extraction network to obtain a target fused feature; The target fusion feature is input into a classification model for classification processing to obtain a lesion detection result of the three-dimensional medical image.

2. The lesion detection method according to claim 1, characterized in that: The first feature extraction network includes a spatial attention layer and a channel attention layer, and determining a first transition feature of the original feature map output by the current first feature extraction network includes: Mapping the input of the first feature extraction network based on the spatial attention layer in the current first feature extraction network to obtain a spatial weighted feature map; Mapping the input of the first feature extraction network based on the channel attention layer in the current first feature extraction network to obtain a channel weighted feature map; The spatial weighted feature map and the channel weighted feature map are fused to obtain a first transition feature output by the current first feature extraction network.

3. The lesion detection method according to claim 2, characterized in that: The spatial attention layer includes a first convolutional layer and a first fusion layer, and the spatial attention layer based on the current first feature extraction network maps the input of the first feature extraction network to obtain a spatial weighted feature map, including: Convolving the input of the current first feature extraction network based on the first convolutional layer in the current first feature extraction network to obtain spatial attention; Based on the first fusion layer, the spatial attention is multiplied pixel by pixel with the input of the current first feature extraction network to obtain a spatial weighted feature map.

4. The lesion detection method according to claim 2, characterized in that: The channel attention layer includes a maximum pooling layer, an average pooling layer and a second fusion layer, and the channel attention layer based on the current first feature extraction network maps the input of the current first feature extraction network to obtain a channel weighted feature map, including: Pooling the input of the current first feature extraction network based on the maximum pooling layer in the current first feature extraction network to obtain a first channel attention; Pooling the input of the current first feature extraction network based on the average pooling layer in the current first feature extraction network to obtain a second channel attention; Based on the second fusion layer, the first channel attention and the second channel attention are respectively multiplied channel by channel with the input of the first feature extraction network to obtain a channel weighted feature map.

5. The lesion detection method according to claim 2, characterized in that: The first feature extraction network further includes a projection convolution layer, and the fusion of the spatial weighted feature map and the channel weighted feature map to obtain the first transition feature currently output by the first feature extraction network includes: Fusing the spatial weighted feature map and the channel weighted feature map to obtain a first intermediate feature; Mapping the input of the current first feature extraction network based on the projected convolutional layer to obtain a second intermediate feature; The first intermediate feature is residually connected with the second intermediate feature to obtain a first transition feature output by the current first feature extraction network.

6. The method for detecting lesions according to claim 1, characterized in that: The second feature extraction network includes a second convolution layer and a third convolution layer, a convolution kernel size of the second convolution layer is smaller than a convolution kernel size of the third convolution layer, and determining a second transition feature of the first transition feature output by the current second feature extraction network includes: Convolving the input of the current second feature extraction network based on the second convolutional layer in the current second feature extraction network to obtain a third transition feature; The third transition feature is convolved based on the third convolutional layer in the current second feature extraction network to obtain a second transition feature.

7. The method for detecting lesions according to claim 6, characterized in that: The number of the second feature extraction networks is the same as the number of the first feature extraction networks, the first feature extraction networks correspond to the second feature extraction networks one by one, and the third convolutional layer in the current second feature extraction network is used to convolve the third transition feature to obtain the second transition feature, including: If the current second feature extraction network is not the first second feature extraction network, fusing the input of the current second feature extraction network with the reference feature to obtain a fourth transition feature, wherein the reference feature is the first transition feature output by the first feature extraction network corresponding to the current second feature extraction network; The fourth transition feature is convolved based on the third convolutional layer in the current second feature extraction network to obtain a second transition feature.

8. A lesion detection device, characterized in that: include: An acquisition module, used for acquiring a three-dimensional medical image and extracting an original feature map of the three-dimensional medical image; A first feature extraction module, configured to input the original feature map into a plurality of first feature extraction networks connected in sequence, and determine, for each of the first feature extraction networks, a first transition feature of the original feature map output by a current first feature extraction network, wherein the output of the current first feature extraction network is the input of a next first feature extraction network; A second feature extraction module, configured to input the first transition feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in sequence, and for each of the second feature extraction networks, determine a second transition feature of the first transition feature output by a current second feature extraction network, wherein the output of the current second feature extraction network is the input of a next second feature extraction network; A feature fusion module, used for fusing the first transition feature output by the last first feature extraction network with the second transition feature output by the last second feature extraction network to obtain a target fused feature; The classification module is used to input the target fusion features into a classification model for classification processing to obtain the lesion detection result of the three-dimensional medical image.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the lesion detection method according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the lesion detection method according to any one of claims 1 to 7 is implemented.

Images