Lesion detection method, device, equipment and storage medium

Abstract

The embodiment of the application provides a kind of focus detection method, device, equipment and storage medium, the method comprises: obtaining the original feature map of three-dimensional medical image;Original feature map is input to the multiple first feature extraction network connected in turn and the multiple second feature extraction network connected in turn, for each first feature extraction network, determine the first transition feature that original feature map is in the first feature extraction network output of current;For each second feature extraction network, determine the second transition feature that first transition feature is in the second feature extraction network output of current;The first transition feature that last first feature extraction network outputs is fused with the second transition feature that last second feature extraction network outputs, and target fusion feature is obtained;Target fusion feature is input into classification model and is classified processing, and the lesion detection result of three-dimensional medical image is obtained.The embodiment of the application can improve the accuracy of lesion detection result.

Description

TECHNICAL FIELD

[0001] The present application relates to, but is not limited to, the technical field of medical image detection, and in particular to a lesion detection method and device, equipment and a storage medium. BACKGROUND

[0002] At present, convolutional neural networks (CNNs) have become the mainstream method in the field of medical image classification due to their strong feature extraction capability. However, traditional convolutional neural networks have limitations in certain specific application scenarios. For example, in the detection of pelvic incomplete fracture (PIF) lesions, the image performance is relatively complex. Since two-dimensional medical images are prone to loss of key information, three-dimensional medical images are often used for more accurate analysis. However, traditional convolutional neural networks mainly recognize two-dimensional medical images, and have limited recognition ability for three-dimensional medical images, resulting in low accuracy of lesion detection. SUMMARY

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The embodiments of the present application provide a lesion detection method, device, equipment and storage medium, which can improve the accuracy of the lesion detection result.

[0005] To achieve the above-mentioned purpose, the first aspect of the embodiments of the present application provides a lesion detection method, comprising: acquiring a three-dimensional medical image and extracting an original feature map of the three-dimensional medical image; inputting the original feature map into a plurality of first feature extraction networks connected in sequence, for each first feature extraction network, determining a first transition feature of the original feature map output by the current first feature extraction network, wherein the output of the current first feature extraction network is the input of the next first feature extraction network; inputting the first transition feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in sequence, for each second feature extraction network, determining a second transition feature of the first transition feature output by the current second feature extraction network, wherein the output of the current second feature extraction network is the input of the next second feature extraction network; fusing the first transition feature output by the last first feature extraction network and the second transition feature output by the last second feature extraction network to obtain a target fusion feature; inputting the target fusion feature into a classification model for classification processing to obtain a lesion detection result of the three-dimensional medical image.

[0006] In some embodiments, the first feature extraction network comprises a spatial attention layer and a channel attention layer, and the determining the first transition feature output by the current first feature extraction network comprises: mapping the input of the first feature extraction network based on the spatial attention layer in the current first feature extraction network to obtain a spatial weighted feature map; mapping the input of the first feature extraction network based on the channel attention layer in the current first feature extraction network to obtain a channel weighted feature map; and fusing the spatial weighted feature map and the channel weighted feature map to obtain the first transition feature output by the current first feature extraction network.

[0007] In some embodiments, the spatial attention layer comprises a first convolution layer and a first fusion layer, and the mapping the input of the first feature extraction network based on the spatial attention layer in the current first feature extraction network to obtain a spatial weighted feature map comprises: performing convolution on the input of the current first feature extraction network based on the first convolution layer in the current first feature extraction network to obtain a spatial attention; and multiplying the spatial attention and the input of the current first feature extraction network pixel by pixel based on the first fusion layer to obtain the spatial weighted feature map.

[0008] In some embodiments, the channel attention layer comprises a max-pooling layer, an average-pooling layer and a second fusion layer, and the mapping the input of the first feature extraction network based on the channel attention layer in the current first feature extraction network to obtain a channel weighted feature map comprises: performing pooling on the input of the current first feature extraction network based on the max-pooling layer in the current first feature extraction network to obtain a first channel attention; performing pooling on the input of the current first feature extraction network based on the average-pooling layer in the current first feature extraction network to obtain a second channel attention; and multiplying the first channel attention and the second channel attention with the input of the first feature extraction network channel by channel respectively based on the second fusion layer to obtain the channel weighted feature map.

[0009] In some embodiments, the first feature extraction network further comprises a projection convolution layer, and the fusing the spatial weighted feature map and the channel weighted feature map to obtain the first transition feature output by the current first feature extraction network comprises: fusing the spatial weighted feature map and the channel weighted feature map to obtain a first intermediate feature; mapping the input of the current first feature extraction network based on the projection convolution layer to obtain a second intermediate feature; and performing residual connection on the first intermediate feature and the second intermediate feature to obtain the first transition feature output by the current first feature extraction network.

[0010] In some embodiments, the second feature extraction network comprises a second convolution layer and a third convolution layer, a size of a convolution kernel of the second convolution layer is smaller than a size of a convolution kernel of the third convolution layer, and the determining the second transition feature of the first transition feature at the current second feature extraction network comprises: performing convolution on an input of the current second feature extraction network based on the second convolution layer in the current second feature extraction network to obtain a third transition feature; and performing convolution on the third transition feature based on the third convolution layer in the current second feature extraction network to obtain the second transition feature.

[0011] In some embodiments, the number of the second feature extraction networks is the same as the number of the first feature extraction networks, the first feature extraction networks correspond to the second feature extraction networks one by one, and the performing convolution on the third transition feature based on the third convolution layer in the current second feature extraction network to obtain the second transition feature comprises: if the current second feature extraction network is not the first second feature extraction network, fusing an input of the current second feature extraction network with a reference feature to obtain a fourth transition feature, wherein the reference feature is the first transition feature output by the first feature extraction network corresponding to the current second feature extraction network; and performing convolution on the fourth transition feature based on the third convolution layer in the current second feature extraction network to obtain the second transition feature.

[0012] To achieve the above object, a second aspect of the embodiment of the present application provides a lesion detection device, comprising: an acquisition module, configured to acquire a three-dimensional medical image and extract an original feature map of the three-dimensional medical image; a first feature extraction module, configured to input the original feature map into a plurality of first feature extraction networks connected in sequence, and for each first feature extraction network, determine a first transition feature of the original feature map output by the current first feature extraction network, wherein an output of the current first feature extraction network is an input of a next first feature extraction network; a second feature extraction module, configured to input the first transition feature output by a first first feature extraction network into a plurality of second feature extraction networks connected in sequence, and for each second feature extraction network, determine a second transition feature of the first transition feature output by the current second feature extraction network, wherein an output of the current second feature extraction network is an input of a next second feature extraction network; a feature fusion module, configured to fuse the first transition feature output by a last first feature extraction network with the second transition feature output by a last second feature extraction network to obtain a target fusion feature; and a classification module, configured to input the target fusion feature into a classification model for classification processing to obtain a lesion detection result of the three-dimensional medical image.

[0013] To achieve the above object, a third aspect of embodiments of the present application provides an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the lesion detection method of the first aspect when executing the computer program.

[0014] To achieve the above object, a fourth aspect of embodiments of the present application provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the lesion detection method of the first aspect.

[0015] The embodiments of the present application at least have the following beneficial effects: by extracting the original feature map of the three-dimensional medical image, and then inputting the original feature map into the plurality of first feature extraction networks connected in sequence, the first transition features output by each first feature extraction network can be determined in sequence, which is equivalent to extracting different levels of feature information in the original feature map through the first feature extraction network, and can perform multi-angle feature extraction on the small target lesion in the three-dimensional medical image, effectively enhancing the sensitivity of the network to small targets, thereby improving the detection capability of the small target lesion, in addition, the first transition feature output by the first first feature extraction network will be input into the plurality of second feature extraction networks connected in sequence, and the second transition features output by each second feature extraction network can be determined in sequence, which is equivalent to extracting different levels of feature information in the first transition feature through the first feature extraction network, and can further enhance the capturing capability of the network to the feature information in the three-dimensional medical image, thereby further improving the detection capability of the small target lesion, then the first transition feature output by the last first feature extraction network is fused with the second transition feature output by the last second feature extraction network to obtain a target fusion feature, and the target fusion feature is classified and processed through a classification model to obtain the lesion detection result of the three-dimensional medical image, and the classification processing is performed by combining the extraction result of the first feature extraction network and the extraction result of the second feature extraction network, which can effectively integrate the feature extraction advantages of the first feature extraction network and the second feature extraction network to the small target lesion, thereby effectively improving the lesion detection accuracy.

[0016] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and achieved by the structure particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the technical solutions of the present application, constitute a part of the specification, and are used to explain the technical solutions of the present application together with the embodiments of the present application, and do not constitute a limitation on the technical solutions of the present application.

[0018] Figure 1 An optional flowchart of a lesion detection method provided by an embodiment of the present application is shown in FIG. 2.

[0019] Figure 2 An optional flowchart of feature extraction provided by an embodiment of the present application is shown in FIG. 3.

[0020] Figure 3 An optional flowchart of spatial attention processing provided by an embodiment of the present application is shown in FIG. 4.

[0021] Figure 4 An optional flowchart of channel attention processing provided by an embodiment of the present application is shown in FIG. 5.

[0022] Figure 5 An optional flowchart of spatial and channel two-dimensional fusion provided by an embodiment of the present application is shown in FIG. 6.

[0023] Figure 6 Another optional flowchart of feature extraction provided by an embodiment of the present application is shown in FIG. 7.

[0024] Figure 7 An optional flowchart of cascade feature extraction provided by an embodiment of the present application is shown in FIG. 8.

[0025] Figure 8 An optional flowchart of a reasoning process provided by an embodiment of the present application is shown in FIG. 9.

[0026] Figure 9 An optional flowchart of spatial and channel attention processing provided by an embodiment of the present application is shown in FIG. 10.

[0027] Figure 10 An optional flowchart of first feature extraction network reasoning provided by an embodiment of the present application is shown in FIG. 11.

[0028] Figure 11 An optional flowchart of convolution layer reasoning provided by an embodiment of the present application is shown in FIG. 12.

[0029] Figure 12 An optional flowchart of second feature extraction network reasoning provided by an embodiment of the present application is shown in FIG. 13.

[0030] Figure 13 Another optional flowchart of second feature extraction network reasoning provided by an embodiment of the present application is shown in FIG. 14.

[0031] Figure 14An optional structural schematic diagram of the lesion detection device provided by the embodiment of the present application is provided.

[0032] Figure 15 An optional hardware structural schematic diagram of the electronic device provided by the embodiment of the present application is provided. DETAILED DESCRIPTION

[0033] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0034] It should be noted that in each specific embodiment of the present application, when relevant processing needs to be performed on data related to the characteristics of the target object, such as target object attribute information or attribute information set, the permission or consent of the target object will be obtained first, and the collection, use and processing of the data will comply with relevant laws, regulations and standards. The target object can be a user. In addition, when the embodiment of the present application needs to obtain target object attribute information, the individual permission or individual consent of the target object will be obtained through a pop-up window or by jumping to a confirmation page, and after obtaining the individual permission or individual consent of the target object, the necessary target object related data for the normal operation of the embodiment of the present application will be obtained.

[0035] In the description of the present application, the meaning of several is one or more, the meaning of multiple is two or more, greater than, less than, more than, etc. are understood as not including the number, above, below, within, etc. are understood as including the number.

[0036] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or above are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0037] In order to facilitate understanding of the technical solutions provided by the embodiments of the present application, some key terms used by the embodiments of the present application are explained first:

[0038] Artificial Intelligence (AI) is the theory, method, technology and application system of using digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0039] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, automatic driving, intelligent transportation and several other major directions.

[0040] Machine Learning (ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It is a branch of computer science that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0041] At present, Convolutional Neural Networks (CNNs) have become the mainstream method in the field of medical image classification due to their powerful feature extraction capability. However, traditional convolutional neural networks have limitations in certain specific application scenarios. For example, in the detection of pelvic incomplete fracture (PIF), the image manifestation is relatively complex. Since two-dimensional medical images are prone to lose key information, three-dimensional medical images are often used for more accurate analysis. However, traditional convolutional neural networks mainly target two-dimensional medical image recognition, and have limited recognition ability for three-dimensional medical images, resulting in low accuracy of lesion detection.

[0042] To solve the problem of low accuracy of lesion detection, the application provides a lesion detection method, device, equipment and storage medium. The method comprises: acquiring a three-dimensional medical image, and extracting an original feature map of the three-dimensional medical image; inputting the original feature map into a plurality of first feature extraction networks connected in sequence, and determining, for each first feature extraction network, a first transition feature of the original feature map output by the current first feature extraction network, wherein the output of the current first feature extraction network is the input of the next first feature extraction network; inputting the first transition feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in sequence, and determining, for each second feature extraction network, a second transition feature of the first transition feature output by the current second feature extraction network, wherein the output of the current second feature extraction network is the input of the next second feature extraction network; fusing the first transition feature output by the last first feature extraction network and the second transition feature output by the last second feature extraction network to obtain a target fusion feature; and inputting the target fusion feature into a classification model for classification processing to obtain a lesion detection result of the three-dimensional medical image. According to the scheme provided in the embodiments of the application, the original feature map of the three-dimensional medical image is extracted, and then the original feature map is input into a plurality of first feature extraction networks connected in sequence, so that the first transition feature output by each first feature extraction network can be determined in sequence, which is equivalent to extracting feature information of different levels in the original feature map through the first feature extraction network, can perform multi-angle feature extraction on small target lesions in the three-dimensional medical image, effectively enhances the sensitivity of the network to small targets, and thus improves the detection capability of small target lesions. In addition, the first transition feature output by the first first feature extraction network is input into a plurality of second feature extraction networks connected in sequence, so that the second transition feature output by each second feature extraction network can be determined in sequence, which is equivalent to extracting feature information of different levels in the first transition feature through the first feature extraction network, can further enhance the capture capability of the network to feature information in the three-dimensional medical image, and thus further improve the detection capability of small target lesions. Then, the first transition feature output by the last first feature extraction network is fused with the second transition feature output by the last second feature extraction network to obtain a target fusion feature, and then the target fusion feature is classified and processed by a classification model to obtain a lesion detection result of the three-dimensional medical image. By combining the extraction results of the first feature extraction network and the extraction results of the second feature extraction network for classification processing, the advantages of feature extraction of small target lesions by the first feature extraction network and the second feature extraction network can be effectively integrated, and thus the lesion detection accuracy is effectively improved.

[0043] The lesion detection method, device, equipment and storage medium provided by the embodiments of the present application are described in detail through the following embodiments. First, the lesion detection method in the embodiments of the present application is described.

[0044] The lesion detection method provided by the embodiments of the present application relates to the technical field of computers. The lesion detection method provided by the embodiments of the present application can be applied to a terminal, can be applied to a server end, and can also be software running in a terminal or a server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc. The server end can be configured as a stand-alone physical server, can be configured as a server cluster or a distributed system composed of multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms, etc. The software can be an application that implements the lesion detection method, but is not limited to the above forms.

[0045] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0046] The embodiments of the present application are further described below with reference to the accompanying drawings.

[0047] As Figure 1 shown, Figure 1 An optional flowchart of the lesion detection method provided by the embodiments of the present application is shown. The lesion detection method can be executed by a server, or can be executed by a terminal, or can be executed by a server in cooperation with a terminal. The lesion detection method includes but is not limited to the following steps S110 to S150:

[0048] In step S110, a three-dimensional medical image is acquired, and an original feature map of the three-dimensional medical image is extracted.

[0049] Step S120, input the original feature map into a plurality of first feature extraction networks connected in sequence, and for each first feature extraction network, determine a first transition feature output by the current first feature extraction network from the original feature map.

[0050] Step S130, input the first transition feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in sequence, and for each second feature extraction network, determine a second transition feature output by the current second feature extraction network from the first transition feature.

[0051] Step S140, fuse the first transition feature output by the last first feature extraction network with the second transition feature output by the last second feature extraction network to obtain a target fusion feature.

[0052] Step S150, input the target fusion feature into a classification model for classification processing to obtain a lesion detection result of the three-dimensional medical image.

[0053] Wherein, the output of the current first feature extraction network is the input of the next first feature extraction network, and the output of the current second feature extraction network is the input of the next second feature extraction network.

[0054] Wherein, the three-dimensional medical image refers to three-dimensional stereoscopic image data obtained through medical imaging technology. These images can provide detailed information about the internal structure of the human body. Unlike traditional two-dimensional images, three-dimensional medical images contain depth information and can observe the internal structure of the human body from different angles and sections.

[0055] Specifically, the three-dimensional medical image can be obtained in various ways. For example, the three-dimensional medical image can be obtained by tomographic scanning of the object to be measured by a computed tomography machine.

[0056] It can be understood that extracting the original feature map of the three-dimensional medical image is equivalent to extracting more abstract and informative features from the three-dimensional medical image. The original feature map can capture rich spatial information in the three-dimensional medical image, which helps to more accurately identify and locate lesions. The original feature map can also capture more contextual information, which helps the model to understand complex internal structures and lesion characteristics.

[0057] In one possible implementation, with reference to Figure 8The original feature map of the three-dimensional medical image can be extracted, specifically, the three-dimensional medical image can be first input into a two-channel 3*3*3 original convolutional layer for convolution, and then the convolution result is input into a two-channel 3*3*3 original max-pooling layer for pooling to obtain the original feature map of the three-dimensional medical image, which can effectively improve the feature extraction effect. Then the original feature map is input into the four first feature extraction networks connected in sequence, the first transition feature output by the first feature extraction network is input into the four second feature extraction networks connected in sequence, the first feature extraction network and the second feature extraction network correspond one by one, finally the first transition feature output by the last first feature extraction network is fused with the second transition feature output by the last second feature extraction network to obtain the target fusion feature. After the target fusion feature is averaged and pooled, it is input into a fully connected layer for mapping to obtain the lesion detection result of the three-dimensional medical image. Here, the fully connected layer is equivalent to a classification model.

[0058] In a possible implementation, with reference to Figure 11 , Figure 11 An optional flowchart of the convolutional layer inference provided by the embodiments of the present application is provided, taking the original convolutional layer as an example. The original convolutional layer includes a three-dimensional convolutional layer, a batch normalization layer and an activation layer. The three-dimensional medical image is input into the three-dimensional convolutional layer, the batch normalization layer and the activation layer connected in sequence to obtain the corresponding convolutional result. The architecture of the convolutional layer in other disclosed embodiments of the present application is consistent with the original convolutional layer.

[0059] It can be understood that the three-dimensional convolutional layer can flexibly control the depth and size of the feature map by adjusting the number of convolutional kernels and the step size; the batch normalization layer can standardize the convolutional output, stabilize the training process, improve the convergence speed, and reduce the overfitting risk of the model; the activation layer uses ReLU (Rectified Linear Unit) as the activation function, which can enhance the nonlinear representation ability of the network and help capture complex image features.

[0060] The first transition feature and the second transition feature are fused, specifically, the first transition feature and the second transition feature can be added, or the first transition feature and the second transition feature can be weighted and then added. The present disclosure does not limit this.

[0061] Based on this, by extracting the original feature map of the three-dimensional medical image, and then inputting the original feature map into a plurality of first feature extraction networks connected in turn, the first transition features output by each first feature extraction network can be determined in turn, which is equivalent to extracting different levels of feature information in the original feature map through the first feature extraction network, can perform multi-angle feature extraction on small target lesions in the three-dimensional medical image, effectively enhance the sensitivity of the network to small targets, thereby improving the detection capability of small target lesions, in addition, the first transition feature output by the first first feature extraction network will be input into a plurality of second feature extraction networks connected in turn, the second transition features output by each second feature extraction network can be determined in turn, which is equivalent to extracting different levels of feature information in the first transition feature through the first feature extraction network, which can further enhance the network's ability to capture feature information in the three-dimensional medical image, thereby further improving the detection capability of small target lesions, then the first transition feature output by the last first feature extraction network is fused with the second transition feature output by the last second feature extraction network to obtain a target fusion feature, and then the target fusion feature is classified by a classification model to obtain the lesion detection result of the three-dimensional medical image. By combining the classification processing of the extraction results of the first feature extraction network and the extraction results of the second feature extraction network, the feature extraction advantages of the first feature extraction network and the second feature extraction network for small target lesions can be effectively integrated, thereby effectively improving the lesion detection accuracy.

[0062] In a possible implementation, the first feature extraction network, the second feature extraction network and the classification model can be obtained through joint training, and the training process can be as follows:

[0063] First, obtain a three-dimensional training image and a class label corresponding to the three-dimensional training image, and extract a training feature map of the three-dimensional training image;

[0064] Then, input the training feature map into a plurality of first feature extraction networks connected in turn, and for each first feature extraction network, determine a first training feature output by the current first feature extraction network from the training feature map, wherein the output of the current first feature extraction network is the input of the next first feature extraction network.

[0065] Then, input the first training feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in turn, and for each second feature extraction network, determine a second training feature output by the current second feature extraction network from the first training feature, wherein the output of the current second feature extraction network is the input of the next second feature extraction network.

[0066] Then, the first training feature output by the last first feature extraction network is fused with the second training feature output by the last second feature extraction network to obtain a training fusion feature.

[0067] Then, the training fusion feature is input into the classification model for classification processing to obtain a predicted classification result of the three-dimensional training image.

[0068] Then, a model loss is determined based on a difference between the predicted classification result and the category label, and the first feature extraction network, the second feature extraction network and the classification model are jointly trained based on the model loss.

[0069] Based on this, by jointly training the first feature extraction network, the second feature extraction network and the classification model, the feature extraction effect of the first feature extraction network and the second feature extraction network and the prediction accuracy of the classification model can be effectively improved, so that an accurate lesion detection result can be predicted in the inference stage.

[0070] Specifically, the model loss can be determined by a Focal loss loss function, so that the model loss can effectively solve the class imbalance problem in medical image classification, and especially performs well in PIF minority class lesion detection. By adjusting the positive and negative sample weights, the attention of the model to the minority class can be significantly improved, and the misclassification rate can be reduced.

[0071] In addition, with reference to Figure 2 In an embodiment, the first feature extraction network includes a spatial attention layer and a channel attention layer, and the first transition feature output by the current first feature extraction network includes but is not limited to the following steps:

[0072] In step S210, the input of the first feature extraction network is mapped based on the spatial attention layer in the current first feature extraction network to obtain a spatial weighted feature map.

[0073] In step S220, the input of the first feature extraction network is mapped based on the channel attention layer in the current first feature extraction network to obtain a channel weighted feature map.

[0074] In step S230, the spatial weighted feature map and the channel weighted feature map are fused to obtain the first transition feature output by the current first feature extraction network.

[0075] Wherein, the input of the current first feature extraction network is named as a first input feature, if the current first feature extraction network is the first first feature extraction network, the first input feature is the original feature map; if the current first feature extraction network is not the first first feature extraction network, the first input feature is the first transition feature output by the previous first feature extraction network.

[0076] wherein, the reference Figure 10 The first feature extraction network further comprises a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer, the convolutional kernel size of the fourth convolutional layer and the sixth convolutional layer is smaller than the convolutional kernel size of the fifth convolutional layer, and the convolutional kernel size of the fourth convolutional layer is the same as that of the sixth convolutional layer, for example, the convolutional kernel size of the fourth convolutional layer and the sixth convolutional layer is 1x1x1, and the convolutional kernel size of the fifth convolutional layer is 3x3x3.

[0077] Before the spatial attention layer and the channel attention layer map the first input feature, the first input feature needs to be mapped by each convolutional layer of the first feature extraction network, specifically, the first input feature is input into the fourth convolutional layer for convolution, while maintaining the quality of the first input feature, the channel number of the first input feature is reduced to obtain a first temporary feature, the first temporary feature is input into the fifth convolutional layer for convolution, the first temporary feature is extracted to obtain a second temporary feature, and the second temporary feature is input into the sixth convolutional layer for convolution, the channel number of the second temporary feature is restored to the channel number of the first input feature to obtain a third temporary feature.

[0078] It can be understood that after obtaining the third temporary feature, the third temporary feature is mapped in the spatial attention layer and the channel attention layer respectively to obtain a spatial weighted feature map and a channel weighted feature map.

[0079] Based on this, by mapping the input of the first feature extraction network based on the spatial attention layer in the current first feature extraction network, a spatial weighted feature map is obtained, which can pay attention to the feature changes of the original feature map in the spatial dimension, and can enhance the response of the network to local key information, such as lesion edge or texture features, so that the spatial weighted feature map can highlight important pixels in the input of the first feature extraction network. At the same time, by mapping the input of the first feature extraction network based on the channel attention layer in the current first feature extraction network, a channel weighted feature map is obtained, which can identify and strengthen important feature channels in the original feature map, while suppressing useless or redundant information, so that the channel weighted feature map can highlight important channels of the input of the first feature extraction network. The first transition feature output by the current first feature extraction network, which is obtained by fusing the spatial weighted feature map and the channel weighted feature map, can highlight important pixels and important channels in the input of the first feature extraction network at the same time, providing reference for the network in two different dimensions, helping the network to focus on important data, reducing the dependence on irrelevant information, improving the performance and robustness of the network, and improving the accuracy of the lesion detection result.

[0080] In addition, with reference to Figure 3In an embodiment, the spatial attention layer includes a first convolutional layer and a first fusion layer, and the input of the current first feature extraction network is mapped based on the spatial attention layer in the current first feature extraction network to obtain a spatial weighted feature map, including but not limited to the following steps:

[0081] In step S310, the input of the current first feature extraction network is convolved based on the first convolutional layer in the current first feature extraction network to obtain spatial attention.

[0082] In step S320, the spatial attention is multiplied with the input of the current first feature extraction network based on the first fusion layer to obtain a spatial weighted feature map.

[0083] Wherein, the reference Figure 9 In the first convolutional layer, the channel compression is performed on the first input feature by using a 1x1x1 three-dimensional convolutional kernel, all channels of the first input feature are merged into one to obtain the spatial attention of the first input feature, and then the spatial attention is multiplied with the first input feature pixel by pixel in the first fusion layer to obtain the spatial weighted feature map. For example, the size of the first input feature is 1xWxHxC, after the first input feature is convolved with the 1x1x1 three-dimensional convolutional kernel, the spatial attention with the size of 1xWxHx1 is obtained, and then the spatial attention is multiplied with the first input feature pixel by pixel in the first fusion layer to obtain the spatial weighted feature map with the size of 1xWxHxC.

[0084] It can be understood that the important pixels in the spatial weighted feature map are more prominent, and the unimportant pixels are relatively more suppressed.

[0085] Therefore, by convolving the input of the current first feature extraction network based on the first convolutional layer in the current first feature extraction network, the spatial attention can be obtained, which can pay attention to the feature changes of the image in the spatial dimension, can enhance the response of the network to the local key information such as the lesion edge or texture feature, and reflects the distribution of important pixels in the input of the first feature extraction network. Then, the spatial attention is multiplied with the input of the current first feature extraction network pixel by pixel based on the first fusion layer to obtain the spatial weighted feature map, so that the important pixels in the spatial weighted feature map are more prominent, and the unimportant pixels are relatively more suppressed, the potential small target lesion is highlighted, the background or irrelevant information is suppressed, the detection ability of the small target lesion is improved, the lesion detection accuracy is effectively improved, and specifically, the spatial weighted feature map can identify the features of the key positions in the image, can improve the performance of the network in processing images with complex background or small target, and is an effective means to improve the target detection and segmentation effect.

[0086] In addition, with reference to Figure 4In an embodiment, the channel attention layer includes a max-pooling layer, an average-pooling layer, and a second fusion layer, and the channel attention layer in the current first feature extraction network is used to map the input of the current first feature extraction network to obtain a channel-weighted feature map, including but not limited to the following steps:

[0087] In step S410, the input of the current first feature extraction network is pooled based on the max-pooling layer in the current first feature extraction network to obtain a first channel attention.

[0088] In step S420, the input of the current first feature extraction network is pooled based on the average-pooling layer in the current first feature extraction network to obtain a second channel attention.

[0089] In step S430, the first channel attention and the second channel attention are respectively multiplied with the input of the first feature extraction network based on the second fusion layer to obtain a channel-weighted feature map.

[0090] In the above embodiment, the reference Figure 9 After the first input feature is max-pooled, the feature map corresponding to each channel of the first input feature is compressed into a value to obtain a first channel feature. For each channel of the first channel feature, the current channel is locally cross-channel interacted with the adjacent k channels. Specifically, the first channel feature is unfolded into a one-dimensional vector, a one-dimensional convolution kernel with a length of k is used to convolve the first channel feature, and then a full connection layer is used to obtain the first channel attention. For example, after the first input feature with a size of 1×W×H×C is max-pooled, the maximum value of each channel of the first input feature is taken as a channel representative value to obtain a first channel feature. The first channel feature is unfolded into a one-dimensional vector to obtain a first channel feature with a size of 1×1×C. If k is 5, a one-dimensional convolution kernel with a length of 5 is used to convolve the first channel feature, and then a full connection layer is used to obtain a first channel attention with a size of 1×1×C.

[0091] In addition, similarly, after the first input feature is average-pooled, the feature map corresponding to each channel of the first input feature is compressed into a value to obtain second channel features. For each channel of the second channel features, the current channel and the adjacent k channels are locally cross-channels interacted. Specifically, the second channel features are unfolded into a one-dimensional vector, a one-dimensional convolution kernel with a length of k is used to convolve the second channel features, and then a full connection layer is passed to obtain the second channel attention. For example, after the first input feature with a size of 1xWxHxC is average-pooled, the average value of each channel of the first input feature is taken as a channel representative value to obtain the second channel features. The second channel features are unfolded into a one-dimensional vector to obtain second channel features with a size of 1x1xC. If k is 5, a one-dimensional convolution kernel with a length of 5 is used to convolve the second channel features, and then a full connection layer is passed to obtain second channel attention with a size of 1x1xC.

[0092] The first channel attention and the second channel attention are respectively multiplied with the input of the first feature extraction network based on the second fusion layer. Specifically, the first channel attention and the second channel attention can be multiplied with the first input feature, or the first channel attention and the second channel attention can be weighted and then multiplied with the first input feature. The present disclosure does not limit this.

[0093] Based on this, the input of the current first feature extraction network is pooled based on the max-pooling layer in the current first feature extraction network to obtain the first channel attention. The first channel attention can assign different weights to each feature channel, identify and strengthen important feature channels, and suppress useless or redundant information, so as to capture the most significant features in the input of the first feature extraction network, which is conducive to detecting edge or mutation features, can enhance the image edges corresponding to small target lesions, and improve the recognition of small target lesions. Then, the input of the current first feature extraction network is pooled based on the average-pooling layer in the current first feature extraction network to obtain the second channel attention. Similarly, the second channel attention can also assign different weights to each feature channel, identify and strengthen important feature channels, and suppress useless or redundant information, so as to capture the most significant features in the input of the first feature extraction network, which can better reflect the overall information of the feature map and has a good ability to capture global context. Finally, the first channel attention and the second channel attention are respectively multiplied with the input of the first feature extraction network based on the second fusion layer to obtain a channel-weighted feature map, so that the channel-weighted feature map can comprehensively utilize the advantages of the two, enhance the information of important channels, and equivalently enhance the selectivity of the network to image features, so that the network can focus resources on important channels, improve the performance and robustness of the network, and improve the accuracy of the lesion detection result.

[0094] In addition, with reference toFigure 5 In an embodiment, the first feature extraction network further comprises a projection convolutional layer configured to fuse the spatial weighted feature map and the channel weighted feature map to obtain a first transition feature output by the first feature extraction network.

[0095] In step S510, the spatial weighted feature map and the channel weighted feature map are fused to obtain a first intermediate feature.

[0096] In step S520, the input of the first feature extraction network is mapped based on the projection convolutional layer to obtain a second intermediate feature.

[0097] In step S530, the first intermediate feature and the second intermediate feature are connected in residual to obtain the first transition feature output by the first feature extraction network.

[0098] In an embodiment, the first feature extraction network further comprises a projection convolutional layer configured to fuse the spatial weighted feature map and the channel weighted feature map to obtain a first transition feature output by the first feature extraction network. Figure 9 The spatial weighted feature map and the channel weighted feature map can be added, or the spatial weighted feature map and the channel weighted feature map can be weighted and then added, to obtain the first intermediate feature, which is not limited in the embodiments of the present disclosure.

[0099] In an embodiment, the projection convolutional layer has a kernel size same as that of the fifth convolutional layer, for example, the kernel size of the projection convolutional layer and the kernel size of the fifth convolutional layer can both be 3x3x3.

[0100] In an embodiment, the first feature extraction network further comprises a projection convolutional layer configured to fuse the spatial weighted feature map and the channel weighted feature map to obtain a first transition feature output by the first feature extraction network. Figure 10 The first input feature is convolved with the kernel of the projection convolutional layer to obtain the second intermediate feature.

[0101] In an embodiment, the first feature extraction network further comprises a projection convolutional layer configured to fuse the spatial weighted feature map and the channel weighted feature map to obtain a first transition feature output by the first feature extraction network.

[0102] Based on this, the first intermediate feature is obtained by fusing the spatial weighted feature map and the channel weighted feature map, and the multi-dimensional feature is fused to enhance the feature, and the detailed features in the medical image are doubly enhanced, the synergistic effect between them is fully tapped, the channel and the space are comprehensively processed in the same module, so that the network can pay attention to the key information of the feature map in different dimensions at the same time, and the input of the current first feature extraction network is mapped based on the projection convolution layer to obtain the second intermediate feature, and finally the first intermediate feature and the second intermediate feature are connected in residual, and the first transition feature output by the current first feature extraction network is obtained. Therefore, through the residual connection, the second intermediate feature closer to the input layer is directly combined with the first intermediate feature closer to the output layer, which can directly transmit the information of the second intermediate feature to the subsequent layer, simplify the information transmission, reduce the training cost of the network, and increase the feature richness, thereby improving the accuracy of the lesion detection result.

[0103] Specifically, the residual connection is introduced to solve the gradient vanishing problem that easily occurs in the training of a deep neural network. The residual connection refers to adding a short circuit path between network layers to directly introduce the input into the output, thereby simplifying the information transmission. This design enables the network to more effectively train deep structures, and has achieved remarkable success in large-scale image classification, detection and other tasks.

[0104] In addition, with reference to Figure 8 , specifically, four first feature extraction networks connected in sequence and four second feature extraction networks connected in sequence can be provided, in the four first feature extraction networks connected in sequence, the first first feature extraction network can include three network modules, the second first feature extraction network can include four network modules, the third first feature extraction network can include six network modules, and the fourth first feature extraction network can include three network modules. Each network module can include a projection convolution layer, a fourth convolution layer, a fifth convolution layer, a sixth convolution layer, a spatial attention layer, a channel attention layer, and a ReLU layer.

[0105] In addition, with reference to Figure 6 In an embodiment, the second feature extraction network includes a second convolution layer and a third convolution layer, the convolution kernel size of the second convolution layer is smaller than the convolution kernel size of the third convolution layer, and the second transition feature of the first transition feature output by the current second feature extraction network includes but is not limited to the following steps:

[0106] Step S610, based on the second convolution layer in the current second feature extraction network, the input of the current second feature extraction network is convolved to obtain a third transition feature.

[0107] Step S620, convolve the third transition feature based on the third convolutional layer in the current second feature extraction network to obtain a second transition feature.

[0108] wherein the input of the current second feature extraction network is named as a second input feature, if the current second feature extraction network is the first second feature extraction network, the second input feature is the first transition feature output by the first first feature extraction network; if the current second feature extraction network is not the first second feature extraction network, the second input feature is the second transition feature output by the previous second feature extraction network.

[0109] wherein the reference Figure 12 The convolution kernel of the second convolutional layer can be a 1x1x1 three-dimensional convolution kernel, and the convolution kernel of the second convolutional layer is convolved with the second input feature to obtain the third transition feature, which reduces the number of channels and can reduce the amount of data to be processed by the second feature extraction network.

[0110] The convolution kernel of the third convolutional layer can be a 3x3x3 three-dimensional convolution kernel, and the convolution kernel of the third convolutional layer is convolved with the third transition feature to perform local feature extraction on the third transition feature to obtain the second transition feature.

[0111] It should be noted that since the convolution kernels of the second convolutional layer and the third convolutional layer can be three-dimensional convolution kernels, and the three-dimensional medical image corresponds to multiple stacked slices, the second convolutional layer and the third convolutional layer can process multiple slices at a time, i.e., the processing efficiency of the network can be increased, and the network's ability to capture spatial information between upper and lower slices can be enhanced.

[0112] Therefore, by convolving the first transition feature based on the second convolutional layer in the current second feature extraction network to obtain the third transition feature, the amount of data processed by the network can be initially reduced, the response speed of the network can be faster, and the inference efficiency can be higher. Then, by convolving the third transition feature based on the third convolutional layer in the current second feature extraction network to obtain the second transition feature, the third transition feature can be locally extracted, the second transition feature can further aggregate deeper image information of the three-dimensional medical image, the response of the network to the target region can be further improved, and thus the accuracy of the lesion detection result can be improved.

[0113] Specifically, by introducing the second feature extraction network, the network's perception ability for three-dimensional structures can be effectively enhanced, and the problem of inaccurate classification caused by loss of spatial information in traditional methods can be effectively avoided. By capturing deep-level spatial features in the image, the detection ability for complex lesions can be greatly improved, especially in the positioning and identification of small target lesions.

[0114] In addition, with reference to Figure 7In an embodiment, the number of the second feature extraction networks is the same as the number of the first feature extraction networks, the first feature extraction network corresponds to the second feature extraction network one by one, the third transition feature is convoluted based on the third convolutional layer in the current second feature extraction network to obtain the second transition feature, including but not limited to the following steps:

[0115] In step S710, if the current second feature extraction network is not the first second feature extraction network, the input of the current second feature extraction network is fused with the reference feature to obtain the fourth transition feature.

[0116] In step S720, the fourth transition feature is convoluted based on the third convolutional layer in the current second feature extraction network to obtain the second transition feature.

[0117] The reference feature is the first transition feature output by the first feature extraction network corresponding to the current second feature extraction network.

[0118] Exemplarily, referring again to Figure 8 , and referring to Figure 13 , each first feature extraction network has a corresponding second feature extraction network, the reference feature and the second input feature need to be up-sampled first to align the sizes of the two, and then the reference feature and the second input feature are added or multiplied, or weighted summation is performed to obtain the fourth transition feature, which is not limited in the embodiments of the present disclosure.

[0119] It should be noted that the third transition feature and the fourth transition feature are fused, and then local feature extraction is performed through the third convolutional layer in the current second feature extraction network, which is equivalent to simultaneously performing local feature extraction on the features extracted by all previous networks to obtain the second transition feature.

[0120] Based on this, if the current second feature extraction network is not the first second feature extraction network, the input of the current second feature extraction network is fused with the reference feature to obtain the fourth transition feature, the fourth transition feature can contain feature information extracted by all previous networks and the corresponding first feature extraction network, and then the fourth transition feature is convoluted based on the third convolutional layer in the current second feature extraction network to obtain the second transition feature. With the multi-level and multi-scale feature information in the fourth transition feature, the second transition feature can aggregate the features of the three-dimensional medical image from multiple angles, can improve the understanding of the network for the three-dimensional medical image, and can strengthen the detection of small target lesions, thereby improving the accuracy of the lesion detection result.

[0121] In a possible implementation, the third transition feature is fused with the reference feature to obtain the fifth transition feature, and then the fifth transition feature is input into the third convolutional layer of the current second feature extraction network for convolution to obtain the second transition feature.

[0122] In addition, with reference to Figure 14 The application further provides a lesion detection device 1400, comprising:

[0123] An acquisition module 1410 is configured to acquire a three-dimensional medical image and extract an original feature map of the three-dimensional medical image.

[0124] A first feature extraction module 1420 is configured to input the original feature map into a plurality of first feature extraction networks connected in sequence, and for each first feature extraction network, determine a first transition feature of the original feature map output by a current first feature extraction network, wherein the output of the current first feature extraction network is the input of a next first feature extraction network.

[0125] A second feature extraction module 1430 is configured to input the first transition feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in sequence, and for each second feature extraction network, determine a second transition feature of the first transition feature output by a current second feature extraction network, wherein the output of the current second feature extraction network is the input of a next second feature extraction network.

[0126] A feature fusion module 1440 is configured to fuse the first transition feature output by the last first feature extraction network and the second transition feature output by the last second feature extraction network to obtain a target fusion feature.

[0127] A classification module 1450 is configured to input the target fusion feature into a classification model for classification processing to obtain a lesion detection result of the three-dimensional medical image.

[0128] It can be understood that the specific implementation of the lesion detection device 1400 is basically the same as that of the above-mentioned specific embodiments of the lesion detection method, and will not be repeated here.

[0129] In addition, with reference to Figure 15 , Figure 15 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device comprises:

[0130] A processor 1501 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute a related program to implement the technical solutions provided by the embodiments of the application.

[0131] The memory 1502 can be implemented in the form of a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a Random Access Memory (RAM), etc. The memory 1502 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 1502 and are called and executed by the processor 1501 to perform the lesion detection method of the embodiments of the present application.

[0132] The input / output interface 1503 is configured to realize information input and output.

[0133] The communication interface 1504 is configured to realize the communication interaction between the device and other devices, and can realize the communication through a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0134] The bus 1505 is configured to transmit information between various components (for example, the processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504) of the device.

[0135] The processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504 are connected to each other through the bus 1505 to realize the communication connection between the device.

[0136] The embodiments of the present application also provide a storage medium, which is a computer readable storage medium, and is used for computer readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned lesion detection method.

[0137] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0138] The embodiments described in the specification of the present application are intended to more clearly illustrate the technical solutions of the present application, and do not constitute a limitation on the technical solutions provided by the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the present application are also applicable to similar technical problems.

[0139] Those skilled in the art can understand that, Figures 1 to 15 The technical solutions shown in the foregoing figures do not constitute a limitation on the present application, and can include more or fewer steps than shown, or combine certain steps, or different steps.

[0140] The device embodiments described above are merely illustrative, and units described as separate components can or can not be physically separated, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0141] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the function modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0142] The terms "first", "second", "third", "fourth" and the like (if any) in the specification of the present application and the above-described figures are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0143] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B, and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, and c can be single or multiple.

[0144] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0145] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0146] In addition, each functional unit in each embodiment of the application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0147] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0148] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A lesion detection method characterized by, The method comprises: acquiring a three-dimensional medical image, and extracting an original feature map of the three-dimensional medical image; inputting the original feature map into a plurality of first feature extraction networks connected in sequence, and determining, for each first feature extraction network, a first transition feature of the original feature map output by the current first feature extraction network, wherein the output of the current first feature extraction network is the input of the next first feature extraction network; inputting the first transition feature output by the first first feature extraction network into a plurality of second feature extraction networks connected in sequence, and determining, for each second feature extraction network, a second transition feature of the first transition feature output by the current second feature extraction network, wherein the output of the current second feature extraction network is the input of the next second feature extraction network; fusing the first transition feature output by the last first feature extraction network and the second transition feature output by the last second feature extraction network to obtain a target fusion feature; inputting the target fusion feature into a classification model for classification processing to obtain a lesion detection result of the three-dimensional medical image; wherein the second feature extraction network comprises a second convolution layer and a third convolution layer, the size of the convolution kernel of the second convolution layer is smaller than the size of the convolution kernel of the third convolution layer, and the determination of the second transition feature of the first transition feature output by the current second feature extraction network comprises: convolving the input of the current second feature extraction network based on the second convolution layer in the current second feature extraction network to obtain a third transition feature; convolving the third transition feature based on the third convolution layer in the current second feature extraction network to obtain a second transition feature; wherein the number of the second feature extraction networks is the same as the number of the first feature extraction networks, the first feature extraction networks correspond to the second feature extraction networks one by one, and the convolution of the third transition feature based on the third convolution layer in the current second feature extraction network to obtain a second transition feature comprises: if the current second feature extraction network is not the first second feature extraction network, fusing the input of the current second feature extraction network with a reference feature to obtain a fourth transition feature, wherein the reference feature is the first transition feature output by the first feature extraction network corresponding to the current second feature extraction network; convolving the fourth transition feature based on the third convolution layer in the current second feature extraction network to obtain a second transition feature.

2. The lesion detection method of claim 1, wherein The first feature extraction network comprises a spatial attention layer and a channel attention layer, and the determination of the first transition feature of the original feature map output by the current first feature extraction network comprises: mapping the input of the first feature extraction network based on the spatial attention layer in the current first feature extraction network to obtain a spatial weighted feature map; mapping, based on the channel attention layer in the current first feature extraction network, an input of the first feature extraction network to obtain a channel-weighted feature map; fusing the spatial-weighted feature map and the channel-weighted feature map to obtain a first transition feature output by the current first feature extraction network.

3. The lesion detection method of claim 2, wherein, The spatial attention layer includes a first convolution layer and a first fusion layer. The mapping, based on the spatial attention layer in the current first feature extraction network, of an input of the first feature extraction network to obtain a spatial-weighted feature map includes: convolving, based on the first convolution layer in the current first feature extraction network, an input of the current first feature extraction network to obtain spatial attention; pixel-by-pixel multiplication, based on the first fusion layer, of the spatial attention and the input of the current first feature extraction network to obtain a spatial-weighted feature map.

4. The lesion detection method of claim 2, wherein The channel attention layer includes a max-pooling layer, an average-pooling layer and a second fusion layer. The mapping, based on the channel attention layer in the current first feature extraction network, of an input of the first feature extraction network to obtain a channel-weighted feature map includes: pooling, based on the max-pooling layer in the current first feature extraction network, of an input of the current first feature extraction network to obtain first channel attention; pooling, based on the average-pooling layer in the current first feature extraction network, of an input of the current first feature extraction network to obtain second channel attention; channel-by-channel multiplication, based on the second fusion layer, of the first channel attention and the second channel attention with the input of the first feature extraction network to obtain a channel-weighted feature map.

5. The lesion detection method of claim 2, wherein, The first feature extraction network further includes a projection convolution layer. The fusing of the spatial-weighted feature map and the channel-weighted feature map to obtain a first transition feature output by the current first feature extraction network includes: fusing the spatial-weighted feature map and the channel-weighted feature map to obtain a first intermediate feature; mapping, based on the projection convolution layer, of an input of the current first feature extraction network to obtain a second intermediate feature; residual connection of the first intermediate feature and the second intermediate feature to obtain a first transition feature output by the current first feature extraction network.

6. A lesion detection apparatus characterized by comprising: The method includes: obtaining a three-dimensional medical image and extracting an original feature map of the three-dimensional medical image; inputting the original feature map into a plurality of first feature extraction networks connected in sequence, and determining, for each first feature extraction network, a first transition feature output by the original feature map in the current first feature extraction network, wherein the output of the current first feature extraction network is the input of the next first feature extraction network; The second feature extraction module is configured to input the first transition feature output by the first feature extraction network into a plurality of second feature extraction networks connected in sequence, and determine, for each second feature extraction network, a second transition feature of the first transition feature output by the current second feature extraction network, wherein the output of the current second feature extraction network is the input of the next second feature extraction network. The feature fusion module is configured to fuse the first transition feature output by the last first feature extraction network and the second transition feature output by the last second feature extraction network to obtain a target fusion feature. The classification module is configured to input the target fusion feature into a classification model for classification processing to obtain a lesion detection result of the three-dimensional medical image. The second feature extraction network includes a second convolution layer and a third convolution layer, the kernel size of the second convolution layer is smaller than the kernel size of the third convolution layer, and the determination of the second transition feature output by the current second feature extraction network includes: convolving the input of the current second feature extraction network based on the second convolution layer in the current second feature extraction network to obtain a third transition feature; convolving the third transition feature based on the third convolution layer in the current second feature extraction network to obtain a second transition feature. The number of the second feature extraction networks is the same as the number of the first feature extraction networks, the first feature extraction network corresponds to the second feature extraction network, and the convolution of the third transition feature based on the third convolution layer in the current second feature extraction network to obtain a second transition feature includes: if the current second feature extraction network is not the first second feature extraction network, fusing the input of the current second feature extraction network with a reference feature to obtain a fourth transition feature, wherein the reference feature is the first transition feature output by the first feature extraction network corresponding to the current second feature extraction network; convolving the fourth transition feature based on the third convolution layer in the current second feature extraction network to obtain a second transition feature.

7. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the lesion detection method in any one of claims 1-5 when executing the computer program.

8. A storage medium storing a computer program, characterized by The computer program is executed by the processor to implement the lesion detection method in any one of claims 1-5.

Images