Convolution kernel generation method and device, and electronic device
By constructing a three-dimensional Gabor convolution kernel based on Gabor filter, the uncertainty problem of convolutional neural network in learning tubular structure features is solved, and efficient feature extraction and lightweight network are achieved, which is suitable for tubular structure segmentation.
Patent Information
- Application Number
- CN202210214564.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-28
- Publication Date
- 2025-09-26
- Estimated Expiration
- 2042-02-28
AI Technical Summary
Existing convolutional neural networks have uncertainty in learning tubular structure features, which makes network training time-consuming and resource-intensive, and conventional convolution kernels impair feature extraction capabilities.
A three-dimensional Gabor convolution kernel generation method based on Gabor filter is adopted. The convolution kernel weights are updated through back propagation. Feature mapping, feature extraction and feature fusion models are constructed to realize feature extraction of tubular structures.
The reliability of tubular structure feature extraction is improved, the redundant parameters of CNN are reduced, and a lightweight network is achieved.
Smart Images
Figure CN114581411B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical image processing, and in particular to a convolution kernel generation method and device, and electronic equipment. Background Art
[0002] Tubular structures are ubiquitous in the human body and are crucial for maintaining human vital signs. In recent years, cardiovascular, cerebrovascular, and tracheal diseases have become major global health threats. Therefore, 3D reconstruction of human tubular structures is crucial for characterizing various diseases.
[0003] While currently used convolutional neural networks (CNNs) offer powerful feature learning capabilities, they cannot predict how CNNs learn features. Furthermore, common CNNs require a large number of channels or sufficient depth to capture sufficient features, resulting in a significant time-consuming acquisition of redundant parameters during network training. Complex network structures can also put pressure on storage and computing resources.
[0004] Currently, the main approach to addressing this issue is to propose lightweight network models for tubular segmentation, but this can compromise the network's feature extraction capabilities. Numerous published studies have demonstrated the advantages of Gabor filters for tubular feature extraction, but these are limited by the cumbersome nature of parameter adjustment. While some research has implemented convolution kernel modulation of Gabor filters, learning of Gabor filters has yet to be achieved. Summary of the Invention
[0005] The purpose of the embodiments of the present invention is to provide a convolution kernel generation method and device, and an electronic device. The generated convolution kernel can be applied to a tubular structure segmentation system, mainly to solve the uncertainty problem of conventional convolution kernels for feature extraction. The convolution kernel generated by the embodiments of the present invention can realize the feature extraction of tubular structures in a more purposeful and targeted manner, and the extracted features are more reliable.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0007] A convolution kernel generation method is provided, wherein the generated convolution kernel is applied to a tubular structure segmentation system, wherein the method comprises:
[0008] Implementing a three-dimensional Gabor filter based on the Gabor filter principle and constructing a Gabor convolution kernel; wherein the single-channel result of the next layer of feature map in the Gabor convolution kernel is generated based on the sampling value of the previous layer of convolution kernel;
[0009] Based on the back propagation principle, the gradient is transferred according to the loss value obtained in each iteration to update the weight of the Gabor convolution kernel;
[0010] Based on the CNN model structure of the Gabor convolution kernel, a target model constructed by combining the Gabor convolution kernel is constructed, wherein the target model includes three parts: feature mapping, feature extraction and feature fusion.
[0011] The steps of implementing a three-dimensional Gabor filter according to the Gabor filter principle and constructing a Gabor convolution kernel include:
[0012] Build C in Gabor convolution kernel Sample the upper layer and convert each C in Sum the sample values obtained in to generate the next layer of feature map M L+1 The single-channel result of in is the upper feature map M L The number of channels;
[0013] In the L-th layer structure, establish C out Group Gabor convolution kernel Among them, each group of Gabor convolution kernels Contains C in Gabor convolution kernel Among them, C out is the next layer feature map M L+1 The number of channels, including C out The next layer feature map M of the number of channels L+1 .
[0014] The step of transferring the gradient according to the loss value obtained in each iteration based on the back propagation principle and updating the weight of the Gabor convolution kernel includes:
[0015] Calculate each channel The gradient under the L function and update G at the learning rate η L (ω,ψ,σ,θ i ), and the following conditions are met:
[0016]
[0017]
[0018] Among them, the feature map of the target model is composed of the convolutional layer M 1 The convolutional layer M 1 By M 0After completing an N-channel convolution and performing 3D batch normalization and ReLU activation processing, it is generated, where M 0 A single-channel feature map is built for the number of samples selected for one training session;
[0019] The feature extraction part introduces the Gabor convolution kernel When introducing M 1 Each channel in the implementation of scale and direction of the feature capture, and when the number of channels is merged, N groups of features are merged into M 2 In any channel;
[0020] The feature fusion part introduces a convolution to achieve feature dimensionality reduction and channel number merging, so that the feature map M L Transfer from feature space to label space.
[0021] A convolution kernel generating device, wherein the generated convolution kernel is applied to a tubular structure segmentation system, wherein the device comprises:
[0022] A construction module is used to implement a three-dimensional Gabor filter based on the Gabor filter principle and construct a Gabor convolution kernel; wherein the single-channel result of the next layer feature map in the Gabor convolution kernel is generated based on the sampling value of the previous layer convolution kernel;
[0023] An update module is used to update the weight of the Gabor convolution kernel by transferring gradients according to the loss value obtained in each iteration based on the back propagation principle;
[0024] A combination module is used to combine the target model constructed by the Gabor convolution kernel based on the CNN model structure of the Gabor convolution kernel, wherein the target model includes three parts: feature mapping, feature extraction and feature fusion.
[0025] Wherein, the building blocks include:
[0026] The first submodule is used to build C in Gabor convolution kernel Sample the upper layer and convert each C in Sum the sample values obtained in to generate the next layer of feature map M L+1 The single-channel result of in is the upper feature map M L The number of channels;
[0027] The second submodule is used to establish C in the L-th layer structure. out Group Gabor convolution kernel Among them, each group of Gabor convolution kernels Contains C in Gabor convolution kernel Among them, C out is the next layer feature map M L+1 The number of channels, including C out The next layer feature map M of the number of channels L+1 .
[0028] The update module is specifically used to:
[0029] Calculate each channel The gradient under the L function and update G at the learning rate η L (ω,ψ,σ,θ i ), and the following conditions are met:
[0030]
[0031]
[0032] Among them, the feature map of the target model is composed of the convolutional layer M 1 The convolutional layer M 1 By M 0 After completing an N-channel convolution and performing 3D batch normalization and ReLU activation processing, it is generated, where M 0 A single-channel feature map is established for the number of samples selected for one training; the feature extraction part introduces the Gabor convolution kernel When introducing M 1 Each channel in the implementation of scale and direction of the feature capture, and when the number of channels is merged, N groups of features are merged into M 2 In any channel of the feature fusion part, a convolution is introduced to realize feature dimension reduction and channel number merging, so that the feature map M L Transfer from feature space to label space.
[0033] An embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and runnable on the processor. When the program or instruction is executed by the processor, the steps of any one of the above-mentioned convolution kernel generation methods are implemented.
[0034] An embodiment of the present invention provides a readable storage medium, which stores a program or instruction. When the program or instruction is executed by a processor, the steps of any one of the above-mentioned convolution kernel generation methods are implemented.
[0035] The convolution kernel generation method provided by the embodiment of the present invention implements a three-dimensional Gabor filter based on the Gabor filter principle and constructs a Gabor convolution kernel. Based on the back propagation principle, the gradient is transferred according to the loss value obtained in each iteration to update the weight of the Gabor convolution kernel. Based on the CNN model structure of the Gabor convolution kernel, the target model constructed by combining the Gabor convolution kernel is combined. The weighted Gabor convolution kernel constructed for tubular structure segmentation in the embodiment of the present invention can directly enhance the characteristics of the tubular structure in terms of scale and direction, reduce the redundant parameters of the CNN, and extract more accurate tubular tissue. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is a flowchart showing the steps of a convolution kernel generation method according to an embodiment of the present application;
[0037] Figure 2 is a structural block diagram showing a convolution kernel generating device according to an embodiment of the present application;
[0038] Figure 3 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0039] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.
[0040] The convolution kernel generation scheme provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.
[0041] The convolution kernel generated in the embodiment of the present application is applied to the tubular structure segmentation system. The convolution kernel generation method in the embodiment of the present application includes the following steps:
[0042] Step 101: Implement a three-dimensional Gabor filter according to the Gabor filter principle and construct a Gabor convolution kernel.
[0043] Among them, the single-channel result of the next layer of feature map in the Gabor convolution kernel is generated based on the sampling value of the previous layer of convolution kernel.
[0044] The Fourier transform is a powerful tool for signal processing that can convert images from the spatial domain to the frequency domain and extract features that are difficult to extract in the spatial domain. However, the Fourier transform lacks local information in time and location.
[0045] The Gabor transform is a short-time windowed Fourier transform (simply put, a Fourier transform performed within a specific time window). It is a special case of the short-time Fourier transform where the window function is a Gaussian function. Therefore, the Gabor filter can extract relevant features at different scales and directions in the frequency domain. Furthermore, because the Gabor function functions similarly to the human eye, it is often used in texture recognition, achieving good results.
[0046] An optional way to construct a Gabor convolution kernel may include the following process:
[0047] First, build C in Gabor convolution kernel Sample the upper layer and convert each C in Sum the sample values obtained in to generate the next layer of feature map M L+1 The single-channel result of in is the upper feature map M L The number of channels. represents the nth feature map of the Lth layer, ijk represents the position of the center point, and L is the number of layers.
[0048] Secondly, in the L-th layer structure, establish C out Group Gabor convolution kernel Among them, each group of Gabor convolution kernels Contains C in Gabor convolution kernel Among them, C out is the next layer feature map M L+1 The number of channels, including C out The next layer feature map M of the number of channels L+1 .
[0049] Step 102: Based on the back propagation principle, the gradient is transferred according to the loss value obtained in each iteration to update the weight of the Gabor convolution kernel.
[0050] When updating the weights of the Gabor convolution kernel, it can be achieved as follows:
[0051] Calculate each channel The gradient under the L function and update G at the learning rate η L (ω,ψ,σ,θ i ), and the following conditions are met:
[0052]
[0053]
[0054] Among them G L(ω,ψ,σ,θ i ) is the Gabor filter function, ω is the sine wave angular frequency, and ψ is the phase offset of the sine wave, which determine the direction of the center frequency of the three-dimensional Gabor filter. σ is the standard deviation of the Gaussian function and the width of the Gaussian envelope in spatial information. θ i Indicates the direction of angular frequency ω.
[0055] Step 103: Based on the CNN model structure of the Gabor convolution kernel, a target model constructed by combining the Gabor convolution kernel.
[0056] Among them, the target model includes three parts: feature mapping, feature extraction and feature fusion.
[0057] The feature map of the target model is composed of the convolutional layer M 1 The convolutional layer M 1 By M 0 After completing an N-channel convolution and performing 3D batch normalization and ReLU activation processing, it is generated, where M 0 A single-channel feature map is created for the number of samples selected for one training session.
[0058] The Gabor convolution kernel is introduced in the feature extraction part When introducing M 1 Each channel in the implementation of scale and direction of the feature capture, and when the number of channels is merged, N groups of features are merged into M 2 in any channel.
[0059] The feature fusion part introduces a convolution to achieve feature dimensionality reduction and channel number merging, making the feature map M L Transfer from feature space to label space.
[0060] The convolution kernel generation method provided in the embodiment of the present application has the following characteristics: first, the created Gabor convolution kernel is different from the general modulation method, and the parameters of the Gabor convolution kernel can be automatically learned during the network back propagation; second, the structure of the Gabor convolution kernel is completely consistent with the conventional convolution kernel and can be added to the convolution kernel of any CNN; third, based on the characteristics of the Gabor convolution kernel, an improved CNN based on the Gabor convolution kernel is applied to tubular structure segmentation. This design concept can be applied to any current CNN-based network structure; fourth, during the convolution process of the network, since the Gabor convolution kernel is controllable for tubular structure feature extraction, there is no need for an extra number of channels to create redundant parameters. Therefore, the parameters of the CNN based on the Gabor convolution kernel can be greatly reduced, realizing a lightweight network.
[0061] Figure 2A structural block diagram of a convolution kernel generation device for implementing an embodiment of the present application.
[0062] The convolution kernel generated by the convolution kernel generation device of the embodiment of the present application is applied to the tubular structure segmentation system. The convolution kernel generation device of the embodiment of the present application includes the following functional modules:
[0063] A construction module 201 is configured to implement a three-dimensional Gabor filter according to the Gabor filter principle and construct a Gabor convolution kernel, wherein the single-channel result of the next layer of feature maps in the Gabor convolution kernel is generated based on the sampling values of the previous layer of convolution kernel;
[0064] An updating module 202 is configured to update the weights of the Gabor convolution kernel by transferring gradients according to the loss value obtained in each iteration based on the back propagation principle;
[0065] The combining module 203 is used to combine the target model constructed by the Gabor convolution kernel based on the CNN model structure of the Gabor convolution kernel, wherein the target model includes three parts: feature mapping, feature extraction and feature fusion.
[0066] Optionally, the building blocks include:
[0067] The first submodule is used to build C in Gabor convolution kernel Sample the upper layer and convert each C in Sum the sample values obtained in to generate the next layer of feature map M L+1 The single-channel result of in is the upper feature map M L The number of channels;
[0068] The second submodule is used to establish C in the L-th layer structure. out Group Gabor convolution kernel Among them, each group of Gabor convolution kernels Contains C in Gabor convolution kernel Among them, C out is the next layer feature map M L+1 The number of channels, including C out The next layer feature map M of the number of channels L+1 .
[0069] Optionally, the update module is specifically configured to:
[0070] Calculate each channel The gradient under the L function and update G at the learning rate η L (ω,ψ,σ,θ i), and the following conditions are met:
[0071]
[0072]
[0073] Optionally, the feature map of the target model is composed of a convolutional layer M 1 The convolutional layer M 1 By M 0 After completing an N-channel convolution and performing 3D batch normalization and ReLU activation processing, it is generated, where M 0 A single-channel feature map is built for the number of samples selected for one training session;
[0074] The feature extraction part introduces the Gabor convolution kernel When introducing M 1 Each channel in the implementation of scale and direction of the feature capture, and when the number of channels is merged, N groups of features are merged into M 2 In any channel;
[0075] The feature fusion part introduces a convolution to achieve feature dimensionality reduction and channel number merging, so that the feature map M L Transfer from feature space to label space.
[0076] The convolution kernel generation device provided in the embodiment of the present application implements a three-dimensional Gabor filter based on the Gabor filter principle and constructs a Gabor convolution kernel; based on the back propagation principle, the gradient is transferred according to the loss value obtained in each iteration to update the weight of the Gabor convolution kernel; based on the CNN model structure of the Gabor convolution kernel, the target model constructed by combining the Gabor convolution kernel, the weighted learning Gabor convolution kernel for tubular structure segmentation constructed in the embodiment of the present invention can directly enhance the characteristics of the tubular structure in terms of scale and direction, reduce the redundant parameters of the CNN, and extract more accurate tubular tissue.
[0077] In the embodiment of the present application Figure 2 The convolution kernel generating device shown can be a device, or a component, integrated circuit, or chip in a server. Figure 2 The convolution kernel generation device shown may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.
[0078] The embodiments of the present application provide Figure 2 The convolution kernel generation device shown can achieve Figure 1To avoid repetition, the various processes implemented in the method embodiment are not described here.
[0079] Alternatively, as Figure 3 As shown, an embodiment of the present application also provides an electronic device 400, including a processor 401, a memory 402, and a program or instruction stored in the memory 402 and executable on the processor 401. When the program or instruction is executed by the processor 401, each process of the above-mentioned convolution kernel generation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0080] It should be noted that the electronic device in the embodiment of the present application includes the server described above.
[0081] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned convolution kernel generation method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0082] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.
[0083] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned convolution kernel generation method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0084] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0085] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.
[0086] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.
Claims
1. A convolution kernel generation method, wherein the generated convolution kernel is applied to a tubular structure segmentation system, characterized in that: The method comprises: Implementing a three-dimensional Gabor filter based on the Gabor filter principle and constructing a Gabor convolution kernel; wherein the single-channel result of the next layer of feature map in the Gabor convolution kernel is generated based on the sampling value of the previous layer of convolution kernel; Based on the back propagation principle, the gradient is transferred according to the loss value obtained in each iteration to update the weight of the Gabor convolution kernel; Based on the CNN model structure of the Gabor convolution kernel, a target model constructed by combining the Gabor convolution kernel, wherein the target model includes three parts: feature mapping, feature extraction and feature fusion; The steps of implementing a three-dimensional Gabor filter according to the Gabor filter principle and constructing a Gabor convolution kernel include: Build C in Gabor convolution kernel Sample the upper feature map and sum the sample values obtained in each channel to generate the next feature map M Q+1 The single-channel result of in is the upper feature map M Q The number of channels, ijk represents the position of the center point, Q is the number of layers, Represents the n-th feature map of the Q-th layer; In the Q-th layer structure, establish C out Groups of Gabor convolution kernels, where each group of Gabor convolution kernels contains C in Gabor convolution kernel Among them, C out is the next layer feature map M Q+1 The number of channels, including C out The next layer feature map M of the number of channels Q+1 ; The step of transferring the gradient according to the loss value obtained in each iteration based on the back propagation principle and updating the weight of the Gabor convolution kernel includes: Calculate each channel The gradient under the L function and update G at the learning rate η L (ω,ψ,σ,θ v ), and the following conditions are met: Among them, G L (ω,ψ,σ,θ v ) is the Gabor filter function, ω is the angular frequency of the sine wave, ψ is the phase shift of the sine wave, σ is the standard deviation of the Gaussian function, θ v represents the direction of angular frequency ω; Among them, the feature map part of the target model is composed of feature map M 1 Composition, the characteristic diagram M 1 By M 0 After completing an N-channel convolution and performing 3D batch normalization and ReLU activation processing, it is generated, where M 0 A single-channel feature map is built for the number of samples selected for one training session; The feature extraction part introduces the Gabor convolution kernel When introducing M 1 Each channel in the implementation of scale and direction of the feature capture, and when the number of channels is merged, N groups of features are merged into M 2 In any channel; The feature fusion part introduces a convolution to achieve feature dimensionality reduction and channel number merging, so that the feature map M Q Transfer from feature space to label space.
2. A convolution kernel generating device, wherein the generated convolution kernel is applied to a tubular structure segmentation system, characterized in that: The device comprises: A construction module is used to implement a three-dimensional Gabor filter based on the Gabor filter principle and construct a Gabor convolution kernel; wherein the single-channel result of the next layer feature map in the Gabor convolution kernel is generated based on the sampling value of the previous layer convolution kernel; An update module is used to update the weight of the Gabor convolution kernel by transferring gradients according to the loss value obtained in each iteration based on the back propagation principle; A combination module, configured to combine the target model constructed by the Gabor convolution kernel with the CNN model structure of the Gabor convolution kernel, wherein the target model includes three parts: feature mapping, feature extraction, and feature fusion; The building blocks include: The first submodule is used to build C in Gabor convolution kernel Sample the upper feature map and sum the sample values obtained in each channel to generate the next feature map M Q+1 The single-channel result of in is the upper feature map M Q The number of channels, ijk represents the position of the center point, Q is the number of layers, Represents the n-th feature map of the Q-th layer; The second submodule is used to establish C in the Q-layer structure. out Groups of Gabor convolution kernels, where each group of Gabor convolution kernels contains C in Gabor convolution kernel Among them, C out is the next layer feature map M Q+1 The number of channels, including C out The next layer feature map M of the number of channels Q+1 ; The update module is specifically used to: Calculate each channel The gradient under the L function and update G at the learning rate η L (ω,ψ,σ,θ v ), and the following conditions are met: Among them, G L (ω,ψ,σ,θ v ) is the Gabor filter function, ω is the angular frequency of the sine wave, ψ is the phase shift of the sine wave, σ is the standard deviation of the Gaussian function, θ v represents the direction of angular frequency ω; Among them, the feature map part of the target model is composed of feature map M 1 Composition, the characteristic diagram M 1 By M 0 After completing an N-channel convolution and performing 3D batch normalization and ReLU activation processing, it is generated, where M 0 A single-channel feature map is built for the number of samples selected for one training session; The feature extraction part introduces the Gabor convolution kernel When introducing M 1 Each channel in the implementation of scale and direction of the feature capture, and when the number of channels is merged, N groups of features are merged into M 2 In any channel; The feature fusion part introduces a convolution to achieve feature dimensionality reduction and channel number merging, so that the feature map M Q Transfer from feature space to label space.
3. An electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the convolution kernel generation method as described in claim 1.