Lightweight self-correcting convolutional feature enhancement system and method

Abstract

The application discloses a lightweight self-correcting convolution feature enhancement system and method, and the system comprises a feature extraction module, a lightweight self-correcting convolution module and a feature merging module; the feature extraction module is used for performing a feature extraction operation on first feature information in an input feature map and transmitting the first feature information to the lightweight self-correcting convolution module and the feature merging module respectively; the lightweight self-correcting convolution module is used for enhancing the first feature information and transmitting obtained second feature information to the feature merging module; and the feature merging module is used for adding the first feature information and the second feature information and merging the first feature information to obtain an output feature map. The first feature information extracted from the input feature map is enhanced by the lightweight self-correcting convolution module, and then the output feature map is obtained by adding and merging the first feature information and the second feature information, so that the neural network becomes lightweight, and the feature recognition capability of the system is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a light-weight self-calibration convolution feature enhancement system and method. BACKGROUND

[0002] Prior art and its defects:

[0003] (1) SCConv:

[0004] Self-Calibrated Convolutions (abbreviated as SCConv) is a multi-scale feature extraction module. The core of the SCConv module is to increase the CNN receptive field (the definition of receptive field is the size of the region on the input picture that each pixel point on the feature map of the output of each layer of the convolutional neural network maps. In the convolutional neural network CNN, the region size of the input layer corresponding to an element in the output result of a certain layer) by feature map down-sampling, so that each spatial position can fuse two different scale spatial information through self-calibration operation. Compared with traditional convolution, self-calibration convolution allows each spatial position to model the inter-channel dependency with the surrounding context information, effectively expanding the receptive field of the convolution layer, and the multi-scale information coding capability is very important for target detection related tasks, which can more accurately distinguish different object regions. Although self-calibration convolution does not introduce additional learnable parameters, the computational complexity will still increase.

[0005] (2) GhostNet (end-side neural network):

[0006] The core of the GhostNet for building a light-weight convolutional neural network for mobile terminals or embedded platforms is the Ghost module. The Ghost module first uses a normal convolution (such as 1x1Point-wise Convolution) to generate intrinsic feature information mapping, and then uses a 3x3 depth convolution as a cheap linear operation (such as 3x3Depth-wise Convolution) to generate more redundant linear feature information. Therefore, half of the features in the feature map generated by the Ghost module are captured by the cheap linear operation to capture spatial information, and the other half of the features are generated only by 1x1Point-wise Convolution and do not interact with other pixels, which limits the ability to capture spatial information and weakens the feature representation ability, which may hinder further performance improvement.

[0007] Therefore, the prior art still needs to be improved and improved. SUMMARY

[0008] The main purpose of the present application is to provide a lightweight self-correcting convolution feature enhancement system and method, aiming to solve the problem of insufficient lightweight of end-side neural network in the prior art, and weak feature representation capability when performing multi-scale feature extraction.

[0009] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0010] A lightweight self-correcting convolution feature enhancement system, comprising a feature extraction module, a lightweight self-correcting convolution module and a feature merging module; the feature extraction module is connected with the lightweight self-correcting convolution module and the feature merging module respectively; the lightweight self-correcting convolution module is also connected with the feature merging module;

[0011] The feature extraction module is used for performing feature extraction operation on the first feature information in the input feature map, and transmitting the internal feature information in the first feature information to the lightweight self-correcting convolution module and the feature merging module respectively; the lightweight self-correcting convolution module is used for enhancing the internal feature information, and transmitting the obtained second feature information to the feature merging module; the feature merging module is used for adding the linear feature information in the first feature information and the second feature information, and then merging with the internal feature information to obtain an output feature map.

[0012] In the lightweight self-correcting convolution feature enhancement system, the feature extraction module comprises a first feature extraction unit and a second feature extraction unit;

[0013] The second feature extraction unit is connected with the first feature extraction unit and the feature merging module respectively; the first feature extraction unit is also connected with the lightweight self-correcting convolution module and the feature merging module;

[0014] The first feature extraction unit is used for performing feature extraction operation on the input feature map, and transmitting the obtained internal feature information to the second feature extraction unit and the feature merging module respectively; the second feature extraction unit is used for performing feature extraction operation on the internal feature information, and transmitting the obtained linear feature information to the feature merging module.

[0015] In the lightweight self-correcting convolution feature enhancement system, the feature merging module comprises a first adding unit and a splicing unit; the first adding unit is connected with the second feature extraction unit, the lightweight self-correcting convolution module and the splicing unit respectively, and the splicing unit is also connected with the first feature extraction unit;

[0016] The first adding unit is configured to add the linear feature information and the second feature information, and transmit the obtained linear enhanced feature information to the splicing unit; and the splicing unit is configured to splice the linear enhanced feature information and the intrinsic feature information to obtain the output feature map.

[0017] The light-weight self-correcting convolution feature enhancement system comprises a downsampling unit, a third feature extraction unit, an upsampling unit, a second adding unit, a mapping unit, a multiplication unit and a fourth feature extraction unit.

[0018] The downsampling unit, the third feature extraction unit, the upsampling unit, the second adding unit, the mapping unit, the multiplication unit and the fourth feature extraction unit are sequentially connected, the second adding unit is further connected to the feature extraction module, the multiplication unit is further connected to the feature extraction module, and the fourth feature extraction unit is further connected to the feature merging module.

[0019] The downsampling unit is configured to perform downsampling operation on the intrinsic feature information, and transmit the obtained downsampling information to the third feature extraction unit; the third feature extraction unit is configured to perform feature extraction operation on the downsampling information, and transmit the obtained third feature information to the upsampling unit; the upsampling unit is configured to perform upsampling operation on the third feature information, and transmit the obtained down-up context feature information to the second adding unit; and the second adding unit is configured to add the down-up context feature information and the intrinsic feature information, and transmit the obtained multi-scale convolution feature information to the mapping unit.

[0020] The mapping unit is configured to perform mapping operation on the multi-scale convolution feature information, and transmit the obtained mapping feature information to the multiplication unit; the multiplication unit is configured to multiply the mapping feature information and the intrinsic feature information, and transmit the obtained corrected convolution feature information to the fourth feature extraction unit; and the fourth feature extraction unit is configured to perform feature extraction operation on the corrected convolution feature information, and transmit the obtained multi-scale feature information to the feature merging module.

[0021] In the light-weight self-correcting convolution feature enhancement system, the first feature extraction unit comprises a 1x1 point-by-point grouped convolution layer; the second feature extraction unit comprises a 3x3 deep convolution layer; and the first feature information comprises the intrinsic feature information and the linear feature information.

[0022] The third feature extraction unit comprises a 1*1 point-by-point grouped convolution layer; the fourth feature extraction unit comprises a 3*3 deep convolution layer; and the second feature information comprises the multi-scale feature information.

[0023] A lightweight self-correcting convolution feature enhancement method of a lightweight self-correcting convolution feature enhancement system as described above, the lightweight self-correcting convolution feature enhancement method comprising the following steps:

[0024] The feature extraction module performs feature extraction operation on the first feature information in the input feature map, and transmits the internal feature information in the first feature information to the lightweight self-correcting convolution module and the feature merging module, respectively;

[0025] The lightweight self-correcting convolution module enhances the internal feature information and transmits the obtained second feature information to the feature merging module;

[0026] The feature merging module adds the linear feature information in the first feature information and the second feature information, and then merges the internal feature information to obtain an output feature map.

[0027] In the lightweight self-correcting convolution feature enhancement method, the feature extraction module performs feature extraction operation on the first feature information in the input feature map, and transmits the internal feature information in the first feature information to the lightweight self-correcting convolution module and the feature merging module, respectively, and specifically comprises:

[0028] The first feature extraction unit performs feature extraction operation on the internal feature information in the input feature map, and transmits the internal feature information to the second feature extraction unit and the splicing unit;

[0029] The second feature extraction unit performs feature extraction operation on the internal feature information, and transmits the obtained linear feature information to the first adding unit;

[0030] The first feature information comprises the internal feature information and the linear feature information.

[0031] In the lightweight self-correcting convolution feature enhancement method, the lightweight self-correcting convolution module enhances the internal feature information, and transmits the obtained second feature information to the feature merging module, and specifically comprises:

[0032] The down-sampling unit performs down-sampling operation on the internal feature information, and transmits the obtained down-sampling information to the third feature extraction unit;

[0033] The third feature extraction unit performs a feature extraction operation on the down-sampling information and transmits the obtained third feature information to an up-sampling unit;

[0034] The up-sampling unit performs an up-sampling operation on the third feature information and transmits the obtained down-up context feature information to a second adding unit;

[0035] The second adding unit adds the down-up context feature information and the intrinsic feature information, and transmits the obtained multi-scale convolution feature information to a mapping unit;

[0036] The mapping unit performs a mapping operation on the multi-scale convolution feature information and transmits the obtained mapping feature information to a multiplication unit;

[0037] The multiplication unit multiplies the mapping feature information and the intrinsic feature information, and transmits the obtained corrected convolution feature information to a fourth feature extraction unit;

[0038] The fourth feature extraction unit performs a feature extraction operation on the corrected convolution feature information and transmits the obtained multi-scale feature information to the first adding unit; wherein the second feature information includes the multi-scale feature information.

[0039] In the light-weight self-correcting convolution feature enhancement method, the feature merging module adds the linear feature information in the first feature information to the second feature information, and then merges the intrinsic feature information to obtain an output feature map, specifically including:

[0040] The first adding unit adds the linear feature information and the multi-scale feature information, and transmits the obtained added feature information to a splicing unit;

[0041] The splicing unit splices the added feature information and the intrinsic feature information to obtain the output feature map.

[0042] Compared with the prior art, the light-weight self-correcting convolution feature enhancement system and method provided by the application, the system comprises a feature extraction module, a light-weight self-correcting convolution module and a feature merging module; the feature extraction module is used for performing feature extraction operation on first feature information in an input feature map, and transmitting the first feature information to the light-weight self-correcting convolution module and the feature merging module respectively; the light-weight self-correcting convolution module is used for enhancing the first feature information, and transmitting obtained second feature information to the feature merging module; and the feature merging module is used for adding the first feature information and the second feature information, and merging the first feature information to obtain an output feature map. The first feature information extracted from the input feature map is enhanced by the light-weight self-correcting convolution module, and then added and merged with the second feature information to obtain the output feature map, so that the neural network becomes light-weight, and the feature recognition capability of the system is improved. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 The structural block diagram of the preferred embodiment of the light-weight self-correcting convolution feature enhancement system provided by the application is shown in the figure.

[0044] Figure 2 The structural schematic diagram of the preferred embodiment of the light-weight self-correcting convolution feature enhancement system provided by the application is shown in the figure.

[0045] Figure 3 The flowchart of the preferred embodiment of the light-weight self-correcting convolution feature enhancement method provided by the application is shown in the figure.

[0046] Figure 4 The flowchart of step S10 of the preferred embodiment of the light-weight self-correcting convolution feature enhancement method provided by the application is shown in the figure.

[0047] Figure 5 The flowchart of step S20 of the preferred embodiment of the light-weight self-correcting convolution feature enhancement method provided by the application is shown in the figure.

[0048] Figure 6 The flowchart of step S30 of the preferred embodiment of the light-weight self-correcting convolution feature enhancement method provided by the application is shown in the figure.

[0049] Reference signs: 100: feature extraction module; 110: first feature extraction unit; 120: second feature extraction unit; 200: light-weight self-correcting convolution module; 210: down-sampling unit; 220: third feature extraction unit; 230: up-sampling unit; 240: second adding unit; 250: mapping unit; 260: multiplying unit; 270: fourth feature extraction unit; 300: feature merging module; 310: first adding unit; 320: splicing unit. DETAILED DESCRIPTION

[0050] In order to make the objects, technical solutions and effects of the present application clearer and more apparent, the present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and not intended to limit the present application.

[0051] Those skilled in the art can understand that the singular forms "a", "an" and "the" used herein include plural forms unless specifically stated otherwise. It should be further understood that the use of the term "include" in the specification of the present application means that the stated features, integers, steps, operations, elements and / or components are present, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be intermediate elements. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0052] Those skilled in the art can understand that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as that generally understood by those skilled in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have meanings consistent with those in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as such.

[0053] The present application provides a lightweight self-correcting convolution feature enhancement system and method. In the present application, the feature extraction module extracts the input feature map to obtain the first feature information, then the lightweight self-correcting convolution module enhances the internal feature information in the first feature information to obtain the second feature information, finally the feature merging module adds the linear feature information in the first feature information to the second feature information and merges the first feature information to obtain the output feature map. By improving the lightweight self-correcting convolution (SCConv), the lightweight self-correcting convolution module is obtained, and is introduced into the Ghost module to enhance the feature discrimination ability of the Ghost module, which not only makes the Ghost module more lightweight, but also generates more diverse linear enhanced features for the Ghost module, thereby not only realizing a lightweight neural network, but also improving the feature recognition ability of the neural network.

[0054] The following describes the design of a lightweight self-correcting convolution feature enhancement system through specific exemplary embodiments. It should be noted that the following embodiments are only used to illustrate the technical solutions of the invention and are not intended to be limiting.

[0055] See also Figure 1 The present invention provides a lightweight self-correcting convolution feature enhancement system, which includes: a feature extraction module 100, a lightweight self-correcting convolution module 200 and a feature merging module 300; the feature extraction module 100 is connected to the lightweight self-correcting convolution module 200 and the feature merging module 300 respectively; the lightweight self-correcting convolution module 200 is also connected to the feature merging module 300.

[0056] See also Figure 2 The feature extraction module 100 is used to perform a feature extraction operation on the first feature information in the input feature map, and convert the intrinsic feature information X1 ( Figure 2 denoted by X1 in the figure), and transmitted to the lightweight self-correction convolution module 200 and the feature merging module 300 respectively; the lightweight self-correction convolution module 200 is used to enhance the intrinsic feature information X1 and transmit the obtained second feature information to the feature merging module 300; the feature merging module 300 is used to enhance the linear feature information X2 ( Figure 2 After adding the second feature information, it is merged with the intrinsic feature information X1 to obtain an output feature map.

[0057] Specifically, in response to the large computational complexity of SCConv and the fact that the Ghost module in GhostNet obtains some features through cheap linear operations, which impairs the model's expressiveness, the present invention improves the lightweight self-correcting convolution (SCConv) and introduces it into the Ghost module to enhance the feature discrimination capability of the Ghost module, thereby constructing the lightweight self-correcting convolution feature enhancement system, that is, the lightweight Ghost module in the present invention. This improvement not only makes the Ghost module more lightweight, but also generates more diverse linear enhancement features for the Ghost module.

[0058] Then, the feature extraction implementation process of the lightweight self-correcting convolutional feature enhancement system is as follows:

[0059] First, the feature extraction module 100 inputs the feature map The first feature information in (C, H, W respectively represent channel, height and width) is subjected to a feature extraction operation, and the internal feature information X1 in the first feature information is transmitted to the lightweight self-correcting convolution module 200 and the feature merging module 300 respectively.

[0060] Then, the lightweight self-correcting convolution module 200 enhances the internal feature information X1, and transmits the obtained second feature information to the feature merging module 300. Finally, the feature merging module 300 adds the linear feature information X2 in the first feature information to the second feature information, and merges the internal feature information X1 to obtain the output feature map

[0061]

[0062] In the present application, the internal convolution module is improved by lightweight self-correcting convolution to obtain the lightweight self-correcting convolution module 200. The first feature information is subjected to feature enhancement by using the lightweight self-correcting convolution module 200, which improves the feature recognition ability of the convolution neural network model, and makes the convolution neural network model lighter.

[0063] Further, the feature extraction module 100 comprises a first feature extraction unit 110 and a second feature extraction unit 120; the second feature extraction unit 120 is connected with the first feature extraction unit 110 and the feature merging module 300 respectively; and the first feature extraction unit 110 is further connected with the lightweight self-correcting convolution module 200 and the feature merging module 300.

[0064] The first feature extraction unit 110 is used for performing a feature extraction operation on the input feature map, and transmitting the obtained internal feature information X1 to the second feature extraction unit 120 and the feature merging module 300 respectively; and the second feature extraction unit 120 is used for performing a feature extraction operation on the internal feature information X1, and transmitting the obtained linear feature information X2 to the feature merging module 300.

[0065] The first feature extraction unit 110 comprises a 1x1 point-by-point grouping convolution layer; the second feature extraction unit 120 comprises a 3x3 depth convolution layer; and the first feature information comprises the internal feature information X1 and the linear feature information X2.

[0066] Specifically, when the input feature map is input into the lightweight self-correcting convolution feature enhancement system, the first feature extraction unit 110 performs a feature extraction operation on the input feature map, that is, extracts features from the input feature map by using a 1x1 pointwise group convolution layer (pointwise group convolution is to perform 1x1 convolution to extract features at a single point). The first feature extraction unit 110 transmits the internal feature information X1 to the second feature extraction unit 120 and the feature merging module 300, respectively. Finally, the second feature extraction unit 120 performs a feature extraction operation on the internal feature information X1 and transmits the obtained linear feature information X2 to the feature merging module 300, so that the feature merging module 300 performs an addition splicing operation on the linear feature information X2.

[0067] The first feature extraction unit 110 performs a feature extraction operation on the input feature map, and the second feature extraction unit 120 further performs a feature extraction operation on the internal feature information X1 extracted by the first feature extraction unit 110, thereby effectively extracting the features in the input feature map. Wherein, as shown in Figure 2 The structure diagram of the lightweight self-correcting convolution feature enhancement system provided by the present application is shown.

[0068] Further, please continue to refer to Figure 1 The feature merging module 300 includes a first addition unit 310 and a splicing unit 320; the first addition unit 310 is connected with the second feature extraction unit 120, the lightweight self-correcting convolution module 200 and the splicing unit 320, respectively; and the splicing unit 320 is further connected with the first feature extraction unit 110.

[0069] The first addition unit 310 is used for adding the linear feature information X2 and the second feature information, and transmitting the obtained linear enhanced feature information X4 to the splicing unit 320; and the splicing unit 320 is used for splicing the linear enhanced feature information X4 and the internal feature information X1 to obtain the output feature map.

[0070] The first addition unit 310 corresponds to the "element-wise addition symbol" in Figure 2 The splicing unit 320 corresponds to "Concat" in Figure 2 , which is used to connect two or more arrays.

[0071] Specifically, after the second feature extraction unit 120 extracts the linear feature information X2 and transmits it to the first adding unit 310, the first adding unit 310 adds the linear feature information X2 and the second feature information to obtain the linear enhanced feature information X4, and transmits the linear enhanced feature information X4 to the splicing unit 320. Then, the splicing unit 320 splices the linear enhanced feature information X4 and the intrinsic feature information X1 to obtain the output feature map.

[0072] By adding the linear feature information X2 and the second feature information through the first adding unit 310, and splicing the linear enhanced feature information X4 and the intrinsic feature information X1 through the splicing unit 320, the output feature map can be quickly and efficiently output.

[0073] Further, the lightweight self-correcting convolution module 200 comprises a downsampling unit 210, a third feature extraction unit 220, an upsampling unit 230, a second adding unit 240, a mapping unit 250, a multiplication unit 260 and a fourth feature extraction unit 270.

[0074] The downsampling unit 210, the third feature extraction unit 220, the upsampling unit 230, the second adding unit 240, the mapping unit 250, the multiplication unit 260 and the fourth feature extraction unit 270 are connected in sequence, the second adding unit 240 is further connected with the lightweight self-correcting convolution module 200, the multiplication unit 260 is further connected with the lightweight self-correcting convolution module 200, the second adding unit 240 is further connected with the feature extraction module 100, the multiplication unit 260 is further connected with the feature extraction module 100, and the fourth feature extraction unit 270 is further connected with the feature merging module 300.

[0075] The downsampling unit 210 is configured to perform downsampling operation on the intrinsic feature information X1 and transmit the obtained downsampling information to the third feature extraction unit 220; the third feature extraction unit 220 is configured to perform feature extraction operation on the downsampling information and transmit the obtained third feature information to the upsampling unit 230; the upsampling unit 230 is configured to perform upsampling operation on the third feature information and transmit the obtained context feature information X C to the second adding unit 240; the second adding unit 240 is configured to add the context feature information X C and the intrinsic feature information X1, and transmit the obtained multi-scale convolution feature information X m to the mapping unit 250.

[0076] The mapping unit 250 is configured to perform a mapping operation on the multi-scale convolution feature information X m and transmit the obtained mapping feature information to the multiplication unit 260; the multiplication unit 260 is configured to multiply the mapping feature information and the intrinsic feature information X1, and transmit the obtained corrected convolution feature information X to the fourth feature extraction unit 270; the fourth feature extraction unit 270 is configured to perform a feature extraction operation on the corrected convolution feature information, and transmit the obtained multi-scale feature information X3 Figure 2 to the feature merging module 300.

[0077] The third feature extraction unit 220 includes a 1x1 point-by-point grouped convolution layer; the fourth feature extraction unit 270 includes a 3x3 depth convolution layer; the multiplication unit 260 corresponds to an "element-by-element multiplication symbol" in Figure 2 ; and the second feature information includes the multi-scale feature information X3.

[0078] Specifically, after the first feature extraction unit 110 extracts the intrinsic feature information X1 and transmits it to the lightweight self-correcting convolution module 200, the down-sampling unit 210 first performs a down-sampling operation on the intrinsic feature information X1, i.e., down-samples it r times using a pooling layer (two-dimensional average pooling, corresponding to Avg Pool2d in Figure 2 ), calculates the output for each element in a fixed shape window (also known as a pooling window) of the input data each time, obtains the down-sampling information, and transmits the obtained down-sampling information to the third feature extraction unit 220. Then, the third feature extraction unit 220 is configured to perform a feature extraction operation on the down-sampling information, and transmit the obtained third feature information to the up-sampling unit 230, i.e., using Conv Block1 to extract spatial domain features and reduce the parameter amount (total weight parameter amount of all parameter layers of the model).

[0079] Secondly, the up-sampling unit 230 performs an up-sampling operation on the third feature information, and transmits the obtained context feature information X C to the second addition unit 240, i.e., the up-sampling unit 230 uses bilinear interpolation to up-sample the third feature information to obtain the size of the original feature map (the input feature map) to obtain the context feature information X C . The second addition unit 240 then adds the context feature information X C and the intrinsic feature information X1 to obtain the multi-scale convolution feature information X m, the second adding unit 240 and the multi-scale convolution feature information X m transmitted to the mapping unit 250. Wherein, bilinear interpolation is also called bilinear interpolation. In mathematics, bilinear interpolation is a linear interpolation extension of the interpolation function with two variables, and its core idea is to perform linear interpolation once in two directions.

[0080] Further, the mapping unit 250 performs mapping operation on the multi-scale convolution feature information X m , that is, using the Sigmoid activation function to map the multi-scale convolution feature information X m , so that the output is always between 0 and 1, and the output will be the input of the weighted sum of a nonlinear function. The mapping unit 250 transmits the mapping feature information to the multiplication unit 260. Finally, the multiplication unit 260 multiplies the mapping feature information and the intrinsic feature information X1, and transmits the obtained corrected convolution feature information to the fourth feature extraction unit 270. The fourth feature extraction unit 270 then performs feature extraction operation on the corrected convolution feature information , and transmits the obtained multi-scale feature information X3 to the feature merging module 300.

[0081] In the present application, the intrinsic feature information X1 is first subjected to downsampling operation by the downsampling unit 210, then the third feature extraction unit 220 performs feature extraction operation on the downsampled information, secondly, the upsampling unit 230 performs upsampling operation on the third feature information, further, the second adding unit 240 adds the context feature information X C and the intrinsic feature information X1, then the mapping unit 250 performs mapping operation on the multi-scale convolution feature information X m , next, the multiplication unit 260 multiplies the mapping feature information and the intrinsic feature information X1, and finally, the fourth feature extraction unit 270 performs feature extraction operation on the corrected convolution feature information , so that each spatial position in the multi-scale feature information X3 is fused with spatial information of different scales through self-correcting convolution operation. Then, compared with the original SCConv, the lightweight SCConv further reduces the learnable parameters, and the corresponding calculation amount is also reduced.

[0082] Further, please refer to Figure 3 , the present application provides a lightweight self-correcting convolution feature enhancement method of the lightweight self-correcting convolution feature enhancement system as described above, the lightweight self-correcting convolution feature enhancement method comprises the following steps:

[0083] S10, the feature extraction module 100 performs feature extraction operation on the first feature information in the input feature map, and transmits the internal feature information in the first feature information to the lightweight self-correcting convolution module 200 and the feature merging module 300 respectively.

[0084] Specifically, in the present application, based on the lightweight self-correcting convolution feature enhancement system, the lightweight self-correcting convolution feature enhancement method, i.e. the feature enhancement method of lightweight Ghost module, is further designed. Then, by analogy, the lightweight self-correcting convolution feature enhancement method is implemented as follows:

[0085] Firstly, the feature extraction module 100 performs feature extraction operation on the first feature information in the input feature map (wherein C, H and W represent channel, height and width respectively) and transmits the internal feature information X1 in the first feature information to the lightweight self-correcting convolution module 200 and the feature merging module 300 respectively, so that the lightweight self-correcting convolution module 200 and the feature merging module 300 operate on the first feature information.

[0086] Further, please refer to Figure 4 S10, the feature extraction module 100 performs feature extraction operation on the first feature information in the input feature map, and transmits the internal feature information in the first feature information to the lightweight self-correcting convolution module 200 and the feature merging module 300, specifically including:

[0087] S11, the first feature extraction unit 110 performs feature extraction operation on the internal feature information X1 in the input feature map and transmits the internal feature information X1 to the second feature extraction unit 120 and the splicing unit 320;

[0088] S12, the second feature extraction unit 120 performs feature extraction operation on the internal feature information X1 and transmits the obtained linear feature information X2 to the first adding unit 310;

[0089] Wherein, the first feature information includes the internal feature information X1 and the linear feature information X2.

[0090] Specifically, when the input feature map is input into the lightweight self-correcting convolution feature enhancement system, the first feature extraction unit 110 performs feature extraction operation on the input feature map, i.e. uses 1x1 point-by-point grouping convolution layer to perform feature extraction operation on the input feature map The internal feature information X1 is obtained by performing the feature extraction processing. Then, the first feature extraction unit 110 transmits the internal feature information X1 to the second feature extraction unit 120 and the splicing unit 320 respectively. Next, the second feature extraction unit 120 performs the feature extraction operation on the internal feature information X1, and transmits the obtained linear feature information X2 to the first adding unit 310.

[0091] Further, please refer to Figure 3 , S20, the lightweight self-correcting convolution module 200 enhances the internal feature information, and transmits the obtained second feature information to the feature merging module 300; wherein the second feature information includes the multi-scale feature information X3.

[0092] Specifically, when the feature extraction module 100 transmits the first feature information to the lightweight self-correcting convolution module 200, the lightweight self-correcting convolution module 200 enhances the internal feature information X1, and transmits the obtained second feature information (the second feature information includes the multi-scale feature information X3) to the feature merging module 300, so that the feature merging module 300 performs the addition and merging operations on the first feature information and the second feature information.

[0093] Further, please refer to Figure 5 , S20, the lightweight self-correcting convolution module 200 enhances the internal feature information, and transmits the obtained second feature information to the feature merging module 300, specifically including:

[0094] S21, the down-sampling unit 210 performs the down-sampling operation on the internal feature information X1, and transmits the obtained down-sampling information to the third feature extraction unit 220;

[0095] S22, the third feature extraction unit 220 performs the feature extraction operation on the down-sampling information, and transmits the obtained third feature information to the up-sampling unit 230;

[0096] S23, the up-sampling unit 230 performs the up-sampling operation on the third feature information, and transmits the obtained context feature information X C to the second adding unit 240;

[0097] S24, the second adding unit 240 adds the context feature information X C and the internal feature information X1, and transmits the obtained multi-scale convolution feature information X m to the mapping unit 250;

[0098] S25, the mapping unit 250 maps the multi-scale convolution feature information X m , and transmits the obtained mapping feature information to the multiplication unit 260;

[0099] S26, the multiplication unit 260 multiplies the mapping feature information and the internal feature information X1, and transmits the obtained corrected convolution feature information X2 to the fourth feature extraction unit 270;

[0100] S27, the fourth feature extraction unit 270 performs feature extraction operation on the corrected convolution feature information, and transmits the obtained multi-scale feature information X3 to the first addition unit 310.

[0101] Specifically, after the first feature extraction unit 110 extracts the internal feature information X1 and transmits it to the downsampling unit 210, the downsampling unit 210 first performs downsampling operation on the internal feature information X1, that is, uses a pooling layer to downsample r times for encoding context information, calculates the output for each element in a fixed shape window of input data each time to obtain the downsampling information, and transmits the obtained downsampling information to the third feature extraction unit 220. Then, the third feature extraction unit 220 is used to perform feature extraction operation on the downsampling information, and transmits the obtained third feature information to the upsampling unit 230, that is, uses Conv Block1 to extract spatial domain features and reduce the parameter amount.

[0102] Secondly, the upsampling unit 230 performs upsampling operation on the third feature information, and transmits the obtained context feature information X2 to the second addition unit 240, that is, the upsampling unit 230 uses bilinear interpolation to upsample the third feature information to obtain the size of the original feature map to obtain the context feature information X2. C C The second addition unit 240 then adds the context feature information X2 and the internal feature information X1 to obtain the multi-scale convolution feature information X. C m , and transmits the multi-scale convolution feature information X to the mapping unit 250. m

[0103] Further, the mapping unit 250 performs mapping operation on the multi-scale convolution feature information X, that is, uses Sigmoid activation function to map the multi-scale convolution feature information X. m m ​​​​​The mapping unit 250 maps the intrinsic feature information X1 to obtain the mapped feature information, so that the mapped feature information is always between 0 and 1, and the output is the weighted sum of the input of a nonlinear function. The mapping unit 250 further transmits the mapped feature information to the multiplication unit 260. Finally, the multiplication unit 260 multiplies the mapped feature information and the intrinsic feature information X1 to obtain the corrected convolution feature information X2, which is transmitted to the fourth feature extraction unit 270. The fourth feature extraction unit 270 performs a feature extraction operation on the corrected convolution feature information X2 to obtain the multi-scale feature information X3, which is transmitted to the first addition unit 310. The fourth feature extraction unit 270 performs a feature extraction operation on the corrected convolution feature information X2 to obtain the multi-scale feature information X3, which is transmitted to the first addition unit 310. The fourth feature extraction unit 270 performs a feature extraction operation on the corrected convolution feature information X2 to obtain the multi-scale feature information X3, which is transmitted to the first addition unit 310.

[0104] Further, please continue to refer to Figure 3 S30, the feature merging module 300 adds the linear feature information in the first feature information and the second feature information, and then merges the intrinsic feature information to obtain an output feature map.

[0105] Specifically, after the lightweight self-correcting convolution module 200 transmits the second feature information to the feature merging module 300, and the feature extraction module 100 transmits the first feature information to the feature merging module 300, the feature merging module 300 adds the first feature information and the second feature information, and then merges the first feature information to obtain the output feature map

[0106] In the present application, the ordinary convolution in the existing self-correcting convolution is further improved to obtain a lightweight self-correcting convolution, i.e. the lightweight self-correcting convolution module 200. The first feature information extracted from the input feature map is enhanced by the lightweight self-correcting convolution module 200, so that the convolutional neural network model becomes lighter, thereby realizing efficient and fast feature extraction from the input feature map to obtain the output feature map.

[0107] In practical applications, different convolutional neural network models use Inverted bottleneck (inverse residual structure) design. Generally, two Ghost modules are used to construct a Ghost bottleneck (ghost residual structure), and then the Ghost bottleneck is used to construct different convolutional neural network structures for image classification, image segmentation, object detection and other visual tasks.

[0108] Further, please refer to Figure 6, S30, the feature merging module 300 adds the linear feature information in the first feature information to the second feature information, and merges the internal feature information to obtain an output feature map, specifically comprising:

[0109] S31, the first adding unit 310 adds the linear feature information X2 to the multi-scale feature information X3, and transmits the obtained added feature information to the splicing unit 320;

[0110] S32, the splicing unit 320 splices the added feature information and the internal feature information X1 to obtain the output feature map.

[0111] Specifically, after the fourth feature extraction unit 270 transmits the multi-scale feature information X3 to the first adding unit 310, and the second feature extraction unit 120 transmits the linear feature information X2 to the first adding unit 310, the first adding unit 310 adds the linear feature information X2 and the second feature information to obtain the linear enhanced feature information X4, and then the first adding unit 310 transmits the linear enhanced feature information X4 to the splicing unit 320. Then, the splicing unit 320 splices the linear enhanced feature information X4 and the internal feature information X1 to obtain the output feature map.

[0112] In summary, the light-weight self-correcting convolution feature enhancement system and method provided by the application, the system comprises: a feature extraction module, a light-weight self-correcting convolution module and a feature merging module; the feature extraction module is used for performing feature extraction operation on the first feature information in the input feature map, and transmitting the first feature information to the light-weight self-correcting convolution module and the feature merging module respectively; the light-weight self-correcting convolution module is used for enhancing the first feature information, and transmitting the obtained second feature information to the feature merging module; the feature merging module is used for adding the first feature information and the second feature information, and merging the first feature information to obtain an output feature map. Through the light-weight self-correcting convolution module, the first feature information extracted from the input feature map is enhanced, and then added and merged with the second feature information to obtain the output feature map, so that the neural network becomes light-weight, and the feature recognition ability of the system is improved.

[0113] It can be understood that for those skilled in the art, equivalent replacements or changes can be made according to the technical solutions and inventive concepts of the application, and all these changes or replacements shall belong to the protection scope of the claims appended to the application.

Claims

1. A lightweight self-correcting convolutional feature enhancement system, characterized in that, The lightweight self-correcting convolution feature enhancement system comprises a feature extraction module, a lightweight self-correcting convolution module and a feature merging module; the feature extraction module is connected with the lightweight self-correcting convolution module and the feature merging module respectively; the lightweight self-correcting convolution module is further connected with the feature merging module; The feature extraction module is used for performing feature extraction operation on first feature information in an input feature map, and transmitting internal feature information in the first feature information to the lightweight self-correcting convolution module and the feature merging module respectively; The feature extraction module comprises a first feature extraction unit and a second feature extraction unit; The second feature extraction unit is connected with the first feature extraction unit and the feature merging module respectively; the first feature extraction unit is further connected with the lightweight self-correcting convolution module and the feature merging module; The lightweight self-correcting convolution module is used for enhancing the internal feature information, and transmitting second feature information obtained to the feature merging module; The lightweight self-correcting convolution module comprises a down-sampling unit, a third feature extraction unit, an up-sampling unit, a second adding unit, a mapping unit, a multiplication unit and a fourth feature extraction unit; The down-sampling unit, the third feature extraction unit, the up-sampling unit, the second adding unit, the mapping unit, the multiplication unit and the fourth feature extraction unit are connected in sequence, the second adding unit is further connected with the feature extraction module, the multiplication unit is further connected with the feature extraction module, and the fourth feature extraction unit is further connected with the feature merging module; The down-sampling unit is used for performing down-sampling operation on the internal feature information, and transmitting down-sampling information obtained to the third feature extraction unit; The third feature extraction unit is used for performing feature extraction operation on the down-sampling information, and transmitting third feature information obtained to the up-sampling unit; The up-sampling unit is used for performing up-sampling operation on the third feature information, and transmitting down-up context feature information obtained to the second adding unit; The second adding unit is used for adding the down-up context feature information and the internal feature information, and transmitting multi-scale convolution feature information obtained to the mapping unit; The mapping unit is used for performing mapping operation on the multi-scale convolution feature information, and transmitting mapping feature information obtained to the multiplication unit; The multiplication unit is used for multiplying the mapping feature information and the internal feature information, and transmitting correction convolution feature information obtained to the fourth feature extraction unit; The fourth feature extraction unit is used for performing feature extraction operation on the correction convolution feature information, and transmitting multi-scale feature information obtained to the feature merging module; The feature merging module is used for adding linear feature information in the first feature information and the second feature information, and merging the internal feature information to obtain an output feature map.

2. The lightweight self-correcting convolutional feature augmentation system of claim 1, wherein, The feature extraction module further comprises: The first feature extraction unit is configured to perform a feature extraction operation on the input feature map and transmit internal feature information obtained to the second feature extraction unit and the feature merging module respectively. The second feature extraction unit is configured to perform a feature extraction operation on the internal feature information and transmit linear feature information obtained to the feature merging module.

3. The lightweight self-correcting convolutional feature augmentation system of claim 2, wherein, The feature merging module comprises a first adding unit and a splicing unit; the first adding unit is connected with the second feature extraction unit, the lightweight self-correcting convolution module and the splicing unit respectively; and the splicing unit is further connected with the first feature extraction unit. The first adding unit is configured to add the linear feature information and the second feature information and transmit linear enhanced feature information obtained to the splicing unit. The splicing unit is configured to splice the linear enhanced feature information and the internal feature information to obtain the output feature map.

4. The lightweight self-correcting convolutional feature augmentation system of claim 2, wherein, The first feature extraction unit comprises a 1*1 point-by-point grouped convolution layer; the second feature extraction unit comprises a 3*3 deep convolution layer; and the first feature information comprises the internal feature information and the linear feature information.

5. The lightweight self-correcting convolutional feature augmentation system of claim 1, wherein, The third feature extraction unit comprises a 1*1 point-by-point grouped convolution layer; the fourth feature extraction unit comprises a 3*3 deep convolution layer; and the second feature information comprises the multi-scale feature information.

6. A lightweight self-correcting convolutional feature enhancement method based on the lightweight self-correcting convolutional feature enhancement system according to any one of claims 1-5, characterized in that, The lightweight self-correcting convolution feature enhancement method comprises the following steps: The feature extraction module performs a feature extraction operation on first feature information in an input feature map and transmits internal feature information in the first feature information to the lightweight self-correcting convolution module and the feature merging module respectively; The lightweight self-correcting convolution module enhances the internal feature information and transmits second feature information obtained to the feature merging module; The feature merging module adds linear feature information in the first feature information to the second feature information and then merges the internal feature information to obtain an output feature map.

7. The lightweight self-correcting convolutional feature enhancement method of claim 6, wherein, The feature extraction module performs a feature extraction operation on first feature information in an input feature map and transmits internal feature information in the first feature information to the lightweight self-correcting convolution module and the feature merging module respectively, specifically comprising: A first feature extraction unit performs a feature extraction operation on internal feature information in the input feature map and transmits the internal feature information to a second feature extraction unit and a splicing unit; The second feature extraction unit performs a feature extraction operation on the internal feature information and transmits linear feature information obtained to a first adding unit; The first feature information comprises the internal feature information and the linear feature information.

8. The lightweight self-correcting convolutional feature enhancement method of claim 7, wherein, The lightweight self-correcting convolution module enhances the internal feature information and transmits second feature information obtained to the feature merging module, specifically comprising: A down-sampling unit performs a down-sampling operation on the internal feature information and transmits down-sampled information obtained to a third feature extraction unit; The third feature extraction unit performs a feature extraction operation on the down-sampling information, and transmits third feature information obtained to an up-sampling unit; The up-sampling unit performs an up-sampling operation on the third feature information, and transmits down-up context feature information obtained to a second adding unit; The second adding unit adds the down-up context feature information and the intrinsic feature information, and transmits multi-scale convolution feature information obtained to a mapping unit; The mapping unit performs a mapping operation on the multi-scale convolution feature information, and transmits mapping feature information obtained to a multiplication unit; The multiplication unit multiplies the mapping feature information and the intrinsic feature information, and transmits correction convolution feature information obtained to a fourth feature extraction unit; The fourth feature extraction unit performs a feature extraction operation on the correction convolution feature information, and transmits multi-scale feature information obtained to the first adding unit; The second feature information includes the multi-scale feature information.

9. The lightweight self-correcting convolutional feature enhancement method of claim 8, wherein, The feature merging module adds linear feature information in the first feature information to the second feature information, and then merges the linear feature information and the intrinsic feature information to obtain an output feature map, and specifically includes: The first adding unit adds the linear feature information and the multi-scale feature information, and transmits addition feature information obtained to a splicing unit; The splicing unit splices the addition feature information and the intrinsic feature information to obtain the output feature map.

Images