Image detection method and device based on neural network

Abstract

The embodiment of the present invention discloses an image detection method and device based on a neural network model. The method includes: obtaining an image to be detected; using each convolution block of a target neural network model obtained by pre-training, performing a convolution operation and a feature recalibration operation on the image to be detected, and determining a target feature map corresponding to the image to be detected, wherein the feature recalibration operation is: multiplying the weight value corresponding to each channel contained in the corresponding convolution block with the feature map obtained after the convolution operation corresponding to each channel; using the fully connected layer of the target neural network model and the target feature map, determining the detection result corresponding to the image to be detected, so as to ensure that the useful features in the feature map corresponding to the image are enhanced according to the importance of each channel and the features that are not very useful are suppressed, while reducing the time and resources consumed by the detection process using the network to a certain extent.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a neural network-based image detection method and device. Background Art

[0002] At present, when using a trained neural network model including an SE (Squeeze-and-Excitation) block to detect an image, after the image or its corresponding feature map passes through the current convolution block and before entering the next block of the current convolution block, it needs to pass through the SE block to determine the importance of the channel where each feature map in the multiple feature maps obtained after passing through the current convolution block is located; wherein a convolution block may include at least one convolution layer, and may also include a nonlinear layer and / or a downsampling layer, wherein the next block of the current convolution block may be: the next convolution block of the current convolution block or the block where the fully connected layer of the neural network model is located; further, based on the multiple feature maps obtained after passing through the current convolution block and the importance of the channel where each feature map is located, the feature maps after each recalibration are determined, wherein the feature map after recalibration can represent the importance of the corresponding channel, and the feature map after recalibration is used as the input of the next block of the current convolution block; and then the detection result of the trained neural network model including the SE block for the image is obtained.

[0003] It can be seen that in the above process, in the actual image detection process, it is necessary to use the SE block to use the feature map corresponding to the image to determine the importance of the channel where the feature map is located, which increases the time required for image detection and the resources consumed to a certain extent. Summary of the invention

[0004] The present invention provides an image detection method and device based on a neural network, so as to ensure that the useful features in the feature map corresponding to the image are enhanced according to the importance of each channel and the less useful features are suppressed, while reducing the time and resources consumed by the detection process using the network to a certain extent. The specific technical solution is as follows.

[0005] In a first aspect, an embodiment of the present invention provides an image detection method based on a neural network model, the method comprising:

[0006] Obtain an image to be detected; use each convolution block of the pre-trained target neural network model to perform a convolution operation and a feature recalibration operation on the image to be detected, and determine a target feature map corresponding to the image to be detected, wherein the feature recalibration operation is: multiplying the weight value corresponding to each channel contained in the corresponding convolution block by the feature map obtained after the convolution operation corresponding to each channel; the weight value corresponding to each channel is used to characterize the importance of the corresponding channel, and the weight value corresponding to each channel is: a value determined according to the target sub-weight value or target sub-compression value corresponding to the channel, and the target sub-weight value or target sub-compression value corresponding to the channel is: a value determined based on a sample image of the target neural network model obtained through training; use the fully connected layer of the target neural network model and the target feature map to determine the detection result corresponding to the image to be detected.

[0007] Optionally, the step of performing a convolution operation and a feature recalibration operation on the image to be detected using each convolution block of the pre-trained target neural network model to determine a target feature map corresponding to the image to be detected includes:

[0008] Using each convolution block of the pre-established target neural network model, perform the following operations to determine the target feature map corresponding to the image to be detected;

[0009] Input the image to be detected or the current feature map corresponding to the image to be detected into the current convolution block, wherein the current convolution block is one of the convolution blocks of the target neural network model. If the current convolution block is the first convolution block of the target neural network model, the input of the current convolution block is the image to be detected; if the current convolution block is not the first convolution block of the target neural network model, the input of the current convolution block is: the current feature map corresponding to the image to be detected, and the output of the previous convolution block of the current convolution block;

[0010] Based on the convolution kernels corresponding to the channels included in the current convolution block, image feature extraction is performed on the image to be detected or the current feature map corresponding to the image to be detected, and the feature maps corresponding to the channels included in the current convolution block are determined;

[0011] For each channel included in the current convolution block, multiply the weight value corresponding to the channel and the feature map corresponding to the channel to determine the feature map after feature recalibration corresponding to each channel included in the current convolution block as a new current feature map corresponding to the image to be detected;

[0012] If the current convolution block is not the last convolution block of the target neural network model, determine the next convolution block of the current convolution block as a new current convolution block, and return to execute the step of inputting the image to be detected or the current feature map corresponding to the image to be detected into the current convolution block;

[0013] If the current convolution block is the last convolution block of the target neural network model, the new current feature map corresponding to the image to be detected is determined as the target feature map corresponding to the image to be detected.

[0014] Optionally, before the step of performing a convolution operation and a feature recalibration operation on the image to be detected using each convolution block of the pre-trained target neural network model to determine a target feature map corresponding to the image to be detected, the method further includes: a process of training the target neural network model, wherein the process includes:

[0015] Obtain multiple sample images and calibration labels corresponding to each sample image; obtain an initial neural network model, wherein the initial neural network model includes multiple convolution blocks, a fully connected layer, and a compression excitation SE block corresponding to each convolution block;

[0016] For each sample image, the sample image and the calibration label corresponding to the sample image are input into the neural network model in the current training process, wherein the neural network model in the current training process is: the initial neural network model or the neural network model after adjusting parameters;

[0017] Using the neural network model in the current training process, the following steps are performed on the input sample image until a preset convergence condition is met, thereby obtaining a target neural network model including the multiple convolution blocks and the fully connected layer, wherein each channel included in each convolution block of the target neural network model corresponds to a fixed weight value;

[0018] Based on the convolution kernel corresponding to the channel included in the current convolution block, image feature extraction is performed on the sample image or the first feature map corresponding to the sample image, and the feature map corresponding to each channel included in the current convolution block is determined, wherein the current convolution block is: one of the convolution blocks of the neural network model in the current training process. If the current convolution block is: the first convolution block of the neural network model in the current training process, the input of the current convolution block is the sample image; if the current convolution block is: a non-first convolution block of the neural network model in the current training process, the input of the current convolution block is: the first feature map corresponding to the sample image, and is a value determined based on the output corresponding to the previous convolution block of the current convolution block and the SE block corresponding to the previous convolution block;

[0019] Input the feature map corresponding to each channel contained in the current convolution block into the SE block corresponding to the current convolution block, and obtain the current sub-weight value corresponding to each channel contained in the current convolution block in the current training process;

[0020] For each channel included in the current convolution block in the current training process, determine the target sub-weight value corresponding to the channel by using the current sub-weight value and the historical sub-weight value corresponding to the channel, wherein the historical sub-weight value is: the sub-weight value corresponding to the channel included in the current convolution block obtained in a training process before the current training process;

[0021] For each channel included in the current convolution block in the current training process, using the specified sub-weight value corresponding to the channel and the feature map corresponding to the channel, determine the recalibrated feature map corresponding to the channel, wherein the specified sub-weight value is: the current sub-weight value or the target sub-weight value;

[0022] If the current convolution block is not the last convolution block of the neural network model in the current training process, determine the next convolution block of the current convolution block as a new current convolution block, return to execute the step of performing image feature extraction on the sample image or the first feature map corresponding to the sample image based on the convolution kernel corresponding to the channel included in the current convolution block, and determine the feature map corresponding to each channel included in the current convolution block;

[0023] If the current convolution block is the last convolution block of the neural network model in the current training process, the recalibrated feature map corresponding to each channel contained in the current convolution block is input into the fully connected layer of the neural network model in the current training process to obtain a predicted label corresponding to the sample image;

[0024] If it is determined that the predicted label corresponding to the obtained sample image reaches a preset threshold, based on the predicted label and the calibration label corresponding to the obtained sample image, adjusting the parameters of the multiple convolution blocks, the fully connected layer, and the SE block corresponding to each convolution block of the neural network model in the current training process;

[0025] If it is determined that the preset convergence condition is not met, return to execute the step of inputting the sample image and the calibration label corresponding to the sample image into the neural network model in the current training process for each sample image.

[0026] Optionally, for each sample image, the sample image and the calibration label corresponding to the sample image are input into a neural network model in a current training process, where the neural network model in the current training process is: the initial neural network model or the neural network model after adjusting parameters;

[0027] Using the neural network model in the current training process, the following steps are performed on the input sample image until a preset convergence condition is met, thereby obtaining a target neural network model including the multiple convolution blocks and the fully connected layer, wherein each channel included in each convolution block of the target neural network model corresponds to a fixed weight value;

[0028] Based on the convolution kernel corresponding to the channel included in the current convolution block, image feature extraction is performed on the sample image or the first feature map corresponding to the sample image, and the feature map corresponding to each channel included in the current convolution block is determined, wherein the current convolution block is: one of the convolution blocks of the neural network model in the current training process. If the current convolution block is: the first convolution block of the neural network model in the current training process, the input of the current convolution block is the sample image; if the current convolution block is: a non-first convolution block of the neural network model in the current training process, the input of the current convolution block is: the first feature map corresponding to the sample image, and is a value determined based on the output corresponding to the previous convolution block of the current convolution block and the SE block corresponding to the previous convolution block;

[0029] Input the feature map corresponding to each channel contained in the current convolution block into the SE block corresponding to the current convolution block, and obtain the current sub-weight value corresponding to each channel contained in the current convolution block in the current training process;

[0030] For each channel included in the current convolution block in the current training process, determine the target sub-weight value corresponding to the channel by using the current sub-weight value and the historical sub-weight value corresponding to the channel, wherein the historical sub-weight value is: the sub-weight value corresponding to the channel included in the current convolution block obtained in a training process before the current training process;

[0031] For each channel included in the current convolution block in the current training process, using the specified sub-weight value corresponding to the channel and the feature map corresponding to the channel, determine the recalibrated feature map corresponding to the channel, wherein the specified sub-weight value is: the current sub-weight value or the target sub-weight value;

[0032] If the current convolution block is not the last convolution block of the neural network model in the current training process, determine the next convolution block of the current convolution block as a new current convolution block, return to execute the step of performing image feature extraction on the sample image or the first feature map corresponding to the sample image based on the convolution kernel corresponding to the channel included in the current convolution block, and determine the feature map corresponding to each channel included in the current convolution block;

[0033] If the current convolution block is the last convolution block of the neural network model in the current training process, the recalibrated feature map corresponding to each channel contained in the current convolution block is determined as the specific feature map corresponding to the sample image, and is input into the fully connected layer of the neural network model in the current training process to obtain a predicted label corresponding to the sample image;

[0034] If it is determined that the predicted label corresponding to the obtained sample image reaches a preset threshold, based on the predicted label and the calibration label corresponding to the obtained sample image, adjusting the parameters of the multiple convolution blocks, the fully connected layer, and the SE block corresponding to each convolution block of the neural network model in the current training process;

[0035] If it is determined that the preset convergence condition is not met, return to execute the step of inputting the sample image and the calibration label corresponding to the sample image into the neural network model in the current training process for each sample image.

[0036] Optionally, before the step of performing a convolution operation and a feature recalibration operation on the image to be detected using each convolution block of the pre-trained target neural network model to determine a target feature map corresponding to the image to be detected, the method further includes:

[0037] The process of training to obtain the target neural network model includes:

[0038] Obtain a plurality of sample images and a calibration label corresponding to each sample image; obtain an initial neural network model, wherein the initial neural network model includes a plurality of convolution blocks, a fully connected layer, and a compression excitation SE block corresponding to each convolution block;

[0039] For each sample image, the sample image and the calibration label corresponding to the sample image are input into the neural network model in the current training process, wherein the neural network model in the current training process is: the initial neural network model or the neural network model after adjusting parameters;

[0040] Using the neural network model in the current training process, the following steps are performed on the input sample image until a preset convergence condition is met, thereby obtaining a target neural network model including the multiple convolution blocks and the fully connected layer, wherein each channel included in each convolution block of the target neural network model corresponds to a fixed compression value;

[0041] Based on the convolution kernel corresponding to the channel included in the current convolution block, image feature extraction is performed on the sample image or the first feature map corresponding to the sample image, and the feature map corresponding to each channel included in the current convolution block is determined, wherein the current convolution block is: one of the convolution blocks of the neural network model in the current training process. If the current convolution block is: the first convolution block of the neural network model in the current training process, the input of the current convolution block is the sample image; if the current convolution block is: a non-first convolution block of the neural network model in the current training process, the input of the current convolution block is: the first feature map corresponding to the sample image, and is a value determined based on the output corresponding to the previous convolution block of the current convolution block and the SE block corresponding to the previous convolution block;

[0042] Input the feature map corresponding to each channel contained in the current convolution block into the compression processing layer of the SE block corresponding to the current convolution block, and obtain the current sub-compression value corresponding to each channel contained in the current convolution block in the current training process;

[0043] For each channel included in the current convolution block in the current training process, determine a target sub-compression value corresponding to the channel by using the current sub-compression value and the historical sub-compression value corresponding to the channel, wherein the historical sub-compression value is: a sub-compression value corresponding to the channel included in the current convolution block obtained in a training process before the current training process;

[0044] Inputting the designated sub-compression value corresponding to each channel contained in the current convolution block in the current training process into the excitation processing layer of the SE block corresponding to the current convolution block, to obtain the target sub-weight value corresponding to each channel contained in the current convolution block in the current training process, wherein the designated sub-compression value is the current sub-compression value or the target sub-compression value;

[0045] For each channel included in the current convolution block in the current training process, using the target sub-weight value corresponding to the channel and the feature map corresponding to the channel, determine the recalibrated feature map corresponding to each channel included in the current convolution block in the current training process;

[0046] If the current convolution block is not the last convolution block of the neural network model in the current training process, determine the next convolution block of the current convolution block as a new current convolution block, return to execute the step of performing image feature extraction on the sample image or the first feature map corresponding to the sample image based on the convolution kernel corresponding to the channel included in the current convolution block, and determine the feature map corresponding to each channel included in the current convolution block;

[0047] If the current convolution block is the last convolution block of the neural network model in the current training process, the recalibrated feature map corresponding to each channel contained in the current convolution block is determined as the specific feature map corresponding to the sample image, and is input into the fully connected layer of the neural network model in the current training process to obtain a predicted label corresponding to the sample image;

[0048] If it is determined that the predicted label corresponding to the obtained sample image reaches a preset threshold, based on the predicted label and the calibration label corresponding to the obtained sample image, adjusting the parameters of the multiple convolution blocks, the fully connected layer, and the SE block corresponding to each convolution block of the neural network model in the current training process;

[0049] If it is determined that the preset convergence condition is not met, return to execute the step of inputting the sample image and the calibration label corresponding to the sample image into the neural network model in the current training process for each sample image.

[0050] Optionally, the step of determining the target sub-compression value corresponding to the channel by using the current sub-compression value and the historical sub-compression value corresponding to the channel is implemented by any one of the following two implementations:

[0051] The first implementation method: using a preset compression value fusion formula, the current sub-compression value corresponding to the channel and the historical sub-compression value, a fusion result is determined as the target sub-compression value corresponding to the channel;

[0052] The second implementation method is: using a preset compression value fusion formula, the current sub-compression value corresponding to the channel and the historical sub-compression value, to determine a fusion result as the intermediate sub-compression value corresponding to the channel;

[0053] A determination result is determined using a preset compression value determination formula, the intermediate sub-compression value corresponding to the channel, and the current sub-compression value, as the target sub-compression value corresponding to the channel.

[0054] Optionally, the preset compression value fusion formula is expressed as:

[0055] Among them, the z i1 Indicates the intermediate sub-compression value corresponding to the i-th channel contained in the current convolution block during the current training process. represents the current sub-compression value corresponding to the i-th channel, represents the historical sub-compression value corresponding to the i-th channel, where a1 and b1 are preset coefficients, a1+b1=1;

[0056] The preset compression value determination formula is expressed as:

[0057] Among them, the z i It represents the target sub-compression value corresponding to the i-th channel contained in the current convolution block in the current training process, the γ represents the parameter to be adjusted, the initial value of γ is 0, and the value is 1 when the target neural network model is obtained.

[0058] Optionally, the image to be detected is an image captured by an image acquisition device set for the target vehicle; the detection result includes: position information corresponding to a preset detection target, and the preset detection target includes at least one of a lane line, a traffic sign, a zebra crossing, and a parking space;

[0059] The method also includes: identifying a map target that matches the preset detection target from a preset automatic navigation map; and determining the current position information of the target vehicle based on the position information corresponding to the preset detection target in the image to be detected and the position information of the map target in the automatic navigation map.

[0060] In a second aspect, an embodiment of the present invention provides an image detection device based on a neural network model, the device comprising:

[0061] The first acquisition module is configured to obtain an image to be detected; the first determination module is configured to use each convolution block of the pre-trained target neural network model to perform a convolution operation and a feature recalibration operation on the image to be detected, and determine the target feature map corresponding to the image to be detected, wherein the feature recalibration operation is: multiplying the weight value corresponding to each channel contained in the corresponding convolution block by the feature map obtained after the convolution operation corresponding to each channel; the weight value corresponding to each channel is used to characterize the importance of the corresponding channel, and the weight value corresponding to each channel is: a value determined according to the target sub-weight value or target sub-compression value corresponding to the channel, and the target sub-weight value or target sub-compression value corresponding to the channel is: a value determined based on a sample image of the target neural network model obtained through training; the second determination module is configured to use the fully connected layer of the target neural network model and the target feature map to determine the detection result corresponding to the image to be detected.

[0062] Optionally, the first determination module includes: a first input unit, configured to input the image to be detected or the current feature map corresponding to the image to be detected into a current convolution block, wherein the current convolution block is one of the convolution blocks of the target neural network model, and if the current convolution block is the first convolution block of the target neural network model, the input of the current convolution block is the image to be detected; if the current convolution block is not the first convolution block of the target neural network model, the input of the current convolution block is: the current feature map corresponding to the image to be detected, and the output of the previous convolution block of the current convolution block;

[0063] A first determining unit is configured to perform image feature extraction on the image to be detected or the current feature map corresponding to the image to be detected based on the convolution kernel corresponding to the channel included in the current convolution block, and determine the feature map corresponding to each channel included in the current convolution block;

[0064] A second determining unit is configured to multiply, for each channel included in the current convolution block, a weight value corresponding to the channel and a feature map corresponding to the channel, and determine a feature map after feature recalibration corresponding to each channel included in the current convolution block as a new current feature map corresponding to the image to be detected;

[0065] A third determining unit is configured to determine the next convolution block of the current convolution block as a new current convolution block if the current convolution block is not the last convolution block of the target neural network model, and return to trigger the first input unit;

[0066] The fourth determination unit is configured to determine the new current feature map corresponding to the image to be detected as the target feature map corresponding to the image to be detected if the current convolution block is the last convolution block of the target neural network model.

[0067] Optionally, the device further includes: a first training module, configured to perform a convolution operation and a feature recalibration operation on the image to be detected in each convolution block of the pre-trained target neural network model, and train the target neural network model before determining the target feature map corresponding to the image to be detected, wherein the first training module includes: a first acquisition submodule, configured to obtain a plurality of sample images and a calibration label corresponding to each sample image;

[0068] A second obtaining submodule is configured to obtain an initial neural network model, wherein the initial neural network model includes a plurality of convolution blocks, a fully connected layer, and a compression excitation SE block corresponding to each convolution block;

[0069] The first input submodule is configured to input, for each sample image, the sample image and the calibration label corresponding to the sample image into the neural network model in the current training process, wherein the neural network model in the current training process is: the initial neural network model or the neural network model after adjusting parameters;

[0070] A third acquisition submodule is configured to use the neural network model in the current training process to perform the following steps on the input sample image until a preset convergence condition is met, thereby obtaining a target neural network model including the multiple convolution blocks and the fully connected layer, wherein each channel included in each convolution block of the target neural network model corresponds to a fixed weight value;

[0071] The third obtaining submodule includes: a fifth determining unit, configured to perform image feature extraction on the sample image or the first feature map corresponding to the sample image based on the convolution kernel corresponding to the channel included in the current convolution block, and determine the feature map corresponding to each channel included in the current convolution block, wherein the current convolution block is: one of the convolution blocks of the neural network model in the current training process, if the current convolution block is: the first convolution block of the neural network model in the current training process, the input of the current convolution block is the sample image; if the current convolution block is: a non-first convolution block of the neural network model in the current training process, the input of the current convolution block is: the first feature map corresponding to the sample image, and is a value determined based on the output corresponding to the previous convolution block of the current convolution block and the SE block corresponding to the previous convolution block;

[0072] A second input unit is configured to input the feature map corresponding to each channel contained in the current convolution block into the SE block corresponding to the current convolution block, so as to obtain the current sub-weight value corresponding to each channel contained in the current convolution block in the current training process;

[0073] A sixth determination unit is configured to determine, for each channel included in the current convolution block in the current training process, a target sub-weight value corresponding to the channel by using the current sub-weight value and the historical sub-weight value corresponding to the channel, wherein the historical sub-weight value is: a sub-weight value corresponding to the channel included in the current convolution block obtained in a training process before the current training process;

[0074] A seventh determination unit is configured to determine, for each channel included in the current convolution block in the current training process, a recalibrated feature map corresponding to the channel using a specified sub-weight value corresponding to the channel and a feature map corresponding to the channel, wherein the specified sub-weight value is: the current sub-weight value or the target sub-weight value;

[0075] an eighth determination unit, configured to determine the next convolution block of the current convolution block as a new current convolution block if the current convolution block is not the last convolution block of the neural network model in the current training process, and trigger the fifth determination unit;

[0076] The third input unit is configured to, if the current convolution block is the last convolution block of the neural network model in the current training process, determine the recalibrated feature map corresponding to each channel contained in the current convolution block as the specific feature map corresponding to the sample image, and input it into the fully connected layer of the neural network model in the current training process to obtain the predicted label corresponding to the sample image;

[0077] The first adjustment unit is configured to adjust the parameters of multiple convolution blocks, fully connected layers and SE blocks corresponding to each convolution block of the neural network model in the current training process based on the predicted labels and calibration labels corresponding to the obtained sample images if it is determined that the predicted labels corresponding to the obtained sample images reach a preset threshold; if it is determined that the preset convergence conditions are not met, trigger the first input submodule.

[0078] Optionally, the sixth determining unit is specifically configured to: determine a fusion result as a target sub-weight value corresponding to the channel by using a preset weight value fusion formula, the current sub-weight value corresponding to the channel, and a historical sub-weight value;

[0079] Or it is specifically configured to: use a preset weight value fusion formula, the current sub-weight value corresponding to the channel, and the historical sub-weight value to determine a fusion result as the intermediate sub-weight value corresponding to the channel;

[0080] A determination result is determined using a preset weight value determination formula, the intermediate sub-weight value corresponding to the channel, and the current sub-weight value, as the target sub-weight value corresponding to the channel.

[0081] From the above content, it can be seen that a method and device provided by an embodiment of the present invention can obtain an image to be detected; use each convolution block of the target neural network model obtained by pre-training to perform a convolution operation and a feature recalibration operation on the image to be detected, and determine the target feature map corresponding to the image to be detected, wherein the feature recalibration operation is: multiplying the weight value corresponding to each channel contained in the corresponding convolution block by the feature map obtained after the convolution operation corresponding to each channel; the weight value corresponding to each channel is used to characterize the importance of the corresponding channel, and the weight value corresponding to each channel is: a value determined according to the target sub-weight value or target sub-compression value corresponding to the channel, and the target sub-weight value or target sub-compression value corresponding to the channel is: a value determined based on the sample image of the target neural network model obtained by training; use the fully connected layer of the target neural network model and the target feature map to determine the detection result corresponding to the image to be detected.

[0082] In the application of the embodiment of the present invention, in the target neural network model obtained by pre-training, each channel included in each convolution block corresponds to a weight value for characterizing the importance of the channel itself, so that in the process of actually using the target neural network model to detect the image, the feature map that can contain the importance of the channel can be directly obtained based on the weight value corresponding to each channel of each convolution block in the target neural network model and the feature map obtained after the convolution operation corresponding to the channel. Through the weight value corresponding to each channel, the useful features in the feature map obtained after the convolution operation corresponding to the channel are promoted and the less useful features are suppressed. In addition, the weight value is a value determined based on the sample image in the process of training the target neural network model, which is a fixed value. In the actual detection process, the time and resource occupancy of calculating the weight value corresponding to each channel are saved, and the time consumption and resource consumption in the detection process are saved to a certain extent. While ensuring that the importance of each channel is obtained, the useful features in the feature map corresponding to the image are promoted according to the importance of each channel and the less useful features are suppressed, and the time and resources consumed in the detection process using the network are reduced to a certain extent. Of course, any product or method implementing the present invention does not necessarily need to achieve all the advantages described above at the same time.

[0083] The innovative features of the embodiments of the present invention include:

[0084] 1. In the target neural network model obtained by pre-training, each channel contained in each convolution block corresponds to a weight value used to characterize the importance of the channel itself, so that in the process of actually using the target neural network model to detect the image, a feature map containing the importance of the channel can be obtained directly based on the weight value corresponding to each channel of each convolution block in the target neural network model and the feature map obtained after the convolution operation corresponding to the channel. Through the weight value corresponding to each channel and the feature map obtained after the convolution operation corresponding to the channel, useful features are promoted and less useful features are suppressed. In addition, the weight value is a value determined based on the sample image in the process of training the target neural network model, which is a fixed value. In the actual detection process, the time and resource occupation of calculating the weight value corresponding to each channel are saved, and the time consumption and resource consumption in the detection process are saved to a certain extent, so as to ensure that the importance of each channel is obtained, so as to promote the useful features in the feature map corresponding to the image according to the importance of each channel and suppress the less useful features, and at the same time, reduce the time and resources consumed in the detection process using the network to a certain extent.

[0085] 2. In the process of training the initial neural network model, after each sample image is input into the neural network model in the current training process, the importance of each channel contained in each convolution block is learned based on the SE block corresponding to each convolution block, and each sample image or the feature map corresponding to each sample image passes through the channels contained in each convolution block to obtain a new feature map, that is, the current sub-weight value representing the importance of each channel is learned, and then based on the historical sub-weight value corresponding to each channel and the current sub-weight value, the target sub-weight value corresponding to each channel is determined, so as to continuously obtain the current target sub-weight value corresponding to each channel of the fused historical sub-weight value during the training process of the neural network model until the convergence condition corresponding to the model is met, and a target neural network model including multiple convolution blocks and a fully connected layer is obtained, and each channel contained in each convolution block of the target neural network model corresponds to a fixed weight value.

[0086] 3. Two methods are provided to determine the target sub-weight value corresponding to the channel. One method is to directly use the preset weight value fusion formula, the current sub-weight value corresponding to the channel, and the historical sub-weight value to determine the fusion result as the target sub-weight value corresponding to the channel, which reduces the amount of calculation to a certain extent. The other method uses the current sub-weight value corresponding to each channel and the intermediate sub-weight value of the fused historical sub-weight value to jointly determine the target sub-weight value corresponding to each channel. The importance of each channel is more accurately reflected, thereby making the detection result of the target neural network model in the actual detection process more accurate.

[0087] 4. In the process of training the initial neural network model, after each sample image is input into the neural network model in the current training process, based on the compression processing layer of the SE block corresponding to each convolution block, and each sample image or the feature map corresponding to each sample image, after passing through the channels contained in each convolution block, the new feature map is obtained to determine the current sub-compression value corresponding to each channel contained in each convolution block, and then, based on the historical sub-compression value corresponding to each channel and the current sub-compression value, the target sub-compression value corresponding to each channel is determined, so that in the process of training the model, the current target sub-compression value corresponding to each channel of the fused historical sub-compression value is continuously obtained; when it is determined that the convergence condition corresponding to the model is met, a target neural network model including multiple convolution blocks and a fully connected layer is obtained, and each channel contained in each convolution block of the target neural network model corresponds to a fixed weight value, and the fixed weight value is: a value determined based on the current target sub-compression value corresponding to each channel of the fused historical sub-compression value.

[0088] 5. Two methods of determining the target sub-compression value corresponding to a channel are provided. One method directly uses the preset compression value fusion formula, the current sub-compression value corresponding to the channel, and the historical sub-compression value to determine the fusion result as the target sub-compression value corresponding to the channel, which reduces the amount of calculation to a certain extent. The other method uses the current sub-compression value corresponding to each channel and the intermediate sub-compression value fused with the historical sub-compression value to jointly determine the target sub-compression value corresponding to each channel, and then uses the target sub-compression value corresponding to each channel to determine the weight value corresponding to each channel. The importance of each channel is more accurately reflected, and thus, the detection result of the target neural network model in the actual detection process is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work.

[0090] Figure 1 A schematic diagram of a process flow of an image detection method based on a neural network model provided by an embodiment of the present invention;

[0091] Figure 2 A schematic diagram of a process of training a target neural network model provided by an embodiment of the present invention;

[0092] Figure 3 Another schematic diagram of a process of training to obtain a target neural network model provided by an embodiment of the present invention;

[0093] Figure 4 A schematic diagram of the structure of an image detection device based on a neural network model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0094] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0095] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0096] The present invention provides an image detection method and device based on a neural network model, so as to achieve the goal of reducing the time and resources consumed by the detection process using the network to a certain extent while ensuring that useful features in the feature map corresponding to the image are enhanced according to the importance of each channel and less useful features are suppressed. The embodiments of the present invention are described in detail below.

[0097] Figure 1 A flowchart of an image detection method based on a neural network model provided by an embodiment of the present invention. The method is applied to an electronic device, which can be a device with strong computing and processing capabilities, such as a server or a terminal device. The electronic device can be set outside the smart vehicle or on the smart vehicle, which is feasible. The method can include the following steps:

[0098] S101: Obtain an image to be detected. In an embodiment of the present invention, the image to be detected can be any type of image, such as an image in RGB (Red, Green, Blue) format, or an image in YUV format, etc. Among them, "Y" represents brightness (Luminance or Luma), that is, the grayscale value; and "U" and "V" represent chrominance (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel.

[0099] S102: Using each convolution block of the pre-trained target neural network model, perform convolution operation and feature recalibration operation on the image to be detected to determine the target feature map corresponding to the image to be detected.

[0100] Among them, the feature recalibration operation is: multiplying the weight value corresponding to each channel contained in the corresponding convolution block by the feature map obtained after the convolution operation corresponding to each channel; the weight value corresponding to each channel is used to characterize the importance of the corresponding channel, and the weight value corresponding to each channel is: a value determined according to the target sub-weight value or target sub-compression value corresponding to the channel, and the target sub-weight value or target sub-compression value corresponding to the channel is: a value determined based on the sample image of the target neural network model obtained through training.

[0101] The electronic device may locally store a pre-trained target neural network model, which may be a neural network model trained based on sample images. The target neural network model includes multiple convolution blocks and fully connected layers, wherein each convolution block may include at least one convolution layer, and may also include a nonlinear layer and / or a downsampling layer; the convolution layer is used to perform convolution processing on the input, which may be the image to be detected or the output of the previous layer of the convolution layer; the nonlinear layer is used to introduce nonlinear features into the system; and the downsampling layer is used to downsample the feature map.

[0102] Each convolution layer may include at least one channel, and each channel corresponds to a convolution kernel, which is used to perform convolution processing on the input of the convolution layer; that is, each convolution block may include at least one channel. In an embodiment of the present invention, each convolution block includes a weight value corresponding to each channel, and the weight value is used to characterize the importance of the corresponding channel, wherein the weight value corresponding to each channel is: in the process of training the target neural network model, the target sub-weight value or target sub-compression value corresponding to the channel is determined based on the sample image required for training, and the value is determined. In order to make the layout clear, the specific process of determining the weight value corresponding to each channel will be introduced later.

[0103] In one case, the larger the weight value, the more important the corresponding channel is, or the larger the weight value, the less important the corresponding channel is. Both are possible.

[0104] The target neural network model may be a model for target detection and / or target positioning, or a model for classifying and / or screening images, etc. The embodiment of the present invention does not limit the specific function of the target neural network model.

[0105] After the electronic device obtains the image to be detected, it can input the image to be detected into the target neural network, so as to use each convolution block of the target neural network model to perform convolution operation and feature recalibration operation on the image to be detected, that is, use each convolution block of each target neural network model to perform convolution operation on the image to be detected, and obtain the feature map corresponding to the image to be detected, wherein each channel included in each convolution block corresponds to a feature map; then, the feature map is multiplied by the weight value corresponding to the corresponding channel to obtain the recalibrated feature map as the output corresponding to each convolution block, and the output of each convolution block is used as the input of the next convolution block of the convolution block, until the target feature map corresponding to the image to be detected is obtained, wherein the target feature map may include at least one. In one case, the target feature map corresponding to the image to be detected may be the output of the last convolution block of the target neural network model, and in another case, the target feature map corresponding to the image to be detected may also be the output of a pre-specified convolution block in the target neural network.

[0106] In the embodiment of the present invention, if the convolution block only includes a convolution layer, the above-mentioned convolution operation on the image to be detected may refer to performing convolution processing on the image to be detected; if the convolution block includes a convolution layer and includes a non-linear layer and / or a pooling layer, the above-mentioned convolution operation on the image to be detected may refer to performing convolution processing on the image to be detected, as well as non-linear processing and / or pooling processing.

[0107] In one implementation of the present invention, the S102 may include: using each convolution block of the pre-established target neural network model, performing the following operations to determine a target feature map corresponding to the image to be detected;

[0108] Input the image to be detected or the current feature map corresponding to the image to be detected into the current convolution block, wherein the current convolution block is one of the convolution blocks of the target neural network model. If the current convolution block is the first convolution block of the target neural network model, the input of the current convolution block is the image to be detected; if the current convolution block is not the first convolution block of the target neural network model, the input of the current convolution block is: the current feature map corresponding to the image to be detected, and the output of the convolution block before the current convolution block;

[0109] Based on the convolution kernel corresponding to the channel contained in the current convolution block, image features are extracted for the image to be detected or the current feature map corresponding to the image to be detected, and the feature maps corresponding to each channel contained in the current convolution block are determined;

[0110] For each channel contained in the current convolution block, the weight value corresponding to the channel and the feature map corresponding to the channel are multiplied to determine the feature map after feature recalibration corresponding to each channel contained in the current convolution block as the new current feature map corresponding to the image to be detected;

[0111] If the current convolution block is not the last convolution block of the target neural network model, the next convolution block of the current convolution block is used as a new current convolution block, and the step of inputting the image to be detected or the current feature map corresponding to the image to be detected into the current convolution block is returned to be executed;

[0112] If the current convolution block is the last convolution block of the target neural network model, the new current feature map corresponding to the image to be detected is determined as the target feature map corresponding to the image to be detected.

[0113] In this implementation, the input processing process of each convolution block in the target neural network model can be the same, wherein, when the convolution layer is the first convolution block of the target neural network model, the input of the convolution block is the image to be detected; when the convolution layer is not the first convolution block of the target neural network model, the input of the convolution block is the output of the previous convolution block of the convolution block, that is, the feature map corresponding to the image to be detected obtained by the previous convolution block of the convolution block.

[0114] The above-mentioned step of extracting image features from the current feature map corresponding to the image to be detected or the image to be detected based on the convolution kernel corresponding to the channel contained in the current convolution block, and determining the feature maps corresponding to each channel contained in the current convolution block may be: if the current convolution block includes a nonlinear layer and / or a pooling layer in addition to the convolution layer, the current feature map corresponding to the image to be detected or the image to be detected is convolved based on the convolution kernel corresponding to the channel contained in the current convolution block, and the result obtained by the convolution processing is nonlinearly processed and / or pooled using the nonlinear layer and / or the pooling layer to obtain the feature maps corresponding to each channel contained in the current convolution block. If the current convolution block only includes a convolution layer, the current feature map corresponding to the image to be detected or the image to be detected is convolved based on the convolution kernel corresponding to the channel contained in the current convolution block, so as to realize image feature extraction of the current feature map corresponding to the image to be detected or the image to be detected, and obtain the feature maps corresponding to each channel contained in the current convolution block.

[0115] S103: Determine the detection result corresponding to the image to be detected by using the fully connected layer of the target neural network model and the target feature map.

[0116] In this step, after the electronic device obtains the target feature map, it inputs the target feature map into the fully connected layer of the target neural network model, and there can be at least one such fully connected layer. The features in the target feature map are integrated and normalized using the fully connected layer of the target neural network model to obtain a vector with a dimension equal to the preset number of categories. Each item in the vector represents the probability that the image to be detected belongs to the category represented by the item, and then the detection result corresponding to the image to be detected is obtained based on the vector. For example: the target neural network model can output the category with the highest probability as the category of the image to be detected, and the detection result includes the category corresponding to the image to be detected.

[0117] According to the embodiment of the present invention, in the target neural network model obtained by pre-training, each channel included in each convolution block corresponds to a weight value used to characterize the importance of the channel itself, so that in the process of actually using the target neural network model to detect an image, a feature map that can contain the importance of the channel can be obtained directly based on the weight value corresponding to each channel of each convolution block in the target neural network model and the feature map obtained after the convolution operation corresponding to the channel. Through the weight value corresponding to each channel, the useful features in the feature map obtained after the convolution operation corresponding to the channel are promoted and the less useful features are suppressed. Moreover, the weight value is a value determined based on the sample image in the process of training the target neural network model, which is a fixed value. In the actual detection process, the time and resource occupation of calculating the weight value corresponding to each channel are saved, and the time consumption and resource consumption in the detection process are saved to a certain extent. While ensuring that the importance of each channel is obtained, the useful features in the feature map corresponding to the image are promoted according to the importance of each channel and the less useful features are suppressed, and the time and resources consumed in the detection process using the network are reduced to a certain extent.

[0118] In view of the fact that the importance of each channel is more correlated with the attributes of the channel itself, but less correlated with the specific content of the feature map passing through the channel, a fixed weight value that can characterize the importance of the channel can be determined based on the sub-weight value corresponding to each channel determined by each sample image in the process of training the initial neural network model, so as to obtain a target neural network model containing the fixed weight value corresponding to each channel, thereby reducing the detection time in the actual image detection process using the target neural network model. Alternatively, a fixed compression value corresponding to each channel fused with the sub-compression value corresponding to each sample image can be determined based on the sub-compression value corresponding to each channel determined by each sample image, and then based on the fixed compression value, a fixed weight value that characterizes the importance of the channel can be determined, so as to obtain a target neural network model containing the fixed weight value corresponding to each channel, thereby reducing the detection time in the actual image detection process using the target neural network model.

[0119] In another embodiment of the present invention, before S102, the method may further include: a process of training to obtain a target neural network model, wherein: Figure 2 As shown, the process includes:

[0120] S201: Obtain multiple sample images and a calibration label corresponding to each sample image.

[0121] S202: Obtain an initial neural network model.

[0122] Among them, the initial neural network model includes multiple convolution blocks, fully connected layers and a compressed excitation SE block corresponding to each convolution block. The SE block is used to determine the sub-weight value that characterizes the importance of each channel contained in the corresponding convolution block. The convolution block is used to extract the image features of the image based on the convolution kernel corresponding to each channel contained. The fully connected layer is used to perform image detection based on the image features of the image.

[0123] S203: For each sample image, the sample image and the calibration label corresponding to the sample image are input into the neural network model in the current training process.

[0124] The neural network model in the current training process is: an initial neural network model or a neural network model after adjusting parameters.

[0125] S204: Using the neural network model in the current training process, perform the following steps on the input sample image until the preset convergence conditions are met, and obtain a target neural network model including multiple convolution blocks and a fully connected layer.

[0126] Among them, each channel contained in each convolution block of the target neural network model corresponds to a fixed weight value.

[0127] S2041: Based on the convolution kernel corresponding to the channel included in the current convolution block, perform image feature extraction on the sample image or the first feature map corresponding to the sample image, and determine the feature maps corresponding to each channel included in the current convolution block.

[0128] Among them, the current convolution block is: one of the convolution blocks of the neural network model in the current training process. If the current convolution block is: the first convolution block of the neural network model in the current training process, the input of the current convolution block is the sample image; if the current convolution block is: a non-first convolution block of the neural network model in the current training process, the input of the current convolution block is: the first feature map corresponding to the sample image, and is a value determined based on the output of the previous convolution block of the current convolution block and the SE block corresponding to the previous convolution block.

[0129] S2042: Input the feature map corresponding to each channel contained in the current convolution block into the SE block corresponding to the current convolution block to obtain the current sub-weight value corresponding to each channel contained in the current convolution block during the current training process.

[0130] S2043: For each channel included in the current convolution block in the current training process, the target sub-weight value corresponding to the channel is determined using the current sub-weight value and the historical sub-weight value corresponding to the channel.

[0131] The historical sub-weight values ​​are: the sub-weight values ​​corresponding to the channels contained in the current convolution block obtained in the previous training process of the current training process.

[0132] S2044: For each channel included in the current convolution block in the current training process, a recalibrated feature map corresponding to the channel is determined using a specified sub-weight value corresponding to the channel and a feature map corresponding to the channel.

[0133] The specified sub-weight value is: the current sub-weight value or the target sub-weight value;

[0134] S2045: If the current convolution block is not the last convolution block of the neural network model in the current training process, determine the next convolution block of the current convolution block as the new current convolution block, and return to execute S2041;

[0135] S2046: If the current convolution block is the last convolution block of the neural network model in the current training process, the recalibrated feature map corresponding to each channel contained in the current convolution block is determined as the specific feature map corresponding to the sample image, and inputs it into the fully connected layer of the neural network model in the current training process to obtain the predicted label corresponding to the sample image.

[0136] S2047: If it is determined that the predicted label corresponding to the obtained sample image reaches a preset threshold, based on the predicted label and calibration label corresponding to the obtained sample image, adjust the parameters of multiple convolution blocks, fully connected layers and SE blocks corresponding to each convolution block of the neural network model in the current training process, and if it is determined that the convergence condition corresponding to the initial neural network model is not met, return to execute S203.

[0137] It can be understood that the calibration label corresponding to each sample image may include but is not limited to: location information identifying the location of the target in the sample image, the category of the target in the sample image, and the classification identification corresponding to the sample image, etc. The embodiment of the present invention does not limit the specific content of the calibration label corresponding to each sample image. The calibration label corresponding to each sample image can be any type of content calibrated by the user according to his or her own needs.

[0138] In an embodiment of the present invention, after the electronic device obtains the initial neural network model, for each sample image, the sample image and the calibration label corresponding to the sample image are input into the neural network model in the current training process, that is, the initial neural network model; the convolution kernel corresponding to each channel contained in the first convolution block of the initial neural network model is used to extract image features of the sample image, and the sample feature map corresponding to each channel contained in the first convolution block is determined.

[0139] Then, the sample feature maps corresponding to each channel contained in the first convolution block are input into the SE (Squeeze-and-Excitation) block corresponding to the first convolution block, and the sample feature maps corresponding to each channel contained in the first convolution block are compressed and squeezed through the compression processing layer of the SE block corresponding to the first convolution block, that is, the global average pooling operation is performed to compress the 2-dimensional sample feature map corresponding to each channel contained in the first convolution block into a 1-dimensional value, that is, the current sub-compression value corresponding to each channel contained in the first convolution block is obtained, and then based on the current sub-compression value corresponding to each channel contained in the first convolution block, a channel vector is obtained, wherein each value in the channel vector represents the feature of each channel contained in the first convolution block, that is, the current sub-compression value corresponding to each channel contained in the first convolution block. Subsequently, the excitation processing layer of the SE block corresponding to the first convolution block is used to perform an excitation excitation operation on the channel vector, that is, the channel vector is input into two pre-set specific fully connected layers and sigmoid functions to model the correlation between channels, and the output obtained is the current sub-weight value corresponding to each channel.

[0140] After the electronic device obtains the current sub-weight value corresponding to each channel contained in the first convolution block, it can obtain the historical sub-weight value corresponding to each channel contained in the first convolution block. It can be understood that the current is the first training process of the initial neural network, and the historical sub-weight value corresponding to each channel contained in the first convolution block is 0. At this time, the current sub-weight value corresponding to each channel contained in the first convolution block can be directly used to determine the target sub-weight value corresponding to each channel contained in the first convolution block. Subsequently, the target sub-weight value or current sub-weight value corresponding to each channel contained in the first convolution block is used as the designated sub-weight value corresponding to each channel contained in the first convolution block.

[0141] Among them, it can be understood that the target sub-weight value corresponding to each channel contained in the first convolution block is the historical sub-weight value corresponding to each channel contained in the first convolution block when the electronic device obtains the next current sub-weight value corresponding to each channel contained in the first convolution block.

[0142] The specified sub-weight value corresponding to each channel contained in the first convolution block is weighted channel by channel to the sample feature map corresponding to each channel contained in the first convolution block through Scale multiplication, so as to recalibrate the features of the sample feature map corresponding to each channel contained in the first convolution block, and obtain the recalibrated sample feature map corresponding to each channel contained in the first convolution block.

[0143] The recalibrated sample feature map corresponding to each channel contained in the first convolution block is used as the current sample feature map corresponding to the image to be detected, and is input into the second convolution block of the initial neural network model, that is, the convolution block after the first convolution block. The convolution kernel corresponding to each channel contained in the second convolution block is used to extract image features from the current sample feature map, and the sample feature maps corresponding to each channel contained in the second convolution block are determined.

[0144] Then, the sample feature maps corresponding to each channel contained in the second convolution block are input into the compression processing layer of the SE block corresponding to the second convolution block. Through the compression processing layer of the SE block corresponding to the second convolution block, the sample feature maps corresponding to each channel contained in the second convolution block are squeezed, that is, the Global Average Pooling operation is performed to compress the 2-dimensional sample feature maps corresponding to each channel contained in the second convolution block into a 1-dimensional value, that is, the current sub-compression value corresponding to each channel contained in the second convolution block is obtained, and then based on the current sub-compression value corresponding to each channel contained in the second convolution block, a channel vector is obtained, wherein each value in the channel vector represents the feature of each channel contained in the second convolution block, that is, the current sub-compression value corresponding to each channel contained in the second convolution block. Subsequently, the excitation processing layer of the SE block corresponding to the second convolution block is used to perform an excitation operation on the channel vector, that is, the channel vector is input into two pre-set specific fully connected layers and sigmoid functions to model the correlation between channels, and the output obtained is the current sub-weight value corresponding to each channel.

[0145] After the electronic device obtains the current sub-weight value corresponding to each channel contained in the second convolution block, it can obtain the historical sub-weight value corresponding to each channel contained in the second convolution block. It can be understood that the current is the first training process of the initial neural network, and the historical sub-weight value corresponding to each channel contained in the second convolution block is 0. At this time, the current sub-weight value corresponding to each channel contained in the second convolution block can be directly used to determine the target sub-weight value corresponding to each channel contained in the second convolution block. Subsequently, the target sub-weight value or current sub-weight value corresponding to each channel contained in the second convolution block is used as the designated sub-weight value corresponding to each channel contained in the second convolution block.

[0146] The specified sub-weight value corresponding to each channel contained in the second convolution block is weighted channel by channel to the sample feature map corresponding to each channel contained in the second convolution block through Scale multiplication, so as to recalibrate the features of the sample feature map corresponding to each channel contained in the second convolution block, and obtain the recalibrated sample feature map corresponding to each channel contained in the second convolution block.

[0147] Similarly, the recalibrated sample feature map corresponding to each channel contained in the second convolution block is used as the new current sample feature map, and input into the third convolution block of the initial neural network model, that is, the convolution block after the second convolution block, until the target sub-weight value corresponding to each channel contained in the last convolution block of the initial neural network model is obtained, and the recalibrated sample feature map corresponding to each channel contained in the last convolution block is obtained.

[0148] The recalibrated sample feature map corresponding to each channel contained in the last convolution block is input into the fully connected layer of the initial neural network model. The fully connected layer is used to perform corresponding processing on the recalibrated feature map corresponding to each channel contained in the last convolution block to obtain the predicted label corresponding to the sample image.

[0149] The electronic device can count the number of predicted labels corresponding to the obtained sample images. If it is determined that the predicted labels corresponding to the obtained sample images do not reach a preset threshold, it directly returns to the step of inputting the sample image and the calibration label corresponding to the sample image into the neural network model of the current training process for each sample image; if it is determined that the predicted labels corresponding to the obtained sample images reach a preset threshold, based on the predicted labels and calibration labels corresponding to each sample image, the parameters of multiple convolution blocks, the fully connected layer and the SE block corresponding to each convolution block of the initial neural network model are adjusted.

[0150] Furthermore, it is possible to continue to determine whether the preset convergence conditions are met. If the preset convergence conditions are not met, then return to the step of inputting the sample image and the calibration label corresponding to the sample image into the neural network model of the current training process for each sample image. If the preset convergence conditions are met, then a target neural network model comprising multiple convolution blocks and a fully connected layer is obtained, wherein each channel contained in each convolution block of the target neural network model corresponds to a fixed weight value, and the fixed weight value is: when the target neural network model is obtained, the target sub-weight value corresponding to each channel contained in each convolution block is obtained.

[0151] The above process of adjusting the parameters of multiple convolution blocks, fully connected layers, and SE blocks corresponding to each convolution block of the neural network model in the current training process based on the prediction labels and calibration labels corresponding to the obtained sample images may be: based on a preset loss function, determining the loss value between the prediction labels and calibration labels corresponding to each obtained sample image, and then adjusting the parameters of multiple convolution blocks, fully connected layers, and SE blocks corresponding to each convolution block of the neural network model in the current training process based on the loss value corresponding to each obtained sample image and a preset optimization algorithm. The preset optimization algorithm may be a gradient descent method.

[0152] The preset threshold is the batch size of sample images set according to user needs, and can be any positive integer greater than or equal to 1. The preset convergence condition can be: whether the number of times the current model is adjusted reaches a predetermined number, or whether the number of current input sample images reaches a predetermined number; or whether the loss value between the predicted label and the calibrated label corresponding to the determined sample image is not greater than a preset loss value threshold.

[0153] In one implementation, the recalibrated feature map corresponding to each channel of each convolution block can also be: the feature map obtained by adding the specified sub-weight value corresponding to the channel and the feature map corresponding to the channel, wherein the addition between the feature maps is the specific result of pixel-by-pixel addition of the recalibrated feature maps corresponding to each channel of each convolution block, which is related to the structure of the network model in which the convolution block is located.

[0154] In one implementation, S2043 may be implemented by any one of the following two implementations:

[0155] The first implementation method: using a preset weight value fusion formula, the current sub-weight value corresponding to the channel and the historical sub-weight value, a fusion result is determined as the target sub-weight value corresponding to the channel;

[0156] The second implementation method is: using the preset weight value fusion formula, the current sub-weight value corresponding to the channel and the historical sub-weight value, to determine the fusion result as the intermediate sub-weight value corresponding to the channel;

[0157] A determination result is determined using a preset weight value determination formula, the intermediate sub-weight value corresponding to the channel, and the current sub-weight value, as the target sub-weight value corresponding to the channel.

[0158] It can be understood that in the first implementation method of determining the target sub-weight value corresponding to the channel, the current sub-weight value corresponding to each channel is directly fused with the corresponding historical sub-weight value. To a certain extent, it is possible to achieve the fusion of the current sub-weight value corresponding to each channel corresponding to each sample image in the training process, and obtain a more accurate weight value that characterizes the importance of each channel.

[0159] Taking into account the problem that the above direct fusion statistics of the current sub-weight value corresponding to each channel corresponding to each sample image blurs the boundaries of different sample images, in the second implementation method of determining the target sub-weight value corresponding to the channel, after the current sub-weight value corresponding to each channel is fused with the corresponding historical sub-weight value, the fused intermediate sub-weight value is fused with the current sub-weight value to determine a more accurate weight value that characterizes the importance of each channel.

[0160] The above preset weight value fusion formula can be expressed as:

[0161] Among them, S j1 Indicates the intermediate sub-weight value corresponding to the j-th channel contained in the current convolution block during the current training process. Indicates the current sub-weight value corresponding to the i-th channel, represents the historical sub-weight value corresponding to the j-th channel, where a2 and b2 are pre-set coefficients, a2+b2=1;

[0162] The preset weight value determination formula is expressed as:

[0163] Among them, S j Indicates the target sub-weight value corresponding to the jth channel contained in the current convolution block during the current training process. α indicates the parameter to be adjusted. The initial value of α is 0, and the value is 1 when the target neural network model is obtained. When the target neural network model is obtained, the S j That is, the weight value corresponding to the jth channel contained in the current convolution block.

[0164] The above-mentioned formula for determining the preset weight value is a formula for determining based on scheduling, that is, by scheduling the value of α, the weight value representing the importance of each channel is adjusted and determined. In an embodiment of the present invention, the above-mentioned formula for determining the preset weight value can also be based on a formula for determining in other forms, such as cosine, step decay, etc.

[0165] In one case, the process of fusing the current sub-weight value corresponding to each channel with the corresponding historical sub-weight value can be called a fusion process, and the network structure including the SE block carrying the fusion process can be called a fusionable SE network structure (Mergeable squeeze and excitation network).

[0166] Among them, each convolution block and its corresponding SE block in the initial neural network model have similar input processing processes. The following takes the first convolution block and its corresponding SE block as an example to illustrate the input processing process:

[0167] The above process of using the convolution kernel corresponding to each channel included in the first convolution block to extract image features from the sample image and determine the sample feature map corresponding to each channel included in the first convolution block can be achieved by F tr Transformation means that F tr It is expressed as follows: F tr :X→U,X∈R W,×H,×C, ,U∈R W×H×C ;

[0168] Among them, X represents the sample image, that is, the input of the first convolutional block, R W,×H,×C， represents the specification of the sample image, W ， Indicates the width of the sample image, H ， Represents the height of the sample image, C ，Indicates the number of channels corresponding to the sample image; U represents the sample feature map corresponding to each channel contained in the first convolution block, that is, it can represent the output of the first convolution block, R W×H×C It represents the output specifications of the first convolution block, that is, the specifications of the sample feature map corresponding to each channel contained in the first convolution block. W represents the width of the output of the first convolution block; H represents the height of the output of the first convolution block; C represents the number of channels of the output of the first convolution block.

[0169] Among them, the above F tr The transformation can be expressed by the following formula (1):

[0170] Among them, u c It represents the sample feature map corresponding to the c-th channel contained in the first convolution block, that is, the c-th output of the first convolution block, which is a two-dimensional matrix with a size of W×H; * represents the convolution operation; Indicates the cth convolution kernel contained in the first convolution block, corresponding to c channels; X = [x 1 ,x 2 ,…x C ] represents the input of the first convolutional block, i.e., the sample image; x s represents the sth input of the first convolutional block, that is, the sth input in X; Represents the 2D spatial kernel in the first convolutional block, v c A single channel of is applied to the s-th input of the first convolutional block.

[0171] The SE block corresponding to the first convolution block performs a Squeeze operation on the sample feature maps corresponding to each channel contained in the first convolution block. The formula (2) used can be expressed as:

[0172]

[0173] Among them, the u c (i, j) represents the element in the i-th row and j-th column of the c-th output of the first convolutional block; z c It represents the current sub-compression value corresponding to the c-th output of the first convolution block, that is, the current sub-compression value corresponding to the c-th channel of the first convolution block.

[0174] The SE block corresponding to the first convolution block is used to perform an excitation operation on the channel vector. The formula (3) used can be expressed as:

[0175] s=F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z)) (3);

[0176] s represents the current sub-weight value of each channel corresponding to the first convolution block, with a dimension of 1*1*C, where C is the number of channels of the first convolution block; W1 and W2 are the parameters in the SE block corresponding to the first convolution block, respectively. W1 and W2 have initial values ​​and are optimized and adjusted during the training process. The dimension of W1 is C / r*C, and the dimension of W2 is C*C / r; r is the preset scaling parameter; z represents the current sub-compression value corresponding to the first convolution block, that is, the corresponding channel vector of the first convolution block mentioned above; δ(W1z) represents a preset specific fully connected layer operation in the SE block corresponding to the first convolution block; σ(W2δ(W1z)) represents another preset specific fully connected layer operation in the SE block corresponding to the first convolution block.

[0177] Subsequently, after obtaining the current sub-weight value corresponding to each channel contained in the first convolution block, the historical sub-weight value corresponding to each channel contained in the first convolution block can be obtained. The current sub-weight value corresponding to each channel contained in the first convolution block and the corresponding historical sub-weight value are used to determine the target sub-weight value corresponding to each channel contained in the first convolution block. The target sub-weight value or current sub-weight value corresponding to each channel contained in the first convolution block is used as the designated sub-weight value corresponding to each channel contained in the first convolution block.

[0178] By using the following formula (4), the current sub-weight value corresponding to each channel contained in the first convolution block is used as the specified sub-weight value corresponding to each channel contained in the first convolution block, and the recalibrated sample feature map corresponding to each channel contained in the first convolution block is obtained:

[0179] in, Represents the recalibrated sample feature map corresponding to the cth channel contained in the first convolutional block, s c Indicates the current sub-weight value corresponding to the c-th channel contained in the first convolutional block, u c Represents the output of the cth channel contained in the first convolutional block.

[0180] In another embodiment of the present invention, before S102, the method may further include: a process of training to obtain a target neural network model, wherein: Figure 3 As shown, the process includes:

[0181] S301: Obtain multiple sample images and a calibration label corresponding to each sample image.

[0182] S302: Obtain an initial neural network model. The initial neural network model includes multiple convolution blocks, a fully connected layer, and a compression excitation SE block corresponding to each convolution block. The SE block is used to: compress the feature map corresponding to each channel of the corresponding convolution block to obtain the feature compression value corresponding to each channel, and excite the feature compression value corresponding to each channel to determine the sub-weight value representing the importance of each channel; the convolution block is used to: extract the image features of the image based on the convolution kernel corresponding to each channel, and the fully connected layer is used to: perform image detection based on the image features of the image.

[0183] S303: For each sample image, the sample image and the calibration label corresponding to the sample image are input into the neural network model in the current training process.

[0184] The neural network model in the current training process is: the initial neural network model or the neural network model after adjusting parameters.

[0185] S304: Using the neural network model in the current training process, perform the following steps S3041 to S3049 on the input sample image until the preset convergence conditions are met, and obtain a target neural network model including multiple convolution blocks and a fully connected layer. Wherein, each channel included in each convolution block of the target neural network model corresponds to a fixed weight value;

[0186] S3041: Based on the convolution kernel corresponding to the channel included in the current convolution block, perform image feature extraction on the sample image or the first feature map corresponding to the sample image, and determine the feature maps corresponding to each channel included in the current convolution block.

[0187] Among them, the current convolution block is: one of the convolution blocks of the neural network model in the current training process. If the current convolution block is: the first convolution block of the neural network model in the current training process, the input of the current convolution block is the sample image; if the current convolution block is: a non-first convolution block of the neural network model in the current training process, the input of the current convolution block is: the first feature map corresponding to the sample image, and is a value determined based on the output of the previous convolution block of the current convolution block and the SE block corresponding to the previous convolution block.

[0188] S3042: Input the feature map corresponding to each channel contained in the current convolution block into the compression processing layer of the SE block corresponding to the current convolution block to obtain the current sub-compression value corresponding to each channel contained in the current convolution block during the current training process.

[0189] S3043: For each channel included in the current convolution block in the current training process, determine the target sub-compression value corresponding to the channel using the current sub-compression value and the historical sub-compression value corresponding to the channel, wherein the historical sub-compression value is: the sub-compression value corresponding to the channel included in the current convolution block obtained in the previous training process of the current training process.

[0190] S3044: Input the designated sub-compression value corresponding to each channel contained in the current convolution block in the current training process into the excitation processing layer of the SE block corresponding to the current convolution block to obtain the target sub-weight value corresponding to each channel contained in the current convolution block in the current training process. The designated sub-compression value is the current sub-compression value or the target sub-compression value.

[0191] S3045: For each channel contained in the current convolution block in the current training process, the recalibrated feature map corresponding to each channel contained in the current convolution block in the current training process is determined using the target sub-weight value corresponding to the channel and the feature map corresponding to the channel.

[0192] S3046: If the current convolution block is not the last convolution block of the neural network model in the current training process, determine the next convolution block of the current convolution block as the new current convolution block, and return to execute S3041.

[0193] S3047: If the current convolution block is the last convolution block of the neural network model in the current training process, the recalibrated feature map corresponding to each channel contained in the current convolution block is determined as the specific feature map corresponding to the sample image, and input into the fully connected layer of the neural network model in the current training process to obtain the predicted label corresponding to the sample image.

[0194] S3048: If it is determined that the predicted label corresponding to the obtained sample image reaches a preset threshold, based on the predicted label and calibration label corresponding to the obtained sample image, adjust the parameters of multiple convolution blocks, fully connected layers and SE blocks corresponding to each convolution block of the neural network model in the current training process.

[0195] S3049: If it is determined that the predicted label corresponding to the obtained sample image reaches the preset threshold, based on the predicted label and calibration label corresponding to the obtained sample image, adjust the parameters of multiple convolutional layers, fully connected layers of the neural network model in the current training process and the SE blocks corresponding to each convolutional block, and if it is determined that the preset convergence condition is not met, return to execute S303.

[0196] In view of the fact that the importance of each channel is more correlated with the attributes of the channel itself, but not closely correlated with the specific content of the feature map passing through the channel, it is possible to use the sub-weight values ​​corresponding to each channel determined based on each sample image in the process of training the initial neural network model to determine a fixed weight value that can characterize the importance of the channel, thereby reducing the detection time in the actual image detection process using the target neural network model.

[0197] Among them, the above S301 is the same as the above S201, the S302 is the same as the above S202, the S303 is the same as the above S203, and the S3041 is the same as the above S2041, which will not be repeated here.

[0198] In this implementation, through the above description of the training process of the initial neural network model, it can be determined that during the training process, the operation flow of the excitation processing corresponding to the SE block corresponding to each convolution block is the same. For the same channel, different sample images or their corresponding feature maps pass through the channel. Different sub-weight values ​​are determined. This is because different sub-compression values ​​are obtained according to different sample images or their corresponding feature maps. In the present implementation, during the initial neural network model training process, the current sub-compression value corresponding to each channel contained in each convolution block is determined, and then the current sub-compression value corresponding to each channel contained in each convolution block and the corresponding historical sub-compression value are used to determine the target sub-compression value corresponding to each channel contained in each convolution block. When it is determined that the initial neural network model has converged, the target sub-compression value corresponding to each channel contained in each convolution block is the fixed compression value corresponding to each channel contained in each convolution block in the target neural network model. For each channel contained in each convolution block, based on the fixed compression value corresponding to each channel contained in each convolution block, the SE block corresponding to each convolution block is used to perform excitation processing on the fixed compression value corresponding to the channel, and a weight value representing the importance of the channel is determined to obtain the target neural network model.

[0199] In the process of training the initial neural network model, a Squeeze operation is performed on the feature maps corresponding to each channel contained in the current convolution block through the compression processing layer of the SE block corresponding to the current convolution block to obtain the current sub-compression value corresponding to each channel contained in the current convolution block; the historical sub-compression value corresponding to each channel contained in the current convolution block is obtained; the target sub-compression value corresponding to each channel contained in the current convolution block is determined by using the current sub-compression value and the historical sub-compression value corresponding to each channel contained in the current convolution block, and then, through the excitation processing layer of the SE block corresponding to the current convolution block, an Excitation operation is performed on the specified sub-compression value corresponding to each channel contained in the current convolution block to obtain the target sub-weight value corresponding to each channel contained in the current convolution block, and then subsequent training operations are performed.

[0200] Among them, if the designated sub-compression value corresponding to each channel contained in the current convolution block is the target sub-compression value corresponding to each channel contained in the current convolution block, when it is determined that the preset convergence condition is met, the designated sub-compression value corresponding to each channel contained in the current convolution block is directly passed through the excitation processing layer of the SE block corresponding to the current convolution block, and the target sub-weight value corresponding to each channel contained in the current convolution block is obtained as the weight value corresponding to each channel contained in the current convolution block. If the designated sub-compression value corresponding to each channel contained in the current convolution block is the current sub-compression value corresponding to each channel contained in the current convolution block, when it is determined that the preset convergence condition is met, the target sub-compression value corresponding to each channel contained in the current convolution block is Excitation operation is performed through the excitation processing layer of the SE block corresponding to the current convolution block when the preset convergence condition is met, and the weight value corresponding to each channel contained in the current convolution block is obtained.

[0201] In one implementation, S3043 can be implemented by any one of the following two implementations: the first implementation: using a preset compression value fusion formula, the current sub-compression value corresponding to the channel, and the historical sub-compression value to determine a fusion result as the target sub-compression value corresponding to the channel; the second implementation: using a preset compression value fusion formula, the current sub-compression value corresponding to the channel, and the historical sub-compression value to determine a fusion result as the intermediate sub-compression value corresponding to the channel; using a preset compression value determination formula, the intermediate sub-compression value corresponding to the channel, and the current sub-compression value to determine a determination result as the target sub-compression value corresponding to the channel.

[0202] It can be understood that in the first implementation method of determining the target sub-compression value corresponding to the channel mentioned above, the current sub-compression value corresponding to each channel is directly fused with the corresponding historical sub-compression value. To a certain extent, it is possible to achieve the fusion of the current sub-compression value corresponding to each channel corresponding to each sample image during the training process, and then determine the fixed weight value corresponding to each channel based on the excitation processing layer of the corresponding SE block and the fused target sub-compression value, so as to obtain a more accurate weight value that characterizes the importance of each channel.

[0203] Taking into account the problem that the boundaries of different sample images are blurred by the above-mentioned direct fusion statistics of the current sub-compression value corresponding to each channel corresponding to each sample image, in the second implementation method of determining the target sub-weight value corresponding to the channel, after the current sub-compression value corresponding to each channel is fused with the corresponding historical sub-compression value, the fused intermediate sub-compression value is fused with the current sub-compression value, and then based on the excitation processing layer of the corresponding SE block and the fused target sub-compression value, the fixed weight value corresponding to each channel is determined to determine a more accurate weight value that characterizes the importance of each channel.

[0204] The above preset compression value fusion formula is expressed as:

[0205] Among them, z i1 Indicates the intermediate sub-compression value corresponding to the i-th channel contained in the current convolution block during the current training process. Indicates the current sub-compression value corresponding to the i-th channel, represents the historical sub-compression value corresponding to the i-th channel, a1 and b1 are preset coefficients, a1+b1=1;

[0206] The preset compression value determination formula is expressed as:

[0207] Among them, z i Indicates the target sub-compression value corresponding to the i-th channel contained in the current convolution block during the current training process, γ represents the parameter to be adjusted, the initial value of γ is 0, and the value is 1 when the target neural network model is obtained. When it is determined that the preset convergence conditions are met, the z i It can represent the compressed value corresponding to the i-th channel contained in the current convolution block.

[0208] The above-mentioned preset compression value determination formula is a determination formula based on scheduling, that is, by scheduling the value of α, the compression value corresponding to each channel is adjusted and determined. In an embodiment of the present invention, the above-mentioned preset compression value determination formula can also be based on a determination formula in other forms, such as cosine, step decay, etc.

[0209] In another embodiment of the present invention, the image to be detected is an image captured by an image acquisition device set for the target vehicle; the detection result includes: position information corresponding to a preset detection target, and the preset detection target includes at least one of a lane line, a traffic sign, a zebra crossing and a parking space; the method may also include: identifying a map target that matches the preset detection target from a preset automatic navigation map; determining the current posture information of the target vehicle based on the position information corresponding to the preset detection target in the image to be detected, and the position information of the map target in the automatic navigation map.

[0210] In an embodiment of the present invention, the target neural network model is a model for target detection, and the corresponding image to be detected can be a target vehicle in the process of driving, and the image acquisition device set by it collects the image of the driving environment in which the target vehicle is located. The detection result corresponding to the image to be detected obtained by using the target neural network model includes at least one of a lane line, a traffic sign, a zebra crossing and a parking space. Subsequently, the electronic device can locate the target vehicle through the detection result and a preset automatic navigation map.

[0211] The electronic device may first obtain a preset automatic navigation map, and match the preset automatic navigation map to determine the location information of the preset detection target in the automatic navigation map as the map location information of the preset detection target. Then, based on the map location information of the preset detection target, the current position information of the target vehicle is determined. The process of determining the current position information of the target vehicle based on the map location information of the preset detection target can be referred to in the relevant technology, and will not be repeated here.

[0212] For a certain map target in a preset automatic navigation map, the preset detection target matched therewith is actually the projection of the map target on the camera imaging plane, that is, the image to be detected, and the specific projection position is determined according to the position and posture of the target vehicle when the camera captures the image to be detected. Therefore, the current position and posture information of the target vehicle can be determined according to the position information of the map target matched with the preset detection target in the preset automatic navigation map, and the position information of the preset detection target in the image to be detected, wherein the current position and posture information includes the current position information and posture information. According to the position information corresponding to the preset detection target identified from the image to be detected, and the position information of the map target matched with each preset detection target in the preset automatic navigation map, multiple position and posture information of the target vehicle can be calculated, and the electronic device can further calculate the current position and posture information of a target vehicle according to the multiple position and posture information of the target vehicle. Specifically, the current position and posture information of the target vehicle can be determined together by combining the position and posture information of the image acquisition device of the target vehicle when acquiring the previous image of the image to be detected, and the data collected by the IMU (Inertial measurement unit) set in the target vehicle.

[0213] The detection result corresponding to the image to be detected obtained by the image detection process based on the neural network model provided by the embodiment of the present invention has higher accuracy and shortens the time to obtain the detection result. Furthermore, the current posture information of the target vehicle is determined using the detection result, which ensures the accuracy of the determined current posture information and shortens the time to determine the current posture information. To a certain extent, the safety of the target vehicle during automatic driving is better guaranteed.

[0214] Figure 4A structural schematic diagram of an image detection device based on a neural network model provided by an embodiment of the present invention. The device includes: a first acquisition module 410, configured to obtain an image to be detected; a first determination module 420, configured to use each convolution block of the target neural network model obtained by pre-training to perform a convolution operation and a feature recalibration operation on the image to be detected, and determine the target feature map corresponding to the image to be detected, wherein the feature recalibration operation is: multiplying the weight value corresponding to each channel contained in the corresponding convolution block with the feature map obtained after the convolution operation corresponding to each channel; the weight value corresponding to each channel is used to characterize the importance of the corresponding channel, and the weight value corresponding to each channel is: a value determined according to the target sub-weight value or target sub-compression value corresponding to the channel, and the target sub-weight value or target sub-compression value corresponding to the channel is: a value determined based on the sample image of the target neural network model obtained by training; a second determination module 430, configured to use the fully connected layer of the target neural network model and the target feature map to determine the detection result corresponding to the image to be detected.

[0215] According to the embodiment of the present invention, in the target neural network model obtained by pre-training, each channel included in each convolution block corresponds to a weight value used to characterize the importance of the channel itself, so that in the process of actually using the target neural network model to detect an image, a feature map that can contain the importance of the channel can be obtained directly based on the weight value corresponding to each channel of each convolution block in the target neural network model and the feature map obtained after the convolution operation corresponding to the channel. Through the weight value corresponding to each channel, the useful features in the feature map obtained after the convolution operation corresponding to the channel are promoted and the less useful features are suppressed. Moreover, the weight value is a value determined based on the sample image in the process of training the target neural network model, which is a fixed value. In the actual detection process, the time and resource occupation of calculating the weight value corresponding to each channel are saved, and the time consumption and resource consumption in the detection process are saved to a certain extent. While ensuring that the importance of each channel is obtained, the useful features in the feature map corresponding to the image are promoted according to the importance of each channel and the less useful features are suppressed, and the time and resources consumed in the detection process using the network are reduced to a certain extent.

[0216] In another embodiment of the present invention, the first determination module 420 includes: a first input unit (not shown in the figure), configured to input the image to be detected or the current feature map corresponding to the image to be detected into the current convolution block, wherein the current convolution block is one of the convolution blocks of the target neural network model, if the current convolution block is the first convolution block of the target neural network model, the input of the current convolution block is the image to be detected; if the current convolution block is not the first convolution block of the target neural network model, the input of the current convolution block is: the current feature map corresponding to the image to be detected, and the output of the convolution block before the current convolution block;

[0217] A first determining unit (not shown in the figure) is configured to perform image feature extraction on the image to be detected or the current feature map corresponding to the image to be detected based on the convolution kernel corresponding to the channel included in the current convolution block, and determine the feature map corresponding to each channel included in the current convolution block;

[0218] A second determining unit (not shown in the figure) is configured to multiply the weight value corresponding to each channel included in the current convolution block and the feature map corresponding to the channel to determine a feature map after feature recalibration corresponding to each channel included in the current convolution block as a new current feature map corresponding to the image to be detected;

[0219] A third determining unit (not shown in the figure) is configured to determine the next convolution block of the current convolution block as a new current convolution block if the current convolution block is not the last convolution block of the target neural network model, and return to trigger the first input unit;

[0220] The fourth determination unit (not shown in the figure) is configured to determine the new current feature map corresponding to the image to be detected as the target feature map corresponding to the image to be detected if the current convolution block is the last convolution block of the target neural network model.

[0221] In another embodiment of the present invention, the device further includes: a first training module (not shown in the figure), which is configured to perform a convolution operation and a feature recalibration operation on the image to be detected by each convolution block of the target neural network model obtained by pre-training, and before determining the target feature map corresponding to the image to be detected, train the target neural network model, wherein the first training module includes:

[0222] A first obtaining submodule (not shown in the figure) is configured to obtain a plurality of sample images and a calibration label corresponding to each sample image;

[0223] A second obtaining submodule (not shown in the figure) is configured to obtain an initial neural network model, wherein the initial neural network model includes a plurality of convolution blocks, a fully connected layer, and a compression excitation SE block corresponding to each convolution block;

[0224] A first input submodule (not shown in the figure) is configured to input, for each sample image, the sample image and the calibration label corresponding to the sample image into the neural network model in the current training process, wherein the neural network model in the current training process is: the initial neural network model or the neural network model after adjusting parameters;

[0225] The third acquisition submodule (not shown in the figure) is configured to use the neural network model in the current training process to perform the following steps on the input sample image until a preset convergence condition is met, thereby obtaining a target neural network model including the multiple convolution blocks and the fully connected layer, wherein each channel included in each convolution block of the target neural network model corresponds to a fixed weight value;

[0226] The third obtaining submodule includes: a fifth determining unit (not shown in the figure), which is configured to perform image feature extraction on the sample image or the first feature map corresponding to the sample image based on the convolution kernel corresponding to the channel contained in the current convolution block, and determine the feature map corresponding to each channel contained in the current convolution block, wherein the current convolution block is: one of the convolution blocks of the neural network model in the current training process, if the current convolution block is: the first convolution block of the neural network model in the current training process, the input of the current convolution block is the sample image; if the current convolution block is: a non-first convolution block of the neural network model in the current training process, the input of the current convolution block is: the first feature map corresponding to the sample image, and is a value determined based on the output corresponding to the previous convolution block of the current convolution block and the SE block corresponding to the previous convolution block;

[0227] A second input unit (not shown in the figure) is configured to input the feature map corresponding to each channel included in the current convolution block into the SE block corresponding to the current convolution block to obtain the current sub-weight value corresponding to each channel included in the current convolution block in the current training process;

[0228] A sixth determination unit (not shown in the figure) is configured to determine, for each channel included in the current convolution block in the current training process, a target sub-weight value corresponding to the channel using the current sub-weight value and the historical sub-weight value corresponding to the channel, wherein the historical sub-weight value is: the sub-weight value corresponding to the channel included in the current convolution block obtained in a training process before the current training process;

[0229] A seventh determination unit (not shown in the figure) is configured to determine, for each channel included in the current convolution block in the current training process, a recalibrated feature map corresponding to the channel using a specified sub-weight value corresponding to the channel and a feature map corresponding to the channel, wherein the specified sub-weight value is: the current sub-weight value or the target sub-weight value;

[0230] an eighth determination unit (not shown in the figure), configured to determine the next convolution block of the current convolution block as a new current convolution block if the current convolution block is not the last convolution block of the neural network model in the current training process, and trigger the fifth determination unit;

[0231] A third input unit (not shown in the figure) is configured to, if the current convolution block is the last convolution block of the neural network model in the current training process, determine the recalibrated feature map corresponding to each channel contained in the current convolution block as the specific feature map corresponding to the sample image, and input it into the fully connected layer of the neural network model in the current training process to obtain a predicted label corresponding to the sample image;

[0232] The first adjustment unit (not shown in the figure) is configured to adjust the parameters of multiple convolution blocks, fully connected layers and SE blocks corresponding to each convolution block of the neural network model in the current training process based on the predicted labels and calibration labels corresponding to the obtained sample images if it is determined that the predicted labels corresponding to the obtained sample images reach a preset threshold; if it is determined that the preset convergence conditions are not met, trigger the first input submodule.

[0233] In another embodiment of the present invention, the sixth determining unit is specifically configured to: determine a fusion result as a target sub-weight value corresponding to the channel by using a preset weight value fusion formula, the current sub-weight value corresponding to the channel, and the historical sub-weight value;

[0234] Or it is specifically configured as: using the preset weight value fusion formula, the current sub-weight value corresponding to the channel and the historical sub-weight value to determine the fusion result as the intermediate sub-weight value corresponding to the channel; using the preset weight value determination formula, the intermediate sub-weight value corresponding to the channel and the current sub-weight value to determine the determination result as the target sub-weight value corresponding to the channel.

[0235] In another embodiment of the present invention, the device further includes: a second training module (not shown in the figure), configured to perform a convolution operation and a feature recalibration operation on the image to be detected in each convolution block of the target neural network model obtained by pre-training, and before determining the target feature map corresponding to the image to be detected, train the target neural network model, wherein the second training module includes: a fourth acquisition submodule (not shown in the figure), configured to obtain a plurality of sample images and a calibration label corresponding to each sample image;

[0236] A fifth obtaining submodule (not shown in the figure) is configured to obtain an initial neural network model, wherein the initial neural network model includes a plurality of convolution blocks, a fully connected layer, and a compression excitation SE block corresponding to each convolution block;

[0237] A second input submodule (not shown in the figure) is configured to input, for each sample image, the sample image and the calibration label corresponding to the sample image into the neural network model in the current training process, wherein the neural network model in the current training process is: the initial neural network model or the neural network model after adjusting parameters;

[0238] A sixth acquisition submodule (not shown in the figure) is configured to use the neural network model in the current training process to perform the following steps on the input sample image until a preset convergence condition is met, thereby obtaining a target neural network model including the multiple convolution blocks and the fully connected layer, wherein each channel included in each convolution block of the target neural network model corresponds to a fixed compression value;

[0239] The sixth acquisition submodule includes: a ninth determination unit (not shown in the figure), which is configured to perform image feature extraction on the sample image or the first feature map corresponding to the sample image based on the convolution kernel corresponding to the channel included in the current convolution block, and determine the feature map corresponding to each channel included in the current convolution block, wherein the current convolution block is: one of the convolution blocks of the neural network model in the current training process, if the current convolution block is: the first convolution block of the neural network model in the current training process, the input of the current convolution block is the sample image; if the current convolution block is: a non-first convolution block of the neural network model in the current training process, the input of the current convolution block is: the first feature map corresponding to the sample image, and is a value determined based on the output corresponding to the previous convolution block of the current convolution block and the SE block corresponding to the previous convolution block;

[0240] A fourth input unit (not shown in the figure) is configured to input the feature map corresponding to each channel contained in the current convolution block into the compression processing layer of the SE block corresponding to the current convolution block, so as to obtain the current sub-compression value corresponding to each channel contained in the current convolution block in the current training process;

[0241] a tenth determining unit (not shown in the figure), configured to determine, for each channel included in the current convolution block in the current training process, a target sub-compression value corresponding to the channel by using the current sub-compression value and the historical sub-compression value corresponding to the channel, wherein the historical sub-compression value is: a sub-compression value corresponding to the channel included in the current convolution block obtained in a training process before the current training process;

[0242] A fifth input unit (not shown in the figure) is configured to input the specified sub-compression value corresponding to each channel included in the current convolution block in the current training process into the excitation processing layer of the SE block corresponding to the current convolution block, so as to obtain the target sub-weight value corresponding to each channel included in the current convolution block in the current training process, wherein the specified sub-compression value is the current sub-compression value or the target sub-compression value;

[0243] An eleventh determining unit (not shown in the figure) is configured to determine, for each channel included in the current convolution block in the current training process, a recalibrated feature map corresponding to each channel included in the current convolution block in the current training process by using the target sub-weight value corresponding to the channel and the feature map corresponding to the channel;

[0244] A twelfth determining unit (not shown in the figure) is configured to determine the next convolution block of the current convolution block as a new current convolution block if the current convolution block is not the last convolution block of the neural network model in the current training process, thereby triggering the ninth determining unit;

[0245] A sixth input unit (not shown in the figure) is configured to, if the current convolution block is the last convolution block of the neural network model in the current training process, determine the recalibrated feature map corresponding to each channel contained in the current convolution block as the specific feature map corresponding to the sample image, and input it into the fully connected layer of the neural network model in the current training process to obtain a predicted label corresponding to the sample image;

[0246] The second adjustment unit (not shown in the figure) is configured to adjust the parameters of multiple convolution blocks, fully connected layers and SE blocks corresponding to each convolution block of the neural network model in the current training process based on the predicted labels and calibration labels corresponding to the obtained sample images if it is determined that the predicted labels corresponding to the obtained sample images reach a preset threshold; if it is determined that the preset convergence conditions are not met, trigger the second input submodule.

[0247] Optionally, the tenth determining unit is specifically configured to determine a fusion result as a target sub-compression value corresponding to the channel by using a preset compression value fusion formula, the current sub-compression value corresponding to the channel, and a historical sub-compression value;

[0248] Or it is specifically configured to use a preset compression value fusion formula, the current sub-compression value corresponding to the channel, and the historical sub-compression value to determine a fusion result as the intermediate sub-compression value corresponding to the channel; use a preset compression value determination formula, the intermediate sub-compression value corresponding to the channel, and the current sub-compression value to determine a determination result as the target sub-compression value corresponding to the channel.

[0249] In another embodiment of the present invention, the preset compression value fusion formula is expressed as:

[0250] Among them, the z i1 Indicates the intermediate sub-compression value corresponding to the i-th channel contained in the current convolution block during the current training process. represents the current sub-compression value corresponding to the i-th channel, represents the historical sub-compression value corresponding to the i-th channel, where a and b are preset coefficients, and a+b=1;

[0251] The preset compression value determination formula is expressed as:

[0252] Among them, the z i It represents the target sub-compression value corresponding to the i-th channel contained in the current convolution block in the current training process, the γ represents the parameter to be adjusted, the initial value of γ is 0, and the value is 1 when the target neural network model is obtained.

[0253] In another embodiment of the present invention, the image to be detected is an image captured by an image acquisition device set for the target vehicle; the detection result includes: location information corresponding to a preset detection target, and the preset detection target includes at least one of a lane line, a traffic sign, a zebra crossing, and a parking space; the device also includes: a recognition module (not shown in the figure), which is configured to identify a map target matching the preset detection target from a preset automatic navigation map;

[0254] The third determination module (not shown in the figure) is configured to determine the current posture information of the target vehicle based on the position information corresponding to the preset detection target in the image to be detected and the position information of the map target in the automatic navigation map.

[0255] The above device embodiment corresponds to the method embodiment and has the same technical effect as the method embodiment. For specific description, please refer to the method embodiment. The device embodiment is obtained based on the method embodiment. For specific description, please refer to the method embodiment part, which will not be repeated here. It can be understood by those skilled in the art that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0256] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0257] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image detection method based on a neural network model, characterized in that: include: Obtaining an image to be detected; Using each convolution block of the pre-trained target neural network model, a convolution operation and a feature recalibration operation are performed on the image to be detected to determine the target feature map corresponding to the image to be detected, wherein the feature recalibration operation is: multiplying the weight value corresponding to each channel contained in the corresponding convolution block by the feature map obtained after the convolution operation corresponding to each channel; the weight value corresponding to each channel is used to characterize the importance of the corresponding channel, and the weight value corresponding to each channel is: a value determined according to the target sub-weight value or target sub-compression value corresponding to the channel, and the target sub-weight value or target sub-compression value corresponding to the channel is: a value determined based on a sample image of the target neural network model obtained through training; Determine the detection result corresponding to the image to be detected by using the fully connected layer of the target neural network model and the target feature map; In the process of iteratively training the initial neural network model to obtain the target neural network model, based on the convolution kernel corresponding to the channel contained in the current convolution block, image feature extraction is performed on the sample image or the first feature map corresponding to the sample image, and the feature map corresponding to each channel contained in the current convolution block is determined, wherein the current convolution block is: one of the convolution blocks of the neural network model in the current training process, if the current convolution block is: the first convolution block of the neural network model in the current training process, the input of the current convolution block is the sample image; if the current convolution block is: a non-first convolution block of the neural network model in the current training process, the input of the current convolution block is: the first feature map corresponding to the sample image, and is a value determined based on the output corresponding to the previous convolution block of the current convolution block and the compressed excitation SE block corresponding to the previous convolution block; Input the feature map corresponding to each channel contained in the current convolution block into the SE block corresponding to the current convolution block to obtain the current sub-weight value corresponding to each channel contained in the current convolution block in the current training process; for each channel contained in the current convolution block in the current training process, use the current sub-weight value corresponding to the channel and the historical sub-weight value to determine the target sub-weight value corresponding to the channel, wherein the historical sub-weight value is: the sub-weight value corresponding to the channel contained in the current convolution block obtained in the previous training process of the current training process; or, The feature maps corresponding to the channels contained in the current convolution block are input into the compression processing layer of the SE block corresponding to the current convolution block to obtain the current sub-compression values ​​corresponding to the channels contained in the current convolution block during the current training process; for each channel contained in the current convolution block during the current training process, the target sub-compression value corresponding to the channel is determined using the current sub-compression value and the historical sub-compression value corresponding to the channel, wherein the historical sub-compression value is: the sub-compression value corresponding to the channel contained in the current convolution block obtained in the previous training process of the current training process.

2. The method according to claim 1, characterized in that The step of using each convolution block of the pre-trained target neural network model to perform a convolution operation and a feature recalibration operation on the image to be detected to determine a target feature map corresponding to the image to be detected includes: Using each convolution block of the pre-established target neural network model, perform the following operations to determine the target feature map corresponding to the image to be detected; Input the image to be detected or the current feature map corresponding to the image to be detected into the current convolution block, wherein the current convolution block is one of the convolution blocks of the target neural network model. If the current convolution block is the first convolution block of the target neural network model, the input of the current convolution block is the image to be detected; if the current convolution block is not the first convolution block of the target neural network model, the input of the current convolution block is: the current feature map corresponding to the image to be detected, and the output of the previous convolution block of the current convolution block; Based on the convolution kernels corresponding to the channels included in the current convolution block, image feature extraction is performed on the image to be detected or the current feature map corresponding to the image to be detected, and the feature maps corresponding to the channels included in the current convolution block are determined; For each channel included in the current convolution block, multiply the weight value corresponding to the channel and the feature map corresponding to the channel to determine the feature map after feature recalibration corresponding to each channel included in the current convolution block as a new current feature map corresponding to the image to be detected; If the current convolution block is not the last convolution block of the target neural network model, determine the next convolution block of the current convolution block as a new current convolution block, and return to execute the step of inputting the image to be detected or the current feature map corresponding to the image to be detected into the current convolution block; If the current convolution block is the last convolution block of the target neural network model, the new current feature map corresponding to the image to be detected is determined as the target feature map corresponding to the image to be detected.

3. The method according to claim 1, characterized in that Before the step of performing a convolution operation and a feature recalibration operation on the image to be detected using each convolution block of the pre-trained target neural network model to determine a target feature map corresponding to the image to be detected, the method further includes: The process of training to obtain the target neural network model includes: Obtain multiple sample images and a calibration label corresponding to each sample image; Obtaining an initial neural network model, wherein the initial neural network model includes a plurality of convolution blocks, a fully connected layer, and a compression excitation SE block corresponding to each convolution block; For each sample image, the sample image and the calibration label corresponding to the sample image are input into the neural network model in the current training process, wherein the neural network model in the current training process is: the initial neural network model or the neural network model after adjusting parameters; Using the neural network model in the current training process, the following steps are performed on the input sample image until a preset convergence condition is met, thereby obtaining a target neural network model including the multiple convolution blocks and the fully connected layer, wherein each channel included in each convolution block of the target neural network model corresponds to a fixed weight value; Based on the convolution kernel corresponding to the channel included in the current convolution block, image feature extraction is performed on the sample image or the first feature map corresponding to the sample image, and the feature map corresponding to each channel included in the current convolution block is determined, wherein the current convolution block is: one of the convolution blocks of the neural network model in the current training process. If the current convolution block is: the first convolution block of the neural network model in the current training process, the input of the current convolution block is the sample image; if the current convolution block is: a non-first convolution block of the neural network model in the current training process, the input of the current convolution block is: the first feature map corresponding to the sample image, and is a value determined based on the output corresponding to the previous convolution block of the current convolution block and the SE block corresponding to the previous convolution block; Input the feature map corresponding to each channel contained in the current convolution block into the SE block corresponding to the current convolution block, and obtain the current sub-weight value corresponding to each channel contained in the current convolution block in the current training process; For each channel included in the current convolution block in the current training process, determine the target sub-weight value corresponding to the channel by using the current sub-weight value and the historical sub-weight value corresponding to the channel, wherein the historical sub-weight value is: the sub-weight value corresponding to the channel included in the current convolution block obtained in a training process before the current training process; For each channel included in the current convolution block in the current training process, using the specified sub-weight value corresponding to the channel and the feature map corresponding to the channel, determine the recalibrated feature map corresponding to the channel, wherein the specified sub-weight value is: the current sub-weight value or the target sub-weight value; If the current convolution block is not the last convolution block of the neural network model in the current training process, determine the next convolution block of the current convolution block as a new current convolution block, return to execute the step of performing image feature extraction on the sample image or the first feature map corresponding to the sample image based on the convolution kernel corresponding to the channel included in the current convolution block, and determine the feature map corresponding to each channel included in the current convolution block; If the current convolution block is the last convolution block of the neural network model in the current training process, the recalibrated feature map corresponding to each channel contained in the current convolution block is determined as the specific feature map corresponding to the sample image, and is input into the fully connected layer of the neural network model in the current training process to obtain a predicted label corresponding to the sample image; If it is determined that the predicted label corresponding to the obtained sample image reaches a preset threshold, based on the predicted label and the calibration label corresponding to the obtained sample image, adjusting the parameters of the multiple convolution blocks, the fully connected layer, and the SE block corresponding to each convolution block of the neural network model in the current training process; If it is determined that the preset convergence condition is not met, return to execute the step of inputting the sample image and the calibration label corresponding to the sample image into the neural network model in the current training process for each sample image.

4. The method according to any one of claims 1 to 3, characterized in that: The step of determining the target sub-weight value corresponding to the channel by using the current sub-weight value and the historical sub-weight value corresponding to the channel is implemented by any one of the following two implementations: The first implementation method: Using the preset weight value fusion formula, the current sub-weight value corresponding to the channel and the historical sub-weight value, a fusion result is determined as the target sub-weight value corresponding to the channel; The second implementation method: Using the preset weight value fusion formula, the current sub-weight value corresponding to the channel and the historical sub-weight value, a fusion result is determined as the intermediate sub-weight value corresponding to the channel; A determination result is determined using a preset weight value determination formula, the intermediate sub-weight value corresponding to the channel, and the current sub-weight value, as the target sub-weight value corresponding to the channel.

5. The method according to claim 1, characterized in that Before the step of performing a convolution operation and a feature recalibration operation on the image to be detected using each convolution block of the pre-trained target neural network model to determine a target feature map corresponding to the image to be detected, the method further includes: The process of training to obtain the target neural network model includes: Obtain multiple sample images and a calibration label corresponding to each sample image; Obtaining an initial neural network model, wherein the initial neural network model includes a plurality of convolution blocks, a fully connected layer, and a compression excitation SE block corresponding to each convolution block; For each sample image, the sample image and the calibration label corresponding to the sample image are input into the neural network model in the current training process, wherein the neural network model in the current training process is: the initial neural network model or the neural network model after adjusting parameters; Using the neural network model in the current training process, the following steps are performed on the input sample image until a preset convergence condition is met, thereby obtaining a target neural network model including the multiple convolution blocks and the fully connected layer, wherein each channel included in each convolution block of the target neural network model corresponds to a fixed weight value; Based on the convolution kernel corresponding to the channel included in the current convolution block, image feature extraction is performed on the sample image or the first feature map corresponding to the sample image, and the feature map corresponding to each channel included in the current convolution block is determined, wherein the current convolution block is: one of the convolution blocks of the neural network model in the current training process. If the current convolution block is: the first convolution block of the neural network model in the current training process, the input of the current convolution block is the sample image; if the current convolution block is: a non-first convolution block of the neural network model in the current training process, the input of the current convolution block is: the first feature map corresponding to the sample image, and is a value determined based on the output corresponding to the previous convolution block of the current convolution block and the SE block corresponding to the previous convolution block; Input the feature map corresponding to each channel contained in the current convolution block into the compression processing layer of the SE block corresponding to the current convolution block, and obtain the current sub-compression value corresponding to each channel contained in the current convolution block in the current training process; For each channel included in the current convolution block in the current training process, determine a target sub-compression value corresponding to the channel by using the current sub-compression value and the historical sub-compression value corresponding to the channel, wherein the historical sub-compression value is: a sub-compression value corresponding to the channel included in the current convolution block obtained in a training process before the current training process; Inputting the designated sub-compression value corresponding to each channel contained in the current convolution block in the current training process into the excitation processing layer of the SE block corresponding to the current convolution block, to obtain the target sub-weight value corresponding to each channel contained in the current convolution block in the current training process, wherein the designated sub-compression value is the current sub-compression value or the target sub-compression value; For each channel included in the current convolution block in the current training process, using the target sub-weight value corresponding to the channel and the feature map corresponding to the channel, determine the recalibrated feature map corresponding to each channel included in the current convolution block in the current training process; If the current convolution block is not the last convolution block of the neural network model in the current training process, determine the next convolution block of the current convolution block as a new current convolution block, return to execute the step of performing image feature extraction on the sample image or the first feature map corresponding to the sample image based on the convolution kernel corresponding to the channel included in the current convolution block, and determine the feature map corresponding to each channel included in the current convolution block; If the current convolution block is the last convolution block of the neural network model in the current training process, the recalibrated feature map corresponding to each channel contained in the current convolution block is determined as the specific feature map corresponding to the sample image, and is input into the fully connected layer of the neural network model in the current training process to obtain a predicted label corresponding to the sample image; If it is determined that the predicted label corresponding to the obtained sample image reaches a preset threshold, based on the predicted label and the calibration label corresponding to the obtained sample image, adjusting the parameters of the multiple convolution blocks, the fully connected layer, and the SE block corresponding to each convolution block of the neural network model in the current training process; If it is determined that the preset convergence condition is not met, return to execute the step of inputting the sample image and the calibration label corresponding to the sample image into the neural network model in the current training process for each sample image.

6. The method according to claim 5, characterized in that The step of determining the target sub-compression value corresponding to the channel by using the current sub-compression value and the historical sub-compression value corresponding to the channel is implemented by any one of the following two implementations: The first implementation method: Using a preset compression value fusion formula, the current sub-compression value corresponding to the channel, and the historical sub-compression value, a fusion result is determined as the target sub-compression value corresponding to the channel; The second implementation method: Using a preset compression value fusion formula, the current sub-compression value corresponding to the channel, and the historical sub-compression value, a fusion result is determined as the intermediate sub-compression value corresponding to the channel; A determination result is determined using a preset compression value determination formula, the intermediate sub-compression value corresponding to the channel, and the current sub-compression value, as the target sub-compression value corresponding to the channel.

7. The method according to claim 6, characterized in that The preset compression value fusion formula is expressed as: Among them, the z i1 Indicates the intermediate sub-compression value corresponding to the i-th channel contained in the current convolution block during the current training process. represents the current sub-compression value corresponding to the i-th channel, represents the historical sub-compression value corresponding to the i-th channel, where a1 and b1 are preset coefficients, a1+b1=1; The preset compression value determination formula is expressed as: Among them, the z i It represents the target sub-compression value corresponding to the i-th channel contained in the current convolution block in the current training process, the γ represents the parameter to be adjusted, the initial value of γ is 0, and the value is 1 when the target neural network model is obtained.

8. The method according to any one of claims 1 to 7, characterized in that: The image to be detected is an image captured by an image acquisition device set for the target vehicle; the detection result includes: position information corresponding to a preset detection target, and the preset detection target includes at least one of a lane line, a traffic sign, a zebra crossing, and a parking space; The method further comprises: Identifying a map target that matches the preset detection target from a preset automatic navigation map; The current position information of the target vehicle is determined according to the position information corresponding to the preset detection target in the image to be detected and the position information of the map target in the automatic navigation map.

9. An image detection device based on a neural network model, characterized in that: include: A first acquisition module is configured to obtain an image to be detected; A first determination module is configured to use each convolution block of the pre-trained target neural network model to perform a convolution operation and a feature recalibration operation on the image to be detected, and determine a target feature map corresponding to the image to be detected, wherein the feature recalibration operation is: multiplying the weight value corresponding to each channel contained in the corresponding convolution block by the feature map obtained after the convolution operation corresponding to each channel; the weight value corresponding to each channel is used to characterize the importance of the corresponding channel, and the weight value corresponding to each channel is: a value determined according to the target sub-weight value or target sub-compression value corresponding to the channel, and the target sub-weight value or target sub-compression value corresponding to the channel is: a value determined based on a sample image of the target neural network model obtained through training; Wherein, in the process of iteratively training the initial neural network model to obtain the target neural network model, based on the convolution kernel corresponding to the channel contained in the current convolution block, image feature extraction is performed on the sample image or the first feature map corresponding to the sample image, and the feature map corresponding to each channel contained in the current convolution block is determined, wherein the current convolution block is: one of the convolution blocks of the neural network model in the current training process, if the current convolution block is: the first convolution block of the neural network model in the current training process, the input of the current convolution block is the sample image; if the current convolution block is: a non-first convolution block of the neural network model in the current training process, the input of the current convolution block is: the first feature map corresponding to the sample image, and is a value determined based on the output corresponding to the previous convolution block of the current convolution block and the compressed excitation SE block corresponding to the previous convolution block; Input the feature map corresponding to each channel contained in the current convolution block into the SE block corresponding to the current convolution block to obtain the current sub-weight value corresponding to each channel contained in the current convolution block in the current training process; for each channel contained in the current convolution block in the current training process, use the current sub-weight value corresponding to the channel and the historical sub-weight value to determine the target sub-weight value corresponding to the channel, wherein the historical sub-weight value is: the sub-weight value corresponding to the channel contained in the current convolution block obtained in the previous training process of the current training process; or, Input the feature map corresponding to each channel included in the current convolution block into the compression processing layer of the SE block corresponding to the current convolution block to obtain the current sub-compression value corresponding to each channel included in the current convolution block in the current training process; for each channel included in the current convolution block in the current training process, use the current sub-compression value corresponding to the channel and the historical sub-compression value to determine the target sub-compression value corresponding to the channel, wherein the historical sub-compression value is: the sub-compression value corresponding to the channel included in the current convolution block obtained in the previous training process of the current training process; The second determination module is configured to use the fully connected layer of the target neural network model and the target feature map to determine the detection result corresponding to the image to be detected.

10. The device according to claim 9, characterized in that The first determining module includes: A first input unit is configured to input the image to be detected or the current feature map corresponding to the image to be detected into a current convolution block, wherein the current convolution block is one of the convolution blocks of the target neural network model. If the current convolution block is the first convolution block of the target neural network model, the input of the current convolution block is the image to be detected; if the current convolution block is not the first convolution block of the target neural network model, the input of the current convolution block is: the current feature map corresponding to the image to be detected, and the output of the previous convolution block of the current convolution block; A first determining unit is configured to perform image feature extraction on the image to be detected or the current feature map corresponding to the image to be detected based on the convolution kernel corresponding to the channel included in the current convolution block, and determine the feature map corresponding to each channel included in the current convolution block; A second determining unit is configured to multiply, for each channel included in the current convolution block, a weight value corresponding to the channel and a feature map corresponding to the channel, and determine a feature map after feature recalibration corresponding to each channel included in the current convolution block as a new current feature map corresponding to the image to be detected; A third determining unit is configured to determine the next convolution block of the current convolution block as a new current convolution block if the current convolution block is not the last convolution block of the target neural network model, and return to trigger the first input unit; The fourth determination unit is configured to determine the new current feature map corresponding to the image to be detected as the target feature map corresponding to the image to be detected if the current convolution block is the last convolution block of the target neural network model.

Images