Image detection method, device, equipment and computer storage medium

Abstract

The embodiment of the present application relates to the technical field of computer data processing, and discloses an image detection method, which comprises the following steps: inputting a to-be-detected image into a target detection model to obtain a target detection result corresponding to the to-be-detected image; the target detection model comprises an improved YOLOv3 model; the improved YOLOv3 model comprises a high-dimensional convolution module; the high-dimensional convolution module is used for performing deep convolution on the to-be-detected image in a high-dimensional space to obtain image features of the to-be-detected image. Through the above manner, the embodiment of the present application realizes the balance between the precision and the detection speed when performing target detection on high-resolution images such as remote sensing images.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer data processing, and particularly relate to an image detection method, device, equipment and computer storage medium. BACKGROUND

[0002] Remote sensing images have the advantages of super-long distance, all-day operation and strong anti-interference capability, and image detection and recognition of remote sensing images are research hotspots in computer vision. In the prior art, the detection task of high-resolution remote sensing images is mainly based on the YOLO (You only look once) series. The YOLO algorithm first divides an image into SxS grids, and determines the center of an object overlapping with a certain grid, so that the prediction of the object will be responsible by the network. YOLOv3 analyzes the data set labels through K-means clustering based on v1, and upgrades the feature extraction network from Darknet19 to Darknet53.

[0003] The inventors have found in the implementation of embodiments of the present application that the YOLOv3 model mainly extracts features through RES and DBL structures. A RES operation mainly consists of a 3x3 convolution with a step of 2 and n ResUnits, and a ResUnit consists of a 1x1 convolution and a 3x3 convolution operation. A large number of 3x3 convolution operations cause a dramatic increase in model parameters. Considering that the resolution of remote sensing images is already very high, YOLOv3 cannot balance the detection speed and accuracy. SUMMARY

[0004] In view of the above problems, embodiments of the present application provide an image detection method for solving the problem that the existing YOLOv3 cannot balance the detection speed and accuracy when performing target detection on high-resolution images.

[0005] According to an aspect of embodiments of the present application, an image detection method is provided, which comprises:

[0006] inputting a to-be-detected image into a target detection model to obtain a target detection result corresponding to the to-be-detected image; the target detection model comprises an improved YOLOv3 model; the improved YOLOv3 model comprises a high-dimensional convolution module; the high-dimensional convolution module is used for performing deep convolution on the to-be-detected image in a high-dimensional space to obtain image features of the to-be-detected image.

[0007] In an optional mode, the high-dimensional convolution module comprises a first deep convolution layer, a dimension adjustment convolution layer and a second deep convolution layer connected in sequence; the first deep convolution layer and the second deep convolution layer are used for deep convolution of input data; and the dimension adjustment convolution layer is used for dimension adjustment of the input data, so that the data input into the second deep convolution layer is high-dimensional data.

[0008] In an optional mode, the improved YOLOv3 model comprises a multi-scale convolution module; the multi-scale convolution module is used for convolution of input data under multi-scale convolution kernels, to obtain multi-scale image features; and the convolution kernel comprises the high-dimensional convolution module.

[0009] In an optional mode, the multi-scale convolution module comprises a plurality of convolution kernels connected in sequence; the size of the convolution kernel decreases in sequence, and the depth of the convolution kernel increases in sequence; and the plurality of convolution kernels adopts grouped convolution, so that each convolution kernel outputs image features of the same number of channels.

[0010] In an optional mode, the improved YOLOv3 model further comprises a feature enhancement module; the feature enhancement module is used for feature enhancement of the image features according to a channel attention mechanism.

[0011] In an optional mode, the feature enhancement module comprises a pooling layer, a plurality of fully connected layers and a weighting calculation layer connected in sequence; the pooling layer is used for extracting image features of a plurality of channels; the fully connected layer is used for determining inter-channel dependency according to the image features of each channel; and the weighting calculation layer is used for weighting calculation of the image features of the plurality of channels according to the inter-channel dependency, to obtain enhanced image features.

[0012] In an optional mode, the image to be detected comprises a high-resolution image; the improved YOLOv3 model is improved based on an original YOLOv3 model; the original YOLOv3 model comprises an original DBL module and an original RES module, the original DBL module comprises an original convolution layer, a normalization layer and an activation function layer connected in sequence, and the original RES module comprises a zero padding layer, the original DBL module and an original residual structure connected in sequence; wherein in the improved YOLOv3 model, the original convolution layer is replaced by a high-dimensional convolution module, a feature enhancement module is added after the activation function layer, and a multi-scale convolution module is added after the original residual structure; the high-dimensional convolution module is used for deep convolution of input data in a high-dimensional space; the feature enhancement module is used for feature enhancement of the image features according to a channel attention mechanism; and the multi-scale convolution module is used for convolution of input data under a multi-scale convolution kernel; and the convolution kernel comprises the high-dimensional convolution module.

[0013] According to another aspect of the embodiments of the present application, an image detection device is provided, comprising:

[0014] a detection module configured to input an image to be detected into a target detection model to obtain a target detection result corresponding to the image to be detected; the target detection model comprises an improved YOLOv3 model; the improved YOLOv3 model comprises a high-dimensional convolution module; and the high-dimensional convolution module is configured to perform deep convolution of the image to be detected in a high-dimensional space to obtain image features of the image to be detected.

[0015] According to another aspect of the embodiments of the present application, an image detection device is provided, comprising:

[0016] a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface complete communication with each other through the communication bus;

[0017] the memory is configured to store at least one executable instruction, and the executable instruction causes the processor to perform the operations of the image detection method embodiments of any one of the preceding aspects.

[0018] According to still another aspect of the embodiments of the present application, a computer readable storage medium is provided, and the storage medium stores at least one executable instruction, and the executable instruction causes the image detection device to perform the operations of the image detection method embodiments of any one of the preceding aspects.

[0019] The embodiment of the application inputs a to-be-detected image into a target detection model to obtain a target detection result corresponding to the to-be-detected image; the target detection model comprises an improved YOLOv3 model; the improved YOLOv3 model comprises a high-dimensional convolution module; wherein the high-dimensional convolution module is used for performing deep convolution on the to-be-detected image in a high-dimensional space to obtain image features of the to-be-detected image, thereby distinguishing from the existing YOLOv3 model which adopts a large number of 3x3 convolution operations for feature extraction, resulting in a problem of excessive model parameter quantity and poor detection speed, the embodiment of the application places the deep convolution at the beginning and end positions of the entire convolution structure, so that the deep convolution operation acts on the high-dimensional space, thereby extracting information-rich feature representations, performing identification mapping and spatial transformation in a higher dimension, reducing the risk of information loss and gradient confusion caused by the traditional mobile network residual structure, thereby effectively improving the processing speed of the model and achieving a balance between detection accuracy and detection speed.

[0020] The above description is only a summary of the technical scheme of the embodiments of the application, in order to more clearly understand the technical means of the embodiments of the application, the content of the specification can be implemented, and in order to make the above and other purposes, characteristics and advantages of the embodiments of the application more obvious and easy to understand, the specific embodiments of the application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings are only used to show the embodiments and are not considered as limiting the application. Moreover, the same reference signs are used to represent the same parts throughout the drawings. In the drawings:

[0022] Figure 1 A model structure diagram of the existing YOLOv3 is shown;

[0023] Figure 2 A flow diagram of the image detection method provided by the embodiment of the application is shown;

[0024] Figure 3 A structure diagram of the high-dimensional convolution model in the image detection method provided by the embodiment of the application is shown;

[0025] Figure 4 A structure diagram of the multi-scale convolution module in the image detection method provided by the embodiment of the application is shown;

[0026] Figure 5 A structure diagram of the feature enhancement module in the image detection method provided by the embodiment of the application is shown;

[0027] Figure 6 A structure diagram of the improved DBL module in the image detection method provided by the embodiment of the application is shown;

[0028] Figure 7 A structure diagram of the improved NRES module in the image detection method provided by the embodiment of the present application is shown;

[0029] Figure 8 A comparison diagram of detection results of the improved YOLOv3 in the image detection method provided by the embodiment of the present application and the prior art is shown;

[0030] Figure 9 A structure diagram of the image detection device provided by the embodiment of the present application is shown;

[0031] Figure 10 A structure diagram of the image detection device provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0032] Exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0033] Related terms are explained as follows:

[0034] IoU value: Intersection over Union, also known as intersection over union, is usually applied in the evaluation of the current target detection algorithm. The higher the IOU value, the higher the prediction accuracy of the algorithm for the target. The value range of IoU is [0, 1], and the larger the IoU value, the better the two boxes coincide.

[0035] Pooling operation: a matrix window is used to scan the input tensor, and the value in each matrix window is reduced by taking the maximum value, average value, etc. (reducing the number of elements is equivalent to extracting the important features of the elements) Because the purpose of convolution is to extract features, after all the features are extracted, the key features need to be further screened out. Therefore, the mechanical pooling operation of the features can very effectively reduce the size of the matrix, such as reducing the length and width of the matrix.

[0036] Grouped Convolution: For normal convolution, if the input feature map size is CxHxW, the number of convolution kernels is N, and the size of each convolution kernel is CxKxK, then the output feature map size is CxHxN, and the total parameter quantity is: NxCxKxK. If grouped convolution is performed, assuming that it is divided into G groups, the number of input feature maps in each group is CG, the number of output feature maps in each group is NG, the size of each convolution kernel is CGxKxK, and the number of convolution kernels in each group is NG, and the convolution kernel only convolves with the input in the same group, then the total parameter quantity is NxCGxKxK, and the total parameter quantity is reduced to 1G of the previous one. When the number of groups is equal to the number of input maps, the number of output maps is also equal to the number of input maps, that is, G=N=C, and the size of each convolution kernel is 1xKxK. It becomes Depthwise convolution (Depthwise Separable Convolution).

[0037] Channel Attention Mechanism: used to explicitly model the interdependence between feature channels, by adopting a brand-new "feature re-scaling" strategy - adaptively recalibrating the feature responses of channels, so that the network pays more attention to the target to be detected, and improves the detection effect.

[0038] Pyramid Convolution (PyConv): processes the input at multiple filter scales. PyConv contains a kernel pyramid, each layer contains different types of filters (the size and depth of the filter can be variable, so it can extract details of different scales). In addition to being able to extract multi-scale information, compared with standard convolution, PyConv is efficient, that is, it does not increase additional computational and parameter quantities. Further, it is more flexible and scalable, providing a larger architecture design space for different applications.

[0039] NWPU VHR-10 dataset: a 10-class geospatial object detection dataset for research and publication. The 10 objects are airplanes, ships, tanks, baseball fields, tennis courts, basketball courts, ground runways, ports, bridges, and vehicles. This dataset contains a total of 800 super high resolution (VHR) remote sensing images, which are cropped from Google Earth and Vaihingen datasets and annotated manually by experts.

[0040] Before describing the embodiments of the present application, the prior art and the problems thereof will be further described:

[0041] Remote sensing images have the advantages of super-long distance, all-day operation, and strong anti-interference capability. Remote sensing image target detection and recognition is a research hotspot in computer vision. At present, optical remote sensing image target detection is attracting much attention, and it has been widely applied in city security, land planning and other fields.

[0042] For the detection task of high-resolution remote sensing images, the YOLO series is mainly used. YOLO algorithm first divides an image into SxS grids, and determines the center of the object overlapping with a certain grid, then the prediction of the object is responsible by the network. YOLO predicts two information of the bounding box for each grid: position and confidence. The position information contains four parameters, which are the center coordinates and width and height of the bounding box. Confidence has two meanings, one is whether there is an object in the predicted bounding box, and the other is the accuracy of the predicted bounding box. Its definition is as follows:

[0043]

[0044] where P r (object) represents whether there is an object in the grid, 1 if there is, otherwise 0; represents the IoU value between the predicted bounding box and the ground truth (standard value). Each bounding box predicts (x, y, w, h) and confidence, a total of 5 values. Each grid also needs to predict a class information, denoted as C classes. Then SxS grids, each grid needs to predict B bounding boxes and C classes. Finally, the network outputs a tensor SxSx(5xB+C).

[0045] YOLOv3 analyzes the dataset label through K-means clustering based on v1, and upgrades the feature extraction network from Darknet19 to Darknet53. Among them, the darknet53 network is the backbone network of YOLOV3, which is used to extract 8, 16, 32 times down-sampling features. The network part uses a large number of 1x1 convolution and 3x3 convolution, in which 1x1 is mainly used for channel expansion and reduction. The overall convolution network adopts the Conv+BN+LeakyReLU structure form, and the residual network first uses a 1x1 convolution kernel to shrink the channel, and then uses a 3x3 convolution kernel to restore the channel, which is essentially a matrix decomposition idea, used to reduce the parameter amount.

[0046] Specifically, the structure of the existing YOLOv3 model is as shown in Figure 1 First, the three basic components of YOLOv3 are described:

[0047] DBL: The smallest component in the YOLOv3 network structure, composed of Conv+BN+Leaky_relu. Among them, Conv is the convolution layer, BN is the Batch Normalization, the normalization layer, and Leaky_relu is the activation function layer; among them, BN will be normalized in the batch size dimension, and the same channel of different samples will be directly normalized to obtain C mean and variance, and C γ, β (γ, β is the learning parameter of the BN layer). The role of the BN layer includes speeding up the training and convergence of the network, controlling gradient explosion to prevent gradient disappearance, and preventing overfitting.

[0048] Res unit: Referring to the residual structure in the Resnet network, the network can be built deeper.

[0049] ResX: composed of a DBL and X residual components, is a large component in YOLOv3. The DBL in front of each Res module plays the role of downsampling, so after 5 Res modules, the feature map obtained is 608->304->152->76->38->19 size.

[0050] Other basic operations in YOLOv3:

[0051] Concat: tensor concatenation, concatenating the darknet intermediate layer and the up-sampling of the subsequent layer. The operation of concatenation is different from the operation of residual layer add. The concatenation will expand the dimension of the tensor, while the add will only directly add without changing the dimension of the tensor.

[0052] add: tensor addition, the tensor is directly added, which does not expand the dimension, for example, 104x104x128 and 104x104x128 are added, the result is still 104x104x128. add and shortcut in the cfg file have the same function.

[0053] The number of convolution layers in the backbone: each ResX contains 1+2xX convolution layers, so the entire backbone network contains 1+(1+2x1)+(1+2x2)+(1+2x8)+(1+2x8)+(1+2x4)=52, plus a FC full connection layer, which can form a Darknet53 classification network. However, in the image detection YOLOv3, the FC layer is removed, but the backbone network of YOLOv3 is still called Darknet53 structure for the convenience of calling.

[0054] As Figure 1As shown, YOLO_v3 uses the first 52 layers of darknet-53 (without the fully connected layer), YOLO_v3 is a fully convolutional network, which uses a lot of residual skip connections, and in order to reduce the negative effects of gradient brought by pooling, it directly abandons the POOLing (pooling), and uses the stride of conv (convolution) to realize the down-sampling (that is, after one convolution, the feature map slides several grids, the default is 1, that is, slides one grid). In this network structure, the convolution with a step of 2 is used for down-sampling.

[0055] In order to enhance the accuracy of the algorithm for small image detection, YOLO v3 uses upsample (upsample) and fusion similar to FPN, and finally fuses 3 scales (scales), and the sizes of the other two scales are 26x26 and 52x52, respectively. Detection is performed on multiple scale feature maps (feature maps).

[0056] The three prediction branches also use a fully convolutional structure, and the number of convolution kernels in the last convolution layer is 255, which is for the 80 classes of COCO dataset: 3x(80+4+1)=255, 3 represents that a grid cell contains 3 bounding boxes (bounding boxes), 4 represents the 4 coordinate information of the detection box, and 1 represents the objectness score (objectness score). The multi-scale image feature detection result is obtained from the three prediction paths, y1, y2 and y3, and the depth of each is 255 (i.e. 3x(5+80)), and the side length is 13:26:52. YOLO v3 sets to predict 3 bounding boxes for each grid cell, so each box needs to have (x, y, w, h, confidence) five basic parameters, and then there are 80 class probabilities.

[0057] Three detections are performed in the network, respectively at 32 times down-sampling, 16 times down-sampling, and 8 times down-sampling, so that detection is performed on multi-scale feature maps, which is similar to SSD. The reason for using up-sample (up-sample) in the network is that the deeper the network, the better the feature expression effect, such as in 16 times down-sampling detection, if the fourth down-sampling feature is directly used for detection, the shallow feature is used, which is generally not good. If you want to use the feature after 32 times down-sampling, but the size of the deep feature is too small, therefore, YOLO_v3 uses up-sample (up-sample) with a step of 2 to increase the size of the feature map obtained by 32 times down-sampling by one, which becomes the dimension after 16 times down-sampling. Similarly, 8 times sampling is also up-sampling with a step of 2 for the feature after 16 times down-sampling, so that deep features can be used for detection.

[0058] Finally, YOLOv3 extracts deep features by upsampling, and the dimension of the extracted features is the same as the dimension of the feature to be fused (but the channel is different). As shown in FIG. 8, the 85th layer upsamples the 13x13x256 feature to obtain a 26x26x256 feature, and then splices the 26x26x256 feature with the feature of the 61st layer to obtain a 26x26x768 feature. In order to obtain the channel 255, a series of 3x3 and 1x1 convolution operations need to be performed, so as to increase the nonlinearity, increase the generalization performance, improve the network precision, and reduce the parameters to improve the real-time performance. The 52x52x255 feature is also obtained by a similar process. Figure 1

[0059] In summary, for an input image, YOLOv3 maps it to three scale output tensors representing the probability of various objects existing at each position of the image. Based on the feature map obtained by Darknet-53, the first feature map is obtained by six DBL structures and the last convolution layer, and the first prediction is performed on the first feature map. On the Y1 branch, the output of the third convolution layer from back to front is connected with the output of the second Res8 structure by a DBL structure and an upsampling, and the second feature map is obtained by six DBL structures and the last convolution layer, and the second prediction is performed on the second feature map. On the Y2 branch, the output of the third convolution layer from back to front is connected with the output of the first Res8 structure by a DBL structure and an upsampling, and the third feature map is obtained by six DBL structures and the last convolution layer, and the third prediction is performed on the third feature map, so as to obtain three scale feature maps as the image feature detection result of the input image.

[0060] The inventors find that the existing YOLOv3 model mainly extracts features through RES and DBL structures in the existing YOLOv3 model. Figure 1 A RES operation mainly consists of a 3x3 convolution with a step of 2 and n ResUnits, and a ResUnit consists of a 1x1 convolution and a 3x3 convolution operation. A large number of 3x3 convolution operations greatly increase the model parameters, and considering that the resolution of a remote sensing image itself is high, the existing YOLOv3 model cannot balance the detection speed and accuracy. Moreover, when different scale detection targets appear in the detection task at the same time, the network is difficult to accurately judge the size change of the larger target.

[0061] Therefore, there is a need for a method for multi-scale target detection for high-resolution images such as remote sensing images, which can balance the detection speed and accuracy.

[0062] Figure 2 ​A flowchart of an image detection method provided by an embodiment of the present application is shown, which is executed by a computer processing device. The computer processing device can include a mobile phone, a notebook computer, etc. As shown in Figure 2 the method includes the following steps:

[0063] Step 10: inputting a to-be-detected image into a target detection model to obtain a target detection result corresponding to the to-be-detected image; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes a high-dimensional convolution module; the high-dimensional convolution module is used for performing deep convolution on the to-be-detected image in a high-dimensional space to obtain image features of the to-be-detected image.

[0064] In an embodiment of the present application, considering that the purpose of convolution is to extract features, the depth of the output feature map = the number of convolution kernels = the number of channels. However, four 3x3 convolution kernels accumulate convolution to reach a 9x9 convolution kernel corresponding to a perception field, but the required parameter amount is less, and more features are generated. Facing a fixed-size picture, if a 9x9 convolution kernel is used, the features are extracted in the 9x9 range, and the number of extracted features is determined data if the picture is properly fitted with a 9-step. Therefore, the smaller the convolution, the more and more fine the extracted features. As described above, the existing YOLOv3 model uses a large number of 3x3 convolutions to extract features of the picture, which causes the parameter amount of the model to be too large, and the model detection speed decreases when the model is used for target detection of high-resolution images such as remote sensing images. Therefore, in the embodiment of the present application, the 3x3 convolution in the YOLOv3 model is replaced by a high-dimensional convolution module, deep convolution is performed on the high-dimensional convolution module in a high-dimensional space, so that the parameter amount of the model is reduced while the information-rich feature representation is extracted, and the risk of information loss and gradient confusion caused by the traditional target detection network residual structure is reduced.

[0065] Specifically, the high-dimensional convolution module can be an hourglass type, that is, a deep convolution structure is arranged at the starting position and the ending position of the high-dimensional convolution module, so as to realize convolution on the input and output high-dimensional data, realize extraction of picture features in a high-dimensional space, and correspondingly, one or more data dimension adjustment structures can be arranged between the starting position and the ending position, so as to adjust the dimension of the data of the deep convolution structure at the ending position to the high-dimensional space. The deep convolution structure can be a convolution kernel of a preset size, such as a 3x3 convolution kernel.

[0066] Therefore, in another embodiment of the present application, the structure of the high-dimensional convolution module can refer to Figure 3 .

[0067] As Figure 3As shown, the high-dimensional convolution module includes a first deep convolution layer, a dimension adjustment convolution layer and a second deep convolution layer connected in sequence; the first deep convolution layer and the second deep convolution layer are used for deep convolution of input data; and the dimension adjustment convolution layer is used for dimension adjustment of the input data, so that the data input into the second deep convolution layer is high-dimensional data.

[0068] The first deep convolution layer can include a convolution kernel of a preset size, such as a 3x3 convolution kernel, that is, Dwise 3x3 (depthwise separable 3x3 convolution kernel) in the following formula. Figure 3 The structure of the second deep convolution layer can be the same as that of the first deep convolution layer. The dimension adjustment convolution layer can include two 1x1 convolution layers connected in sequence, that is, Conv 1x1, 1x1 convolution layer in the following formula. Figure 3 The 1x1 convolution layer connected with the first deep convolution layer is used for reduction operation of data, and the 1x1 convolution layer connected with the second deep convolution layer is used for expansion operation of data. Optionally, the input of the high-dimensional convolution module can be connected with an activation function in the existing YOLOv3 model.

[0069] Further, considering that the YOLOv3 model is used for extracting multi-scale target detection results of an input image, and the existing RES module is used for multi-scale feature extraction in the existing YOLOv3 model, and the DBL module is included in the RES module, and a plurality of 3x3 convolution kernels are used for feature extraction of a picture in the DBL module, the DBL module has the problems of too many parameters and poor model processing speed, therefore, in another embodiment of the present application, the feature extraction in the RES module in the existing YOLOv3 model can be realized by the high-dimensional convolution module in the foregoing embodiment, so as to perform identity mapping and spatial transformation in a higher dimension, reduce the risk of information loss and gradient confusion caused by the traditional target detection network residual structure, and balance the processing speed and processing accuracy of the model.

[0070] Therefore, in another embodiment of the present application, the improved YOLOv3 model includes a multi-scale convolution module; the multi-scale convolution module is used for convolution of input data under a multi-scale convolution kernel, to obtain multi-scale image features; and the convolution kernel includes the high-dimensional convolution module.

[0071] Specifically, the multi-scale convolution module can include a plurality of convolution kernels of different scales and depths connected in sequence, wherein the multi-scale convolution module can be in a pyramid structure, the size of the convolution kernel decreases sequentially from top to bottom of the pyramid structure, and the number of channels of the convolution kernel increases sequentially from top to bottom of the pyramid structure in the channel dimension. Each convolution kernel in the multi-scale convolution module finally splices the obtained feature map. When splicing the feature maps output by the convolution kernels of different sizes, in order to achieve the same number of output channels, the method of grouped convolution can be used. Further, in order to improve the processing and detection speed of the network on the basis of obtaining multi-scale feature maps, the high-dimensional convolution module in the foregoing method embodiment can be used to extract features in each layer of convolution and grouped convolution in the multi-scale convolution module.

[0072] Therefore, in another embodiment of the present application, referring to Figure 4 , the multi-scale convolution module includes a plurality of convolution kernels connected in sequence; the size of the convolution kernel decreases sequentially, and the depth of the convolution kernel increases sequentially; the plurality of convolution kernels use grouped convolution to make each convolution kernel output the same number of channel image features.

[0073] As Figure 4 shown, the feature map (Input Feature Maps in Figure 4 ) input into the multi-scale convolution module will pass through convolution kernels of different sizes (Pyramidal Convolution Kernels in Figure 4 , including Level1-n PyConv), and then each feature map will be connected by channel to obtain the output (Onput Feature Maps in Figure 4 ). The size of the convolution kernel is constantly increasing, while the depth of the convolution kernel is constantly decreasing. In order to be able to use convolution kernels of different depths, the method of grouped convolution is used, thereby obtaining feature maps of the same number of channels. In order to further improve the performance of feature extraction, the high-dimensional convolution module (Sandglass Block) can be used to extract features when performing convolution processing in the pyramid structure.

[0074] Further, considering that high-resolution images such as remote sensing images have high image accuracy and can extract relatively more features, in order to filter the large number of extracted features and enhance the target features that can represent the input image, thereby improving the accuracy of feature extraction in the existing YOLOv3 model, in another embodiment of the present application, the improved YOLOv3 model further includes a feature enhancement module: the feature enhancement module is used to perform feature enhancement on the image features according to the channel attention mechanism.

[0075] Specifically, the feature enhancement module is connected after the aforementioned high-dimensional convolutional module. This module enhances the image features extracted by the high-dimensional convolutional module, suppressing unimportant redundant features and thus enhancing the target features in image detection. Specifically, the feature enhancement module can employ a channel attention mechanism to enhance the image features extracted by the high-dimensional convolutional module. Specifically, the feature enhancement module first uses average pooling to obtain feature vectors along the channel directions, then uses a fully connected layer to capture the dependencies between different channels. Finally, the weights corresponding to the dependencies between different channels are mapped, and the weights are multiplied by the original features to obtain the final enhanced features.

[0076] In one embodiment of the present invention, a schematic diagram of the feature enhancement module is shown below. Figure 5 As shown, for reference Figure 5 The feature enhancement module includes a pooling layer (Global Pooling), multiple fully connected (FC) layers, and a weighted calculation layer connected in sequence. The pooling layer is used to extract image features from multiple channels. The fully connected layers are used to determine the inter-channel dependencies based on the image features of each channel. The weighted calculation layer is used to perform weighted calculations on the image features of the multiple channels based on the inter-channel dependencies to obtain enhanced image features.

[0077] The system can have two fully connected layers, which determine the inter-channel dependencies based on the image features of each channel. The weighted calculation layer can map the weights corresponding to the inter-channel dependencies to [0, 1] using the Sigmoid function, and then multiply these weights by the original input image features to obtain the enhanced features.

[0078] In another embodiment of the present invention, for target detection scenarios of high-resolution images such as remote sensing images, when applying the YOLOv3 model, in order to further improve the processing performance of the model and the accuracy of multi-scale feature extraction, the high-dimensional convolution module, feature enhancement module and multi-scale feature extraction module in the foregoing embodiments can be integrated into the existing YOLOv3 model to obtain an improved YOLOv3 model.

[0079] Specifically, the image to be detected includes a high-resolution image; the improved YOLOv3 model is derived from the original YOLOv3 model; the original YOLOv3 model includes an original DBL module and an original RES module, the original DBL module includes a sequentially connected original convolutional layer, a normalization layer, and an activation function layer, and the original RES module includes a sequentially connected zero-padding layer, the original DBL module, and an original residual structure; wherein, in the improved YOLOv3 model, the original convolutional layer is replaced with a high-dimensional convolutional module, a feature enhancement module is added after the activation function layer, and a multi-scale convolutional module is added after the original residual structure; the high-dimensional convolutional module is used to perform depth convolution on the input data in a high-dimensional space; the feature enhancement module is used to enhance the image features according to the channel attention mechanism; the multi-scale convolutional module is used to convolve the input data under multi-scale convolutional kernels; the convolutional kernels include the high-dimensional convolutional module.

[0080] like Figure 1 As shown, in the original YOLOv3 model, the original convolutional layer includes a conv layer, the normalization layer includes a BN layer, the activation function layer includes a Leaky ReLU layer, and the zero padding layer includes a zero padding layer; the original residual structure includes ResUnit*n.

[0081] against Figure 1 In the original YOLOv3 model shown, in one embodiment of the present invention, as Figure 6 As shown, the first improvement is to replace the original DBL module with a high-dimensional convolutional module, namely the sandglass block in the figure. At the same time, a feature enhancement module, namely the SE (Squeeze and excitation) module in the figure, is added after the activation function layer to obtain the improved DBL module (denoted as NDBL).

[0082] Furthermore, improvements to the original RES module, such as Figure 7 As shown, firstly, the original DBL module in the original RES module is replaced with the improved DBL module, and then a multi-scale convolution module, namely hourglass-pyconv in the figure, is added after the original residual structure to obtain the improved RES module (denoted as NRES).

[0083] After the above improvements to the original DBL module and the original RES module of the existing YOLOv3, an improved YOLOv3 model is obtained. Its overall network structure is the same as YOLOv3, and NDBL and NRES are used to extract features.

[0084] The improved YOLOv3 model is tested on the NWPU VHR-10 dataset in the embodiment of the application. The comparison results of the detection results of the improved YOLOv3 model and the existing YOLOv3 model of the embodiment of the application can be referred to Figure 8 .

[0085] In an embodiment of the application, the training process of the improved YOLOv3 model can be as follows:

[0086] Step 1: Data augmentation is performed on the remote sensing images in the dataset.

[0087] The data augmentation process includes: 1024x1024 size series of blocks are cut from the remote sensing images in the dataset according to a stride of 512.

[0088] In addition, the image size is adjusted according to a ratio of 1:0.5 for the target classes with a small number of classes, and the random flipping and rotation method is used to process the data set class imbalance problem.

[0089] Step 2: The augmented data is labeled to obtain training samples.

[0090] Specifically, the K-means clustering algorithm is used to determine the prior box, 9 clusters are set and uniformly distributed on 3 scales, and the prior box is obtained from the NWPU VHR-10.

[0091] Step 3: The improved YOLOv3 model is trained according to the training samples.

[0092] The processed picture is input into the improved YOLOv3 model, and two branches are independently output after the third and fourth NRES in the improved YOLOv3 model, and another two features are extracted, and finally three target feature maps are obtained.

[0093] The image detection device provided by the embodiment of the present application inputs a to-be-detected image into a target detection model to obtain a target detection result corresponding to the to-be-detected image; the target detection model comprises an improved YOLOv3 model; the improved YOLOv3 model comprises a high-dimensional convolution module; the high-dimensional convolution module is used for performing deep convolution on the to-be-detected image in a high-dimensional space to obtain image features of the to-be-detected image, thereby distinguishing from the existing YOLOv3 model which adopts a large number of 3x3 convolution operations for feature extraction, resulting in a problem of excessive model parameter quantity and poor detection speed; the embodiment of the present application places the deep convolution at the beginning and end positions of the entire convolution structure, so that the deep convolution operation acts on the high-dimensional space, thereby extracting information-rich feature representations, performing identification mapping and spatial transformation in a higher dimension, reducing the risk of information loss and gradient confusion caused by the traditional mobile network residual structure, thereby effectively improving the processing speed of the model and achieving a balance between detection accuracy and detection speed.

[0094] Figure 9 The structure of the image detection device provided by the embodiment of the present application is shown. As shown in the figure, Figure 9 The device 20 comprises a detection module 201. The detection module 201 is used for inputting a to-be-detected image into a target detection model to obtain a target detection result corresponding to the to-be-detected image; the target detection model comprises an improved YOLOv3 model; the improved YOLOv3 model comprises a high-dimensional convolution module; the high-dimensional convolution module is used for performing deep convolution on the to-be-detected image in a high-dimensional space to obtain image features of the to-be-detected image.

[0095] The operation process of the image detection device provided by the embodiment of the present application is substantially the same as the foregoing method embodiment, and will not be described again.

[0096] The image detection device provided in this embodiment of the invention obtains the target detection result corresponding to the image to be detected by inputting the image to be detected into a target detection model. The target detection model includes an improved YOLOv3 model. The improved YOLOv3 model includes a high-dimensional convolution module. The high-dimensional convolution module is used to perform depth convolution on the image to be detected in a high-dimensional space to obtain the image features of the image to be detected. This is different from the existing YOLOv3 model, which uses a large number of 3x3 convolution operations for feature extraction, resulting in an excessive number of model parameters and poor detection speed. This embodiment of the invention places depth convolution at the beginning and end of the entire convolution structure, so that the depth convolution operation is applied in a high-dimensional space. This allows for the extraction of information-rich feature representations and the execution of identifier mapping and spatial transformation in a higher dimension. This reduces the risk of information loss and gradient confusion caused by the inverse residual structure of traditional mobile networks, thereby effectively improving the processing speed of the model and achieving a balance between detection accuracy and detection speed.

[0097] Figure 10 The diagram shows a structural schematic of an image detection device provided in an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the image detection device.

[0098] like Figure 10 As shown, the image detection device may include: a processor 302, a communication interface 304, a memory 306, and a communication bus 308.

[0099] The processor 302, communication interface 304, and memory 306 communicate with each other via communication bus 308. Communication interface 304 is used to communicate with other network elements such as clients or other servers. The processor 302 executes program 310, specifically performing the relevant steps described above in the image detection method embodiment.

[0100] Specifically, program 310 may include program code, which includes computer-executable instructions.

[0101] Processor 302 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The image detection device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0102] A memory 306 for storing a program 310. The memory 306 can include a high-speed RAM memory and can also include a non-volatile memory, such as at least one disk memory.

[0103] The program 310 can be specifically invoked by the processor 302 to enable the image detection device to perform the following operations:

[0104] inputting a to-be-detected image into a target detection model to obtain a target detection result corresponding to the to-be-detected image; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes a high-dimensional convolution module; the high-dimensional convolution module is used for performing deep convolution on the to-be-detected image in a high-dimensional space to obtain image features of the to-be-detected image.

[0105] The operation process of the image detection device provided in the embodiment of the present application is substantially the same as that of the foregoing method embodiment, and will not be described again.

[0106] The image detection device provided in the embodiment of the present application inputs a to-be-detected image into a target detection model to obtain a target detection result corresponding to the to-be-detected image; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes a high-dimensional convolution module; wherein the high-dimensional convolution module is used for performing deep convolution on the to-be-detected image in a high-dimensional space to obtain image features of the to-be-detected image, thereby distinguishing from the existing YOLOv3 model which adopts a large number of 3x3 convolution operations for feature extraction, resulting in too large a parameter quantity of the model and poor detection speed, the embodiment of the present application places the deep convolution at the beginning and end positions of the entire convolution structure, so that the deep convolution operation acts on the high-dimensional space, thereby extracting information-rich feature representations, performing identification mapping and spatial transformation in a higher dimension, reducing the risk of information loss and gradient confusion caused by the traditional mobile network residual structure, thereby effectively improving the processing speed of the model and achieving a balance between detection accuracy and detection speed.

[0107] The embodiment of the present application provides a computer-readable storage medium, the storage medium stores at least one executable instruction, the executable instruction runs on the image detection device, and the image detection device executes the image detection method in any method embodiment.

[0108] The executable instruction can be specifically used to enable the image detection device to perform the following operations:

[0109] input the to-be-detected image into a target detection model to obtain a target detection result corresponding to the to-be-detected image; the target detection model comprises an improved YOLOv3 model; the improved YOLOv3 model comprises a high-dimensional convolution module; the high-dimensional convolution module is configured to perform deep convolution on the to-be-detected image in a high-dimensional space to obtain image features of the to-be-detected image.

[0110] The computer readable storage medium provided in the embodiment of the present application stores executable instructions, and the operation process of the executable instructions is substantially the same as that of the foregoing method embodiment, and thus will not be described in detail.

[0111] The computer readable storage medium provided in the embodiment of the present application stores executable instructions, and the operation process of the executable instructions is substantially the same as that of the foregoing method embodiment, and thus will not be described in detail.

[0112] The computer readable storage medium provided in the embodiment of the present application stores executable instructions, and the operation process of the executable instructions is substantially the same as that of the foregoing method embodiment, and thus will not be described in detail.

[0113] The computer program provided in the embodiment of the present application can be called by a processor to enable an image detection device to execute the image detection method in any method embodiment.

[0114] The computer program product provided in the embodiment of the present application comprises a computer program stored on a computer readable storage medium, and the computer program comprises program instructions, which, when executed on a computer, enable the computer to execute the image detection method in any method embodiment.

[0115] The algorithms and displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems can be used with programs in accordance with the teachings herein, or it can prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will be apparent from the description above. In addition, the present embodiments are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the teachings of the application as described herein, and any references below to specific languages are provided for disclosure of enablement of the best mode of the application.

[0116] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been described in detail in order to not obscure the understanding of this description.

[0117] Similarly, it is to be understood that the mechanical details of the application can be adapted to a wide variety of situations and, as such, details described herein should not be construed as limiting but merely as illustrating a particular implementation of the application. Accordingly, what is desired to be secured by Letters Patent is set forth in the following claims.

[0118] Those skilled in the art will appreciate that modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and can be further split into more modules or units or components. Any combination of all the features disclosed in the specification (including the accompanying claims, abstract and drawings), and any method or process or steps of any method or process so disclosed, can be employed in any combination, unless the context clearly indicates otherwise. Unless explicitly stated otherwise, each feature disclosed in the specification (including the accompanying claims, abstract and drawings) is implicated by a replacement feature serving the same, equivalent or similar purpose.

[0119] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unitary claim, several devices, apparatuses or means can be listed, comprising means for carrying out a certain task. The use of the term'means' in a claim is intended to refer to a combination of devices, apparatuses or means for carrying out a task. The word 'first','second', 'third', etc. do not imply any order. The use of these terms is to be construed as an indication of particular embodiments. Steps in the above-described embodiments, unless otherwise specified, are not to be construed as necessarily limiting the order in which the steps are performed.

Claims

1. An image detection method characterized by, The method comprises: inputting a to-be-detected image into a target detection model to obtain a target detection result corresponding to the to-be-detected image; the target detection model comprises an improved YOLOv3 model; the improved YOLOv3 model comprises a high-dimensional convolution module; the high-dimensional convolution module is used for performing deep convolution on the to-be-detected image in a high-dimensional space to obtain image features of the to-be-detected image; the high-dimensional convolution module comprises a first deep convolution layer, a dimension adjustment convolution layer and a second deep convolution layer connected in sequence; the first deep convolution layer and the second deep convolution layer are used for performing deep convolution on input data; the dimension adjustment convolution layer is used for adjusting the dimension of the input data so that the data input into the second deep convolution layer is high-dimensional data; the improved YOLOv3 model comprises a multi-scale convolution module; the multi-scale convolution module is used for performing convolution on input data under multi-scale convolution kernels to obtain multi-scale image features; wherein the convolution kernel comprises the high-dimensional convolution module; the multi-scale convolution module comprises a plurality of convolution kernels connected in sequence; the size of the convolution kernel decreases in sequence, and the depth of the convolution kernel increases in sequence; a plurality of the convolution kernels adopt grouped convolution so that each convolution kernel outputs image features of the same number of channels.

2. The method of claim 1, wherein, The improved YOLOv3 model further comprises a feature enhancement module: the feature enhancement module is used for performing feature enhancement on the image features according to a channel attention mechanism.

3. The method of claim 2, wherein, The feature enhancement module comprises a pooling layer, a plurality of fully connected layers and a weighting calculation layer connected in sequence; the pooling layer is used for extracting image features of a plurality of channels; the fully connected layer is used for determining inter-channel dependency according to image features of each channel; and the weighting calculation layer is used for performing weighting calculation on image features of the plurality of channels according to the inter-channel dependency to obtain enhanced image features.

4. The method of claim 1, wherein, The to-be-detected image comprises a high-resolution image; the improved YOLOv3 model is improved based on an original YOLOv3 model; the original YOLOv3 model comprises an original DBL module and an original RES module, the original DBL module comprises an original convolution layer, a normalization layer and an activation function layer connected in sequence, and the original RES module comprises a zero padding layer, the original DBL module and an original residual structure connected in sequence; wherein in the improved YOLOv3 model, the original convolution layer is replaced by a high-dimensional convolution module, a feature enhancement module is added after the activation function layer, and a multi-scale convolution module is added after the original residual structure; the high-dimensional convolution module is used for performing deep convolution on input data in a high-dimensional space; the feature enhancement module is used for performing feature enhancement on the image features according to a channel attention mechanism; and the multi-scale convolution module is used for performing convolution on input data under multi-scale convolution kernels; the convolution kernel comprises the high-dimensional convolution module.

5. An image detection apparatus characterized by comprising: The device comprises: The detection module is configured to input a to-be-detected image into a target detection model to obtain a target detection result corresponding to the to-be-detected image. The target detection model comprises an improved YOLOv3 model. The improved YOLOv3 model comprises a high-dimensional convolution module. The high-dimensional convolution module is configured to perform deep convolution on the to-be-detected image in a high-dimensional space to obtain image features of the to-be-detected image. The high-dimensional convolution module comprises a first deep convolution layer, a dimension adjustment convolution layer and a second deep convolution layer connected in sequence. The first deep convolution layer and the second deep convolution layer are configured to perform deep convolution on input data. The dimension adjustment convolution layer is configured to perform dimension adjustment on the input data, so that the data input into the second deep convolution layer is high-dimensional data. The improved YOLOv3 model comprises a multi-scale convolution module. The multi-scale convolution module is configured to perform convolution on input data under multi-scale convolution kernels to obtain multi-scale image features. The convolution kernels comprise the high-dimensional convolution module. The multi-scale convolution module comprises a plurality of convolution kernels connected in sequence. The sizes of the convolution kernels decrease in sequence, and the depths of the convolution kernels increase in sequence. The plurality of convolution kernels perform grouped convolution, so that each convolution kernel outputs image features of the same number of channels.

6. An image detection device, characterized by, Comprise: a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction makes the processor execute the operations of the image detection method according to any one of claims 1-4.

7. A computer readable storage medium characterized in that, The storage medium stores at least one executable instruction, and the executable instruction makes the image detection device execute the operations of the image detection method according to any one of claims 1-4 when the image detection device runs.

Images