Image Processing Method, Apparatus, Device, and Medium

Abstract

An embodiment of the present invention discloses an image processing method for detecting vehicles on a road, including: a camera device collects images of the road in real time, and the collected images are input into a detection unit; the detection unit is based on a convolutional neural network model to: extract features of the images to obtain a feature map of the images; and based on the feature map, detect vehicles on the road and output vehicle detection results; wherein, the convolutional neural network model involves a convolutional operation, and the convolutional operation includes: performing a convolutional operation using a convolutional kernel of size m rows by n columns, and in each convolutional operation, calculating the values at each position in the m rows by n columns, determining the outlier values among the values at each position, and after filtering out the outlier values, adding the remaining values as the result of this convolutional operation. The embodiment of the present invention can improve the accuracy of image recognition.

Description

Technical Field

[0001] The present invention relates to image processing technology, and in particular to an image processing method, apparatus, device, and storage medium. Background Art

[0002] In artificial intelligence, it has become increasingly common to use a convolutional neural network to process images, extract image features, and achieve functions such as image recognition. Among them, the essence of a convolutional neural network is to move a convolutional kernel on an image and perform multiplication and addition operations to extract image features. However, the inventors of the present application have found that in some scenarios, when using a convolutional neural network to extract image features, the extracted features are quite different from the true features of the image, thus affecting the accuracy of image recognition. Therefore, there is a need for improvement in the prior art. Summary of the Invention

[0003] In view of this, the present application provides an image processing method, apparatus, device, and storage medium, which can improve the accuracy of image recognition.

[0004] An image processing method according to an embodiment of the present invention for detecting vehicles on a road includes: using a camera device to collect images of the road in real time, and inputting the collected images into a detection unit; the detection unit based on a convolutional neural network model: extracting features of the images to obtain a feature map of the images; and based on the feature map, detecting the vehicles on the road and outputting a vehicle detection result; wherein, the convolutional neural network model involves a convolutional operation, and the convolutional operation includes: performing a convolutional operation using a convolutional kernel of size m rows and n columns, and in each convolutional operation, calculating the values at each position in the m rows and n columns, determining the outlier values among the values at each position, and after filtering out the outlier values, adding the remaining values as the result of this convolutional operation.

[0005] Wherein, the method further includes: training the convolutional neural network model using road image data with noise and road image data without noise; and pre-deploying the trained convolutional neural network model into the detection unit.

[0006] Wherein, the convolutional neural network sequentially includes: a network backbone, a network neck, and a detection head, which are respectively used for feature extraction, feature fusion, and extracting vehicle information as the vehicle detection result, and the convolutional operation is involved in the network backbone, the network neck, and the detection head.

[0007] Wherein, the vehicle information includes at least one of: the category of the vehicle, the position of the vehicle, and the size of the vehicle.

[0008] Among them, the outliers include: the maximum value and the minimum value.

[0009] An image processing method according to an embodiment of the present invention includes: obtaining an image to be processed; inputting the image into a convolutional neural network model to identify the image; wherein, the convolutional neural network model involves a convolutional operation, and the convolutional operation includes: performing a convolutional operation using a convolutional kernel of a predetermined size, and when performing the convolutional operation, calculating the values at each position in the convolutional kernel, determining the outliers among the values at each position, and after filtering out the outliers, adding the remaining values as the result of this convolutional operation.

[0010] Among them, the outliers include: the maximum value and the minimum value.

[0011] An image processing device according to an embodiment of the present invention for detecting vehicles on a road includes: an acquisition module for acquiring an image of the road; a detection module for, based on a convolutional neural network model: extracting features of the image to obtain a feature map of the image; and based on the feature map, identifying the vehicles on the road and outputting a vehicle detection result; wherein, the convolutional neural network model involves a convolutional operation, and the convolutional operation includes: performing a convolutional operation using a convolutional kernel of m rows and n columns, and in each convolutional operation, calculating the values at each position in m rows and n columns, determining the outliers among the values at each position, and after filtering out the outliers, adding the remaining values as the result of this convolutional operation.

[0012] An electronic device according to an embodiment of the present invention includes: a processor; and a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the executable instructions to implement the method according to an embodiment of the present invention.

[0013] A computer-readable storage medium according to an embodiment of the present invention stores a computer program thereon, and the computer program includes executable instructions, and when the executable instructions are executed by a processor, the method according to an embodiment of the present invention is implemented.

[0014] Advantageous effects of the embodiments of the present invention:

[0015] In the embodiments of the present invention, when using a convolutional neural network model to identify an image, the convolutional operation involved in the convolutional neural network model is improved, that is, when performing the convolutional operation, an operation of filtering outliers is added, so as to effectively prevent the influence of abnormal points in the image, improve the fitting ability of the convolutional neural network model (that is, the accuracy of the neural network output), and further improve the accuracy of image recognition. Description of the Drawings

[0016] Other details and advantages of the present invention will become apparent from the following detailed description. It should be understood that the following drawings are merely schematic and thus cannot be considered as limiting the present invention. The following will refer to the drawings for a detailed description, wherein:

[0017] Figure 1 is a schematic flowchart of an embodiment of the image processing method of the present invention;

[0018] Figure 2 is a schematic structural diagram of an embodiment of the image processing system of the present invention;

[0019] Figure 3 is a schematic structural diagram of an embodiment of the convolutional neural network model of the present invention;

[0020] Figure 4 is a schematic structural diagram of an embodiment of the image processing apparatus of the present invention;

[0021] Figure 5 is a schematic structural diagram of an embodiment of the electronic device of the present invention. Detailed Embodiments

[0022] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer and more obvious, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not used to limit the present invention.

[0023] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. Moreover, the terms "first", "second", etc. are applicable to distinguish similar objects and do not have to be used to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0024] When using a convolutional neural network model for image recognition, the commonly involved operation is the convolution operation. The so-called convolution operation mainly refers to moving a convolution kernel on an image and performing multiplication and addition operations with image pixels to extract features. Among them, the convolution kernel is a matrix composed of weight values of a certain size. Assuming that the size of the convolution kernel is m rows and n columns, and w ij represents the weight value at the i-th row and j-th column in the convolution kernel, and p ij represents the pixel value at the corresponding position in the image. Then the convolution operation can be expressed as: Among them, y is the eigenvalue. When extracting image features, the convolution kernel is moved on the image, and the convolution operation is performed on each m*n block in the image to obtain the feature map of the image.

[0025] However, as can be seen from the above formula, the size of the eigenvalue is obtained by adding the products of each weight value and pixel value. If the result of one of the products is too large or too small, the y value will be dominated by this result, and the influence of other results will be greatly weakened. For example, assume that the size of the convolution kernel is 2 rows and 2 columns. During a convolution operation, the values calculated at four positions are -1000, 0.5, 0.8, and 1 respectively. Therefore, the final y value will be dominated by the value -1000, and in reality, the pixel point corresponding to -1000 is very likely to be noise, and the eigenvalue determined thereby cannot truly reflect the situation of the image.

[0026] Therefore, in order to eliminate the influence of such outliers (i.e., noise), this application improves the convolution operation and performs an operation to remove outliers before the addition in the convolution operation. The outlier can be the maximum value and / or the minimum value. Taking the removal of the maximum value and the minimum value as an example, the formula for the convolution operation of this application is:

[0027]

[0028] Among them, max_ind is the function to find the maximum value, and min_ind is the function to find the minimum value. ind max is the position of the maximum value, and ind min is the position of the minimum value. Based on the above convolution operation, the embodiments of the present invention will not affect the situation where all pixel values contribute effectively to the eigenvalue, and have a good optimization effect on those situations with outliers. Therefore, compared with the traditional convolution operation, it can improve the fitting ability of the network and the processing ability of noise. Therefore, when the convolution neural network model based on the above convolution operation is used for image processing, such as for the recognition of vehicles on the road, the accuracy of image recognition can be improved.

[0029] Next, in combination with the accompanying drawings and specific application examples, the image processing method of the embodiments of the present invention will be described.

[0030] As Figure 1 and 2 shown, it is a specific example of detecting vehicles on the road based on the convolution neural network model. Of course, the convolution neural network model of the embodiments of the present invention can still be used for image recognition in other aspects in addition to being applied to road detection. As Figure 1 and 2As shown, the imaging device (such as a camera) 20 is used to collect images of the road in real time (step S10). The imaging device 20 can be installed on a vehicle, such as a front camera of the vehicle, or installed on the roadside, such as on a traffic pole at an intersection, to monitor vehicles on the road, such as whether a vehicle is illegally parked. The image of the road collected by the imaging device 20 is transmitted to the detection unit 22.

[0031] Specifically, the detection unit 22 processes the image based on a convolutional neural network model (step S12), for example, extracts the features of the image to obtain a feature map of the image; and based on the obtained feature map, detects the vehicles on the road and outputs a vehicle detection result.

[0032] Among them, the detection unit 22 can be, for example, an ECU (Electronic Control Unit). A trained convolutional neural network model is pre-deployed in the detection unit 22, and the image recognition operation is performed through this convolutional neural network model. Among them, the convolutional neural network model can be trained first using road image data with noise and road image data without noise, and then the trained convolutional neural network model is deployed in the detection unit 22, so that the detection unit 22 can perform image recognition based on the convolutional neural network model.

[0033] Among them, the main operations involved in the convolutional neural network model include convolutional operations. As Figure 1 shown in steps S121 - S124, the convolutional operation of the embodiment of the present invention mainly includes: performing a convolutional operation using a convolutional kernel of size m rows and n columns (step S121). In each convolutional operation, calculate the values at each position in the m rows and n columns (step S122), then determine the outlier values among the values at each position (step S123), and after filtering out the outlier values, add the remaining values as the result of this convolutional operation (step S124). Among them, the specific process of the convolutional operation can refer to the formula (1) shown above and will not be elaborated here.

[0034] Convolutional operations are involved at multiple positions in the convolutional neural network model. In the embodiments of the present application, all convolutional operations in the convolutional neural network model can be replaced with the convolutional operations proposed in the embodiments of the present application, or only a part can be replaced with the convolutional operations of the present application, while the other part can still use traditional convolutional operations. The present application does not make any limitations on these.

[0035] Specifically, as Figure 3As shown in the figure, the convolutional neural network model mainly includes four parts: the network backbone, the network neck, the detection head, and the loss function. Among them, the network backbone is composed of several convolutional blocks, and each convolutional block is mainly composed of a convolutional layer, a pooling layer, a residual structure, etc. The network backbone is mainly used to extract features from the original image to generate multiple original feature maps. The network neck is mainly composed of a convolutional layer and sampling (such as upsampling, downsampling) structures. The original feature maps from the network backbone are fused at the network neck to obtain multiple new feature maps. These multiple new feature maps are input into the detection head (mainly composed of a convolutional layer) to extract the required information from the new feature maps, such as the category, position, and size of the vehicle. Among them, the loss function is mainly used during the training process of the convolutional neural network model. It is used to compare the difference between the results output by the detection head and the real results, so as to obtain the gradient to update the weight parameters in the convolutional neural network model.

[0036] As Figure 4 As shown in the figure, it is a schematic structural diagram of an embodiment of the image processing device 4 of the present invention. The image processing device 4 can be used to detect vehicles on the road, and it includes: an acquisition module 40 for acquiring an image of the road; a detection module 42 for extracting features of the image based on a convolutional neural network model to obtain a feature map of the image; and based on the feature map, identifying vehicles on the road and outputting a vehicle detection result. Among them, the above-mentioned convolution operation mainly includes: performing a convolution operation using a convolution kernel of size m rows and n columns, and in each convolution operation, calculating the values at each position in m rows and n columns, determining the outliers among the values at each position, and after filtering out the outliers, adding the remaining values as the result of this convolution operation.

[0037] As Figure 5 As shown in the figure, an embodiment of the present invention also provides an electronic device, including: a processor 50; and a memory 52 for storing executable instructions of the processor; wherein, the processor is configured to execute the executable instructions to implement the method according to the embodiment of the present invention.

[0038] In addition, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes executable instructions, and when the executable instructions are executed by the processor, the processing method according to the embodiment of the present invention is implemented.

[0039] The descriptions of the above storage medium, device, and device embodiments are similar to the descriptions of the above method embodiments, and have beneficial effects similar to those of the method embodiments. For the technical details not disclosed in the storage medium, device, and device embodiments of the present application, please refer to the descriptions of the method embodiments of the present application for understanding.

[0040] The above-mentioned processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, etc. It can be understood that the electronic device implementing the functions of the above-mentioned processor may also be others, which are not specifically limited in the embodiments of the present application.

[0041] The above-mentioned computer storage medium / memory may be a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Ferromagnetic Random Access Memory (FRAM), a Flash Memory, a magnetic surface memory, an optical disc, or a Compact Disc Read-Only Memory (CD-ROM), etc.; it may also be various terminals including one or any combination of the above-mentioned memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0042] It should be noted that the above description is only an example and not a limitation of the present invention. In other embodiments of the present invention, the method may have more, fewer, or different steps, and the order, inclusion, and functional relationships between the steps may be different from those described and illustrated. For example, usually multiple steps may be combined into a single step, and a single step may also be split into multiple steps. For those of ordinary skill in the art, without creative efforts, the changes in the sequence of each step are also within the protection scope of the present invention.

[0043] Essentially, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.), a processor, or a microcontroller to execute all or part of the steps of the methods described in various embodiments of the present invention.

[0044] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments.

[0045] Although the present invention has been disclosed above with preferred embodiments, the present invention is not limited thereto. Any person skilled in the art, without departing from the spirit and scope of the present invention, makes various changes and modifications, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the scope defined by the claims.

Claims

1. An image processing method for detecting vehicles on a road, characterized in that, Including: Using a camera device to collect images of a road in real time, and inputting the collected images into a detection unit; The detection unit is based on a convolutional neural network model: Extracting features of the image to obtain a feature map of the image; And Based on the feature map, detecting vehicles on the road and outputting a vehicle detection result; Wherein, the convolutional neural network model involves a convolution operation, and the convolution operation includes: performing a convolution operation using a convolution kernel of size m rows by n columns, and in each convolution operation, calculating the values at each position in the m rows by n columns, determining the outlier values among the values at each position, and after filtering out the outlier values, adding the remaining values as the result of this convolution operation.

2. The image processing method according to claim 1, wherein The method further includes: Using road image data with noise and road image data without noise to train the convolutional neural network model; and Pre-deploying the trained convolutional neural network model into the detection unit.

3. The image processing method according to claim 2, wherein The convolutional neural network sequentially includes: a network backbone, a network neck, and a detection head, which are respectively used for feature extraction, feature fusion, and extracting vehicle information as the vehicle detection result, and the convolution operation is involved in both the network backbone, the network neck, and the detection head.

4. The image processing method according to claim 3, wherein The vehicle information includes at least one of: the category of the vehicle, the position of the vehicle, and the size of the vehicle.

5. The image processing method according to claim 1, characterized in that The outlier values include: the maximum value and the minimum value.

6. An image processing method, characterized in that, Including: Obtaining an image to be processed; Inputting the image into a convolutional neural network model to identify the image; Wherein, the convolutional neural network model involves a convolution operation, and the convolution operation includes: performing a convolution operation using a convolution kernel of a predetermined size, and when performing the convolution operation, calculating the values at each position in the convolution kernel, determining the outlier values among the values at each position, and after filtering out the outlier values, adding the remaining values as the result of this convolution operation.

7. The image processing method according to claim 6, wherein The outlier values include: the maximum value and the minimum value.

8. An image processing device for detecting vehicles on a road, characterized in that, Including: An acquisition module for acquiring an image of a road; A detection module for, based on a convolutional neural network model: extracting features of the image to obtain a feature map of the image; And based on the feature map, identifying vehicles on the road and outputting a vehicle detection result; Wherein, a convolution operation is involved in the convolutional neural network model, and the convolution operation includes: performing a convolution operation using a convolution kernel of size m rows by n columns, and in each convolution operation, calculating the values at each position in the m rows by n columns, determining the outlier values among the values at each position, and after filtering out the outlier values, adding the remaining values as the result of this convolution operation.

9. An electronic device, characterized in that, Including: A processor; And A memory for storing executable instructions of the processor; Wherein, the processor is configured to execute the executable instructions to implement the processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, the computer program includes executable instructions, and when the executable instructions are executed by a processor, the processing method according to any one of claims 1 to 7 is implemented.

Images