A method for identifying snow-throwing areas of snow truck cargo boxes based on machine vision

Abstract

The present invention discloses a method for identifying a target snow throwing area of ​​a snow truck cargo box based on machine vision, which comprises the following steps: 1) constructing an image data set including images of various loading states of a snow truck cargo box; 2) constructing a snow truck cargo box loading state recognition model; 3) performing binarization processing on the image data; 4) training and testing the recognition model; 5) continuously and in real time collecting loading images of the snow truck cargo box; 6) inputting the image into the recognition model to identify the full-load area and the under-load area of ​​the cargo box; 7) dividing the next snow throwing target area adjacent to the full-load area in the under-load area according to the recognition result. The present invention automatically identifies the loading status of the cargo box through the snow truck cargo box loading state recognition model, distinguishes the full-load area and the under-load area of ​​the cargo box, and divides the next snow throwing target area in the under-load area, thereby replacing the human eye of the snow thrower driver to observe the status of the snow truck cargo box device, and is ready for controlling the steering of the snow throwing tube of the snow thrower.

Description

Technical Field

[0001] The present invention relates to the technical field of snow throwers, and in particular to a method for controlling the distance between a snow carrier and a snow thrower. Background Art

[0002] A snow thrower is a vehicle that gathers the accumulated snow by rotating its auger and throws the snow out through its snow throwing impeller, thereby achieving the purpose of snow removal. The snow carrier works in conjunction with the snow thrower. The snow carrier drives beside the snow thrower and the snow thrower throws the snow into the cargo box of the snow carrier.

[0003] However, the snow throwing direction of the snow thrower in the prior art is entirely controlled by the driver's experience. The driver controls the direction of the snow throwing pipe while driving the vehicle, which is difficult to operate. It is easy for the snow to accumulate too much in one part of the snow truck's cargo box and overflow the cargo box, while other parts of the cargo box are not full of snow. Moreover, distraction is not conducive to safe driving. In addition, when performing snow removal operations at night, especially at night, the driving difficulty of the snow thrower is further increased due to the low ambient brightness.

[0004] Allowing the snow thrower to automatically control the direction of the snow thrower is an effective measure to reduce the difficulty of driving the snow thrower and improve driving safety. To achieve automatic control of the snow thrower direction, it is necessary to be able to identify the loading conditions of different parts of the snow truck cargo box, so that the next step is to control the snow thrower to throw snow to the unloaded area. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a method for identifying target snow-throwing areas in a snow-carrying vehicle cargo box based on machine vision, so as to solve the technical problem of automatically identifying the loading conditions of different areas in the snow-carrying vehicle cargo box.

[0006] The present invention provides a method for identifying a target snow throwing area of ​​a snow truck cargo box based on machine vision, which comprises the following steps:

[0007] 1) Collect image data of various loading states of snow truck cargo boxes during the day and at night, and make the collected image data into image data sets according to a certain format, and divide the image data sets into training data sets and test data sets;

[0008] 2) Construct a snow truck cargo box loading status recognition model based on 3D convolutional neural network;

[0009] 3) The images in the training data set and the test data set are first subjected to discrete Laplace transform and then binarized; the operator of discrete Laplace transform is shown in the following formula:

[0010] l(x,y)=[f(x+1,y)+f(x-1,y)+f(x,y+1)+f(x,y-1)]-4(x,y)

[0011] Where x, y are the x, y coordinates of a pixel in the image respectively; l(x, y) is the Laplace value at the coordinate point x, y;

[0012] In the binarization process of each pixel, the image Laplacian values ​​in the horizontal and vertical directions are defined as l x and l y When the Laplace amplitude is binarized, the (min(l x |+|ly|,255) / 255) is rounded to get a binary amplitude;

[0013] 4) using the training data set processed in step 3) to train the parameter weights of the snow truck cargo box loading state recognition model, and then using the test data set to test the trained snow truck cargo box loading state recognition model, and finally obtaining a qualified snow truck cargo box loading state recognition model;

[0014] 5) An image acquisition device is arranged on the snow thrower to continuously and in real time acquire images of the snow truck cargo box loading, and the acquired images are binarized using the method described in step 3);

[0015] 6) inputting the image obtained in step 5) into the snow truck cargo box loading state recognition model obtained in step 4), identifying the loading conditions of various areas of the cargo box in the image through the snow truck cargo box loading state recognition model, and dividing the cargo box into fully loaded areas and underloaded areas;

[0016] 7) According to the recognition result of step 6), the next snow throwing target area adjacent to the fully loaded area is divided in the not fully loaded area.

[0017] Furthermore, the snow truck cargo box loading status recognition model constructed in the step 2) is composed of a single-layer sparse autoencoder layer and a multi-layer convolutional neural network, and the multi-layer convolutional neural network includes a convolutional layer connected to the single-layer sparse autoencoder layer, a sampling layer connected to the convolutional layer, a feature convolutional layer connected to the sampling layer, a feature downsampling layer connected to the feature convolutional layer, and a fully connected layer connected to the feature downsampling layer.

[0018] Beneficial effects of the present invention:

[0019] The present invention discloses a method for identifying a target snow throwing area for a snow truck cargo box based on machine vision. The method automatically identifies the loading status of the cargo box through a snow truck cargo box loading status recognition model, distinguishes between a full load area and an underloaded area of ​​the cargo box, and divides the next snow throwing target area in the underloaded area. This can replace the human eye observation of the snow truck cargo box device, and prepare for controlling the steering of the snow throwing tube of the snow truck. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1Schematic diagram of the feature extraction process of the snow truck cargo box loading status recognition model. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0022] The method for identifying a target snow throwing area of ​​a snow truck cargo box based on machine vision in this embodiment includes the following steps:

[0023] 1) Collect image data of various loading states of the snow truck cargo box during the day and at night, and make the collected image data into an image data set according to a certain format, and divide the image data set into a training data set and a test data set. In this embodiment, night vision technology is specifically used to collect image data of various loading states of the snow truck cargo box, such as empty cargo box, partially underloaded cargo box, and fully loaded cargo box during the day and at night.

[0024] 2) Construct a snow truck cargo box loading state recognition model based on a 3D convolutional neural network. The constructed snow truck cargo box loading state recognition model consists of a single-layer sparse autoencoding layer and a multi-layer convolutional neural network. The multi-layer convolutional neural network includes a convolution layer connected to the single-layer sparse autoencoding layer, a sampling layer connected to the convolution layer, a feature convolution layer connected to the sampling layer, a feature downsampling layer connected to the feature convolution layer, and a fully connected layer connected to the feature downsampling layer. In this embodiment, the snow truck cargo box loading state recognition model constructed based on the 3D convolutional neural network can effectively identify time series feature information. By processing the target time series features, while improving the recognition effect and real-time performance, it can also divide the snow truck cargo box into full and empty state areas.

[0025] 3) The images in the training data set and the test data set are first subjected to discrete Laplace transform, which can enhance the underlying features of the image, and then binarized. The operator of discrete Laplace transform is shown in the following formula:

[0026] l(x,y)=[f(x+1,y)+f(x-1,y)+f(x,y+1)+f(x,y-1)]-4(x,y)

[0027] Where x, y are the x, y coordinates of a pixel in the image respectively; l(x, y) is the Laplace value at the coordinate point x, y;

[0028] In the binarization process of each pixel, the image Laplacian values ​​in the horizontal and vertical directions are defined as l x and l y When the Laplace amplitude is binarized, the (min(|l x |+|ly|,255) / 255) is rounded to get a binary amplitude.

[0029] 4) Use the training data set processed in step 3) to train the parameter weights of the snow truck cargo box loading status recognition model, and then use the test data set to test the trained snow truck cargo box loading status recognition model, and finally obtain a qualified snow truck cargo box loading status recognition model.

[0030] 5) An image acquisition device is arranged on the snow thrower to continuously and in real time acquire images of the snow truck cargo box loading, and the acquired images are binarized using the method described in step 3);

[0031] 6) inputting the image obtained in step 5) into the snow truck cargo box loading state recognition model obtained in step 4), identifying the loading conditions of various areas of the cargo box in the image through the snow truck cargo box loading state recognition model, and dividing the cargo box into fully loaded areas and underloaded areas;

[0032] 7) According to the recognition result of step 6), the next snow throwing target area adjacent to the fully loaded area is divided in the not fully loaded area.

[0033] In the present embodiment, the method for identifying the target snow-throwing area of ​​the snow-carrying truck cargo box based on machine vision automatically identifies the loading status of the cargo box through the snow-carrying truck cargo box loading status recognition model, distinguishes the full load area and the underloaded area of ​​the cargo box, and divides the next snow-throwing target area in the underloaded area, thereby replacing the human eye of the snow-throwing truck driver to observe the status of the snow-carrying truck cargo box device, and preparing for controlling the steering of the snow-throwing truck's snow-throwing tube.

[0034] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A method for identifying a target snow throwing area of ​​a snow truck cargo box based on machine vision, characterized in that: The following steps are involved: 1) Collect image data of various loading states of snow truck cargo boxes during the day and at night, and make the collected image data into image data sets according to the format, and divide the image data sets into training data sets and test data sets; 2) Construct a snow truck cargo box loading status recognition model based on 3D convolutional neural network; 3) The images in the training data set and the test data set are first subjected to discrete Laplace transform and then binarized; the operator of discrete Laplace transform is shown in the following formula: l(x,y)=[f(x+1,y)+f(x-1,y)+f(x,y+1)+f(x,y-1)]-4f(x,y) Where x, y are the x, y coordinates of a pixel in the image; l(x, y) is the Laplace value at the coordinate point x, y; In the binarization process of each pixel, the image Laplacian values ​​in the horizontal and vertical directions are defined as l x and l y When the Laplace amplitude is binarized, the (min(l x| + | l y| ,255) / 255) is rounded to get a binary amplitude; 4) using the training data set processed in step 3) to train the parameter weights of the snow truck cargo box loading state recognition model, and then using the test data set to test the trained snow truck cargo box loading state recognition model, and finally obtaining a qualified snow truck cargo box loading state recognition model; 5) An image acquisition device is arranged on the snow thrower to continuously and in real time acquire images of the snow truck cargo box loading, and the acquired images are binarized using the method described in step 3); 6) inputting the image obtained in step 5) into the snow truck cargo box loading state recognition model obtained in step 4), identifying the loading conditions of various areas of the cargo box in the image through the snow truck cargo box loading state recognition model, and dividing the cargo box into fully loaded areas and underloaded areas; 7) Based on the recognition result of step 6), the next snow throwing target area adjacent to the fully loaded area is divided in the not fully loaded area.

2. The method for identifying a target snow throwing area of ​​a snow truck cargo box based on machine vision according to claim 1 is characterized in that: The snow truck cargo box loading status recognition model constructed in the step 2) is composed of a single-layer sparse autoencoding layer and a multi-layer convolutional neural network, wherein the multi-layer convolutional neural network includes a convolutional layer connected to the single-layer sparse autoencoding layer, a sampling layer connected to the convolutional layer, a feature convolutional layer connected to the sampling layer, a feature downsampling layer connected to the feature convolutional layer, and a fully connected layer connected to the feature downsampling layer.

Images