Open-pit mine front vehicle recognition method based on discriminative dictionary learning

Abstract

The present application relates to the open-pit mine front vehicle recognition method based on discriminant dictionary learning, including the following steps: S1, vehicle image extraction step: the vehicle image existing in the image is extracted from the obtained image information;S2, image feature extraction step: the vehicle image is input into the pre-trained neural network to obtain several convolution features of the vehicle image;S3, vehicle based on feature dictionary discriminant step: the convolution features of the vehicle image are compared with the pre-generated feature dictionary, and the matching degree between the convolution features and the feature dictionary is used to determine the vehicle category corresponding to the vehicle image. The open-pit mine front vehicle recognition method based on discriminant dictionary learning has high front vehicle recognition precision, and the recognition time is significantly less than that of the traditional method. Discriminant dictionary learning can achieve high recognition rate under the condition of less sample and low signal-to-noise ratio.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology for autonomous vehicles, and specifically to a method for recognizing vehicles in front of an open-pit mine based on discriminant dictionary learning. Background Technology

[0002] Smart mines present a limited scenario with a limited variety of road obstacles, posing high demands on the speed and accuracy of algorithms for identifying vehicle types ahead during unmanned mining truck operations. While vision-based vehicle recognition has been studied for many years, challenges arising from vehicle posture, lighting, occlusion, and scale variations in real-world road environments persist. Image-based target recognition using onboard cameras is currently the mainstream research method. Commonly used methods for forward vehicle recognition typically employ multi-sensor information fusion technologies, including cameras, LiDAR, thermal infrared, and ultrasonic radar. These methods perform well in environments with good road conditions and clear vehicle features. Furthermore, while traditional machine learning and neural network algorithms such as ANN, SVM, R-FCN, Faster R-CNN, and YOLO offer high accuracy, they require significant computational resources, resulting in slower processing speeds for onboard computing platforms. Summary of the Invention

[0003] To improve the ability of autonomous vehicles to recognize surrounding vehicles and provide faster recognition efficiency while ensuring recognition accuracy, this application provides a method for recognizing vehicles in front of an open-pit mine based on discriminant dictionary learning.

[0004] The method for identifying vehicles ahead in open-pit mines based on discriminant dictionary learning in this application includes the following steps: S1. Vehicle image extraction steps: Extract the vehicle images present in the acquired image information; S2, Image Feature Extraction Step: Input the information obtained from the vehicle image into the pre-trained neural network to obtain several convolutional features of the vehicle image; S3. Vehicle identification based on feature dictionary: The convolutional features of the vehicle image are compared with the pre-generated feature dictionary, and the vehicle category corresponding to the vehicle image is determined by the degree of matching between the convolutional features and the feature dictionary.

[0005] Preferably, in the vehicle image extraction step S1, the image information is video stream information. The moving object is determined based on the comparison between multiple frames of the video stream information, and the image region corresponding to the moving object that meets the requirements is extracted to obtain the vehicle image.

[0006] Preferably, in the vehicle image extraction step S1, the extracted vehicle image or the acquired image information is subjected to time-frequency transformation to obtain a time-frequency image after discrete Fourier transform, which is then filtered and restored to obtain a vehicle image or image information with reduced resolution.

[0007] Preferably, the information obtained from the vehicle image includes the vehicle image and / or the geometric features of the vehicle image.

[0008] Preferably, the S2 image feature extraction step uses the data from the last convolutional layer of the LeNet-5 network of the neural network as the convolutional features of the vehicle image.

[0009] Preferably, step S3, which involves identifying vehicles based on a feature dictionary, involves constructing a convolutional feature sub-dictionary based on a redundant dictionary and determining the feature dictionary and category determination criteria. Based on the category determination criteria, the matching degree between the convolutional features and the feature dictionary is determined to identify the vehicle category.

[0010] This method for identifying vehicles ahead in open-pit mines based on discriminant dictionary learning achieves high accuracy and significantly reduces the recognition time compared to traditional methods. By utilizing discriminant dictionary learning, it can achieve a high recognition rate even with limited samples and low signal-to-noise ratios. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a schematic diagram of the pre-trained neural network of the present invention. Detailed Implementation

[0012] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. In this specification, the dimensions of the drawings do not represent the actual dimensions. They are only used to illustrate the relative positional and connection relationships between the components. Components with the same name or the same reference numeral represent similar or identical structures and are limited to illustrative purposes.

[0013] Figure 1 This diagram illustrates the specific steps of the open-pit mine vehicle recognition method based on discriminant dictionary learning proposed in this application. The steps include: S1. Vehicle Image Extraction Steps. Vehicle images are extracted from localized areas of images acquired by camera sensors or other image sensors. These images can be single frames, multi-frame image streams, or video streams. The process of filtering these images to obtain the vehicle image involves removing irrelevant portions of the image other than the vehicle to be identified, retaining only the image of the vehicle itself. This process removes irrelevant information and improves recognition accuracy.

[0014] S2, Image Feature Extraction Step. The vehicle image obtained in step S1 and its geometric features are input into a pre-trained neural network to obtain several convolutional features of the vehicle image.

[0015] S3. Vehicle identification based on feature dictionary. The convolutional features of the vehicle image obtained in step S2 are compared with the pre-generated feature dictionary. The vehicle category corresponding to the vehicle image is determined by the degree of matching between the convolutional features and the feature dictionary.

[0016] Based on the characteristics of open-pit mine roads, the geometric information of the road scene in a single image captured by the vehicle-mounted camera can be mainly divided into three categories: sky, road surface, and moving objects. The sky is usually located at the top of the image, the road surface typically refers to the road surface area, and moving objects are larger moving objects on the road surface, including pedestrians and vehicles. Therefore, considering the high real-time requirements of unmanned mining trucks and the limited computing power of the onboard computing platform, an image planar information extraction method is used to select the area of ​​the vehicle ahead from the road scene as the subsequent range for local vehicle feature extraction.

[0017] Autonomous vehicles typically perceive their surroundings using onboard imaging devices or other sensors. Taking image sensors as an example, this article presents a method for acquiring images of vehicles in the environment. When an autonomous vehicle acquires image data of its surroundings, other vehicles in the environment are also moving relative to it. This means that other vehicles are dynamically moving relative to the background. Therefore, after acquiring image information based on a video stream, moving objects in the scene can be identified by comparing multiple frames. These moving objects that meet the criteria are then marked as vehicles, and their corresponding image regions are extracted to obtain vehicle images.

[0018] Images are acquired via an in-vehicle camera and then processed by a stabilization module. Based on the summation of absolute errors algorithm, optical flow is applied to eliminate video interference signals caused by translational motion and jitter, thus obtaining a stable video stream. The SLIC algorithm is used to geometrically segment the in-vehicle images, initially screening image regions containing vehicles. Through this screening, regions belonging to the sky and ground are set to full black (BGR(0,0,0), retaining only moving objects within them. Only image regions with the attribute of moving objects participate in subsequent calculations of local features of vehicles ahead. The moving objects are then extracted to obtain the vehicle image. This method has good discriminative and detection rates, and the forward vehicle region screening method shortens the recognition time to a certain extent, meeting the real-time and computing power requirements of the in-vehicle computing platform. Optionally, the extracted vehicle image or the acquired image information can be subjected to time-frequency transformation to obtain a discrete Fourier transform time-frequency image. After filtering, the image is restored to its original state, achieving pixel reset and obtaining a resolution-compressed vehicle image or image information.

[0019] S2 Image Feature Extraction Steps. The obtained vehicle images are processed using a pre-trained CNN convolutional neural network to obtain several image features as output. The neural network architecture is as follows: Figure 2 As shown. In the specific processing, the data from the last convolutional layer of the LeNet-5 network is used as image features, i.e., convolutional features. Furthermore, to address the problem that the network training often gets stuck at saddle points or local extrema, the network is pre-trained using a dataset of images collected from a two-month-old open-pit mine vehicle camera. Then, the parameters of layers 2-6 are transferred to the new LeNet-5 network to extract features.

[0020] This paper utilizes the LeNet-5 convolutional neural network to describe deep image features, avoiding the complexity of manual feature selection. Compared to other shallow features, the extracted convolutional features can largely represent the characteristics of different vehicles in front of the image, fully leveraging the advantages of deep learning networks in feature extraction and recognition. For convolutional feature extraction, this paper uses training samples to retrain the new LeNet-5 network. When the recognition rate reaches its highest level, it indicates that the parameters in layers 2-6 of the network have reached their optimal state. The network parameters of layers 2-6 in the pre-trained LeNet-5 convolutional neural network are transferred to the new LeNet-5, and the data from the 6th convolutional layer is extracted as the convolutional features. Then, a discriminative dictionary learning method is used to recognize these convolutional features.

[0021] To enhance the specificity of convolutional features in the results, the geometric features of the vehicle image are also input simultaneously when the vehicle image is fed into the convolutional neural network. That is, the pre-trained convolutional neural network is actually designed for the vehicle image itself and the set of corresponding geometric features. A pre-trained LeNet-5 convolutional neural network is used, followed by transfer learning to extract high-level features. Because convolutional neural networks excel in image processing and can automatically extract multi-level features such as edges and textures, vehicle detection is performed using the LeNet-5 convolutional neural network, and the ability to recognize small objects is improved through multi-level feature fusion.

[0022] S3. Vehicle Discrimination Based on Feature Dictionary. A convolutional feature sub-dictionary is constructed based on a redundant dictionary and sparsely represented to obtain the feature dictionary. The Fisher criterion is used to make the sub-dictionary more discriminative. Furthermore, reconstruction error and sparse coding coefficient similarity are introduced to enhance class discrimination ability, thereby improving the accuracy and reliability of recognition. The discriminative dictionary learning part requires constructing a discriminative dictionary and optimizing the objective function to make the dictionary better classify different features. "Discriminative Dictionary Learning (DDL) optimizes the discriminativeness of the dictionary, making the sparse codes of similar samples more similar, thus improving classification performance. It can reduce interference from irrelevant samples. It can enhance the discriminative ability of features and is suitable for handling intra-class differences and inter-class similarities in vehicle images." The dictionary construction goal is to learn a class-sensitive dictionary that makes the sparse representations of similar features as similar as possible, and the differences between dissimilar features significant. Mathematically, this is achieved by optimizing the objective function.

[0023] The image is partitioned into low-level superpixels based on the SLIC method. Each superpixel is represented by information such as color, position, and perspective, and then input into a pre-trained regression classifier to obtain the category of each superpixel: sky, road surface, or road object. Finally, regions in the image pixels belonging to the sky and ground are set to all black BGR(0,0,0), and only image regions with the attribute of moving objects participate in the subsequent local feature calculation process for vehicles ahead.

[0024] The LeNet-5 network is iteratively trained using training samples to extract convolutional features. The network parameters are optimized based on the recognition rate. Data from layer C5 is then extracted as convolutional features for the training samples. Test samples are directly convolutionally pooled through the optimized LeNet-5 network, and data from layer C5 is extracted as convolutional features.

[0025] The basic steps of the recognition process are as follows: Images captured by the vehicle-mounted camera undergo time-frequency transformation to obtain a time-frequency image, which is then resized to a 28x28 pixel image. The resulting training and testing sample convolutional features are used as training and testing samples for dictionary learning. Finally, the vehicle in front is identified based on the category determination criteria.

[0026] This algorithm uses images captured by vehicle-mounted cameras under different lighting conditions as its foundation. It obtains two-dimensional time-frequency data of the captured images through time-frequency transformation and inputs it into a LeNet-5 convolutional neural network. The parameters of layers 2-6 from the pre-trained network are transferred to the new LeNet-5, and the data from the 6th convolutional layer is extracted as the convolutional features. The horizontal features at the bottom of the vehicle in front, the vertical features on the left and right sides of the vehicle in front, the local grayscale features of the vehicle in front, the local gradient features of the vehicle in front, and the local fluctuation features of the vehicle in front are each used as a signal feature sub-dictionary. A discriminant dictionary learning method is then used to identify the vehicle in front. The fusion of convolutional features and the discriminant dictionary involves inputting the convolutional features into the discriminant dictionary learning model after feature extraction; that is, features are extracted first, and then the dictionary is used for classification. Test results show that the method described in this patent has high accuracy in identifying vehicles in front, and the recognition time is significantly shorter than traditional methods. The use of discriminant dictionary learning can achieve a high recognition rate even with few samples and low signal-to-noise ratios.

[0027] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for identifying vehicles ahead in an open-pit mine based on discriminant dictionary learning, characterized in that, Includes the following steps: S1. Vehicle image extraction steps: Extract the vehicle images present in the acquired image information; S2. Image feature extraction step: Input the information obtained from the vehicle image into the pre-trained neural network to obtain several convolutional features of the vehicle image; S3. Vehicle identification based on feature dictionary: The convolutional features of the vehicle image are compared with the pre-generated feature dictionary, and the vehicle category corresponding to the vehicle image is determined by the degree of matching between the convolutional features and the feature dictionary.

2. The method for identifying vehicles ahead in an open-pit mine based on discriminant dictionary learning as described in claim 1, characterized in that, In the S1 vehicle image extraction step, the image information is video stream information. Based on the comparison between multiple frames of the video stream information, the moving object is determined, and the image region corresponding to the moving object that meets the requirements is extracted to obtain the vehicle image.

3. The method for identifying vehicles ahead in an open-pit mine based on discriminant dictionary learning as described in claim 1, characterized in that, In the S1 vehicle image extraction step, the extracted vehicle image or the collected image information is subjected to time-frequency transformation to obtain a time-frequency image after discrete Fourier transform. After filtering, it is restored to obtain a vehicle image or image information with reduced resolution.

4. The method for identifying vehicles ahead in an open-pit mine based on discriminant dictionary learning as described in claim 1, characterized in that, The information obtained from the vehicle image includes the vehicle image and / or the geometric features of the vehicle image.

5. The method for identifying vehicles ahead in an open-pit mine based on discriminant dictionary learning as described in claim 1, characterized in that, The S2 image feature extraction step uses the data from the last convolutional layer of the LeNet-5 network of the neural network as the convolutional features of the vehicle image.

6. The method for identifying vehicles ahead in an open-pit mine based on discriminant dictionary learning as described in claim 1, characterized in that, The S3 step of vehicle identification based on feature dictionary involves constructing a convolutional feature sub-dictionary based on a redundant dictionary and determining the feature dictionary and category determination criteria. Based on the category determination criteria, the matching degree between the convolutional features and the feature dictionary is determined to determine the vehicle category.

Images