Unmanned mine car driving obstacle image recognition method

Abstract

The invention discloses an unmanned mine car driving obstacle image recognition method, which comprises the steps of acquiring a video image and performing segmentation processing to obtain a segmented image; performing video vectorization processing on the segmented image based on polar coordinate transformation to obtain a processed image; performing candidate region feature extraction according to the processed image to obtain image extraction features; carrying out feature classification on the obtained image extraction features based on an LSSVM and an evidence theory, and outputting a feature classification result; and judging a feature classification result, and outputting an obstacle image recognition result. According to the method, fusion is carried out through image classification results, the obstacle type can be better described, and the image classification training speed and the classification efficiency are improved by introducing the LSSVM.

Description

Technical Field

[0001] The present invention relates to the technical field of smart open-pit mines, and in particular to a method for recognizing images of obstacles encountered by unmanned mine vehicles. Background Art

[0002] In the smart mine scenario, due to the relatively closed environment, relatively fixed road structure and relatively limited types of obstacles, the unmanned mine vehicles have extremely high speed and accuracy requirements for the obstacle recognition algorithm during driving. These unmanned mine vehicles not only need to quickly identify targets to determine whether there are suspicious obstacles ahead, but also need to accurately determine the type of obstacles so as to take corresponding obstacle avoidance measures or response strategies.

[0003] Currently, the use of on-board cameras for image target recognition is the main method for unmanned mining vehicles in smart mines to achieve obstacle recognition. On-board cameras can capture information about roads and surrounding environments in real time, and identify potential obstacles through image processing and analysis technology. However, among the existing mainstream image target detection algorithms, such as R-FCN, FasterR-CNN, and YOLO series, although they perform well in terms of accuracy, they often require a large amount of computing resources.

[0004] For unmanned mining vehicles, these high-computing algorithms mean that the onboard computing platform needs to be equipped with high-performance processors and large-capacity memory to support real-time, accurate image processing and target detection. However, such a configuration not only increases the manufacturing cost of the mining vehicle, but also under certain extreme conditions, such as high temperatures in the mine or unstable power supply, the heat dissipation and stability of the high-performance computing platform may become a bottleneck restricting the normal operation of the unmanned mining vehicle.

[0005] More importantly, algorithms with high computing power requirements often result in slower computing speeds on the onboard computing platform. In a high-speed mining car, this delay in computing speed may greatly affect the efficiency of the driverless mining car's behavioral decision-making. Once the mining car fails to promptly and accurately identify obstacles ahead and make correct obstacle avoidance decisions, it may lead to serious safety accidents. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art. To achieve the above purpose, a method for identifying obstacles in an unmanned mining vehicle is adopted to solve the problems raised in the above background technology.

[0007] A method for identifying an obstacle image when an unmanned mining vehicle is traveling comprises the following steps:

[0008] Step S1, collecting video images and performing segmentation processing to obtain images after segmentation processing;

[0009] Step S2, performing video vectorization processing on the segmented image based on polar coordinate transformation to obtain a processed image;

[0010] Step S3, extracting candidate region features based on the processed image to obtain image extraction features;

[0011] Step S4, performing feature classification on the obtained image extraction features based on LSSVM and evidence theory, and outputting the feature classification results;

[0012] Step S5: Determine the feature classification result and output the obstacle image recognition result.

[0013] As a further solution of the present invention: the specific steps in step S1 include:

[0014] The unmanned mining car's onboard camera equipment is used to collect mining car driving videos, and the onboard computing platform is used to process the videos in frames, and the clarity is improved through image enhancement processing;

[0015] Image segmentation is then achieved based on the inter-frame difference method to extract relevant candidate areas from the image.

[0016] As a further solution of the present invention: the specific steps in step S2 include:

[0017] Use polar coordinate transformation to process video vectorization and remove the interference of feature extraction caused by nonlinear distortion in the collected video;

[0018] Among them, the driving video expression is f(x,y);

[0019] The polar coordinate expression is f(r,θ);

[0020] Wherein, x=r*sinθ, y=r*cosθ represent the video image and its polar coordinate transformation expression respectively; the radial length of the image transformation is r, and the rotation angle is θ.

[0021] As a further solution of the present invention: the specific steps in step S3 include:

[0022] A plurality of candidate region features are extracted according to the processed image, and the feature with the largest class distance is selected as the video image extraction feature to obtain the image extraction feature.

[0023] As a further solution of the present invention: the specific steps in step S4 include:

[0024] The least squares support vector machine (LSSVM) is used as the image classifier, and then the rules of synthesizing multiple features are used according to the evidence DS theory to fuse the image classification results of multiple features to achieve image classification.

[0025] As a further solution of the present invention: the specific steps in the rule of synthesizing multiple features according to the evidence DS theory include:

[0026] Define an exhaustive set of all hypotheses and a hypothesis space, where all hypotheses are mutually exclusive; that is, let the set Ω contain n elements, and Ω can be expressed as follows: Ω = {a1, a2, ..., an};

[0027] Among them, the subset A of Ω is called a proposition, and the power set 2 Ω is composed of all subsets of Ω, including 2 N elements; define function m: 2 Ω →[0, 1] is the basic probability distribution function mapping, satisfying and

[0028] In the formula, the value of m(A) represents the confidence of the result A; the trust function and likelihood function are:

[0029]

[0030] Pl(A)=Σ B∩A≠Ω m(B)

[0031] Then the obstacle type classification integrating multiple features is obtained, and the confidence calculation formula is as follows:

[0032]

[0033] The conflict factor k value reflects the degree of evidence conflict, and its calculation formula is as follows:

[0034]

[0035] When there is a conflict in the classification of obstacle types with multiple features, the corresponding confidence distributions are fused based on the DS evidence theory to obtain the corrected confidence distribution and sort it, and the type corresponding to the maximum confidence is taken as the final obstacle type.

[0036] As a further solution of the present invention: the specific steps in step S5 include:

[0037] The image classification result is judged. If it is judged to be a sensitive target, it is output to the unmanned mining vehicle decision module; if it is judged not to be a sensitive target, it is directly output as a non-sensitive target.

[0038] Compared with the prior art, the present invention has the following technical effects:

[0039] The above technical solution uses a convolutional neural network to classify images, which has a stronger ability to automatically learn features than traditional classification methods. However, designing a convolutional neural network with good classification effect requires a large amount of data to optimize the neural network parameters. In view of the mutual limitation of image speed and accuracy of the on-board camera of unmanned mining vehicles, and the fact that a single feature can only describe the local category and partial information of the target in the image, multi-feature fusion can more accurately and comprehensively reflect the image category. The DS evidence theory is used to fuse the image classification results of a single feature, which can better characterize the obstacle type, and the introduction of LSSVM can accelerate the image classification training speed and classification efficiency. The method in this paper is less affected by noise, mainly because the method in this paper uses the evidence confidence to define the range to greatly avoid the interference of irrelevant noise, and has strong image classification stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings:

[0041] Figure 1 A schematic diagram of the steps of the image recognition method disclosed in the present application;

[0042] Figure 2 A schematic diagram of the image recognition algorithm architecture of the embodiment disclosed in this application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Please refer to Figure 1 In an embodiment of the present invention, a method for identifying an image of an obstacle when an unmanned mining vehicle is traveling comprises the following steps:

[0045] Step S1, collecting video images and performing segmentation processing to obtain images after segmentation processing, the specific steps include:

[0046] The unmanned mining car's onboard camera equipment is used to collect mining car driving videos, and the onboard computing platform is used to process the videos in frames, and the clarity is improved through image enhancement processing;

[0047] Image segmentation is then achieved based on the inter-frame difference method to extract relevant candidate areas from the image.

[0048] like Figure 2 As shown, the figure is a schematic diagram of the obstacle image recognition algorithm architecture of an unmanned mine car in an open-pit mine environment;

[0049] Step S2, performing video vectorization processing on the segmented image based on polar coordinate transformation to obtain a processed image, the specific steps include:

[0050] Use polar coordinate transformation to process video vectorization and remove the interference of feature extraction caused by nonlinear distortion in the collected video;

[0051] Among them, the driving video expression is f(x,y);

[0052] The polar coordinate expression is f(r,θ);

[0053] Wherein, x=r*sinθ, y=r*cosθ represent the video image and its polar coordinate transformation expression respectively; the radial length of the image transformation is r, and the rotation angle is θ.

[0054] Step S3: extracting features of candidate regions according to the processed image to obtain image extraction features. The specific steps include:

[0055] A plurality of candidate region features are extracted according to the processed image, and the feature with the largest class distance is selected as the video image extraction feature to obtain the image extraction feature.

[0056] Step S4: classify the obtained image extraction features based on LSSVM and evidence theory, and output the feature classification results. The specific steps include:

[0057] The least squares support vector machine (LSSVM) is used as the image classifier, and then the rules of synthesizing multiple features are used according to the evidence DS theory to fuse the image classification results of multiple features to achieve image classification.

[0058] Specifically, the specific steps in the rule of synthesizing multiple features according to the evidence DS theory include:

[0059] Define an exhaustive set of all hypotheses and a hypothesis space, where all hypotheses are mutually exclusive; that is, let the set Ω contain n elements, and Ω can be expressed as follows: Ω = {a1, a2, ..., an};

[0060] Among them, the subset A of Ω is called a proposition, and the power set 2 Ω is composed of all subsets of Ω, including 2 N elements; define function m: 2 Ω →[0, 1] is the basic probability distribution function mapping, satisfying and

[0061] In the formula, the value of m(A) represents the confidence of the result A; the trust function and likelihood function are:

[0062]

[0063] Pl(A)=Σ B∩A≠Ω m(B)

[0064] Then the obstacle type classification integrating multiple features is obtained, and the confidence calculation formula is as follows:

[0065]

[0066] The conflict factor k value reflects the degree of evidence conflict, and its calculation formula is as follows:

[0067]

[0068] When there is a conflict in the classification of obstacle types with multiple features, the corresponding confidence distributions are fused based on the DS evidence theory to obtain the corrected confidence distribution and sort it, and the type corresponding to the maximum confidence is taken as the final obstacle type.

[0069] Step S5: determine the feature classification result and output the obstacle image recognition result. The specific steps include:

[0070] The image classification result is judged. If it is judged to be a sensitive target, it is output to the unmanned mining vehicle decision module; if it is judged not to be a sensitive target, it is directly output as a non-sensitive target.

[0071] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents, and all should be included in the scope of protection of the present invention.

Claims

1. A method for identifying obstacles in unmanned mining vehicles, characterized in that: The following steps are involved: Step S1, collecting video images and performing segmentation processing to obtain images after segmentation processing; Step S2, performing video vectorization processing on the segmented image based on polar coordinate transformation to obtain a processed image; Step S3, extracting candidate region features based on the processed image to obtain image extraction features; Step S4, performing feature classification on the obtained image extraction features based on LSSVM and evidence theory, and outputting the feature classification results; Step S5: Determine the feature classification result and output the obstacle image recognition result.

2. According to claim 1, a method for identifying obstacles when an unmanned mining vehicle is traveling is characterized in that: The specific steps in step S1 include: The unmanned mining car's onboard camera equipment is used to collect mining car driving videos, and the onboard computing platform is used to process the videos in frames, and the clarity is improved through image enhancement processing; Image segmentation is then achieved based on the inter-frame difference method to extract relevant candidate areas from the image.

3. According to claim 1, a method for identifying obstacles when an unmanned mining vehicle is traveling is characterized in that: The specific steps in step S2 include: Use polar coordinate transformation to process video vectorization and remove the interference of feature extraction caused by nonlinear distortion in the collected video; Among them, the driving video expression is f(x,y); The polar coordinate expression is f(r,θ); Wherein, x=r*sinθ, y=r*cosθ represent the video image and its polar coordinate transformation expression respectively; the radial length of the image transformation is r, and the rotation angle is θ.

4. According to claim 1, a method for identifying obstacles in unmanned mining vehicles is characterized in that: The specific steps in step S3 include: A plurality of candidate region features are extracted according to the processed image, and the feature with the largest class distance is selected as the video image extraction feature to obtain the image extraction feature.

5. According to claim 1, a method for identifying obstacles in unmanned mining vehicles is characterized in that: The specific steps in step S4 include: The least squares support vector machine (LSSVM) is used as the image classifier, and then the rules of synthesizing multiple features are used according to the evidence DS theory to fuse the image classification results of multiple features to achieve image classification.

6. According to claim 5, a method for identifying obstacles in unmanned mining vehicles is characterized in that: The specific steps in the rule of synthesizing multiple features according to the evidence DS theory include: Define an exhaustive set of all hypothesis results and a hypothesis space, where all hypotheses are mutually exclusive; that is, let the set Ω contain n elements, and Ω can be expressed as follows: Ω = {a1, a2, ..., an}; Among them, the subset A of Ω is called a proposition, and the power set 2 Ω is composed of all subsets of Ω, including 2 N elements; define function m: 2 Ω →[0, 1] is the basic probability distribution function mapping, satisfying and In the formula, the value of m(A) represents the confidence of the result A; the trust function and likelihood function are: Pl(A)=∑ B∩A≠Ω m(B) Then the obstacle type classification integrating multiple features is obtained, and the confidence calculation formula is as follows: The conflict factor k value reflects the degree of evidence conflict, and its calculation formula is as follows: When there is a conflict in the classification of obstacle types with multiple features, the corresponding confidence distributions are fused based on the DS evidence theory to obtain the corrected confidence distribution and sort it, and the type corresponding to the maximum confidence is taken as the final obstacle type.

7. According to claim 1, a method for identifying obstacles in unmanned mining vehicles is characterized in that: The specific steps in step S5 include: The image classification result is judged. If it is judged to be a sensitive target, it is output to the unmanned mining vehicle decision module; if it is judged not to be a sensitive target, it is directly output as a non-sensitive target.

Images