Agricultural machinery automatic driving obstacle identification method and system

Abstract

The invention provides an agricultural machinery automatic driving obstacle identification method. The method comprises the following steps: acquiring a panoramic image in a forward direction in an agricultural machinery automatic driving process; dividing obstacles into dynamic obstacles and static obstacles according to the difference between adjacent frames of the panoramic image; obtaining a dynamic vector through the brightness of the dynamic obstacle; obtaining a dynamic space relationship between the dynamic obstacle and the agricultural machine through the dynamic vector and the motion state of the agricultural machine; acquiring labeling box information of the static obstacle; obtaining a depth value of the static obstacle through a recognition model based on the labeling box information; obtaining a static space relationship between the static obstacle and the agricultural machine through the depth value; and obtaining an overall spatial relationship through the dynamic spatial relationship and the static spatial relationship. According to the method, the obstacles in the image are distinguished, and different recognition methods are adopted according to the dynamic and static states of the obstacles, so that the accuracy and precision of obstacle recognition are improved.

Description

Technical Field

[0001] The present invention relates to the field of visual recognition technology, and in particular to a method and system for identifying obstacles in automatic driving of agricultural machinery. Background Art

[0002] Agricultural machinery autonomous driving technology is of great significance in reducing agricultural labor intensity, improving work efficiency and increasing agricultural production capacity. In order for agricultural machinery vehicles to achieve autonomous driving, it is necessary to accurately perceive the vehicle's surrounding environment and identify and locate surrounding obstacles. Computer vision and deep learning are key perception technologies in the field of autonomous driving, and play a vital role in achieving effective recognition of the vehicle's surrounding environment.

[0003] Traditional obstacle perception methods mainly rely on a combination of machine vision, lidar, and millimeter-wave radar. However, in the agricultural machinery driving environment, although the radar system performs well in identifying dynamic objects, the perception effect of low-speed or static obstacles is relatively poor, which cannot meet the high requirements of agricultural machinery for environmental accuracy in automatic driving. This makes it impossible for agricultural machinery to accurately identify the position of static obstacles when encountering them, resulting in untimely avoidance, thus affecting the effect of automatic driving. At the same time, the existing technology has low efficiency and accuracy in identifying dynamic obstacles in panoramic images. Therefore, it is very necessary to design an obstacle recognition method and system for automatic driving of agricultural machinery. Summary of the invention

[0004] The purpose of the present invention is to provide an obstacle recognition method and system for automatic driving of agricultural machinery, which distinguishes obstacles in the image and adopts different recognition methods according to the dynamic and static states of the obstacles to improve the accuracy and precision of obstacle recognition.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for identifying obstacles in automatic driving of agricultural machinery comprises the following steps:

[0007] Obtain panoramic images in the forward direction of the agricultural machinery during automatic driving;

[0008] Obstacles are classified into dynamic obstacles and static obstacles by the differences between adjacent frames of the panoramic image;

[0009] The dynamic vector is obtained by the brightness of the dynamic obstacle; the dynamic vector includes the dynamic direction and the dynamic speed;

[0010] The dynamic spatial relationship between the dynamic obstacle and the agricultural machinery is obtained through the dynamic vector and the motion state of the agricultural machinery;

[0011] Get the annotation box information of static obstacles;

[0012] Based on the annotation box information, the depth value of the static obstacle is obtained through the recognition model;

[0013] The static spatial relationship between static obstacles and agricultural machinery is obtained through depth values;

[0014] The overall spatial relationship is obtained through dynamic spatial relationship and static spatial relationship.

[0015] Optionally, obstacles are classified into dynamic obstacles and static obstacles by differences between adjacent frames of the panoramic image, including:

[0016] Convert each frame of the panoramic image into a grayscale image;

[0017] Perform image alignment operation on the grayscale images to obtain an initial image set;

[0018] Calculate the difference between any two adjacent frames in the initial image set;

[0019] Obstacles are classified into dynamic obstacles and static obstacles according to the comparison result between the difference and the preset difference threshold.

[0020] Optionally, a dynamic vector is obtained by the brightness of the dynamic obstacle, including:

[0021] The light intensity vector is obtained through the pixel position and brightness of the dynamic obstacle;

[0022] The optical flow constraint equation is constructed through the light intensity vector, and the optical flow constraint equation is solved to obtain the optical flow vector;

[0023] The optical flow vectors are filtered and fused to obtain the dynamic vector.

[0024] Optionally, the dynamic spatial relationship between the dynamic obstacle and the agricultural machinery is obtained through the dynamic vector and the motion state of the agricultural machinery, including:

[0025] Real-time positioning of agricultural machinery to obtain the two-dimensional spatial position of agricultural machinery;

[0026] The agricultural machinery motion vector is obtained by changing the two-dimensional spatial position of the agricultural machinery; the agricultural machinery motion vector includes the agricultural machinery motion direction and the agricultural machinery motion speed;

[0027] The real vector of the dynamic obstacle is obtained according to the angle between the agricultural machinery motion vector and the dynamic vector; the real vector includes: real direction and real speed;

[0028] The distance between the dynamic obstacle and the agricultural machinery is obtained according to the dynamic vector;

[0029] The dynamic spatial relationship is obtained according to the distance between the dynamic obstacle and the agricultural machinery, the true vector and the movement vector of the agricultural machinery.

[0030] Optionally, the real vector of the dynamic obstacle is obtained according to the angle between the agricultural machinery motion vector and the dynamic vector, including:

[0031] The dynamic vector starting point is taken as the real vector starting point, the agricultural machinery motion vector end point is taken as the real vector end point, and the real direction is obtained according to the real vector starting point and the real vector end point;

[0032] When the angle between the agricultural machinery motion vector and the dynamic vector is an obtuse angle, the calculation formula for the true speed is: Among them, V z is the true speed, V D is the dynamic speed, V n is the speed of the agricultural machinery, θ1 is the angle between the agricultural machinery movement direction and the dynamic direction, and θ2 is the angle between the dynamic direction and the true direction;

[0033] When the angle between the agricultural machinery motion vector and the dynamic vector is an acute angle, the calculation formula of the true speed is:

[0034]

[0035] Optionally, the distance between the dynamic obstacle and the agricultural machinery is calculated as follows: Among them, S DN is the distance between the dynamic obstacle and the agricultural machinery, B t-1 is the total number of pixels of dynamic obstacles at the last moment, B t is the total number of pixels of dynamic obstacles at the current moment, V D is the dynamic speed, Δt is the time interval between the previous moment and the current moment, and F is the focal length of the camera.

[0036] Optionally, based on the annotation box information, the depth value of the static obstacle is obtained through the recognition model, including:

[0037] Build recognition models;

[0038] Obtain a depth map of the marked box area through the recognition model;

[0039] Calculate the depth value according to the depth map;

[0040] Optionally, the recognition model includes an input layer, an encoder, a bottleneck layer, a decoder, and an output layer connected in sequence;

[0041] The number of channels of the input layer is 3; the encoder consists of multiple convolutional layers and activation layers. The convolutional layer is used to reduce the dimension and extract features of the output of the input layer, and the activation layer is used to activate the convolutional layer. The bottleneck layer is used to integrate the features extracted by the convolutional layer; the decoder is used to restore the low-dimensional features to the original spatial dimension; the output layer is used to output the depth map.

[0042] Optionally, the static spatial relationship between the static obstacle and the agricultural machinery is obtained through the depth value, including:

[0043] The distance between the static obstacle and the agricultural machinery is obtained according to the depth value;

[0044] The agricultural machinery motion vector is obtained by changing the two-dimensional spatial position of the agricultural machinery; the agricultural machinery motion vector includes the agricultural machinery motion direction and the agricultural machinery motion speed;

[0045] Taking the static obstacle as the origin, the static spatial relationship is obtained through the origin and the agricultural machinery motion vector.

[0046] An agricultural machinery automatic driving obstacle recognition system, comprising:

[0047] An image acquisition module is used to obtain a panoramic image in the forward direction of the agricultural machinery during automatic driving;

[0048] An obstacle differentiation module, for classifying obstacles into dynamic obstacles and static obstacles by differences between adjacent frames of the panoramic image;

[0049] A dynamic recognition module is used to obtain a dynamic vector through the brightness of the dynamic obstacle; and obtain a dynamic spatial relationship between the dynamic obstacle and the agricultural machinery through the dynamic vector and the motion state of the agricultural machinery;

[0050] Image detection module, used to obtain the annotation box information of static obstacles;

[0051] The depth estimation module is used to obtain the depth value of the static obstacle through the recognition model based on the annotation box information;

[0052] The static recognition module is used to obtain the static spatial relationship between static obstacles and agricultural machinery through depth values.

[0053] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the agricultural machinery automatic driving obstacle recognition method provided by the present invention includes: obtaining a panoramic image in the forward direction of the agricultural machinery automatic driving process; dividing the obstacles into dynamic obstacles and static obstacles according to the differences between adjacent frames of the panoramic image; obtaining a dynamic vector according to the brightness of the dynamic obstacle; obtaining the dynamic spatial relationship between the dynamic obstacle and the agricultural machinery according to the dynamic vector and the motion state of the agricultural machinery; obtaining the annotation box information of the static obstacle; based on the annotation box information, obtaining the depth value of the static obstacle through the recognition model; obtaining the static spatial relationship between the static obstacle and the agricultural machinery through the depth value; obtaining the overall spatial relationship through the dynamic spatial relationship and the static spatial relationship. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0055] Figure 1 is a flow chart of the obstacle identification method of the present invention;

[0056] Figure 2 It is a flow chart of the method for distinguishing between dynamic and static obstacles of the present invention;

[0057] Figure 3 The dynamic vector calculation flow chart of the present invention;

[0058] Figure 4 A flow chart for obtaining dynamic spatial relationships of the present invention;

[0059] Figure 5 A flow chart for obtaining the static spatial relationship of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] like Figure 1 As shown, an embodiment of the present invention provides an obstacle recognition method for automatic driving of agricultural machinery, comprising the following steps:

[0063] Step 100: Acquire a panoramic image in the forward direction of the agricultural machine during automatic driving;

[0064] Specifically, a panoramic camera (such as a fisheye lens or an array of multiple wide-angle cameras) is used as a sensing device to achieve 360-degree panoramic field of view capture and provide rich environmental information. The resolution of the camera in this embodiment is a configuration of 3840x2160 pixels, which ensures rich image details and helps in subsequent obstacle identification and positioning. The camera array of this embodiment uses multiple wide-angle cameras to form a ring configuration to achieve panoramic imaging. Each camera is responsible for capturing a certain area, and has a built-in image stitching algorithm to synthesize the images captured by each camera into an overall panoramic image.

[0065] It should be noted that panoramic cameras can capture a 360-degree field of view at one time, effectively reducing blind spots and providing rich environmental information. High resolution ensures rich image details and improves the accuracy of obstacle recognition. At the same time, compared with sensors such as lidar, panoramic cameras are less expensive and suitable for large-scale promotion, reducing the overall cost of autonomous driving systems.

[0066] Step 200: Classify obstacles into dynamic obstacles and static obstacles by differences between adjacent frames of the panoramic image; the specific steps are as follows: Figure 2 As shown, including:

[0067] Step 201: convert each frame of the panoramic image into a grayscale image;

[0068] Specifically, use an image processing library (such as OpenCV) to read the panoramic image file and obtain each frame of the image. Then convert each frame of the image into a grayscale image. The expression of the conversion process is: t (x,y)=0.2989·R(x,y)+0.5870·G(x,y)+0.1140·B(x,y); where, I' t 9x,y) is the grayscale value, R(x,y) is the pixel value of the red channel, G(x,y) is the pixel value of the green channel, and B(x,y) is the pixel value of the blue channel.

[0069] Step 202: performing an image alignment operation on the grayscale images to obtain an initial image set;

[0070] Specifically, the grayscale images obtained within each second are clustered to obtain a grayscale data set. The first frame image in the grayscale data set is used as the anchor image, and the remaining images are used as the images to be aligned. The feature points of each frame image are extracted by the SIFT feature detection algorithm, and the features of each frame are matched. The 3x3 homography matrix obtained from the matched feature points using the RANSAC algorithm is used as the transformation matrix to determine the transformation relationship between the anchor image and the image to be aligned. All the images to be aligned are transformed into the coordinate system of the anchor image through the Homography matrix corresponding to the image to be aligned to obtain the aligned initial image set.

[0071] Step 203: Calculate the difference between any two adjacent frames of images in the initial image set;

[0072] Specifically, any two adjacent frames in the initial image set are integrated into an image pair, and the absolute difference between the pixels in each image pair is calculated. The obstacle area in the image is divided into regions to obtain multiple small windows. All pixels in the small window are then traversed to calculate the average difference of the small window.

[0073] More specifically, the absolute difference is calculated as: D(x,y) = |I' t+1 (x,y)-I' t (x,y)|; where I' t+1 (x, y) is the pixel value at the next moment, I' t (x, y) is the pixel value at the current moment, and D(x, y) is the absolute difference value.

[0074] The formula for calculating the mean difference is: Wherein, N is the total number of pixels in the small window, W is the width of the small window, and H is the height of the small window.

[0075] Step 204: Classify the obstacles into dynamic obstacles and static obstacles according to the comparison result between the difference and the preset difference threshold.

[0076] Specifically, the obstacle area in the image is binarized, and when the average difference of the small window exceeds the preset threshold, the small window is assigned a value of 1; when the average difference of the small window does not exceed the preset threshold, the small window is assigned a value of 0. Obstacles are divided into dynamic obstacles and static obstacles according to the binarization ratio of the obstacle area.

[0077] More specifically, the expression of the binarization operation is: Among them, M(x, y) is a binary value, 1 represents a moving area, 0 represents a stationary area, and Y is a threshold value. The threshold value is adjusted according to the actual working environment of the agricultural machinery. This embodiment does not limit the threshold value.

[0078] It should be noted that through grayscale conversion, image alignment and difference calculation, the environmental changes around agricultural machinery can be monitored in real time, which improves the accuracy and response speed of obstacle detection. The rapid identification of dynamic obstacles ensures the safety of agricultural machinery operations and avoids potential collision risks. The rapid identification of static obstacles can be used to optimize the driving path and improve the efficiency of agricultural machinery operations.

[0079] Step 300: Obtain a dynamic vector by the brightness of the dynamic obstacle; the dynamic vector includes a dynamic direction and a dynamic speed; the specific steps are as follows Figure 3 As shown, including:

[0080] Step 301: Obtain a light intensity vector through the pixel position and brightness of the dynamic obstacle;

[0081] Specifically, it is assumed that the brightness of the same target does not change when it moves between different frames, and the light intensity vector I(x, y, t) of the same pixel in different frames is constructed, where I is the pixel brightness and t represents the time dimension of the light intensity vector.

[0082] Step 302: construct an optical flow constraint equation through the light intensity vector, and solve the optical flow constraint equation to obtain an optical flow vector;

[0083] Specifically, the constraint equation is expressed as: in, and are the brightness gradients of the pixels in the x and y directions, u and v are the horizontal and vertical optical flows, respectively. is the brightness change of the pixel in time.

[0084] More specifically, for a small window with dynamic obstacles, the constraint equation is transformed into a least squares problem and solved to obtain the optical flow vector The expression of the least squares problem is: AV = 0; where,

[0085] Step 303: Filter and fuse the optical flow vectors to obtain a dynamic vector.

[0086] Specifically, an optical flow threshold is preset, and the optical flow vectors with an activity range greater than the threshold are retained, and the optical flow vectors with an activity range less than the threshold are removed to obtain a valid vector. The expression of the valid vector is: Where T is the effective threshold.

[0087] It should be noted that the analysis based on optical flow can track dynamic obstacles in real time and respond quickly, which greatly enhances the driving safety of agricultural machinery and improves the efficiency of distinguishing different types of moving objects.

[0088] Step 400: Obtain the dynamic spatial relationship between the dynamic obstacle and the agricultural machine through the dynamic vector and the motion state of the agricultural machine; the specific steps are as follows: Figure 4 As shown, including:

[0089] Step 401: positioning the agricultural machinery in real time to obtain the two-dimensional spatial position of the agricultural machinery;

[0090] Specifically, a positioning device is installed on the agricultural machinery to obtain the position coordinates of the agricultural machinery in two-dimensional space in real time.

[0091] Step 402: obtaining an agricultural machine motion vector through a change in the two-dimensional spatial position of the agricultural machine; the agricultural machine motion vector includes the agricultural machine motion direction and the agricultural machine motion speed;

[0092] Specifically, a two-dimensional plane coordinate system is established on the ground, and the position of the agricultural machinery in the first frame of the image time frame within one second is used as the starting point of the vector, and the position of the agricultural machinery in the last frame is used as the end point of the vector. The position coordinates of the starting point and the end point are connected to obtain the agricultural machinery movement vector. The speed of the agricultural machinery movement is obtained by dividing the vector size by the time interval from the starting point to the end point, and the direction of the vector is the direction of the agricultural machinery movement.

[0093] Step 403: obtaining the real vector of the dynamic obstacle according to the angle between the agricultural machine motion vector and the dynamic vector; the real vector includes: real direction and real speed;

[0094] Specifically, the dynamic vector is the relative motion state of the dynamic obstacle relative to the agricultural machine calculated with the agricultural machine itself as a fixed target, and is not the real motion state. The real vector of the dynamic obstacle is calculated by the relative relationship between the agricultural machine motion vector and the dynamic vector.

[0095] Furthermore, the starting point of the agricultural machinery movement vector and the end point of the dynamic vector are converted into the same position coordinates. The starting point of the dynamic vector is used as the starting point of the real vector, and the end point of the agricultural machinery movement vector is used as the end point of the real vector. The starting point and end point of the real vector are connected to obtain the real direction.

[0096] The true speed is greater or less than the dynamic speed according to the angle between the direction of the agricultural machinery motion vector and the dynamic vector. The specific calculation criteria for the true speed are as follows:

[0097] When the angle between the agricultural machinery motion vector and the dynamic vector is an obtuse angle, the calculation formula for the true speed is:

[0098] When the angle between the agricultural machinery motion vector and the dynamic vector is an acute angle, the calculation formula of the true speed is: Among them, V z is the true speed, V D is the dynamic speed, V nis the speed of the agricultural machinery, θ1 is the angle between the agricultural machinery movement direction and the dynamic direction, and θ2 is the angle between the dynamic direction and the true direction.

[0099] Step 404: obtaining the distance between the dynamic obstacle and the agricultural machine according to the dynamic vector;

[0100] Specifically, the distance between the dynamic obstacle and the agricultural machinery is calculated according to the scaling ratio of the dynamic obstacle within a certain time interval. The scaling ratio of the dynamic obstacle is calculated by the difference in the number of pixels occupied by the dynamic obstacle in different frames. The calculation formula for the distance between the dynamic obstacle and the agricultural machinery is:

[0101]

[0102] Among them, S DN is the distance between the dynamic obstacle and the agricultural machinery, B t-1 is the total number of pixels of dynamic obstacles at the last moment, B t is the total number of pixels of dynamic obstacles at the current moment, V D is the dynamic speed, Δt is the time interval between the previous moment and the current moment, and F is the focal length of the camera.

[0103] Step 405: Obtain a dynamic spatial relationship based on the distance between the dynamic obstacle and the agricultural machinery, the true vector, and the motion vector of the agricultural machinery.

[0104] Specifically, a dynamic spatial relationship model is constructed based on the distance between the dynamic obstacle and the agricultural machinery, the motion vector of the agricultural machinery, and the real vector of the dynamic obstacle. The model includes: the distance relationship, speed relationship, and motion direction relationship between the dynamic obstacle and the agricultural machinery. The distance relationship describes the physical distance between the two, the speed relationship is used to determine the relative speed between the agricultural machinery and the obstacle, and the direction relationship is used to analyze the angle between the motion directions of the two and determine the potential collision risk in combination with the speed relationship.

[0105] It should be noted that real-time positioning and motion vector calculations enable precise monitoring of the movement of agricultural machinery, ensuring safe driving. The combination of dynamic vectors and motion states can accurately assess the risk of potential collisions, allowing timely avoidance measures to be taken, greatly improving the safety of autonomous driving. The dynamic spatial relationship between dynamic obstacles and agricultural machinery represented by the dynamic spatial relationship model greatly improves the intelligence level of the autonomous driving system.

[0106] Step 500: Obtain the labeling box information of the static obstacle;

[0107] Specifically, this embodiment uses the YOLO target detection algorithm to obtain the annotation box coordinates and category labels of static obstacles.

[0108] Step 600: Based on the annotation box information, obtain the depth value of the static obstacle through the recognition model;

[0109] Specifically, a recognition model is first constructed, a depth map of the marked box area is obtained through the recognition model, and finally a depth value is calculated based on the depth map.

[0110] More specifically, the recognition model includes an input layer, an encoder, a bottleneck layer, a decoder, and an output layer connected in sequence. The input layer receives an input image, which is generally in RGB format, with a height of H, a width of W, and a number of channels of 3. The encoder consists of multiple convolutional layers and activation layers. The convolutional layer is used to reduce the dimension and extract features of the output of the input layer, and the activation layer is used to activate the convolutional layer. The encoder uses the output of the input layer as input to extract high-level features in the input image, gradually reducing the spatial dimensions (height and width) of the image and increasing the depth of the feature layer. The layer distribution of the encoder has multiple hierarchical distributions. The initial layer uses a small number of filters, and the number of filters gradually increases as the level deepens. A pooling layer is also connected after the convolutional layer, which further reduces the size of the feature map through the maximum pooling operation while maintaining key feature information to reduce the computational cost. The bottleneck layer is used to integrate the features extracted by the convolutional layer to avoid over-computation. The decoder restores the low-dimensional features to the original spatial dimensions through the built-in transposed convolutional layer. The encoder and the decoder use a jump connection to directly pass the output of some layers of the encoder to the decoder, helping the model to restore clearer depth information. The jump connection effectively reduces the loss of information and retains high-resolution details. The output layer processes the decoded feature map into the depth value of each pixel through a built-in convolution layer to obtain a depth map. The depth map is a single-channel image in which the value of each pixel represents the actual distance from the camera to the pixel.

[0111] Step 700: Obtain the static spatial relationship between the static obstacle and the agricultural machinery through the depth value; the specific steps are as follows: Figure 5 As shown, including:

[0112] Step 701: Obtain the distance between the static obstacle and the agricultural machinery according to the depth value;

[0113] Specifically, the distance calculation formula between the static obstacle and the agricultural machinery is:

[0114] Where D is the distance between the static obstacle and the agricultural machinery, (X d , Y d ) is the two-dimensional space coordinate represented by the depth value, (X j , Y j ) is the two-dimensional spatial coordinate of the agricultural machinery.

[0115] Step 702: Obtain the agricultural machine motion vector by changing the two-dimensional spatial position of the agricultural machine; the agricultural machine motion vector includes the agricultural machine motion direction and the agricultural machine motion speed. The method used in step 402 is the same.

[0116] Step 703: Taking the static obstacle as the origin, the static spatial relationship is obtained through the origin and the agricultural machinery motion vector.

[0117] Specifically, a static two-dimensional coordinate system is constructed with the static obstacle as the origin, and the corresponding coordinates of the agricultural machinery in the static two-dimensional coordinate system are calculated according to the distance between the static obstacle and the agricultural machinery and the motion vector of the agricultural machinery. The real-time relative position relationship between the agricultural machinery and the static obstacle is obtained through real-time updating of the coordinates.

[0118] Step 800: Obtaining the overall spatial relationship through the dynamic spatial relationship and the static spatial relationship.

[0119] Specifically, the coordinates of the agricultural machinery, static obstacles and dynamic obstacles in the dynamic spatial relationship and the static spatial relationship are represented in the same coordinate system to achieve overall recognition of dynamic and static obstacles.

[0120] The present invention also provides an agricultural machinery automatic driving obstacle recognition system, comprising:

[0121] An image acquisition module is used to obtain a panoramic image in the forward direction of the agricultural machinery during automatic driving;

[0122] An obstacle differentiation module, for classifying obstacles into dynamic obstacles and static obstacles by differences between adjacent frames of the panoramic image;

[0123] A dynamic recognition module is used to obtain a dynamic vector through the brightness of the dynamic obstacle; and obtain a dynamic spatial relationship between the dynamic obstacle and the agricultural machinery through the dynamic vector and the motion state of the agricultural machinery;

[0124] Image detection module, used to obtain the annotation box information of static obstacles;

[0125] The depth estimation module is used to obtain the depth value of the static obstacle through the recognition model based on the annotation box information;

[0126] The static recognition module is used to obtain the static spatial relationship between static obstacles and agricultural machinery through depth values.

[0127] The beneficial effects of the present invention are as follows:

[0128] 1) By acquiring the location information of static and dynamic obstacles in real time, the distance to the obstacles can be accurately determined, and potential collisions can be avoided in time, greatly improving driving safety;

[0129] 2) The use of advanced intelligent technology has realized real-time position monitoring and status analysis of agricultural machinery in complex environments, which can respond promptly to dynamic changes and greatly improve work efficiency.

[0130] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0131] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for identifying obstacles in automatic driving of agricultural machinery, characterized in that: The steps include: Obtain panoramic images in the forward direction of the agricultural machinery during automatic driving; Classifying obstacles into dynamic obstacles and static obstacles by differences between adjacent frames of the panoramic image; A dynamic vector is obtained by the brightness of the dynamic obstacle; the dynamic vector includes a dynamic direction and a dynamic speed; Obtaining a dynamic spatial relationship between the dynamic obstacle and the agricultural machine through the dynamic vector and the motion state of the agricultural machine; Obtaining the labeling box information of the static obstacle; Based on the annotation box information, obtaining the depth value of the static obstacle through a recognition model; Obtaining a static spatial relationship between the static obstacle and the agricultural machinery through the depth value; The overall spatial relationship is obtained through the dynamic spatial relationship and the static spatial relationship.

2. The method for identifying obstacles in automatic driving of agricultural machinery according to claim 1, characterized in that: Classifying obstacles into dynamic obstacles and static obstacles by differences between adjacent frames of the panoramic image, including: Convert each frame of the panoramic image into a grayscale image; Performing an image alignment operation on the grayscale images to obtain an initial image set; Calculate the difference between any two adjacent frames of images in the initial image set; The obstacles are divided into the dynamic obstacles and the static obstacles according to a comparison result between the difference and a preset difference threshold.

3. The method for identifying obstacles in automatic driving of agricultural machinery according to claim 1, characterized in that: Obtaining a dynamic vector according to the brightness of the dynamic obstacle includes: Obtaining a light intensity vector through the pixel position and brightness of the dynamic obstacle; Constructing an optical flow constraint equation through the light intensity vector, and solving the optical flow constraint equation to obtain an optical flow vector; The optical flow vectors are screened and fused to obtain the dynamic vector.

4. The method for identifying obstacles in automatic driving of agricultural machinery according to claim 1, characterized in that: The dynamic spatial relationship between the dynamic obstacle and the agricultural machine is obtained through the dynamic vector and the motion state of the agricultural machine, including: Positioning the agricultural machinery in real time to obtain the two-dimensional spatial position of the agricultural machinery; The agricultural machine motion vector is obtained by changing the two-dimensional spatial position of the agricultural machine; the agricultural machine motion vector includes the agricultural machine motion direction and the agricultural machine motion speed; The real vector of the dynamic obstacle is obtained according to the angle between the agricultural machinery motion vector and the dynamic vector; the real vector includes: real direction and real speed; Obtaining a distance between the dynamic obstacle and the agricultural machine according to the dynamic vector; The dynamic spatial relationship is obtained according to the distance between the dynamic obstacle and the agricultural machinery, the real vector and the motion vector of the agricultural machinery.

5. The method for identifying obstacles in automatic driving of agricultural machinery according to claim 4, characterized in that: Obtaining the real vector of the dynamic obstacle according to the angle between the agricultural machinery motion vector and the dynamic vector includes: Taking the dynamic vector starting point as the real vector starting point, taking the agricultural machinery motion vector end point as the real vector end point, and obtaining the real direction according to the real vector starting point and the real vector end point; When the angle between the agricultural machinery motion vector and the dynamic vector is an obtuse angle, the calculation formula of the true speed is: Among them, V z is the true speed, V D is the dynamic speed, V n is the speed of the agricultural machinery, θ1 is the angle between the agricultural machinery movement direction and the dynamic direction, and θ2 is the angle between the dynamic direction and the true direction; When the angle between the agricultural machinery motion vector and the dynamic vector is an acute angle, the calculation formula of the true speed is:

6. The method for identifying obstacles in automatic driving of agricultural machinery according to claim 4, characterized in that: The calculation formula of the distance between the dynamic obstacle and the agricultural machinery is: Among them, S DN is the distance between the dynamic obstacle and the agricultural machinery, B t-1 is the total number of pixels of dynamic obstacles at the last moment, B t is the total number of pixels of dynamic obstacles at the current moment, V D is the dynamic speed, Δt is the time interval between the previous moment and the current moment, and F is the focal length of the camera.

7. The method for identifying obstacles in automatic driving of agricultural machinery according to claim 1, characterized in that: Based on the annotation box information, the depth value of the static obstacle is obtained through a recognition model, including: Constructing the recognition model; Obtaining a depth map of the marked box area through the recognition model; The depth value is calculated according to the depth map.

8. The method for identifying obstacles in automatic driving of agricultural machinery according to claim 7, characterized in that: The recognition model includes an input layer, an encoder, a bottleneck layer, a decoder and an output layer connected in sequence; The number of channels of the input layer is 3; the encoder is composed of multiple convolutional layers and activation layers, the convolutional layer is used to reduce the dimension and extract features of the output of the input layer, and the activation layer is used to perform activation training on the convolution device; the bottleneck layer is used to integrate the features extracted by the convolutional layer; The decoder is used to restore the low-dimensional features to the original spatial dimensions; and the output layer is used to output the depth map.

9. The method for identifying obstacles in automatic driving of agricultural machinery according to claim 1, characterized in that: Obtaining the static spatial relationship between the static obstacle and the agricultural machine through the depth value includes: Obtaining the distance between the static obstacle and the agricultural machinery according to the depth value; The agricultural machine motion vector is obtained by changing the two-dimensional spatial position of the agricultural machine; the agricultural machine motion vector includes the agricultural machine motion direction and the agricultural machine motion speed; The static obstacle is taken as the origin, and the static spatial relationship is obtained through the origin and the agricultural machinery motion vector.

10. An obstacle recognition system for automatic driving of agricultural machinery, characterized in that: include: An image acquisition module is used to obtain a panoramic image in the forward direction of the agricultural machinery during automatic driving; An obstacle differentiation module, for classifying obstacles into dynamic obstacles and static obstacles by differences between adjacent frames of the panoramic image; A dynamic recognition module is used to obtain a dynamic vector through the brightness of the dynamic obstacle; and obtain a dynamic spatial relationship between the dynamic obstacle and the agricultural machinery through the dynamic vector and the motion state of the agricultural machinery; Image detection module, used to obtain the annotation box information of static obstacles; The depth estimation module is used to obtain the depth value of the static obstacle through the recognition model based on the annotation box information; The static recognition module is used to obtain the static spatial relationship between static obstacles and agricultural machinery through depth values.

Images