An agricultural machine automatic driving obstacle identification method and system

Abstract

The application provides an agricultural machine automatic driving obstacle identification method, which comprises the following steps: acquiring a panoramic image in the advancing direction during the agricultural machine automatic driving process; dividing obstacles into dynamic obstacles and static obstacles through the difference between adjacent frames of the panoramic image; obtaining a dynamic vector through the brightness of the dynamic obstacles; obtaining a dynamic spatial relationship between the dynamic obstacles and the agricultural machine through the dynamic vector and the motion state of the agricultural machine; acquiring label box information of the static obstacles; obtaining a depth value of the static obstacles through an identification model based on the label box information; obtaining a static spatial relationship between the static obstacles and the agricultural machine through the depth value; and obtaining an overall spatial relationship through the dynamic spatial relationship and the static spatial relationship. The method improves the accuracy and precision of obstacle identification by distinguishing obstacles in the image and using different identification methods according to the dynamic and static states of the obstacles.

Description

Technical Field

[0001] This invention relates to the field of visual recognition technology, and in particular to a method and system for obstacle recognition in automated agricultural machinery. Background Technology

[0002] Autonomous driving technology for agricultural machinery plays a vital role in reducing agricultural labor intensity, improving work efficiency, and increasing agricultural productivity. To enable agricultural machinery to drive autonomously, it is essential to accurately perceive the surrounding environment and identify and locate obstacles. Computer vision and deep learning are key perception technologies in the field of autonomous driving, playing a crucial role in achieving effective recognition of the vehicle's surroundings.

[0003] Traditional obstacle perception methods primarily rely on a combination of machine vision, lidar, and millimeter-wave radar. However, in agricultural machinery driving environments, while radar systems excel at recognizing dynamic objects, their perception of low-speed or static obstacles is relatively poor, failing to meet the high environmental accuracy requirements of autonomous driving in agricultural machinery. This results in agricultural machinery being unable to accurately identify the location of static obstacles, leading to delayed avoidance and impacting the effectiveness of autonomous driving. Furthermore, existing technologies are not efficient or accurate in recognizing dynamic obstacles in panoramic images. Therefore, designing an obstacle recognition method and system for autonomous agricultural machinery is essential. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for obstacle recognition in agricultural machinery automatic driving. By distinguishing obstacles in images and using different recognition methods according to the dynamic and static states of the obstacles, the accuracy and precision of obstacle recognition can be improved.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] An obstacle recognition method for automated agricultural machinery includes the following steps:

[0007] Acquire panoramic images of the direction of travel during the automatic driving of agricultural machinery;

[0008] Obstacles are classified into dynamic and static obstacles based on the differences between adjacent frames of a panoramic image;

[0009] The dynamic vector is obtained by measuring the brightness of dynamic obstacles; the dynamic vector includes dynamic direction and dynamic velocity.

[0010] The dynamic spatial relationship between dynamic obstacles and agricultural machinery is obtained by using dynamic vectors and the motion state of agricultural machinery.

[0011] Obtain the bounding box information of static obstacles;

[0012] Based on the bounding box information, the depth value of static obstacles is obtained by recognizing the model;

[0013] The static spatial relationship between static obstacles and agricultural machinery can be obtained through depth values;

[0014] The overall spatial relationship is obtained through dynamic and static spatial relationships.

[0015] Optionally, obstacles are categorized into dynamic and static obstacles based on differences between adjacent frames of the panoramic image, including:

[0016] Convert each frame of the panoramic image to a grayscale image;

[0017] Perform image alignment on the grayscale images to obtain the initial image set;

[0018] Calculate the difference between any two adjacent frames in the initial image set;

[0019] Based on the comparison results between the difference and the preset difference threshold, obstacles are divided into dynamic obstacles and static obstacles.

[0020] Optionally, the dynamic vector is obtained through the brightness of the dynamic obstacle, including:

[0021] The light intensity vector is obtained by analyzing the pixel position and brightness of dynamic obstacles.

[0022] The optical flow constraint equation is constructed by using the light intensity vector, and the optical flow constraint equation is solved to obtain the optical flow vector;

[0023] The optical flow vectors are filtered and fused to obtain the dynamic vector.

[0024] Optionally, the dynamic spatial relationship between the obstacle and the agricultural machinery is obtained through dynamic vectors and the machinery's motion state, including:

[0025] Real-time positioning of agricultural machinery to obtain its two-dimensional spatial location;

[0026] The motion vector of the agricultural machinery is obtained by the change of its two-dimensional spatial position; the motion vector of the agricultural machinery includes the direction of motion and the speed of motion.

[0027] The true vector of the dynamic obstacle is obtained by the angle between the agricultural machinery's motion vector and its dynamic vector; the true vector includes: true direction and true velocity.

[0028] The distance between the dynamic obstacle and the agricultural machinery is obtained based on the dynamic vector.

[0029] The dynamic spatial relationship is obtained based on the distance between the dynamic obstacle and the agricultural machinery, the real vector, and the motion vector of the agricultural machinery.

[0030] Optionally, the true vector of the dynamic obstacle is obtained based on the angle between the agricultural machinery's motion vector and the dynamic vector, including:

[0031] The starting point of the dynamic vector is taken as the starting point of the real vector, and the ending point of the agricultural machinery movement vector is taken as the ending point of the real vector. The real direction is obtained based on the starting point and the ending point of the real vector.

[0032] When the angle between the agricultural machinery's motion vector and dynamic vector is obtuse, the formula for calculating the true speed is: Among them, V z For true speed, V D For dynamic speed, V n Let θ1 be the angle between the direction of the agricultural machinery's movement and the dynamic direction, and θ2 be the angle between the dynamic direction and the real direction.

[0033] When the angle between the agricultural machinery's motion vector and dynamic vector is acute, the formula for calculating the actual speed is:

[0034]

[0035] Optionally, the formula for calculating the distance between the dynamic obstacle and the agricultural machinery is: Among them, S DN B represents the distance between the dynamic obstacle and the agricultural machinery. t-1 B represents the total number of pixels of the dynamic obstacles in the previous time step. t V represents the total number of pixels of the dynamic obstacle at the current moment. D The dynamic velocity is denoted by Δt, which is the time interval between the previous moment and the current moment, and F is the camera focal length.

[0036] Optionally, based on the bounding box information, the depth value of the static obstacle is obtained through a recognition model, including:

[0037] Build a recognition model;

[0038] The depth map of the labeled area is obtained by recognizing the model;

[0039] The depth value is calculated based on the depth map;

[0040] Optionally, the recognition model includes an input layer, an encoder, a bottleneck layer, a decoder, and an output layer connected in sequence;

[0041] The input layer has 3 channels; the encoder consists of multiple convolutional layers and activation layers. The convolutional layers are used to reduce the dimensionality and extract features from the output of the input layer, and the activation layers are used to activate and train the convolutional layer; the bottleneck layer is used to integrate the features extracted by the convolutional layers; the decoder is used to restore the low-dimensional features to the original spatial dimension; and the output layer is used to output the depth map.

[0042] Optionally, the static spatial relationship between static obstacles and agricultural machinery can be obtained through depth values, including:

[0043] The distance between static obstacles and agricultural machinery can be determined based on the depth value;

[0044] The motion vector of the agricultural machinery is obtained by the change of its two-dimensional spatial position; the motion vector of the agricultural machinery includes the direction of motion and the speed of motion.

[0045] Using the static obstacle as the origin, the static spatial relationship is obtained through the origin and the agricultural machinery motion vector.

[0046] An obstacle recognition system for automatic agricultural machinery operation includes:

[0047] The image acquisition module is used to acquire panoramic images along the forward direction during the automatic driving process of agricultural machinery;

[0048] The obstacle differentiation module is used to classify obstacles into dynamic obstacles and static obstacles based on the differences between adjacent frames of the panoramic image;

[0049] The dynamic recognition module is used to obtain dynamic vectors by analyzing the brightness of dynamic obstacles; and to obtain the dynamic spatial relationship between dynamic obstacles and agricultural machinery by analyzing the dynamic vectors and the movement state of the agricultural machinery.

[0050] The image detection module is used to acquire bounding box information of static obstacles;

[0051] The depth estimation module is used to obtain the depth value of static obstacles based on the bounding box information and the recognition model;

[0052] The static recognition module is used to determine the static spatial relationship between static obstacles and agricultural machinery through depth values.

[0053] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: The present invention provides an obstacle recognition method for automatic agricultural machinery driving, the method comprising: acquiring a panoramic image in the forward direction during automatic agricultural machinery driving; classifying obstacles into dynamic obstacles and static obstacles based on the differences between adjacent frames of the panoramic image; obtaining a dynamic vector based on the brightness of the dynamic obstacles; obtaining the dynamic spatial relationship between the dynamic obstacles and the agricultural machinery based on the dynamic vector and the motion state of the agricultural machinery; acquiring the bounding box information of the static obstacles; obtaining the depth value of the static obstacles based on the bounding box information through a recognition model; obtaining the static spatial relationship between the static obstacles and the agricultural machinery based on the depth value; and obtaining the overall spatial relationship based on the dynamic spatial relationship and the static spatial relationship. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart of the obstacle recognition method of the present invention;

[0056] Figure 2 This is a flowchart of the method for distinguishing between dynamic and static obstacles according to the present invention;

[0057] Figure 3 This is a flowchart of the dynamic vector calculation process of the present invention;

[0058] Figure 4 This is a flowchart of the dynamic spatial relationship acquisition process of the present invention;

[0059] Figure 5 This is a flowchart of the static spatial relationship acquisition process of the present invention. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] like Figure 1 As shown, this embodiment of the invention provides a method for obstacle recognition in automated agricultural machinery, comprising the following steps:

[0063] Step 100: Acquire a panoramic image of the agricultural machinery in the forward direction during automatic driving;

[0064] Specifically, a panoramic camera (such as a fisheye lens or an array of multiple wide-angle cameras) is used as the sensing device to achieve 360-degree panoramic field of view capture and provide rich environmental information. In this embodiment, the camera is configured with a resolution of 3840x2160 pixels, ensuring rich image details and aiding in subsequent obstacle recognition and localization. The camera array in this embodiment uses multiple wide-angle cameras in a ring configuration to achieve panoramic imaging. Each camera is responsible for capturing a specific area and has a built-in image stitching algorithm to combine the images captured by each camera into a unified panoramic image.

[0065] It's worth noting that panoramic cameras can capture a 360-degree field of view in a single shot, effectively reducing blind spots and providing rich environmental information. High resolution ensures rich image detail, improving the accuracy of obstacle recognition. Furthermore, compared to sensors like LiDAR, panoramic cameras are less expensive, making them suitable for large-scale deployment and reducing the overall cost of autonomous driving systems.

[0066] Step 200: Classify obstacles into dynamic and static obstacles based on the differences between adjacent frames of the panoramic image; specific steps are as follows. Figure 2 As shown, it includes:

[0067] Step 201: Convert each frame of the panoramic image to a grayscale image;

[0068] Specifically, an image processing library (such as OpenCV) is used to read the panoramic image file and obtain each frame. Each frame is then converted to grayscale; the conversion process is expressed as: I' t (x,y)=0.2989·R(x,y)+0.5870·G(x,y)+0.1140·B(x,y); where, I' t R(x,y) represents the grayscale value, R(x,y) represents the pixel value of the red channel, G(x,y) represents the pixel value of the green channel, and B(x,y) represents the pixel value of the blue channel.

[0069] Step 202: Perform image alignment on the grayscale images to obtain an initial image set;

[0070] Specifically, grayscale images acquired every second are clustered to obtain a grayscale dataset. The first frame in the grayscale dataset is used as the anchor image, and the remaining images are used as images to be aligned. Feature points of each frame are extracted using the SIFT feature detection algorithm, and the features of each frame are matched. The 3x3 homography matrix obtained from the matched feature points using the RANSAC algorithm is used as the transformation matrix to determine the transformation relationship between the anchor image and the images to be aligned. All images to be aligned are transformed to the coordinate system of the anchor image using the homography matrix corresponding to the images to be aligned, resulting in the initial aligned image set.

[0071] Step 203: Calculate the difference between any two adjacent frames in the initial image set;

[0072] Specifically, any two adjacent frames in the initial image set are combined into an image pair, and the absolute difference between pixels in each image pair is calculated. The obstacle region in the image is divided into multiple small windows. Then, all pixels in each small window are traversed, and the average difference between the small windows is calculated.

[0073] More specifically, the formula for calculating the absolute difference is: D(x,y)=|I' t+1 (x,y)-I' t (x,y)|;where I' t+1 (x,y) represents the pixel value at the next time step, I' t (x,y) represents the pixel value at the current moment, and D(x,y) represents the absolute difference value.

[0074] The formula for calculating the average difference is: Where N is the total number of pixels in the small window, W is the width of the small window, and H is the height of the small window.

[0075] Step 204: Based on the comparison results between the difference and the preset difference threshold, the obstacles are divided into dynamic obstacles and static obstacles.

[0076] Specifically, the obstacle regions in the image are binarized. When the average difference of the small windows exceeds a preset threshold, the small window is assigned a value of 1; when the average difference of the small windows does not exceed the preset threshold, the small window is assigned a value of 0. Based on the binarization ratio of the obstacle regions, obstacles are distinguished into dynamic obstacles and static obstacles.

[0077] More specifically, the expression for the binarization operation is: Where M(x,y) is a binary value, 1 represents the moving region and 0 represents the stationary region, and Y is a threshold value. The threshold value is adjusted according to the actual working environment of the agricultural machinery. In this embodiment, the threshold value is not limited.

[0078] It should be noted that by using grayscale conversion, image alignment, and difference calculation, changes in the environment around agricultural machinery can be monitored in real time, improving the accuracy and response speed of obstacle detection. Rapid identification of dynamic obstacles ensures the safety of agricultural machinery operations and avoids potential collision risks. Rapid identification of static obstacles can be used to optimize driving paths and improve the efficiency of agricultural machinery operations.

[0079] Step 300: Obtain the dynamic vector by analyzing the brightness of the dynamic obstacle; the dynamic vector includes dynamic direction and dynamic velocity; specific steps are as follows. Figure 3 As shown, it includes:

[0080] Step 301: Obtain the light intensity vector by using the pixel position and brightness of the dynamic obstacle;

[0081] Specifically, the brightness of the same target does not change when it moves between different frames. A light intensity vector I(x,y,t) of the same pixel point is constructed in different frames, where I is the pixel brightness and t represents the time dimension of the light intensity vector.

[0082] Step 302: Construct the optical flow constraint equation using the light intensity vector, and solve the optical flow constraint equation to obtain the optical flow vector;

[0083] Specifically, the expression for the constraint equation is: in, and , , represent the brightness gradient of the pixel in the x and y directions, respectively, and u and v represent the optical flow in the horizontal and vertical directions, respectively. This represents the brightness change of a pixel over time.

[0084] More specifically, for the small window with dynamic obstacles, the constraint equations are transformed into a least-squares problem to be solved, yielding the optical flow vector. The expression for the least squares problem is: AV = 0; where,

[0085] Step 303: Filter and fuse the optical flow vectors to obtain the dynamic vector.

[0086] Specifically, a preset optical flow threshold is used to retain optical flow vectors with an activity range greater than this threshold, while those with a range less than the threshold are discarded, resulting in effective vectors. The expression for the effective vector is: Where T is the effective threshold.

[0087] It should be noted that optical flow-based analysis can track dynamic obstacles in real time and respond quickly, thereby greatly enhancing the driving safety of agricultural machinery and improving the efficiency of distinguishing different types of moving objects.

[0088] Step 400: Obtain the dynamic spatial relationship between the obstacle and the agricultural machinery through dynamic vectors and the machinery's motion state; specific steps are as follows. Figure 4 As shown, it includes:

[0089] Step 401: Perform real-time positioning of the agricultural machinery to obtain its two-dimensional spatial position;

[0090] Specifically, a positioning device is installed on the agricultural machinery to obtain the real-time position coordinates of the agricultural machinery in two-dimensional space.

[0091] Step 402: Obtain the agricultural machinery motion vector by measuring the change in the two-dimensional spatial position of the agricultural machinery; the agricultural machinery motion vector includes the direction of agricultural machinery motion and the speed of agricultural machinery motion;

[0092] Specifically, a two-dimensional coordinate system is established on the ground. The position of the agricultural machinery in the first frame of a one-second image time frame is taken as the starting point of the vector, and the position of the agricultural machinery in the last frame is taken as the ending point of the vector. Connecting the position coordinates of the starting point and the ending point yields the agricultural machinery's motion vector. The speed of the agricultural machinery is obtained by dividing the magnitude of the vector by the time interval between the starting and ending points, and the direction of the vector is the direction of the agricultural machinery's motion.

[0093] Step 403: Obtain the true vector of the dynamic obstacle based on the angle between the agricultural machinery's motion vector and the dynamic vector; the true vector includes: true direction and true velocity;

[0094] Specifically, the dynamic vector represents the relative motion state of the obstacle to the agricultural machinery, calculated with the machinery itself as a fixed target; it is not the actual motion state. The true vector of the dynamic obstacle is calculated by the relative relationship between the machinery's motion vector and the dynamic vector.

[0095] Furthermore, the starting point of the agricultural machinery motion vector and the ending point of the dynamic vector are transformed into the same position coordinates. The starting point of the dynamic vector is taken as the starting point of the real vector, and the ending point of the agricultural machinery motion vector is taken as the ending point of the real vector. The starting point and ending point of the real vector are connected to obtain the real direction.

[0096] The actual speed is determined by the angle between the directions of the agricultural machinery's motion vector and dynamic vector; the actual speed is greater or less than the dynamic speed. The specific calculation standards for the actual speed are as follows:

[0097] When the angle between the agricultural machinery's motion vector and dynamic vector is obtuse, the formula for calculating the true speed is:

[0098] When the angle between the agricultural machinery's motion vector and dynamic vector is acute, the formula for calculating the actual speed is: Among them, V z For true speed, V D For dynamic speed, V nLet θ1 be the speed of the agricultural machinery, θ2 be the angle between the direction of the agricultural machinery's movement and the dynamic direction, and θ3 be the angle between the dynamic direction and the real direction.

[0099] Step 404: Obtain the distance between the dynamic obstacle and the agricultural machinery based on the dynamic vector;

[0100] Specifically, the distance between the dynamic obstacle and the agricultural machinery is calculated based on the scaling ratio of the dynamic obstacle over a certain time interval. The scaling ratio of the dynamic obstacle is calculated by the difference in the number of pixels occupied by the dynamic obstacle in different frames. The formula for calculating the distance between the dynamic obstacle and the agricultural machinery is:

[0101]

[0102] Among them, S DN B represents the distance between the dynamic obstacle and the agricultural machinery. t-1 B represents the total number of pixels of the dynamic obstacles in the previous time step. t V represents the total number of pixels of the dynamic obstacle at the current moment. D The dynamic velocity is denoted by Δt, which is the time interval between the previous moment and the current moment, and F is the camera focal length.

[0103] Step 405: Obtain the dynamic spatial relationship based on the distance between the dynamic obstacle and the agricultural machinery, the real vector, and the motion vector of the agricultural machinery.

[0104] Specifically, a dynamic spatial relationship model is constructed based on the distance between the dynamic obstacle and the agricultural machinery, the motion vector of the agricultural machinery, and the actual vector of the dynamic obstacle. This model includes: the distance relationship, velocity relationship, and direction of motion relationship between the dynamic obstacle and the agricultural machinery. The distance relationship describes the physical distance between the two, the velocity relationship is used to determine the relative velocity between the agricultural machinery and the obstacle, and the direction relationship is used to analyze the angle between the two motion directions and, combined with the velocity relationship, to determine the potential collision risk.

[0105] It should be noted that real-time positioning and motion vector calculation enable precise monitoring of agricultural machinery movement, ensuring safe operation. The combination of dynamic vectors and motion status allows for accurate assessment of potential collision risks, enabling timely avoidance measures and significantly improving the safety of autonomous driving. Representing the dynamic spatial relationship between dynamic obstacles and agricultural machinery using a dynamic spatial relationship model greatly enhances the intelligence level of the autonomous driving system.

[0106] Step 500: Obtain the bounding box information of static obstacles;

[0107] Specifically, this embodiment uses the YOLO object detection algorithm to obtain the bounding box coordinates and category labels of static obstacles.

[0108] Step 600: Based on the bounding box information, obtain the depth value of the static obstacle by recognizing the model;

[0109] Specifically, a recognition model is first constructed, then a depth map of the labeled area is obtained through the recognition model, and finally the depth value is calculated based on the depth map.

[0110] More specifically, the recognition model comprises an input layer, an encoder, a bottleneck layer, a decoder, and an output layer connected in sequence. The input layer receives the input image, typically in RGB format, with a height of H, a width of W, and 3 channels. The encoder consists of multiple convolutional layers and activation layers. The convolutional layers are used to reduce the dimensionality and extract features from the input layer's output, while the activation layers are used to train the convolutional layers. The encoder uses the output of the input layer as input, extracting high-level features from the input image, progressively reducing the image's spatial dimensions (height and width) and increasing the depth of the feature layers. The encoder's layer distribution has multiple hierarchical distributions, with the initial layers using a small number of filters, gradually increasing the number of filters as the layers deepen. Pooling layers are connected after the convolutional layers, further reducing the size of the feature maps through max pooling while preserving key feature information to reduce computational cost. The bottleneck layer integrates the features extracted by the convolutional layers, avoiding overcomputation. The decoder restores the low-dimensional features to their original spatial dimensions using a built-in transposed convolutional layer. A skip connection is used between the encoder and decoder, directly passing the output of certain layers of the encoder to the decoder to help the model recover clearer depth information. Skip connections effectively reduce information loss and preserve high-resolution details. The output layer processes the decoded feature map into depth values ​​for each pixel through a built-in convolutional layer, thus obtaining a depth map. The depth map is a single-channel image where the value of each pixel represents the actual distance from the camera to that pixel.

[0111] Step 700: Obtain the static spatial relationship between static obstacles and agricultural machinery using depth values; specific steps are as follows: Figure 5 As shown, it includes:

[0112] Step 701: Calculate the distance between the static obstacle and the agricultural machinery based on the depth value;

[0113] Specifically, the formula for calculating the distance between a static obstacle and agricultural machinery is as follows:

[0114] Where D is the distance between the static obstacle and the agricultural machinery, (X d Y d (X) represents the two-dimensional spatial coordinates represented by the depth value. j Y j ) represents the two-dimensional spatial coordinates of the agricultural machinery.

[0115] Step 702: Obtain the agricultural machinery's motion vector by observing the changes in its two-dimensional spatial position; the agricultural machinery's motion vector includes the direction of motion and the speed of motion. This is the same method used in step 402.

[0116] Step 703: Using the static obstacle as the origin, obtain the static spatial relationship through the origin and the agricultural machinery motion vector.

[0117] Specifically, a static two-dimensional coordinate system is constructed with the static obstacle as the origin. The corresponding coordinates of the agricultural machinery in the static two-dimensional coordinate system are calculated based on the distance between the static obstacle and the agricultural machinery and the movement vector of the agricultural machinery. The real-time relative positional relationship between the agricultural machinery and the static obstacle is obtained through real-time updates of the coordinates.

[0118] Step 800: Obtain the overall spatial relationship through dynamic and static spatial relationships.

[0119] Specifically, the coordinates of agricultural machinery, static obstacles, and dynamic obstacles in dynamic and static spatial relationships are represented in the same coordinate system to achieve overall identification of dynamic and static obstacles.

[0120] The present invention also provides an obstacle recognition system for automatic agricultural machinery driving, comprising:

[0121] The image acquisition module is used to acquire panoramic images along the forward direction during the automatic driving process of agricultural machinery;

[0122] The obstacle differentiation module is used to classify obstacles into dynamic obstacles and static obstacles based on the differences between adjacent frames of the panoramic image;

[0123] The dynamic recognition module is used to obtain dynamic vectors by analyzing the brightness of dynamic obstacles; and to obtain the dynamic spatial relationship between dynamic obstacles and agricultural machinery by analyzing the dynamic vectors and the movement state of the agricultural machinery.

[0124] The image detection module is used to acquire bounding box information of static obstacles;

[0125] The depth estimation module is used to obtain the depth value of static obstacles based on the bounding box information and the recognition model;

[0126] The static recognition module is used to determine the static spatial relationship between static obstacles and agricultural machinery through depth values.

[0127] The beneficial effects of this invention are as follows:

[0128] 1) By acquiring the location information of static and dynamic obstacles in real time, the distance between the vehicle and the obstacle can be accurately determined, and potential collisions can be avoided in time, greatly improving driving safety;

[0129] 2) Advanced intelligent technology is adopted to realize real-time location monitoring and status analysis of agricultural machinery in complex environments, which can respond to dynamic changes in a timely manner and greatly improve work efficiency.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0131] Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. Furthermore, those skilled in the art will recognize that, based on the ideas of this invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for obstacle recognition in automated agricultural machinery, characterized in that, Includes the following steps: Acquire panoramic images of the direction of travel during the automatic driving of agricultural machinery; Obstacles are classified into dynamic obstacles and static obstacles based on the differences between adjacent frames of the panoramic image; A dynamic vector is obtained by measuring the brightness of the dynamic obstacle; the dynamic vector includes dynamic direction and dynamic velocity. The dynamic spatial relationship between the dynamic obstacle and the agricultural machinery is obtained through the dynamic vector and the motion state of the agricultural machinery; The specific steps include: The agricultural machinery is located in real time to obtain its two-dimensional spatial position. The agricultural machinery motion vector is obtained by the change of its two-dimensional spatial position; the agricultural machinery motion vector includes the agricultural machinery motion direction and the agricultural machinery motion speed; The true vector of the dynamic obstacle is obtained based on the angle between the agricultural machinery's motion vector and the dynamic vector; the true vector includes: true direction and true velocity; the specific steps include: taking the starting point of the dynamic vector as the starting point of the true vector, taking the ending point of the agricultural machinery's motion vector as the ending point of the true vector, and obtaining the true direction based on the starting point and the ending point of the true vector; when the angle between the agricultural machinery's motion vector and the dynamic vector is an obtuse angle, the formula for calculating the true velocity is: ;in, For actual speed, For dynamic speed, For the speed of agricultural machinery movement, The angle between the direction of movement and the dynamic direction of the agricultural machinery. The angle between the dynamic direction and the true direction; when the angle between the agricultural machinery motion vector and the dynamic vector is acute, the formula for calculating the true speed is: ; The distance between the dynamic obstacle and the agricultural machinery is obtained based on the dynamic vector; the formula for calculating the distance between the dynamic obstacle and the agricultural machinery is: ;in, The distance between the dynamic obstacle and the agricultural machinery. This represents the total number of pixels of the dynamic obstacles at the previous time step. This represents the total number of pixels of the dynamic obstacle at the current moment. For dynamic speed, The time interval between the previous moment and the current moment. The focal length of the camera; The dynamic spatial relationship is obtained based on the distance between the dynamic obstacle and the agricultural machinery, the real vector, and the motion vector of the agricultural machinery; Obtain the bounding box information of the static obstacle; Based on the labeled box information, the depth value of the static obstacle is obtained through a recognition model; The static spatial relationship between the static obstacle and the agricultural machinery can be obtained through the depth value; The overall spatial relationship is obtained through the dynamic spatial relationship and the static spatial relationship.

2. The obstacle recognition method for automatic agricultural machinery operation according to claim 1, characterized in that, Obstacles are categorized into dynamic and static obstacles based on the differences between adjacent frames of the panoramic image, including: Each frame of the panoramic image is converted into a grayscale image; The grayscale images are then aligned to obtain an initial image set. Calculate the difference between any two adjacent frames in the initial image set; Based on the comparison result between the difference and the preset difference threshold, the obstacle is divided into the dynamic obstacle and the static obstacle.

3. The obstacle recognition method for automatic agricultural machinery operation according to claim 1, characterized in that, The dynamic vector is obtained by measuring the brightness of the dynamic obstacle, including: The light intensity vector is obtained by analyzing the pixel positions and brightness of the dynamic obstacle. An optical flow constraint equation is constructed using the light intensity vector, and the optical flow constraint equation is solved to obtain the optical flow vector. The optical flow vectors are filtered and fused to obtain the dynamic vector.

4. The obstacle recognition method for automatic agricultural machinery operation according to claim 1, characterized in that, Based on the labeled bounding box information, the depth value of the static obstacle is obtained through a recognition model, including: Construct the recognition model; The depth map of the labeled region is obtained through the recognition model; The depth value is calculated based on the depth map.

5. The obstacle recognition method for automatic agricultural machinery operation according to claim 4, characterized in that, The recognition model comprises an input layer, an encoder, a bottleneck layer, a decoder, and an output layer connected in sequence. The input layer has 3 channels; the encoder consists of multiple convolutional layers and activation layers, the convolutional layers are used to reduce the dimensionality and extract features from the output of the input layer, and the activation layers are used to activate and train the convolutional layers; the bottleneck layer is used to integrate the features extracted by the convolutional layers. The decoder is used to restore low-dimensional features to their original spatial dimensions; The output layer is used to output the depth map.

6. The obstacle recognition method for automatic agricultural machinery operation according to claim 1, characterized in that, The static spatial relationship between the static obstacle and the agricultural machinery can be obtained through the depth value, including: The distance between the static obstacle and the agricultural machinery can be obtained based on the depth value; The agricultural machinery motion vector is obtained by the change of its two-dimensional spatial position; the agricultural machinery motion vector includes the agricultural machinery motion direction and the agricultural machinery motion speed; The static spatial relationship is obtained by taking the static obstacle as the origin and the agricultural machinery motion vector through the origin.

7. An obstacle recognition system for automated agricultural machinery, used to implement the obstacle recognition method for automated agricultural machinery as described in any one of claims 1-6, characterized in that, include: The image acquisition module is used to acquire panoramic images along the forward direction during the automatic driving process of agricultural machinery; The obstacle differentiation module is used to classify obstacles into dynamic obstacles and static obstacles based on the differences between adjacent frames of the panoramic image; The dynamic recognition module is used to obtain dynamic vectors by analyzing the brightness of dynamic obstacles; and to obtain the dynamic spatial relationship between dynamic obstacles and agricultural machinery by analyzing the dynamic vectors and the movement state of the agricultural machinery. The image detection module is used to acquire bounding box information of static obstacles; The depth estimation module is used to obtain the depth value of static obstacles based on the bounding box information and the recognition model; The static recognition module is used to determine the static spatial relationship between static obstacles and agricultural machinery through depth values.

Images