Grain transport vehicle-oriented vehicle-mounted visual tractor track passable area evaluation method

Abstract

The invention provides a grain transport truck-oriented vehicle-mounted visual tractor lane passable area evaluation method, which comprises the following steps: S1, constructing a binocular-grain transport truck-environment low-blind-area multi-view model: differentially constructing a grain transport truck visual information acquisition system, acquiring images and constructing a data set; an image-camera-world coordinate system is established, and environment three-dimensional space information is obtained; s2, non-structural tractor track area and obstacle detection: using a transparency mask to remove redundancy; detecting a tractor track area and an obstacle by adopting a weight sharing lightweight multistage deep network model, and calculating three-dimensional space information of the obstacle; and S3, assessing the trafficability of the tractor track in front of the grain transport vehicle: designing a self-adaptive transverse dynamic grid partitioning method, preliminarily judging the trafficability of the tractor track in combination with multi-source information, and creating a comprehensive scoring system to assess the trafficability of the tractor track region. According to the method, the defects of the traditional technology in the aspects of low blind area coverage, obstacle recognition and trafficability evaluation are effectively overcome.

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving of grain transport vehicles based on machine vision, and more specifically to a method for evaluating the passable area of ​​a mechanized farming road based on vehicle-mounted vision for grain transport vehicles. Background Art

[0002] In recent years, with the continuous advancement of agricultural modernization, the role of grain trucks in agricultural production has become increasingly prominent. As an important part of agricultural machinery, grain trucks not only bear the heavy task of large-scale agricultural product transportation, but also play an irreplaceable role in improving agricultural operation efficiency and reducing labor intensity. However, the traditional manual driving method can no longer meet the needs of modern agriculture for efficient and precise management. Therefore, the intelligent transformation of grain trucks has become a research hotspot in the field of agricultural automation.

[0003] In the research of intelligent grain transport vehicles, the on-board vision system is one of the key technologies. The on-board vision system can accurately detect road conditions and identify obstacles by perceiving the surrounding environment in real time, thereby ensuring the safety and efficiency of vehicle driving. However, due to the unstructured characteristics of the road environment in rural areas, the detection of the passage area of ​​tractor roads faces complex and changing environmental conditions, such as the blind spot problem of low-floor vehicles, the variety of obstacles, and the uncertainty of passability assessment. These problems put forward higher real-time and accuracy requirements for existing technologies.

[0004] At present, the methods for detecting road drivable areas are mainly divided into three categories: methods based on road features, methods based on road models, and methods based on deep learning.

[0005] ① Method based on road characteristics:

[0006] The detection of drivable areas is achieved by extracting the salient features of the road (such as color, texture, and edges). Typical techniques of this method include threshold segmentation and edge detection. For example, by converting an RGB image to an HSV image and using the S component to perform an inverse perspective transform, the road surface area can be detected in physical space. However, for unstructured mechanized farming roads with small color differences or unclear texture features, this method has poor detection effects.

[0007] ② Road model-based method:

[0008] Assuming that the road edge has a certain regularity, the road area and non-road area are distinguished by establishing an edge structure model. For example, the road vanishing point detection method based on video images uses the Harris corner point optimization score function to reduce the computational burden, and combines the optical flow method to extract motion feature points, and determines the road area through the optimized clustering algorithm. However, this method is less applicable in unstructured road scenes with fuzzy boundaries and unclear structures.

[0009] ③Methods based on deep learning:

[0010] With the rapid development of deep learning technology, deep learning models based on semantic segmentation have shown superior performance in complex scenarios. For example, a bilateral network model that uses a lightweight network (such as ResNet18) combined with an attention mechanism can not only extract deep features, but also enhance image detail information. Deep learning methods, with their high robustness to environmental changes, can effectively cope with complex and changeable unstructured road scenes, and have become the mainstream direction of current research.

[0011] Although deep learning methods have shown good results in unstructured road detection, their application in grain truck scenarios still faces many challenges. For example, the variety of obstacle types in mechanized farming roads, the large blind spots of vehicle chassis, and the complexity of unstructured roads make it difficult to simultaneously meet the real-time and accuracy of trafficability assessment. Therefore, there is an urgent need for a traffic area assessment method that can comprehensively consider the complexity of the mechanized farming road environment and the diversity of obstacles to provide more reliable support for the autonomous driving of grain trucks. Summary of the invention

[0012] The present invention proposes a vehicle-mounted visual mechanized road passable area assessment method for grain transport vehicles, which aims to solve the problems of low chassis, large blind spots of small-sized vehicles, unstructured, complex and changeable mechanized roads, diverse obstacles, inaccurate passability assessment and low decision-making accuracy. It can be widely used in autonomous grain transport vehicles in the field of agricultural automation, has high practical value and broad application prospects, and can provide important technical support for assisted automatic driving of grain transport vehicles in the field of agricultural automation. The specific scheme is as follows:

[0013] A method for evaluating the traversable area of ​​a mechanized farming road using on-board visual technology for a grain transport vehicle, the method comprising the following steps:

[0014] S1. Construction of low blind spot multi-view model of binocular-grain truck-environment: Four depth cameras are installed at different angles and heights on the top four sides of the grain tank of the grain truck through adjustable brackets to collect images of the tractor road and obstacles and build a data set; based on the positional relationship between the camera, grain truck and environment, an image-camera-world coordinate system is established to obtain the three-dimensional spatial information of the environment;

[0015] S2. Unstructured tractor road area and obstacle detection: A transparency mask is used to remove the background area above the vanishing line in the tractor road, and a lightweight multi-level deep network model with weight sharing is proposed. The RGB image and depth image are used as input to simultaneously detect the visible road surface in the tractor road area and identify obstacles; the three-dimensional spatial information of the obstacle is calculated by combining the image-camera-world coordinate system and the internal and external parameters of the camera;

[0016] S3. Assessment of the passability of the mechanized road ahead of the grain truck: An adaptive lateral dynamic grid partitioning method is constructed based on the speed of the grain truck and the width of the mechanized road. The multi-source information of the grain truck’s own characteristics, the mechanized road characteristics, and the attributes of obstacles is combined to preliminarily judge the passability of the mechanized road. A comprehensive scoring system is created and the impact of multi-source attributes on the regional accessibility of the mechanized road is quantitatively analyzed to further divide the accessibility level of the mechanized road.

[0017] Furthermore, S3 includes the following sub-steps:

[0018] S301, constructing adaptive horizontal dynamic grid partitioning: by sensing the width of the mechanized farming road in real time, the horizontal partition size of the grid is adjusted proportionally; at the same time, based on the movement speed of the grain transport vehicle, the time interval and density of the longitudinal grid are dynamically adjusted: when the vehicle speed is high, the grid division interval is reduced to enhance the responsiveness to obstacles and changes in the mechanized farming road; when the speed is low, the grid interval is increased to reduce the computational complexity;

[0019] S302. Initial judgment of passability based on multi-source information of grain transport vehicle and environment: The multi-source information of grain transport vehicle and environment is integrated for preliminary evaluation. The RGBD image of the mechanized farming road is input. First, it is judged whether the maximum slope value of the mechanized farming road is less than the climbing limit value of the grain transport vehicle. According to the type of obstacles, all hard obstacles in the image are removed, and the remaining mechanized farming road area in the image after the hard obstacles are removed is adaptively partitioned into horizontal dynamic grids. Secondly, the relationship between the characteristics of the grain transport vehicle itself, the characteristics of the mechanized farming road, and the attributes of the obstacles in each grid is judged in turn to make initial judgments on different passabilities.

[0020] S303. Evaluate the tractor road passable area based on the tractor road-obstacle attributes: After completing the initial judgment, create a comprehensive scoring system based on the attributes of the slope of the tractor road, the size and height of the obstacles, quantitatively analyze the impact of the attributes on the passability of the tractor road area, and further divide the passability of the tractor road area for grain transport vehicles into ten discrete evaluation levels, thereby realizing the on-board visual tractor road passable area evaluation for grain transport vehicles.

[0021] Furthermore, S2 uses a transparency mask to obtain the vanishing line position through depth estimation, removes the background area above the vanishing line, and retains only the mechanized farming road and related areas to reduce image redundancy; the weight-sharing lightweight multi-level deep network model includes semantic segmentation and target detection, and the two networks share the weights of the shallow feature extraction layer; the mechanized farming road passable area mask is extracted through the semantic segmentation network, and obstacles are identified in the target detection network based on the shared network weights. At the same time, the data source of the network model is a multi-channel image fused with RGB and depth.

[0022] Furthermore, the three-dimensional spatial information of the obstacle described in S2 is obtained through the binocular vision principle, the disparity map is used to construct the depth image, and each pixel point is converted into the actual space coordinate in combination with the camera calibration parameters to calculate the three-dimensional position, size and suspension state of the obstacle.

[0023] Furthermore, the S303 tractor road-obstacle attribute calculation method is as follows:

[0024]

[0025] In formula (1), Δh represents the height difference of the mechanized farming road within the partition, Δd represents the horizontal distance of the mechanized farming road within the partition, and F1 is the slope;

[0026] In formula (2), N2 is the number of obstacles, t i is the size fraction of the ith obstacle, and F2 is the size of the obstacle;

[0027] In formula (3), N3 represents the number of obstacles, h i represents the height of the ith obstacle, and F3 is the height of the obstacle;

[0028] In formula (4), S is the comprehensive score, n is the total number of attributes, and W i is the weight of the i-th attribute, F i is the score of the i-th attribute.

[0029] Furthermore, the judging step in S302 is:

[0030] S302a: First, determine whether the maximum value of the slope of the mechanized farming road exceeds the maximum climbing capacity of the grain transport vehicle: if it exceeds, the area ahead is an impassable area; if not, proceed to the next step of determination;

[0031] S302b: Determine whether the area contains hard obstacles: if so, remove the area where the obstacles are located and proceed to the next step of determination; perform adaptive horizontal dynamic grid partitioning on the mechanized farming road area that does not contain hard obstacles, and perform determination on each area after partitioning;

[0032] S302c: Determine whether there are obstacles in each area. If not, end the preliminary determination of passability. If yes, proceed to the next step of determination.

[0033] S302d: Determine whether the obstacle is a suspended obstacle. If so, determine whether the distance between the bottom of the suspended obstacle and the mechanized farming road is greater than the height of the grain transport vehicle, so as to determine whether the grain transport vehicle can pass through. If not, proceed to the next step of determination.

[0034] S302e: Determine whether there is a road area larger than the width of the grain transport vehicle. If yes, end the preliminary determination of drivability. If no, proceed to the next step of determination.

[0035] S302f: Determine whether there are multiple road areas in the area that are larger than the track width of the grain transport vehicle. If not, the area is impassable. If so, proceed to the next step of determination.

[0036] S302g: Determine whether there are two areas that meet the conditions for grain truck tracks to pass, that is, the maximum distance between areas > the outer width of the track, and the minimum distance < the inner width of the track, and the height of the obstacles between areas is lower than the chassis of the grain truck. If so, the preliminary judgment of passability ends. If not, the area is impassable. The above steps can complete the preliminary judgment of the passability of the mechanized farming road and eliminate the impassable area with the highest probability.

[0037] Furthermore, the passability is divided into multiple levels: through the analysis of a large amount of historical data and actual experience, the threshold of each scoring segment is determined through statistical analysis. According to the score, the final comprehensive score is finally divided into 10 discrete evaluation levels, from 0 to 9, representing different pass weights: 0-2, easy to pass; 3-4, slightly difficult; 5-6, relatively difficult; 7-8, very difficult; 9, impossible to pass.

[0038] Furthermore, the characteristics of the grain transport vehicle in S3 include size and maximum climbing ability; the characteristics of the mechanized farming road include slope and width; the attributes of obstacles include presence, size, hardness, height, whether they are suspended, and the distance from the bottom of the mechanized farming road when suspended.

[0039] Beneficial effects:

[0040] The present invention provides a vehicle-mounted visual mechanized farming road passable area assessment method for grain transport vehicles, which specifically includes the following advantages:

[0041] (1) Aiming at the characteristics of large size and high chassis of grain transport trucks, the present invention innovatively designs a differentiated visual information acquisition system. Compared with the traditional 360° surround view camera, the present invention uses stereo vision technology to construct a low blind spot multi-view model. By flexibly adjusting the number, position and angle of the cameras and rationally arranging four binocular cameras, comprehensive coverage and accurate perception of mechanized farming roads and obstacles are achieved. Seamless coverage of the detection area is achieved through overlapping fields of view, effectively eliminating blind spots. It exhibits higher robustness and detection efficiency in different scenarios, providing strong support for the autonomous driving of grain transport trucks. This design significantly improves the reliability and detection accuracy of the visual system, and enhances the autonomous navigation capability of grain transport trucks in complex rural environments.

[0042] (2) In view of the complex structure and diverse obstacles of mechanized farming roads in field environments, the present invention proposes a lightweight multi-level deep network model with shared weights. The background area is eliminated by using a transparency mask, and only the mechanized farming roads and obstacles are focused on, effectively reducing the computational burden. Combining RGB and depth images, the model can obtain richer spatial information and achieve accurate delineation and three-dimensional modeling of obstacle boundaries. The obstacle elevation model is established using a point-line method, which enhances the adaptability to complex environments. While maintaining real-time performance, this method improves the accuracy of obstacle detection and recognition, providing accurate support for grain truck path planning in complex scenarios.

[0043] (3) The traditional road passable area detection method has a single information source and evaluation criteria, which is difficult to quantitatively analyze. In the complex and changeable mechanized farming roads, it can often only give a simple "passable" or "impassable" conclusion. This binary judgment mode lacks flexibility and is difficult to reflect the difficulty and risk level of passage. Grain transport vehicles are prone to unnecessary stagnation or risks in actual operation. Therefore, the present invention proposes an adaptive horizontal dynamic grid partitioning method. In each partition, the passability of the mechanized farming road area is initially judged by combining the size and maximum climbing ability of the grain transport vehicle, the slope and width of the mechanized farming road, and the presence, size, hardness, height, whether the obstacle is suspended, and the distance between the bottom of the suspended object and the mechanized farming road. The impassable area with the highest probability is eliminated. Then, a comprehensive scoring system is created based on the slope of the mechanized farming road, the size and height of the obstacle, and the impact of multiple source attributes on the passability of the mechanized farming road area is quantitatively analyzed. The passability of the grain transport vehicle to the mechanized farming road area is further divided to realize the on-board visual mechanized farming road passable area evaluation for grain transport vehicles.

[0044] This invention effectively solves the shortcomings of traditional technologies in low blind area coverage, obstacle identification and trafficability assessment. Through the coordinated application of differentiated visual information acquisition system, multi-level deep network model and comprehensive scoring system, the detection accuracy and trafficability analysis ability of grain transport vehicles in complex unstructured mechanized farming road environments have been significantly improved, providing important technical support for the intelligent development of agricultural machinery. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The present invention is a flowchart of a method for evaluating the traversable area of ​​a mechanized farming road using on-board vision for grain transport vehicles.

[0046] Figure 2 It is a low blind spot multi-view system diagram of binoculars-grain transport truck-environment.

[0047] Figure 3 It is the original image of the tractor road.

[0048] Figure 4 This is the result of the mechanized farming road inspection.

[0049] Figure 5 It is the obstacle detection result map.

[0050] Figure 6 It is a three-dimensional image of the obstacle.

[0051] Figure 7 It is a flowchart for the overall assessment of the area accessible by mechanized farming roads.

[0052] Figure 8 It is a schematic diagram of linear division of horizontal partitions. Description of the drawings:

[0054] 1. Front camera; 2. Right camera; 3. Rear camera; 4. Left camera; 5. Adjustable bracket; 6. Grain transport vehicle. DETAILED DESCRIPTION

[0055] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with examples. The examples are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.

[0056] Example:

[0057] The present invention discloses a vehicle-mounted visual mechanized farming road traversable area assessment method for grain transport vehicles, so as to solve the problems of large detection blind spots and inaccurate traversability assessment of large-sized agricultural machinery caused by the unstructured, complex and changeable mechanized farming roads and various obstacles. The method can be widely used in the assisted automatic driving of grain transport vehicles in the field of agricultural automation.

[0058] Figure 1 This is a flowchart of a method for evaluating the traversable area of ​​a grain transporter's mechanized farming road using on-board visual technology. Figure 1 As shown, the method comprises the following steps:

[0059] S1. Construction of low blind area multi-view model of binocular-grain transport vehicle-environment:

[0060] Construct a low blind spot multi-perspective model of binocular-grain truck-environment. According to the autonomous driving requirements of the grain truck 6, a differentiated grain truck visual information acquisition system is constructed. Four depth cameras are installed at different angles and heights on the top four sides of the grain box of the grain truck 6 through an adjustable bracket 5 to collect images of the mechanized farming roads and obstacles in the environment where the grain truck 6 is located, and construct a mechanized farming road and obstacle data set; at the same time, based on the position relationship between the camera, the grain truck 6 and the environment, an image-camera-world coordinate system is constructed to obtain the three-dimensional spatial information of the environment. The main steps are as follows:

[0061] Four high-resolution RGBD binocular cameras are installed on the left, right, front, and rear of the top of the grain tank of the grain truck 6 to ensure that there are no blind spots in the surrounding field of view: Front camera 1: installed 12.5 cm in front of the grain truck 6, with a depression angle of 23° and a resolution of 2K. Rear camera 3: installed 5 cm behind the grain truck 6, with a depression angle of 35° and a resolution of 1K. Left and right cameras 2: symmetrically installed 1.5 cm on the side of the grain truck 6, with a depression angle of 43° and a resolution of 1K. Each camera is installed at a height of 36 cm to ensure that the surrounding fields of view overlap and meet the requirements of multi-view data collection.

[0062] Figure 3 is the original image of the tractor road, such as Figure 3 As shown in the figure, images of the mechanized farming roads and obstacles in the environment where the grain transport vehicle is located are collected to construct and expand the mechanized farming roads and obstacle datasets.

[0063] Figure 2 It is a binocular-grain truck 6-environment low blind spot multi-view system, such as Figure 2 As shown, it includes a front camera 1; a right camera 2; a rear camera 3; a left camera 4; an adjustable bracket 5; and a grain truck 6. Based on the positional relationship between the camera, the grain truck 6, and the environment, an image-camera-world coordinate system is constructed. The world coordinate system is located in front of the grain truck 6, and the grain truck 6 coordinate system is located at the center of the chassis of the grain truck 6. The spatial position of each camera is determined by calibration and actual measurement of the position difference, laying the foundation for obtaining the three-dimensional spatial information of the environment based on binocular vision.

[0064] S2. Non-structured mechanized farming road area and obstacle detection within the area:

[0065] A transparency mask is used to remove the background area above the vanishing line in the mechanized farming road to reduce image redundancy; a multi-level deep network model for detecting traversable areas with weight sharing and lightweight is proposed, and both the RGB image and the depth image data obtained by the depth camera are used as input data sources to detect the visible road surface in the mechanized farming road area, and further detect obstacles based on the visible road surface area of ​​the mechanized farming road by merging shared weights; at the same time, the three-dimensional spatial information of obstacles is calculated by combining the image-camera-world coordinate system and the internal and external parameters of the camera. The main steps are as follows:

[0066] 2.1: Transparency masking refers to obtaining the vanishing line position through depth estimation, using the transparency mask to remove the background information above the vanishing line, and only retaining the tractor road and the related areas above it.

[0067] 2.2: A lightweight multi-level deep network model with weight sharing, including semantic segmentation and object detection. The two networks share the weights of the shallow feature extraction layer. Figure 4As shown in Figure 1, the model first detects the visible road surface of the mechanized farming road through a semantic segmentation network, and then uses the segmentation network to extract the passable area mask of the mechanized farming road. Figure 5 As shown in the figure, based on the shared network weights, the target detection network quickly identifies obstacles based on the detected visible road mask, thereby achieving accurate detection of obstacles without increasing the computational burden, and improving the lightweight and real-time performance of the system. In view of the characteristics of obstacles on the mechanized farming road having different distances, heights and sizes, the present invention uses the multi-channel image combining RGB and depth images as the data source of the multi-level deep network model to further enhance the detection accuracy.

[0068] 2.3: If Figure 6 As shown, the core of the three-dimensional spatial information of the obstacle lies in the acquisition of depth information. The present invention uses the binocular vision principle to construct a disparity map using the horizontal position difference of the same object captured by the left and right cameras in the image. According to the internal and external parameters of the camera obtained by calibration, the disparity map is converted into a depth image to obtain the depth information of each pixel in the scene. Based on the depth information, each pixel in the RGB image is converted into the actual space coordinates in the world coordinate system to represent the three-dimensional position of the obstacle, with the coordinates of the center point of the obstacle used as the position identifier. The obstacle outline is outlined by dots and lines, and a three-dimensional model of the obstacle is constructed to calculate its size, height and distance parameters from the bottom of the suspended state to the mechanized farming road, thereby improving the accuracy and stability of obstacle detection. The main steps are as follows:

[0069] 2.3.1. Obtain the disparity images of the left and right cameras: According to the principle of binocular vision, in the disparity map, the corresponding positions of the same object in the left and right camera images will have different horizontal positions. The disparity map can be obtained by calculating the horizontal displacement difference d of the object points on the left and right camera images.

[0070] 2.3.2. Calculate depth information based on disparity map: Based on the disparity map and the basic parameters of the camera (baseline length b, focal length f, etc.), the depth image is calculated using the formula. The formula is as follows:

[0071]

[0072] 2.3.3. Convert the depth information and the calibration parameters of the binocular camera into actual coordinates: Convert the depth image into a point cloud based on the depth information and the calibration parameters of the binocular camera. The coordinates of each point in the point cloud can be calculated by the formula as follows:

[0073]

[0074] Among them, (X w ,Y w ,Z w ) is a point in the world coordinate system; Z cis the depth in the depth image; K1 is the camera intrinsic parameter, K2 is the camera extrinsic parameter; (u, ν) is the pixel coordinate in the image coordinate system.

[0075] 2.4. Extract the location information and three-dimensional coordinate features of the obstacle: Use the above conversion formula to convert the RGB image pixels of the obstacle into pixel coordinates in the world coordinate system to obtain the location information and elevation image of the obstacle.

[0076] 2.5. Detect elevation differences: Use the grayscale histogram scanning method to obtain the maximum elevation value H corresponding to each obstacle M and the mean elevation H of the ground area D , by calculating the difference between the maximum elevation value corresponding to the obstacle and the average elevation value corresponding to the ground area, as the differentiated height of each obstacle area, the formula is as follows:

[0077] H DS =H M -H D

[0078] Among them, H DS is the height difference; H M is the maximum height of the obstacle area; H D is the average height of the ground area.

[0079] S3. Assessment of the traversability of the mechanized farming road in front of the grain transport vehicle:

[0080] An adaptive lateral dynamic grid partitioning method is designed based on the speed of the grain truck and the width of the mechanized farming road. In each partition, the passability of the mechanized farming road area is initially judged based on the size and maximum climbing ability of the grain truck itself, the slope and width of the mechanized farming road, and the presence, size, hardness, height, whether it is suspended, and the distance from the bottom of the suspended state to the mechanized farming road of multiple sources of information such as the obstacle, and the maximum probability of the inaccessible area is eliminated; and a comprehensive scoring system is created based on the attributes of the slope of the mechanized farming road, the size and height of the obstacle, and the influence of the attributes on the passability of the mechanized farming road area is quantitatively analyzed. The passability of the grain truck 6 in the mechanized farming road area is further divided, and the on-board visual mechanized road passable area assessment for the grain truck 6 is realized.

[0081] like Figure 7 As shown, the main steps are as follows:

[0082] 3.1 Adaptive horizontal dynamic grid partitioning:

[0083] like Figure 8As shown in the figure, the adaptive horizontal dynamic grid refers to the horizontal area grid dynamically divided according to the width of the mechanized farming road and the vehicle movement speed during the movement of the grain transport vehicle 6. This method adjusts the horizontal partition size of the grid in proportion by sensing the width of the mechanized farming road in real time. At the same time, based on the movement speed of the grain transport vehicle 6, the time interval and density of the longitudinal grid are dynamically adjusted: when the vehicle speed is high, the grid division interval is reduced to enhance the responsiveness to obstacles and changes in the mechanized farming road; when the speed is low, the grid interval is increased to reduce the computational complexity.

[0084] 3.2 Initial judgment of passability based on multi-source information of grain transport vehicle and environment:

[0085] Combine multi-source information such as grain trucks and the environment to make an initial judgment on the passability of the mechanized farming road area. Input the RGBD image of the mechanized farming road, first determine whether the maximum slope value of the mechanized farming road is less than the climbing limit value of the grain truck, remove all hard obstacles in the image according to the type of obstacles, and perform adaptive horizontal dynamic grid partitioning on the remaining mechanized farming road area after removing the hard obstacles in the image; secondly, judge the relationship between the characteristics of the grain truck itself, the characteristics of the mechanized farming road, and the attributes of the obstacles in each grid in turn to make an initial judgment on different passabilities. The specific steps are as follows:

[0086] S3.2.1: First, determine whether the maximum slope of the tractor road exceeds the maximum climbing capacity of the grain transport vehicle. If so, the area ahead is an impassable area. If not, proceed to the next step.

[0087] S3.2.2: Determine whether the area contains hard obstacles: If so, remove the area where the obstacles are located and proceed to the next step of determination;

[0088] S3.2.3: Adaptively partition the mechanized farming road area without hard obstacles into horizontal dynamic grids, and make a judgment on each area after partitioning; whether there are obstacles in each area, if not, then end the preliminary judgment of passability, if yes, then proceed to the next step of judgment;

[0089] S3.2.4: Determine whether the obstacle is a suspended obstacle. If so, determine whether the distance between the bottom of the suspended obstacle and the mechanized farming road is greater than the height of the grain truck body, so as to determine whether the grain truck can pass through. If not, proceed to the next step of determination;

[0090] S3.2.5: Determine whether there is a road area larger than the width of the grain truck. If so, end the preliminary determination of drivability. If not, proceed to the next step.

[0091] S3.2.6: Determine whether there are multiple road areas in the area that are larger than the track width of the grain transport vehicle. If not, the area is impassable. If so, proceed to the next step of determination.

[0092] S3.2.6: Determine whether there are two areas that meet the conditions for grain truck tracks to pass, that is, the maximum distance between areas > the outer width of the track, and the minimum distance < the inner width of the track, and the height of the obstacles between areas is lower than the chassis of the grain truck. If so, the preliminary judgment of passability ends. If not, the area is impassable. The above steps can complete the preliminary judgment of the passability of the mechanized farming road and eliminate the impassable area with the highest probability.

[0093] 3.3 Assessment of accessible areas of mechanized farming roads:

[0094] After completing the initial judgment, a comprehensive scoring system is created based on the attributes of the slope of the mechanized farming road, the size and height of the obstacles, and the impact of the attributes on the traffic capacity of the mechanized farming road area is quantitatively analyzed. The trafficability of the mechanized farming road area for the grain transport vehicle 6 is further divided into ten discrete evaluation levels, thereby realizing the on-board visual mechanized road trafficable area assessment for the grain transport vehicle 6.

[0095] 3.3.1 Calculation method of farmland-obstacle attributes:

[0096] a. Slope (F1): Calculate the maximum slope of the tractor road surface in each zone.

[0097]

[0098] In formula (1), Δh represents the height difference of the mechanized farming road within the partition, and Δd represents the horizontal distance of the mechanized farming road within the partition.

[0099] b. Size of obstacles (F2): Obstacles are divided into three categories according to their size and each category is assigned a score.

[0100] Small obstacles (such as rocks): 1 point

[0101] Medium obstacles (such as tree stumps): 2 points

[0102] Large obstacles (such as large rocks): 3 points

[0103] The obstacle size score can be calculated as follows:

[0104]

[0105] In formula (2), N2 is the number of obstacles, t i is the size fraction of the i-th obstacle.

[0106] c. Height of obstacle (F3): Use the height of the obstacle directly.

[0107]

[0108] In formula (3), N3 represents the number of obstacles, h i Represents the height of the i-th obstacle.

[0109] 3.3.2 Weight distribution:

[0110] Weights can be assigned based on the degree of influence of each attribute on the passability.

[0111] The weight of the tractor road slope is W1 = 0.25

[0112] Obstacle size weight W2 = 0.4

[0113] Obstacle height weight W3 = 0.35

[0114] 3.3.3 Calculation of comprehensive score:

[0115] The scores of each attribute are weighted and summed according to the weight to calculate the comprehensive score. The formula is as follows:

[0116]

[0117] In formula (4), S is the comprehensive score, n is the total number of attributes, and W i is the weight of the i-th attribute, F i is the score of the i-th attribute.

[0118] 3.3.4 Multi-level classification of accessibility:

[0119] Through the analysis of a large amount of historical data and actual experience, the threshold of each scoring segment is determined through statistical analysis. The final comprehensive score is finally divided into 10 discrete evaluation levels according to the score, from 0 to 9, representing different pass weights: 0-2, easy to pass; 3-4, slightly difficult; 5-6, relatively difficult; 7-8, very difficult; 9, impossible to pass.

[0120] The above steps can realize the assessment of the passable area of ​​the farm road.

[0121] As a further improvement, the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A vehicle-mounted visual mechanized farming road passable area assessment method for grain transport vehicles, characterized in that: The evaluation method comprises the following steps: S1. Construction of low blind spot multi-view model of binocular-grain truck-environment: Four depth cameras are installed at different angles and heights on the top four sides of the grain tank of the grain truck through adjustable brackets to collect images of the tractor road and obstacles and build a data set; based on the positional relationship between the camera, grain truck and environment, an image-camera-world coordinate system is established to obtain the three-dimensional spatial information of the environment; S2. Unstructured tractor road area and obstacle detection: A transparency mask is used to remove the background area above the vanishing line in the tractor road, and a lightweight multi-level deep network model with weight sharing is proposed. The RGB image and depth image are used as input to simultaneously detect the visible road surface in the tractor road area and identify obstacles; the three-dimensional spatial information of the obstacle is calculated by combining the image-camera-world coordinate system and the internal and external parameters of the camera; S3. Assessment of the passability of the mechanized road ahead of the grain truck: An adaptive lateral dynamic grid partitioning method is constructed based on the speed of the grain truck and the width of the mechanized road. The multi-source information of the grain truck’s own characteristics, the mechanized road characteristics, and the attributes of obstacles is combined to preliminarily judge the passability of the mechanized road. A comprehensive scoring system is created and the impact of multi-source attributes on the regional accessibility of the mechanized road is quantitatively analyzed to further divide the accessibility level of the mechanized road.

2. According to claim 1, a method for evaluating the traversable area of ​​a grain transport vehicle using on-board visual mechanized farming road is characterized in that: S3 includes the following sub-steps: S301, constructing adaptive horizontal dynamic grid partitioning: by sensing the width of the mechanized farming road in real time, the horizontal partition size of the grid is adjusted proportionally; at the same time, based on the movement speed of the grain transport vehicle, the time interval and density of the longitudinal grid are dynamically adjusted: when the vehicle speed is high, the grid division interval is reduced to enhance the responsiveness to obstacles and changes in the mechanized farming road; when the speed is low, the grid interval is increased to reduce the computational complexity; S302. Initial judgment of passability based on multi-source information of grain transport vehicle and environment: The multi-source information of grain transport vehicle and environment is integrated for preliminary evaluation. The RGBD image of the mechanized farming road is input. First, it is judged whether the maximum slope value of the mechanized farming road is less than the climbing limit value of the grain transport vehicle. According to the type of obstacles, all hard obstacles in the image are removed, and the remaining mechanized farming road area in the image after the hard obstacles are removed is adaptively partitioned into horizontal dynamic grids. Secondly, the relationship between the characteristics of the grain transport vehicle itself, the characteristics of the mechanized farming road, and the attributes of the obstacles in each grid is judged in turn to make initial judgments on different passabilities. S303. Evaluate the tractor road passable area based on the tractor road-obstacle attributes: After completing the initial judgment, create a comprehensive scoring system based on the attributes of the slope of the tractor road, the size and height of the obstacles, quantitatively analyze the impact of the attributes on the passability of the tractor road area, and further divide the passability of the tractor road area for grain transport vehicles into ten discrete evaluation levels, thereby realizing the on-board visual tractor road passable area evaluation for grain transport vehicles.

3. The method for evaluating the traversable area of ​​a grain transport vehicle using on-board visual mechanized farming road according to claim 1 is characterized in that: A transparency mask is used in S2, and the position of the vanishing line is obtained through depth estimation. The background area above the vanishing line is eliminated, and only the mechanized farming road and related areas are retained to reduce image redundancy; the weight-sharing lightweight multi-level deep network model includes two parts: semantic segmentation and target detection, and the two networks share the weights of the shallow feature extraction layer; the mask of the passable area of ​​the mechanized farming road is extracted through the semantic segmentation network, and obstacles are identified in the target detection network based on the shared network weights; at the same time, the data source of the network model is a multi-channel image fused with RGB and depth.

4. The method for evaluating the traversable area of ​​a grain transporter's tractor road based on on-board visual technology according to claim 1 is characterized in that: The three-dimensional spatial information of the obstacle described in S2 is obtained through the binocular vision principle, and the depth image is constructed using the disparity map. Combined with the camera calibration parameters, each pixel point is converted into the actual space coordinates to calculate the three-dimensional position, size and suspension state of the obstacle.

5. The method for evaluating the traversable area of ​​a grain transporter's tractor road based on on-board visual technology according to claim 2 is characterized in that: The calculation method of S303 tractor road-obstacle attribute is as follows: In formula (1), Δh represents the height difference of the mechanized farming road within the partition, Δd represents the horizontal distance of the mechanized farming road within the partition, and F1 is the slope; In formula (2), N2 is the number of obstacles, t i is the size fraction of the ith obstacle, and F2 is the size of the obstacle; In formula (3), N3 represents the number of obstacles, h i represents the height of the ith obstacle, and F3 is the height of the obstacle; In formula (4), S is the comprehensive score, n is the total number of attributes, and W i is the weight of the i-th attribute, F i is the score of the i-th attribute.

6. The method for evaluating the traversable area of ​​a grain transport vehicle using on-board visual mechanized farming road according to claim 2 is characterized in that: The steps of judging in S302 are: S302a: First, determine whether the maximum value of the slope of the mechanized farming road exceeds the maximum climbing capacity of the grain transport vehicle: if it exceeds, the area ahead is an impassable area; if not, proceed to the next step of determination; S302b: Determine whether the area contains hard obstacles: if so, remove the area where the obstacles are located and proceed to the next step of determination; perform adaptive horizontal dynamic grid partitioning on the mechanized farming road area that does not contain hard obstacles, and perform determination on each area after partitioning; S302c: Determine whether there are obstacles in each area. If not, end the preliminary determination of passability. If yes, proceed to the next step of determination. S302d: Determine whether the obstacle is a suspended obstacle. If so, determine whether the distance between the bottom of the suspended obstacle and the mechanized farming road is greater than the height of the grain transport vehicle, so as to determine whether the grain transport vehicle can pass through. If not, proceed to the next step of determination. S302e: Determine whether there is a road area larger than the width of the grain transport vehicle. If yes, end the preliminary determination of drivability. If no, proceed to the next step of determination. S302f: Determine whether there are multiple road areas in the area that are larger than the track width of the grain transport vehicle. If not, the area is impassable. If so, proceed to the next step of determination. S302g: Determine whether there are two areas that meet the conditions for the grain truck tracks to pass through, that is, the maximum distance between areas > the outer width of the tracks, and the minimum distance < the inner width of the tracks, and the height of the obstacles between areas is lower than the chassis of the grain truck. If so, end the preliminary judgment of passability. If not, the area is impassable.

7. The method for evaluating the traversable area of ​​a grain transporter's tractor road based on on-board visual technology according to claim 1 is characterized in that: Multi-level classification of passability: Through the analysis of a large amount of historical data and actual experience, the threshold of each scoring segment is determined through statistical analysis. The final comprehensive score is finally divided into 10 discrete evaluation levels according to the score, from 0 to 9, representing different pass weights: 0-2, easy to pass; 3-4, slightly difficult; 5-6, relatively difficult; 7-8, very difficult; 9, impossible to pass.

8. The method for evaluating the traversable area of ​​a grain transport vehicle using on-board visual mechanized farming road according to claim 1 is characterized in that: The characteristics of the grain transport vehicle in S3 include its size and maximum climbing ability; the characteristics of the mechanized farming road include slope and width; the attributes of obstacles include presence, size, hardness, height, whether it is suspended, and the distance from the bottom of the mechanized farming road when suspended.

Images