Unstructured road identification method and system suitable for agricultural unmanned vehicle, medium and terminal

Abstract

The invention provides an unstructured road identification method and system suitable for an agricultural unmanned vehicle, a medium and a terminal, on one hand, the method does not depend on color threshold segmentation, is not limited by illumination and surface texture change any more, is suitable for scenes with dense trees on two sides of a road, can also adapt to tree-free open scenes, has a very strong generalization ability, and is suitable for popularization and application. The method does not need to depend on the weed coverage condition, color difference and other external characteristics of roads. And on the other hand, the method does not depend on artificial experience like a traditional machine learning technology, can well adapt to various farmland scenes, greatly increases the universality, and greatly improves the recognition rate in a large-range unstructured environment. And finally, large-scale accurate data labeling is not needed, and the generalization performance under different crop, soil texture and weather conditions is excellent. In short, the spatial perception precision can be improved; the influence caused by illumination is avoided; the method is suitable for scenes with many or no trees; the path is prevented from being identified only through trees; and the generalization ability is strong, and no requirement on road characteristics exists.

Description

Technical Field

[0001] This application relates to the field of unstructured road recognition and planning technology, and in particular to unstructured road recognition methods, systems, media and terminals applicable to agricultural unmanned vehicles. Background Technology

[0002] With the development of agricultural automation and intelligence, agricultural unmanned vehicles have received widespread attention in scenarios such as farmland operations, material transportation, sowing, and spraying. Unlike autonomous driving on urban roads, the farmland environment is a typical unstructured scenario with blurred road boundaries, complex surface textures, and significant dynamic changes in the environment (such as mud, weeds, shadows, and crop residues).

[0003] In the core technology system of agricultural unmanned vehicles, accurate road detection and segmentation are the primary prerequisites and core foundations for achieving autonomous navigation, path planning, and operational decision-making. Only by accurately identifying passable areas in farmland can unmanned vehicles effectively avoid crop areas, obstacles, and dangerous terrain, ensuring the rationality of the operational path and the safety of the process. However, the special characteristics of farmland scenarios present many challenges to this technical aspect. Traditional autonomous driving perception algorithms developed based on urban road scenarios, which rely on clear lane lines, curbs, and other structured markers, are often difficult to apply in farmland environments. There is an urgent need to develop dedicated detection and segmentation technologies for the unstructured features of farmland.

[0004] One of the difficulties in detecting farmland roads is the lack of clear road boundaries: urban roads form clear traffic boundaries through standardized lane lines, curbs and other facilities, while the so-called roads in farmland are mostly dirt roads or grass roads that facilitate the passage of agricultural machinery. They have neither artificially marked lines nor clear curb boundaries, and their transition with the surrounding crop areas and weed areas often presents a gradual feature, making it difficult for the perception system to quickly and accurately define the scope of the passable area.

[0005] The second difficulty in detecting farmland roads lies in the similarity between the surface texture and the crop background: the soil, weeds, crop residues and shadows formed at different times on the farmland surface are highly similar in color grayscale and texture features. This makes it easy for traditional algorithms that rely on color threshold segmentation or simple texture feature extraction to make misjudgments, identifying non-travelable areas as roads, or conversely, missing effective travel areas.

[0006] The third challenge in farmland road inspection lies in the highly dynamic nature of the environment: the dynamic nature of the farmland environment further exacerbates the technical difficulty of road inspection, becoming a significant factor restricting the stability of inspection accuracy. Changes in weather conditions directly alter the visual characteristics of the ground surface. For example, strong shadows easily form under bright sunlight on sunny days, poor uniformity of light on cloudy days leads to reduced color contrast, and rainy days cause reflections due to water accumulation on the road surface, while also causing the color characteristics of soil and weeds to blend together. Dynamic changes in lighting conditions are equally significant. Differences in the angle and intensity of light from dawn to dusk, as well as dynamic shadows formed by crop shading, all lead to large fluctuations in the visual appearance of the same area. Furthermore, changes in crop growth status and soil moisture levels due to seasonal transitions systematically alter the overall visual characteristics of the farmland surface. These dynamic factors collectively require road inspection technology to possess extremely strong environmental adaptability and robustness to meet the all-weather, full-cycle operational needs of agricultural unmanned vehicles.

[0007] Currently, to address the challenges of detecting farmland roads, various methods are employed, including traditional image processing, machine learning-based feature classification, and even deep learning-based semantic segmentation. However, traditional image processing methods rely on techniques such as color thresholding, making them extremely sensitive to changes in lighting and surface texture; they struggle to handle complex backgrounds and shadows; and they lack robustness and generalization ability. While machine learning-based feature classification methods improve detection performance, feature design depends on human experience, making it difficult to adapt to diverse farmland scenarios, resulting in poor model transferability and low recognition rates in large-scale unstructured environments. Deep learning-based semantic segmentation methods offer automatic feature extraction capabilities, but require large-scale, accurately labeled data. However, farmland image acquisition and labeling are costly, and the models exhibit poor generalization performance under different crop, soil, and weather conditions. Summary of the Invention

[0008] In view of the shortcomings of the prior art described above, the purpose of this application is to provide an unstructured road recognition method, system, medium and terminal suitable for agricultural unmanned vehicles, to solve the technical problems of unclear road boundaries, similarity between ground texture and crop background and strong environmental dynamics in existing farmland road detection.

[0009] To achieve the above and other related objectives, a first aspect of this application provides an unstructured road recognition method suitable for agricultural unmanned vehicles, comprising: acquiring binocular images; detecting target crops using a target detection model based on one of the binocular images, and extracting the coordinate data of the target detection boxes of the target crops; calculating a disparity map based on the binocular images, obtaining a depth map based on the disparity map and combined with camera intrinsic parameters, selecting an initial growth point of the region and obtaining an initial path region; dividing the initial path region into several non-overlapping sub-regions, each containing target crops, sorting the target crops in the divided sub-regions by spacing, and retaining the path region between the group of target crops with the largest spacing as the final path region.

[0010] In some embodiments of the first aspect of this application, the extraction process of the coordinate data of the target crop's target detection box includes: standardizing the left-eye image acquired by the binocular camera according to the input requirements of the target detection model; the standardization process includes adjusting the image resolution, normalizing pixel values, and eliminating image distortion, or one or more of these; inputting the pre-processed left-eye image into the trained target detection model, wherein the target detection model extracts multi-scale deep features of the target crop in the left-eye image through the backbone network, fuses the multi-scale features through the neck network, and outputs a prediction result through the head network, the prediction result including the target category, confidence level, and detection box coordinate data; filtering out detection boxes with a confidence level lower than a preset threshold, and using the coordinate data of the remaining valid detection boxes as the coordinate data of the target detection box.

[0011] In some embodiments of the first aspect of this application, the extraction process of the coordinate data of the target crop's target detection box includes: standardizing the left-eye image acquired by the binocular camera according to the input requirements of the target detection model; the standardization process includes adjusting the image resolution, normalizing pixel values, and eliminating image distortion, or one or more of these; inputting the pre-processed left-eye image into the trained target detection model, wherein the target detection model extracts multi-scale deep features of the target crop in the left-eye image through the backbone network, fuses the multi-scale features through the neck network, and outputs a prediction result through the head network, the prediction result including the target category, confidence level, and detection box coordinate data; filtering out detection boxes with a confidence level lower than a preset threshold, and using the coordinate data of the remaining valid detection boxes as the coordinate data of the target detection box.

[0012] In some embodiments of the first aspect of this application, the method further includes: calculating a disparity map based on the binocular images using a lightweight binocular detection model, the execution process of which is as follows: preprocessing the input binocular images, inputting the preprocessed binocular images into a lightweight backbone network, splitting the standard convolution into depthwise convolution and pointwise convolution through depthwise separable convolution in the network to extract multi-scale features; adopting a lightweight matching strategy, selecting the optimal matching point in the right image that best matches the reference pixel point based on the reference pixel point in the left image; using the difference in the horizontal coordinate between the reference pixel point and the optimal matching point as the disparity value corresponding to the reference pixel point; completing the batch disparity value calculation by traversing all pixels in the left image to generate a disparity map.

[0013] In some embodiments of the first aspect of this application, the lightweight matching strategy is configured to: set a limited search range only in a preset local region around a reference pixel in the right eye image; use only pixel grayscale similarity and feature distance as filtering conditions; and filter the optimal matching point by a threshold.

[0014] In some embodiments of the first aspect of this application, setting a limited search range only in a preset local region surrounding a reference pixel in the right eye image includes: delineating a local search region in the right eye image based on the reference pixel in the left eye image and the epipolar constraint of the binocular camera; dividing the local search region into several candidate point neighborhoods according to a preset neighborhood window; determining the reference point neighborhood where the reference pixel is located according to the preset neighborhood window; calculating the fit between the reference point neighborhood and each candidate point neighborhood based on grayscale similarity and feature distance; and selecting the optimal matching point from among them.

[0015] In some embodiments of the first aspect of this application, the step of selecting an initial growth point and obtaining an initial path region includes: selecting an initial growth point based on the shooting range of the unmanned vehicle and the environmental perception requirements, and generating a path region based on the initial growth point using a region growth algorithm.

[0016] In some embodiments of the first aspect of this application, the region growing algorithm is configured to: store the initial growth point in a seed point queue, sequentially retrieve the pixels in the queue, and traverse their preset neighboring pixels; determine whether a preset growth criterion is met, and if so, add the pixel to the path region as a new seed point and store it in the queue, repeating this process until the queue is empty; wherein, the preset growth criterion includes: the depth difference between the adjacent pixel and the pixel in the current region must be less than a preset threshold, and the depth change rate of the pixel is lower than a safety threshold.

[0017] To achieve the above and other related objectives, a second aspect of this application provides an unstructured road recognition system suitable for agricultural unmanned vehicles, comprising: an image acquisition module for acquiring binocular images; a detection box extraction module for detecting target crops using a target detection model based on one of the binocular images, and extracting the coordinate data of the target detection boxes of the target crops; and a path region generation module for calculating a disparity map based on the binocular images, obtaining a depth map based on the disparity map and combined with camera intrinsic parameters, selecting an initial growth point of the region and obtaining an initial path region; dividing the initial path region into several non-overlapping sub-regions, each containing target crops, sorting the target crops within the divided sub-regions by spacing, and retaining the path region between the group of target crops with the largest spacing as the final path region.

[0018] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the unstructured road recognition method applicable to agricultural unmanned vehicles.

[0019] To achieve the above and other related objectives, a fourth aspect of this application provides a computer program product comprising computer program code that, when executed on a computer, causes the computer to implement the unstructured road recognition method applicable to agricultural unmanned vehicles.

[0020] To achieve the above and other related objectives, a fifth aspect of this application provides a computer device / equipment / system / electronic terminal, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the unstructured road recognition method applicable to agricultural unmanned vehicles.

[0021] As described above, the unstructured road recognition method, system, medium, and terminal applicable to agricultural unmanned vehicles of this application have the following beneficial effects: The agricultural unmanned vehicle of this application can automatically maintain its path during operation, effectively preventing it from deviating from the preset road. Firstly, this application does not rely on traditional image processing methods such as color threshold segmentation, and is no longer limited by changes in lighting and surface texture. It is applicable to both densely wooded scenes on both sides of the road and open, treeless scenes. Furthermore, it possesses strong generalization ability, requiring no reliance on external features such as road weed coverage or color differences, and can accurately lock the operation path in various complex road environments, significantly improving the stability and adaptability of the operation process. Secondly, unlike traditional machine learning techniques, this application does not rely on human experience, and can adapt well to diverse farmland scenes, greatly increasing its universality and significantly improving the recognition rate in large-scale unstructured environments. Finally, this application does not require large-scale, accurately labeled data, and exhibits excellent generalization performance under different crop, soil, and weather conditions. Therefore, this application improves spatial perception accuracy; avoids the influence of lighting; is applicable to various scenarios, including those with many trees on both sides or those without trees; avoids identifying paths solely through trees; has strong generalization ability and does not require specific road features. Attached Figure Description

[0022] Figure 1 The diagram shown is a flowchart illustrating an unstructured road recognition method applicable to agricultural unmanned vehicles in one embodiment of this application.

[0023] Figure 2 The diagram shows a flowchart of how a lightweight binocular detection model is used to calculate a disparity map based on a binocular image in one embodiment of this application.

[0024] Figure 3 The diagram shown is a schematic diagram illustrating the calculation of the sum of grayscale differences between the neighborhood of the reference point and the neighborhood of the candidate point in one embodiment of this application.

[0025] Figure 4 The diagram shown is a flowchart illustrating an unstructured road recognition method applicable to agricultural unmanned vehicles in one embodiment of this application.

[0026] Figure 5 The diagram shown is a schematic representation of a path region generated by an unstructured road recognition method in one embodiment of this application.

[0027] Figure 6 The diagram shown is a structural schematic of an unstructured road recognition system for agricultural unmanned vehicles according to an embodiment of this application.

[0028] Figure 7 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of this application. Detailed Implementation

[0029] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0030] To facilitate understanding of the embodiments of this application, firstly, in conjunction with Figure 1 Detailed explanation. Figure 1 This document illustrates a flowchart of an unstructured road recognition method for agricultural unmanned vehicles, as described in an embodiment of the present invention. The unstructured road recognition method in this embodiment mainly includes the following steps:

[0031] Step S1: Acquire stereo images.

[0032] In some optional implementations, to accurately capture the three-dimensional spatial information and visual features of farmland operation scenarios, this application employs a binocular camera for binocular image acquisition. The binocular camera, through its left and right lenses simulating the principle of human vision, simultaneously captures paired images of the farmland operation area. This not only records two-dimensional visual information of traversable areas, crop distribution, and obstacles (such as trees and rocks), but also, based on the parallax principle of binocular vision, provides raw data support for subsequent three-dimensional reconstruction, distance measurement, and obstacle localization.

[0033] It should be noted that various types of binocular cameras can be used in the embodiments of this application, and this application does not specifically limit them. For example, binocular cameras can be divided into short-baseline and long-baseline cameras based on baseline distance. Short-baseline cameras have a compact lens spacing (e.g., centimeter level), are highly portable, and have high accuracy in close-range measurement, making them suitable for close-range perception scenarios such as crop spacing detection and small obstacle identification in farmland. Long-baseline cameras have a larger lens spacing (e.g., decimeter level and above), a longer measurement range, and can meet the needs of large-scale three-dimensional perception such as farmland boundary mapping and long-distance obstacle localization. Based on imaging spectrum, visible light, infrared, and multispectral binocular cameras can be selected. Visible light binocular cameras have moderate cost and good color reproduction, making them suitable for conventional farmland operations with good lighting conditions. Infrared binocular cameras are not limited by visible light intensity and can capture target thermal radiation information in complex environments such as night, fog, and shadows, ensuring all-weather perception. Multispectral binocular cameras can simultaneously acquire visible light and specific band spectral images, combining three-dimensional spatial information and spectral characteristics, making them suitable for scenarios requiring multi-dimensional data support, such as crop growth monitoring and pest and disease identification. In addition, depending on the application requirements, you can choose between factory-calibrated cameras (no additional calibration required, high stability) or field-calibrated cameras (adapted to customized installation scenarios, more flexible). Targeted selection can fully meet the diverse sensing needs of unstructured farmland scenarios.

[0034] Step S2: Based on one of the binocular images, use a target detection model to detect the target crop and extract the coordinate data of the target detection box of the target crop.

[0035] Preferably, in this embodiment, the left eye image from the binocular images is selected as the data source for target detection. It should be noted that both the left and right eye images in a binocular system can clearly present the visual features of the target crop and can both be used as data sources for target detection. However, this application preferably sets the left eye image as the default reference viewpoint. This is to establish a unified coordinate reference system with subsequent processes such as 3D reconstruction and distance calculation in binocular vision. If the right eye image is selected, an additional step of converting the left and right eye coordinates is required, which not only increases the complexity of data processing but may also affect the subsequent perception accuracy due to coordinate mapping errors. Furthermore, selecting the left eye as the reference viewpoint conforms to the design specifications of most binocular vision systems and is more compatible with the training data distribution of existing target detection models (most public datasets and self-developed labeled data often use the left eye as the reference), avoiding a decrease in model detection performance due to viewpoint differences.

[0036] Object detection models are deep learning models that can automatically identify specific object categories in images and output the object's location (represented by the coordinates of the detection box). Their core function is to simultaneously perform the two major tasks of object classification and object localization. The object detection models selected in the embodiments of this application include single-stage models (such as the YOLO series, SSD, etc.) and two-stage models (such as Faster R-CNN). Object detection models automatically extract deep visual features from images through convolutional neural networks (CNNs), eliminating the need for manual feature design and enabling them to adapt to the morphological and textural variations of target crops in farmland scenes.

[0037] For example, the training process of the object detection model includes the following steps: First, a dataset specifically for farmland scenes is constructed. Left-eye images are collected under different lighting conditions, weather conditions, and crop growth stages. Target crops (such as trees, wheat, and corn) in the images are manually labeled, including the target category label and the corresponding detection box coordinates (usually the upper left and lower right diagonal coordinates). Second, a pre-trained model (such as a backbone network like ResNet or DarkNet trained on the ImageNet dataset) is selected as the foundation. The model parameters are initialized through transfer learning to reduce the amount of data and computational cost required for training. Finally, the labeled dataset is divided into training, validation, and test sets. Appropriate hyperparameters such as learning rate, batch size, and number of iterations are set. The gradient descent algorithm is used to minimize the model's classification loss (such as cross-entropy loss) and localization loss (such as smoothing L1 loss). The parameters are adjusted in real time using the validation set to avoid overfitting. Finally, an object detection model that achieves both classification accuracy and localization accuracy on the test set is obtained.

[0038] To facilitate understanding by those skilled in the art, the following describes the detection and coordinate extraction process based on the left-eye image, using trees as the target crop, and can be divided into the following specific steps:

[0039] Step a1) Data preprocessing: Standardize the left eye image acquired by the binocular camera according to the input requirements of the target detection model, including adjusting the image resolution (e.g., scaling to 640×640 pixels to match the model input size), normalizing pixel values ​​(e.g., converting the pixel range from 0-255 to 0-1 to accelerate model convergence), and eliminating image distortion (using binocular camera calibration parameters to correct lens distortion).

[0040] Step a2) Model inference: Input the preprocessed left eye image into the trained target detection model (such as YOLOv8). The model extracts multi-scale features such as texture and contour of trees in the image through the backbone network. After the neck network fuses the multi-scale features, the head network outputs the prediction result. The prediction result includes the target category (determined to be a tree), confidence score (such as a confidence score of 0.92, reflecting the reliability of the detection result), and detection box coordinates.

[0041] Step a3) Coordinate filtering and extraction: Set a confidence threshold (e.g., 0.5) to filter out false detection results with confidence below the threshold and retain valid tree detection boxes. The detection box coordinates output by the model are usually the top left vertex (x1, y1) and the bottom right vertex (x2, y2). If specific coordinates (e.g., the bottom left vertex) are needed, they can be calculated through coordinate transformation.

[0042] Step a4) Coordinate output and storage: Output the coordinates of the filtered tree detection boxes (such as (x1,y1,x2,y2) or the converted lower left coordinates (x1,y2)) in a structured format (such as JSON, XML) to facilitate subsequent binocular parallax calculation, tree distance measurement and obstacle avoidance path of the unmanned vehicle.

[0043] Preferably, after detecting the target crop in the left eye image, instead of extracting all four coordinates of the target detection box, only one set of diagonal coordinates is extracted. This is because the target detection box is usually uniquely determined by the diagonal coordinates of the upper left and lower right, or the lower left and upper right. There is no need to store all four coordinates, as redundant coordinates would increase the cost of data storage and transportation. Moreover, in practical applications (such as localization and obstacle avoidance), only the key diagonal coordinates are needed to complete the calculation.

[0044] Step 3: Calculate the disparity map based on the binocular image, obtain the depth map based on the disparity map and combined with the camera intrinsic parameters, select the initial growth point of the region, and obtain the initial path region.

[0045] In the embodiments of this application, a lightweight binocular detection model is specifically used to calculate the disparity map based on the binocular images. The implementation process is as follows: Figure 2 As shown, it includes the following steps:

[0046] Step S2a: Preprocess the input stereo image. Input the preprocessed stereo image into a lightweight backbone network. Use depthwise separable convolution in the network to split the standard convolution into depthwise convolution and pointwise convolution to extract multi-scale features.

[0047] Specifically, the left image in the binocular imagery is used as the reference image, and the right image is used as the matching image. Preprocessing methods include synchronous scaling and distortion correction for both images to ensure uniform image size and eliminate imaging deviations caused by optical characteristics. The processed images are then input into a lightweight backbone network. This type of network can efficiently extract multi-scale feature maps, preserving key details such as crop edges and textures while significantly reducing resource consumption during computation through special convolutional structures such as depthwise separable convolution and grouped convolution. Depthwise separable convolution breaks down standard convolution into depthwise and pointwise convolutions, requiring only a small number of parameters for feature extraction; grouped convolution divides the input feature map into multiple groups for separate convolution, reducing parameter interactions between groups.

[0048] The lightweight backbone networks include, but are not limited to, the MobileNet series focused on efficient inference on mobile devices, the ShuffleNet series which reduces computational costs through channel rearrangement, the EfficientNet-Lite series which optimizes performance with a composite scaling strategy, GhostNet which combines depthwise separable convolution with attention, and ResNet-Lite which is based on residual structure simplification. This application does not limit these to any particular type. Taking MobileNetV3 of the MobileNet series as an example, the process of extracting multi-scale features is illustrated: MobileNetV3 splits the standard convolution into depthwise convolution and pointwise convolution through depthwise separable convolution. The depthwise convolution performs convolution on each input channel individually to capture spatial features, while the pointwise convolution fuses features from different channels. Equivalent feature extraction can be completed with only about 1 / 8 of the parameters of the standard convolution. At the same time, combined with the SE attention mechanism, the weights of key features such as crop edges and textures are dynamically strengthened, while background noise interference is weakened.

[0049] It's worth noting that the lightweight binocular detection model uses a lightweight backbone network instead of the heavy backbone networks (such as ResNet and VGG) used in conventional binocular detection models. This is to better adapt to the application scenarios of agricultural unmanned vehicles. Firstly, agricultural unmanned vehicles are mostly equipped with embedded edge terminals (such as NVIDIA Jetson Nano and embedded CPUs), which have very limited computing power, storage resources, and power consumption. Conventional binocular detection models have a large number of parameters and are computationally intensive, with inference latency typically exceeding 100ms on edge devices. The lightweight model, however, can control inference latency to within 30ms. Secondly, agricultural unmanned vehicles need to respond quickly to environmental changes during farmland operations. Disparity map calculation and depth perception, as preliminary steps in navigation decision-making, must achieve low-latency closed-loop processing. Only by leveraging the efficient inference capabilities of the lightweight model can the real-time updating of perceived data be guaranteed. Third, the core perception targets in farmland scenarios (such as trees, crop areas, road boundaries, etc.) are mostly medium to large-sized targets, and the requirements for fine-grained parallax accuracy are not extreme. Although lightweight models sacrifice some extreme accuracy, they can still meet the core requirements of 3D perception in farmland through targeted optimization (such as controlling the distance measurement error within 5%). Moreover, the real-time response capability brought by its high efficiency is far more in line with the actual operation requirements of agricultural unmanned vehicles than the extreme accuracy of heavy models, avoiding navigation lag or safety risks caused by slow model inference.

[0050] Step S2b: Using a lightweight matching strategy, the optimal matching point that best matches the reference pixel in the left eye image is selected in the right eye image based on the reference pixel; the difference in the horizontal coordinate between the reference pixel and the optimal matching point is used as the disparity value corresponding to the reference pixel; the batch disparity value is calculated by traversing all pixels in the left eye image to generate a disparity map.

[0051] It should be understood that a disparity map is a grayscale image obtained by calculating the positional differences of corresponding pixels in the left and right eye images. Essentially, it represents the offset of the projected coordinates of the same 3D point in the left and right eye images, serving as a crucial bridge connecting binocular 2D images with 3D spatial information. In a disparity map, the grayscale value (or pixel value) of each pixel corresponds to the magnitude of the disparity at that location. A larger disparity value indicates that the point is closer to the camera, while a smaller disparity value indicates that the point is farther away. When the disparity value is 0, the target can be considered to be at infinity. For example, in a farmland scene, trees near the autonomous vehicle have a larger difference in projected position in the left and right eye images, resulting in a higher pixel value (brighter color) in the corresponding area of ​​the disparity map; while distant farmland boundaries or the horizon have a smaller difference in projection between the left and right eyes, resulting in a lower pixel value (darker color) in the corresponding area of ​​the disparity map.

[0052] The lightweight matching strategy is an efficient matching scheme used in binocular vision matching tasks. It aims to minimize computational consumption and time costs while ensuring basic matching accuracy by simplifying computational logic, limiting the search range, and selecting low-complexity matching evaluation indicators. Specifically, in this application, the lightweight matching strategy is configured as follows: (1) Only a limited search range of a certain size is set in a preset local area around the reference pixel in the right eye image, thus avoiding redundant computation caused by global traversal. (2) Only pixel grayscale similarity and feature distance are selected as screening conditions, eliminating the need for complex feature extraction (such as convolution operation, feature encoding, etc.) evaluation dimensions, thereby reducing the computational amount of single-pixel matching. (3) The optimal matching point is selected by threshold, eliminating the need for complex iteration or model inference processes.

[0053] Therefore, the specific implementation of the lightweight matching strategy for selecting the optimal matching point is as follows: based on the preset reference pixel in the left-eye image, the optimal matching point that best matches the reference pixel is selected within the preset search range of the right-eye image by calculating pixel gray-level similarity and feature distance. The specific implementation process is as follows: First, based on the reference pixel in the left-eye image, a local search region is delineated in the right-eye image based on the epipolar constraints of the binocular camera, and the local search region is divided into several candidate point neighborhoods according to a preset neighborhood window. Second, the reference point neighborhood where the reference pixel is located is determined according to the preset neighborhood window, and the fit between the reference point neighborhood and each candidate point neighborhood is calculated based on gray-level similarity and feature distance, and the optimal matching point is selected from them.

[0054] The gray-level similarity calculation refers to calculating the sum of gray-level differences (SAD), the sum of squared gray-level differences (SSD), or the normalized correlation coefficient (NCC) between the neighborhood of the reference point and the neighborhood of the candidate point. A smaller SAD value indicates a closer similarity in gray-level distribution, a smaller SSD value indicates a closer similarity in gray-level distribution, and an NCC value closer to 1 indicates a stronger gray-level correlation. This is used to initially screen candidate point neighborhoods with high gray-level compatibility. For ease of understanding, let's use... Figure 3 Taking this example, the neighborhood window size is 3×3. The pixel in the center (value 15) is the reference pixel. The nine pixels formed by the pixel itself, its top, bottom, left, right, and diagonal lines constitute the reference point neighborhood in the left eye image (labeled as neighborhood A). Each number represents the brightness of a pixel (value between 0 and 255). Based on the epipolar constraints of the stereo camera, assume that neighborhood A has six candidate point neighborhoods in the right eye image: neighborhood B1, neighborhood B2, ..., neighborhood B6. Each candidate point neighborhood has the same size as neighborhood A, also a 3×3 window. Taking neighborhood B1 as an example (see...) Figure 3 (As shown in the right figure), calculate the SAD value between neighborhood A and neighborhood B1:

[0055] First row: |10-11|=1, |12-12|=0, |11-10|=1;

[0056] Second line: |13-14|=1, |15-15|=0, |14-13|=1;

[0057] Third row: |12-13|=1, |11-10|=1, |10-11|=1;

[0058] Adding the nine differences together gives 7, so the SAD value between neighborhood A and neighborhood B1 is 7. Similarly, calculate the SAD values ​​between neighborhood A and each of the other neighborhoods, and select the one with the smallest SAD value as the optimal matching point.

[0059] Furthermore, to avoid mismatches caused by changes in lighting and repetition of local textures, and to eliminate mismatched candidate points with similar gray levels but large feature differences, this embodiment of the application also performs feature distance calculation between the neighborhood of the reference point and the neighborhood of each candidate point. The specific calculation process is as follows: calculate the lightweight local features of the neighborhood of the reference point, arrange the calculated feature parameters in a certain order to form the first feature vector of the neighborhood of the reference point, and calculate the second feature vector of each candidate point neighborhood in a corresponding manner; calculate the distance between the first feature vector and each second feature vector respectively, and eliminate the candidate points corresponding to feature vectors whose distance values ​​exceed a preset threshold.

[0060] For example, for the neighborhood A of the left eye reference point, calculate its lightweight local features (such as the statistical values ​​of gradient magnitude and direction, the bin distribution values ​​of the gray-level histogram, etc.), and arrange these feature parameters in a fixed order to form the feature vector V of A. a Similarly, perform the same operation on the 3×3 neighborhood of each candidate point B1~B6 in the right eye to obtain their respective feature vectors Vᵦ1~Vᵦ6. Calculate the feature vectors Vᵦ1~Vᵦ6 using either Euclidean distance or Manhattan distance. a The distance values ​​between Vᵦ1 and Vᵦ6 are used; the smaller the distance, the higher the feature similarity between the two neighborhoods. A reasonable distance threshold is set (e.g., determined based on experimental data or scene requirements). Candidate points with distance values ​​greater than this threshold are directly eliminated. This is because even if these candidate points have high grayscale similarity to the baseline point, their large feature differences suggest a high probability of mismatch due to lighting variations or texture repetition, and they need to be excluded to improve matching accuracy.

[0061] In the embodiments of this application, after calculating the disparity map of the binocular image, the depth map is further calculated by combining the camera intrinsic parameters.

[0062] It should be understood that a binocular camera consists of two parallel cameras that mimic the parallax principle of the human eye. It calculates depth by capturing two images of the same scene, one on the left and one on the right. Using triangulation, it analyzes the pixel displacement (parallax) of corresponding points in the two images to reconstruct the 3D scene and generate a depth map. A depth map is an image that records the distance information of various points in an agricultural scene. The value of each pixel (usually encoded in grayscale or color) represents the physical distance from that point to the camera plane. Essentially, it compresses the geometric information of three-dimensional space into a two-dimensional image.

[0063] In this application, the conversion relationship between depth and disparity of the stereo camera after epipolar correction is as follows: It is assumed that the stereo camera has completed epipolar correction, that is, the left and right eye images have achieved row alignment, and the imaging points of the same spatial point in the left and right eye images are in the same row. Therefore, the disparity only changes in the horizontal direction, without having to consider the offset in the vertical direction, which simplifies matching and calculation.

[0064] Camera intrinsic parameter matrix ;

[0065] The intrinsic parameter matrix describes the imaging characteristics of the camera, where Indicates that the camera is horizontal ( Focal length (in pixels) in the direction of ) , This represents the pixel coordinates of the principal point of the image (the intersection of the optical axis and the imaging plane), but since parallax only varies horizontally after epipolar correction, it is not actually used in depth calculation. , .

[0066] For any pixel in the image The corresponding depth Parallax The relationship is represented as:

[0067] ;

[0068] in, Represents pixels The corresponding depth value, that is, the vertical distance from the camera to the corresponding spatial point of the pixel, is in the same unit as the baseline B, usually in meters; Indicates that the camera is horizontal ( The focal length in the direction of ) is derived from the intrinsic parameter matrix, as explained above; B is the binocular baseline, representing the distance between the optical centers of the two cameras; It is a pixel. The corresponding parallax (unit: pixels) is defined as the lateral pixel difference between corresponding points in the left and right eyes, i.e. ; It is the x-coordinate of the point on the left eye. This is the x-coordinate of the point corresponding to the right eye. It can be seen that the greater the parallax, the smaller the depth, which manifests as greater parallax in near scenes and smaller parallax in distant scenes; that is, depth and parallax are inversely proportional.

[0069] In the embodiments of this application, after obtaining the depth map, an initial growth point for the region is selected based on the shooting range of the unmanned vehicle and the environmental perception requirements, and an initial path region is generated based on the initial growth point using a region growth algorithm.

[0070] Before selecting initial growth points, it's crucial to define the boundaries of the autonomous vehicle's field of view, i.e., the pixel range covered by the camera's field of view. Invalid depth data outside the field of view must be eliminated to avoid ineffective calculations. Simultaneously, constraints should be defined based on environmental perception requirements. These include: growth points must fall within a passable area (preliminarily determined through depth maps, excluding areas with too-close obstacles and too-far-away areas with blurred perception), and they must align with the autonomous vehicle's driving dynamics (e.g., proximity to the current trajectory extension line to ensure path smoothness). Therefore, the depth map can be preprocessed, such as through Gaussian filtering for noise reduction and threshold segmentation to initially extract the outline of the passable area. Then, multiple evenly distributed pixels far from obstacle edges are selected from the preprocessed candidate set of passable areas as initial growth points, ensuring comprehensive coverage and no blind spots in the subsequently generated path area.

[0071] Subsequently, based on the initial growth point of the selected region, a region growth algorithm is used to generate the initial path region. The region growth algorithm refers to starting from the initial growth point and gradually merging adjacent pixels with similar characteristics to form a continuous region. Similar characteristics can be determined based on depth value and depth change rate: First, growth criteria are set: the depth difference between adjacent pixels and pixels within the current region must be less than a preset threshold (this threshold is adjusted according to road surface smoothness requirements; for example, the threshold is smaller for urban roads and can be appropriately increased for off-road roads), and the depth change rate of pixels (i.e., the ratio of the depth difference between adjacent pixels to the pixel distance) must be lower than a safety threshold to avoid including obstacle edges or areas with abrupt slope changes in the path. Second, the initial growth point is stored in a seed point queue. Pixels in the queue are sequentially retrieved, and their preset neighboring pixels (such as 8-neighborhood or 4-neighborhood pixels) are traversed to determine if they meet the growth criteria. If they do, the pixel is added to the path region as a new seed point and stored in the queue. This process is repeated until the queue is empty. At this point, the preliminary path region has been formed, and all pixels within this region meet the passable depth feature requirements.

[0072] Finally, after generating the initial path region, post-processing and validity verification are required to obtain the final region that meets the driving requirements of the autonomous vehicle. The initially generated path region may contain isolated noise points (such as occasional abnormal pixels in the depth map) or local connectivity defects (such as region breaks caused by abrupt changes in depth), so post-processing optimization is required. Optimization methods include, but are not limited to: (1) using morphological filtering (such as dilatational erosion) to eliminate small noise areas and fill in small holes in the area; (2) identifying and repairing broken areas through connectivity analysis to ensure the continuity of the path region. Validity verification can be performed in conjunction with the physical constraints of the autonomous vehicle, specifically including: checking whether the width of the path region meets the vehicle's passage requirements, whether the safe distance between the edge of the region and obstacles meets the standard, and whether the curvature of the region is within the vehicle's turning limit range. After the verification is passed, the path region is converted from the pixel coordinate system to the world coordinate system and output to the path planning module of the autonomous vehicle to provide regional basis for the generation of accurate paths.

[0073] Step S4: Divide the initial path region into several non-overlapping sub-regions, each containing the target crops. Sort the target crops in the divided sub-regions by spacing and retain the path region between the target crops with the largest spacing as the final path region.

[0074] In the specific implementation process: First, the boundaries of the path region (i.e., the width range of the image) are determined. According to preset equidistant or adaptive rules (such as based on path width, tree density, etc.), the image is divided into multiple non-overlapping rectangular sub-regions along the horizontal or vertical direction. This breaks down the large path region into multiple small-scale units, reducing the complexity of subsequent coordinate analysis and avoiding coordinate statistical bias caused by overly scattered tree distribution, ensuring that the tree coordinates within each sub-region are locally representative. Second, the coordinate data of the target crops along the length of each sub-region are recorded and sorted, and the adjacent differences between target crops are calculated. The smaller the adjacent difference, the denser the distribution of crops, and vice versa. The path region usually corresponds to the sparsely distributed crop area. Then, the path region between the target crops with the largest adjacent difference is selected, which is the sparsest crop distribution within that sub-region and also the core candidate region of the path. Finally, the maximum difference intervals selected from all sub-regions are summarized, and the complete path region is obtained through coordinate stitching and boundary smoothing (e.g., removing discrete outlier intervals and fitting the overall path contour).

[0075] To facilitate understanding by those skilled in the art, the following text is combined with Figure 4 and Figure 5 To further explain, Figure 4 Here is a flowchart of an example of an unstructured road recognition method applicable to agricultural unmanned vehicles. Figure 5This is a schematic diagram of the path region generated by an unstructured road recognition method.

[0076] exist Figure 4 The execution process in the unstructured road recognition method is as follows:

[0077] 1. Images are acquired using a binocular camera.

[0078] 2. Use the left eye image; use an object detection model to detect trees; extract the coordinates of the bounding boxes of the trees.

[0079] 3. Use binocular images; calculate the disparity map using a lightweight binocular detection model; obtain the depth map based on camera intrinsic parameters; select region growth points based on the autonomous vehicle's viewpoint and usage characteristics; obtain the path region using a region growing method.

[0080] 4. In the path region with trees, divide the image horizontally into multiple regions; record the x-values ​​of the tree coordinates in each region and sort them; keep only the path region with the largest difference between adjacent values; update the path regions.

[0081] The above method and process are used to obtain Figure 5 The diagram illustrates the path region. The red box represents the image, the yellow boxes represent the segmented sub-regions, the green circles represent trees (i.e., the target crop), and the black area represents the final fitted path region. First, the image area is defined as indicated by the red box, and the image is divided vertically into multiple non-overlapping rectangular sub-regions (i.e., the yellow rectangles). Then, the x-coordinate data of the green trees within each yellow rectangle is recorded, and the difference between adjacent trees is calculated. The group with the largest adjacent difference is identified (the diagram shows the group of trees with the largest adjacent difference). Next, path regions are fitted according to the selected groups of trees, as shown by the black curve.

[0082] The above section, with reference to specific embodiments, provides a detailed explanation of the unstructured road recognition method for agricultural unmanned vehicles provided in this application. The following section will further describe the unstructured road recognition system, medium, and terminal for agricultural unmanned vehicles provided in this application.

[0083] Figure 6 This illustration shows a schematic diagram of an unstructured road recognition system for agricultural unmanned vehicles provided by an embodiment of the present invention. The unstructured road recognition system 600 includes: an image acquisition module 601, a detection box extraction module 602, an initial path region generation module 603, and a final path region generation module 604.

[0084] Image acquisition module 601 is used to acquire stereo images. Detection box extraction module 602 is used to detect target crops using a target detection model based on one of the stereo images, and extract the coordinate data of the target detection boxes for the target crops. Initial path region generation module 603 is used to calculate a disparity map based on the stereo images, obtain a depth map based on the disparity map and camera intrinsic parameters, select initial growth points for the region, and obtain an initial path region. Final path region generation module 604 is used to divide the initial path region into several non-overlapping sub-regions, each containing target crops, sort the target crops within the divided sub-regions by spacing, and retain the path region between the group of target crops with the largest spacing as the final path region.

[0085] It should be noted that the implementation method and principle of the unstructured road recognition system applicable to agricultural unmanned vehicles are similar to those of the unstructured road recognition method applicable to agricultural unmanned vehicles described above, and will not be repeated here. It should also be understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division; other division methods may be used in actual implementation. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0086] In the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0087] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0088] Figure 7This is a schematic block diagram of the electronic terminal provided in an embodiment of this application. Figure 7 As shown, the electronic terminal includes at least one processor 701, a memory 702, at least one network interface 703, and a user interface 705. The various components in the device are coupled together via a bus system 704. It is understood that the bus system 704 is used to implement communication between these components. In addition to a data bus, the bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 7 The general will label all buses as bus systems.

[0089] The user interface 705 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0090] It is understood that memory 702 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0091] In this embodiment of the invention, the memory 702 is used to store various types of data to support the operation of the electronic terminal 700. Examples of this data include: any executable program for operation on the electronic terminal 700, such as the operating system 7021 and application program 7022; the operating system 7021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 7022 may contain various applications, such as a media player, browser, etc., for implementing various application services. The unstructured road recognition method for agricultural unmanned vehicles provided in this embodiment of the invention can be included in the application program 7022.

[0092] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 701. Processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 701 or by instructions in software form. The processor 701 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 701 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 701 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0093] In an exemplary embodiment, the electronic terminal 700 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0094] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute the above-described unstructured road recognition method applicable to agricultural unmanned vehicles.

[0095] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when run on a computer, causes the computer to perform the above-described method.

[0096] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0097] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0098] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0100] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0102] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs, DVDs), or semiconductor media (e.g., solid-state disks, SSDs, etc.).

[0103] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0105] In summary, this application provides a method, system, medium, and terminal for unstructured road recognition applicable to agricultural unmanned vehicles. The agricultural unmanned vehicle described in this application can automatically maintain its path during operation, effectively preventing it from deviating from the preset road. Firstly, this application does not rely on traditional image processing methods such as color threshold segmentation, and is no longer limited by changes in lighting and surface texture. It is applicable to both densely wooded roadside scenes and treeless open scenes. Furthermore, it possesses strong generalization ability, requiring no reliance on external features such as road weed coverage or color differences, and can accurately lock the operation path in various complex road environments, significantly improving the stability and adaptability of the operation process. Secondly, unlike traditional machine learning techniques, this application does not rely on human experience, allowing it to adapt well to diverse farmland scenarios, greatly increasing its universality and significantly improving the recognition rate in large-scale unstructured environments. Finally, this application does not require large-scale, precisely labeled data and exhibits excellent generalization performance under different crop, soil, and weather conditions. Therefore, this application improves spatial perception accuracy; avoids the influence of lighting; is applicable to various scenarios, whether there are many trees on both sides or no trees; avoids identifying paths solely through trees; and has strong generalization ability, requiring no specific road features. Thus, this application effectively overcomes the various shortcomings of existing technologies and has high industrial applicability.

[0106] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for unstructured road recognition suitable for agricultural unmanned vehicles, characterized in that, include: Acquire binocular images; Based on one of the binocular images, a target detection model is used to detect the target crop, and the coordinate data of the target detection box of the target crop is extracted. A disparity map is calculated based on the binocular image. A depth map is obtained based on the disparity map and combined with camera intrinsic parameters. An initial growth point for the region is selected, and an initial path region is obtained. The initial path region is divided into several non-overlapping sub-regions, each containing the target crop. The target crops within the sub-regions are sorted by spacing, and the path region between the target crops with the largest spacing is retained as the final path region.

2. The unstructured road recognition method according to claim 1, characterized in that, The extraction process of the coordinate data of the target detection box of the target crop includes: The left eye image acquired by the binocular camera is standardized according to the input requirements of the target detection model; the standardization process includes adjusting the image resolution, normalizing pixel values, and eliminating image distortion, or one or more of these. The preprocessed left eye image is input into the trained target detection model. The target detection model extracts multi-scale deep features of the target crops in the left eye image through the backbone network. After the multi-scale features are fused by the neck network, the head network outputs the prediction result, which includes the target category, confidence level and detection box coordinate data. Detection boxes with confidence levels below a preset threshold are filtered out, and the coordinate data of the remaining valid detection boxes are used as the coordinate data of the target detection box.

3. The unstructured road recognition method according to claim 1, characterized in that, The method further includes: calculating a disparity map based on the binocular images using a lightweight binocular detection model, the execution process of which is shown below: The input binocular images are preprocessed and then input into a lightweight backbone network. The standard convolution is split into depthwise convolution and pointwise convolution through depthwise separable convolution in the network to extract multi-scale features. A lightweight matching strategy is adopted. Based on the reference pixel in the left eye image, the optimal matching point that best matches the reference pixel is selected in the right eye image. The difference in the horizontal coordinate between the reference pixel and the optimal matching point is used as the disparity value corresponding to the reference pixel. The batch disparity value is calculated by traversing all pixels in the left eye image to generate a disparity map.

4. The unstructured road recognition method according to claim 3, characterized in that, The lightweight matching strategy is configured to: set a limited search range only in a preset local region around the reference pixel in the right eye image; use only pixel grayscale similarity and feature distance as filtering conditions; and filter the optimal matching point through a threshold.

5. The unstructured road recognition method according to claim 4, characterized in that, The method of setting a limited search range only within a preset local region surrounding the reference pixel in the right eye image includes: Based on the reference pixel in the left eye image, a local search region is delineated in the right eye image based on the epipolar constraint of the binocular camera, and the local search region is divided into several candidate point neighborhoods according to a preset neighborhood window. The reference point neighborhood where the reference pixel is located is determined according to the preset neighborhood window. The degree of fit between the reference point neighborhood and the neighborhood of each candidate point is calculated based on gray-level similarity and feature distance, and the optimal matching point is selected from them.

6. The unstructured road recognition method according to claim 1, characterized in that, The process of selecting an initial growth point and obtaining an initial path region includes: selecting an initial growth point based on the autonomous vehicle's shooting range and environmental perception requirements, and generating a path region based on the initial growth point using a region growth algorithm.

7. The unstructured road recognition method according to claim 6, characterized in that, The region growing algorithm is configured as follows: The initial growth point is stored in the seed point queue. Pixels in the queue are taken out one by one and their preset neighboring pixels are traversed. It is determined whether the preset growth criteria are met. If they are met, the pixel is added to the path region as a new seed point and stored in the queue. This process is repeated until the queue is empty. The preset growth criteria include: the depth difference between the adjacent pixel and the pixel in the current region must be less than a preset threshold, and the depth change rate of the pixel must be lower than a safety threshold.

8. An unstructured road recognition system suitable for agricultural unmanned vehicles, characterized in that, include: The image acquisition module is used to acquire stereo images; The detection box extraction module is used to detect target crops based on one of the binocular images using a target detection model, and to extract the coordinate data of the target detection box of the target crops. The initial path region generation module is used to calculate a disparity map based on the binocular image, obtain a depth map based on the disparity map and combined with camera intrinsic parameters, select an initial growth point for the region, and obtain an initial path region. The final path region generation module is used to divide the initial path region into several non-overlapping sub-regions, each of which contains the target crops. The target crops in the divided sub-regions are sorted by spacing, and the path region between the target crops with the largest spacing is retained as the final path region.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the unstructured road recognition method for agricultural unmanned vehicles as described in any one of claims 1 to 7.

10. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the unstructured road recognition method for agricultural unmanned vehicles as described in any one of claims 1 to 7.

Images