Self-propelled tea garden management machine obstacle visual automatic identification system and identification method

Abstract

The application relates to the field of visual identification, and discloses a visual automatic obstacle identification system and method for a self-propelled tea garden management machine, which comprises the following steps: obtaining the inherent parameters of the self-propelled tea garden management machine, and completing the calibration of a visual sensor; synchronously collecting multi-modal image data of a tea garden working environment through the visual sensor; performing fusion processing on the multi-modal image data to generate fused image data; performing tea garden background stripping processing based on the fused image data to extract an obstacle candidate region; inputting the obstacle candidate region into a target detection network to identify the obstacle, and outputting the category and pixel position information of the obstacle; compensating the ranging result based on the binocular parallax ranging principle and in combination with the real-time posture parameters of the self-propelled tea garden management machine, and solving the spatial position of the obstacle relative to the self-propelled tea garden management machine; and then matching a preset danger level according to the spatial position and the category, and outputting an obstacle avoidance control instruction corresponding to the danger level.

Description

Technical Field

[0001] This invention relates to the field of visual recognition, specifically to an automatic visual recognition system and method for obstacle recognition in self-propelled tea garden management machines. Background Technology

[0002] Tea garden management machines face an extremely complex visual perception environment when operating between rows. On the one hand, the dense canopy of tea trees causes drastic fluctuations in light intensity, with alternating strong midday leaf reflections and morning / evening backlighting, causing traditional monocular visible light sensors to experience a sharp deterioration in image quality under such conditions. On the other hand, the work area is frequently obscured by various obstacles, including tea pickers, field stones, pruned dead branches, fallen tea branches, and abandoned farm tools. These obstacles vary widely in size and are often partially obscured by tea tree branches and leaves. Furthermore, the machine's movement is often bumpy, and lateral swaying is significant when operating on slopes, severely interfering with the accurate detection and ranging of obstacles. In addition, the onboard industrial control motherboards of tea garden management machines generally have low computing power, limiting the direct deployment of high-precision deep learning models on the machine.

[0003] Currently, infrared beam and ultrasonic obstacle avoidance solutions can only detect the presence of obstacles, but cannot distinguish the type and size of obstacles. They can only perform a one-size-fits-all emergency stop, resulting in low operational efficiency and a high risk of accidental stopping and damage to tea trees. General monocular vision solutions trained on open field datasets are not adapted to the high-density green background of tea gardens, with high missed detection rates under foliage obstruction and high false detection rates under strong light. Furthermore, monocular vision lacks depth ranging capabilities and cannot determine the dangerous distance of obstacles. Existing binocular vision solutions for agricultural machinery lack dynamic attitude compensation mechanisms for bumpy conditions on tea garden slopes, resulting in large ranging errors. At the same time, the model size is too large, making it difficult to deploy and run on low-computing-power industrial control boards. Existing tea garden vision technologies only target the identification of tea buds and shoots, and do not involve the hierarchical identification and autonomous obstacle avoidance linkage of multiple types of obstacles in the entire operation scenario. Summary of the Invention

[0004] The purpose of this invention is to provide an automatic visual recognition system and method for obstacles in self-propelled tea garden management machines, thereby solving at least one of the above-mentioned technical problems.

[0005] The objective of this invention can be achieved through the following technical solutions: An automatic obstacle visual recognition method for self-propelled tea garden management machines includes the following steps: Obtain the inherent parameters of the self-propelled tea garden management machine and complete the calibration of the vision sensor.

[0006] Multimodal image data of the tea garden working environment are collected synchronously using calibrated vision sensors. The multimodal image data includes visible light image data and infrared image data.

[0007] The acquired multimodal image data is fused to generate fused image data. The fusion weight is determined based on the ambient illuminance value at the time of acquisition: when the ambient illuminance value is higher than the high illuminance threshold, the fusion weight of the infrared image data is configured to be higher than that of the visible light image data; when the ambient illuminance value is lower than the low illuminance threshold, the fusion weight of the visible light image data is configured to be higher than that of the infrared image data; the high illuminance threshold is greater than the low illuminance threshold.

[0008] Based on the fused image data, a tea garden background stripping process is performed to extract obstacle candidate regions. Specifically, a fixed background region of the tea garden is extracted according to a preset color space threshold segmentation method. The preset color space is the HSV color space, and the fixed background region includes a green tea tree ridge area and a brown ridge soil area. Pixels falling into this fixed background region are masked and removed, and the remaining image region is the obstacle candidate region.

[0009] The candidate obstacle regions are input into the target detection network to perform obstacle recognition and output the obstacle category and its pixel location information in the image. Before inputting into the network, it is preferable to perform dynamic region of interest cropping on the fused image data to remove pixel regions whose ground clearance exceeds a preset height range. The target detection network adopts an improved CFD-YOLOv8 network, which uses the native YOLOv8n as the backbone, retains the overall architecture of the CSPDarknet backbone, PAN-FPN neck, and decoupled detection head, and implements the following three targeted improvements: (1) An RFCA (Receptive Field Attention Convolution) module is embedded at the bottleneck of the C2f module in the second and third stages of the backbone network. This module is composed of a multi-branch dilated convolution unit and a channel attention unit cascaded together. The multi-branch dilated convolution unit sets up three parallel branches and uses 3×3 dilated convolution with dilation rates of 1, 3 and 5 to sample the input feature map. Under the premise of no increase in the number of parameters, the shallow receptive field is expanded to capture the multi-scale spatial context of small target obstacles. The channel attention unit performs global average pooling and global max pooling on the multi-branch fused features. After two fully connected layers, the channel weight vector is generated and the features of each channel are weighted and calibrated to enhance the channel response of small target features and suppress redundant background channels, thereby realizing the enhanced extraction of spatial features of small-scale obstacles at the 5-12cm level.

[0010] (2) Replace all the 3×3 standard convolutional layers used for feature fusion in the neck PAN-FPN structure with Ghost lightweight convolutional modules. This module first generates intrinsic feature maps through a small number of 1×1 standard convolutions, and then performs cheap linear transformations such as 3×3 depthwise separable convolutions on the intrinsic feature maps to generate phantom feature maps. The two types of feature maps are then spliced ​​together to obtain the complete output. The floating-point operation of the neck network is significantly compressed while maintaining expressive power, so as to adapt to the deployment of low-computing embedded industrial control motherboards.

[0011] (3) The bounding box regression loss of the detection head adopts the occlusion adaptive loss function. This function is based on CIoU loss and introduces an additional branch and leaf occlusion weight factor ω. The occlusion ratio r is estimated by the intersection-union ratio of the predicted bounding box and the green background area of ​​the tea garden. The value range is 0 to 1. The occlusion weight ω = 1 + λ × r, where λ is the preset occlusion gain coefficient (value 0.2 to 0.5). The final regression loss L = ω × L (CIoU). When the obstacle is partially occluded by the branches and leaves of the tea garden, the occlusion ratio increases and the weight increases accordingly, thereby strengthening the loss weight of the bounding box regression, guiding the network to focus on the edge features of the occluded target, and reducing the localization deviation caused by branch and leaf occlusion.

[0012] During the identification process, the classification confidence threshold of the target detection network is dynamically determined based on the ambient illuminance value at the time of acquisition: when the illuminance value is greater than the high illuminance threshold, the confidence threshold is set to the first threshold; when the illuminance value is less than the low illuminance threshold, the confidence threshold is set to the second threshold; when the illuminance value is between the low illuminance threshold and the high illuminance threshold, the confidence threshold is set to the third threshold; wherein the first threshold is greater than the third threshold, and the third threshold is greater than the second threshold. After the obstacle candidate region is input into the network, the output results with a confidence value lower than the current classification confidence threshold are eliminated, and finally, the valid obstacle category and pixel location information are retained.

[0013] Based on the principle of binocular parallax ranging, and combined with the real-time attitude parameters of the self-propelled tea garden management machine, the ranging results are compensated to calculate the spatial position of the obstacle relative to the management machine. The specific compensation process is as follows: The pitch angle α, lateral yaw angle β, vertical bump height h, and travel speed v of the management machine are obtained, and the obstacle distance D is calculated using the following formula: D=(1+K1×α+K2×β-K3×h+K4×v)×(B×f) / d Where B is the baseline distance of the visual sensor, f is the focal length of the visual sensor, d is the subpixel parallax value, and K1, K2, K3, and K4 are the tilt angle-error fitting coefficient, deflection-error fitting coefficient, height-error fitting coefficient, and velocity-error fitting coefficient obtained through field calibration experiments on tea garden slopes, respectively.

[0014] Based on the calculated spatial location and obstacle type, a preset hazard level is matched. The hazard level is divided into multiple levels, each corresponding to a different distance threshold range. The specific matching rules are as follows: if the distance value is greater than or equal to the first distance threshold, the first hazard level is matched; if the distance value is less than the first distance threshold but greater than or equal to the second distance threshold, the second hazard level is matched; if the distance value is less than the second distance threshold but greater than or equal to the third distance threshold, the third hazard level is matched; if the distance value is less than the third distance threshold, the fourth hazard level is matched; where the first distance threshold > the second distance threshold > the third distance threshold.

[0015] Finally, based on the matched hazard level, the corresponding obstacle avoidance control command is output: the first hazard level corresponds to the constant speed passage command, the second hazard level corresponds to the deceleration warning command, the third hazard level corresponds to the detour planning command, and the fourth hazard level corresponds to the emergency parking command.

[0016] The self-propelled tea garden management machine obstacle visual automatic recognition system includes: The hardware sensing module includes a binocular RGB-infrared fusion camera, which is used to simultaneously acquire visible light image data and infrared image data; An environmental adaptive control module is used to collect ambient illuminance values ​​and adaptively turn on or off supplementary lighting or filter operations based on the ambient illuminance values. The vehicle body linkage correction module includes an IMU attitude sensor, which is used to collect the real-time attitude parameters of the self-propelled tea garden management machine; The vehicle-mounted edge computing module has a built-in embedded industrial control motherboard, which is loaded with program instructions. The agricultural machinery interactive execution module is used to electrically connect to the original vehicle electronic control unit of the self-propelled tea garden management machine, so as to output the obstacle avoidance control command to the original vehicle electronic control unit.

[0017] The beneficial effects of this invention are: (1) This invention uses a binocular RGB-infrared fusion camera to simultaneously acquire visible light and infrared dual-path images. Combined with an adaptive fusion strategy based on ambient illumination, it can obtain clear obstacle images under complex lighting conditions such as strong light reflection in tea gardens, tree shadows, and backlighting at dawn and dusk. Compared with a single visible light or fixed fusion scheme, it can effectively reduce the interference of image noise caused by leaf reflection and shadow on subsequent recognition algorithms and improve the imaging consistency under all-weather working conditions. (2) The present invention collects attitude parameters such as vehicle pitch angle and lateral deflection in real time through IMU attitude sensor, and introduces attitude compensation mechanism to correct the distance measurement results in the process of binocular parallax distance measurement. This can offset the parallax offset caused by tea garden slope operation and travel bumps, so that the spatial position calculation results of obstacles are closer to the actual scene, and avoid distance measurement distortion caused by vehicle attitude change and the resulting accidental obstacle avoidance action.

[0018] (3) This invention improves the CFD-YOLOv8 network structure by embedding an RFCA receptive field attention convolution module in the backbone network to enhance the spatial feature response of small-scale obstacles, and replaces the ordinary convolution of the neck network with Ghost lightweight convolution to compress the floating-point operation of the model. This can reduce the computing power requirements of the vehicle industrial control motherboard while maintaining the accuracy of obstacle classification and positioning, and is suitable for the embedded deployment of low computing power edge computing devices. At the same time, with the four-level hazard level judgment and corresponding obstacle avoidance command output, the system can perform passage, warning, detour or parking actions according to the differences in obstacle category and spatial location. This reduces invalid downtime while ensuring the safety of human and machine operation, and adapts to the automatic operation needs of narrow passage space between tea garden rows. Attached Figure Description

[0019] The invention will now be further described with reference to the accompanying drawings.

[0020] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1: Please see Figure 1 This invention provides a technical solution: a method for automatic obstacle visual recognition in a self-propelled tea garden management machine, comprising: S1. Machine calibration steps: complete the epipolar correction of the binocular camera, the calibration of the ridge spacing coordinate system, and the zero-point calibration of the IMU attitude sensor, and input the inherent parameters of the self-propelled tea garden management machine. S2, Multimodal image synchronous acquisition step: Visible light color images and infrared grayscale images are acquired in parallel by a binocular RGB-infrared fusion camera. The environment adaptive control module reduces noise in real time and filters out high light noise on the leaf surface, and fuses them to generate an anti-interference dual-channel fusion image. S3, Tea Garden Background Adaptive Stripping Step: Based on HSV color space threshold segmentation, extract the fixed background of the tea garden, remove static and harmless background objects by masking, and retain candidate areas for dynamic and hard obstacles. S4. Improve the target recognition steps of CFD-YOLOv8 by inputting the obstacle candidate region into the lightweight target detection network to complete the classification and pixel box localization of the obstacle, and output the category, pixel coordinates and confidence of the obstacle. S5. Binocular depth attitude compensation ranging step, combined with the principle of binocular parallax triangulation, superimposed the real-time attitude parameters of the agricultural machinery to correct the three-dimensional coordinates, and calculate the spatial position of the obstacle relative to the machine body. S6. Hazard level determination and linkage execution steps: Combine the real-time travel speed of the agricultural machinery with the spatial position of the obstacle to match the hazard level, and send the corresponding control command to the original vehicle electronic control unit of the tea garden management machine to complete the obstacle avoidance linkage. S7. Iterative identification steps: Repeatedly collect and identify data in a closed loop at a predetermined frame rate, dynamically update obstacle position information, and adapt to the dynamic travel conditions of agricultural machinery.

[0023] It should be noted that when the self-propelled tea garden management machine is operating between rows, the system must first undergo initial calibration. Specifically, the binocular RGB-infrared fusion camera is fixedly installed at the front centerline of the management machine frame, 75cm above the row surface, with a fixed camera pitch angle of 18° and a baseline distance of 120mm, covering a full field of view from 0.3m to 8m in front of the machine. After powering on, the system automatically performs epipolar correction on the binocular camera to ensure that the imaging planes of the left and right images are strictly aligned, eliminating parallax deviation caused by installation errors. Simultaneously, the IMU six-axis attitude sensor automatically completes attitude zero-point calibration after being stationary on flat ground for 3 seconds, recording the standard pitch angle, roll angle, and yaw angle of the vehicle at this time as the reference zero point. The operator inputs the machine's inherent parameters through the human-machine interface, including the body width, minimum turning radius, and braking response time. These parameters will be used in subsequent hazard level assessment and obstacle avoidance path planning. Taking an 18-horsepower tracked self-propelled tea garden pruning machine as an example, its body width is 85cm and its braking response time is 0.3s. The system stores these parameters in the non-volatile memory of the on-board edge computing motherboard.

[0024] In the S2, the binocular camera simultaneously acquires visible light images and 850nm infrared grayscale images at a frame rate of 30fps. The environment adaptive control module reads the ambient illuminance value in real time through the camera's built-in image sensor. When the illuminance is below 350 lux, the system determines it to be a low-light or backlight condition and automatically activates the soft infrared supplementary light group to enhance the texture information of the infrared image. When the illuminance is above 800 lux, the system determines it to be a strong light condition and automatically activates a pixel-level strong light filtering algorithm to suppress overexposure noise caused by specular reflection from the tea leaves. When the normal illuminance is between 350 lux and 800 lux, the system performs balanced fusion of the visible light and infrared images with a 1:1 weight. The fusion process is based on wavelet transform to complete pixel-level fusion of the two images. The pixel variance of the denoised image is reduced by about 36% compared to a single visible light image, effectively improving the imaging stability of subsequent recognition algorithms under complex lighting conditions.

[0025] In S3, the system adaptively strips the tea garden background from the fused image based on the HSV color space. Specifically, according to the actual scenario of tea garden inter-row operations, it presets the hue threshold range (H value between 35° and 85°) for the green tea tree ridge area and the hue threshold range (H value between 5° and 25°) for the brown ridge surface soil area, while setting lower limits for saturation and brightness. Pixel areas that meet the above conditions are marked as fixed background areas. Subsequently, the system masks and removes pixels falling into the fixed background areas in the fused image, retaining only the remaining image areas as obstacle candidate areas. This processing effectively eliminates interference from the green branches and leaves of the tea trees and the ridge surface soil, reducing the inference computation load of the subsequent object detection network. At the same time, the system removes weeds with a height of less than or equal to 3 cm and areas with an area of ​​less than 120 cm². 2 Invalid interference targets such as ground clods of soil were excluded, and only targets with a height greater than or equal to 5 cm and a bounding rectangle area greater than or equal to 120 cm² were retained. 2 Effective operational obstacles significantly reduce false alarm sources.

[0026] In S4, the obstacle candidate regions, after background stripping, are input into the improved CFD-YOLOv8 object detection network. This network is based on the YOLOv8n architecture. An RFCA receptive field attention convolution module is embedded in the C2f module of the second and third stages of the backbone network. The receptive field of the shallow feature map is expanded by multi-branch dilated convolution with dilation rates of 1, 3 and 5. Combined with channel attention weighting, the spatial feature response to small-scale obstacles (such as small dead branches and gravel) of 5cm to 12cm is enhanced. In the neck network, all the original 3×3 ordinary convolutional layers were replaced with Ghost lightweight convolutional layers, which reduced the model's floating-point operation volume from about 12G in the native YOLOv8 to 7.8G, making it suitable for deployment on embedded industrial control motherboards with 4TOPS computing power. In the detection head, an occlusion adaptive loss function is used instead of the traditional CIoU loss function. This loss function calculates the occlusion ratio of branches and leaves based on the intersection-union ratio of the predicted box and the green background. An occlusion weight factor is introduced to weight the CIoU loss. For typical working conditions where obstacles are partially occluded by tea tree branches and leaves and multiple obstacles overlap, the regression accuracy of the bounding box is optimized, which can effectively distinguish the edges of tea trees from the edges of obstacles.

[0027] The network training relies on the self-built TeaObstacle2026 tea garden real-world obstacle dataset. This dataset was collected from 12 standardized mountain tea gardens in western Zhejiang and southern Anhui, covering five typical working conditions: sunny days with strong sunlight, cloudy days with tree shade, morning and evening backlight, wet leaves after rain, and foliage occlusion. It contains a total of 18,624 labeled images, categorized into five types: living people, hard rocks, dead wood branches, fallen tea branches, and farm tools. The dataset is divided into training, validation, and test sets in a 7:2:1 ratio. During the inference phase, the network outputs the obstacle's category label, pixel bounding box coordinates, and classification confidence score. The system adaptively adjusts the confidence threshold based on ambient light levels, setting it to 0.82 under normal conditions to filter out false positives with low confidence.

[0028] In the S5, the system calculates the spatial position of obstacles based on the principle of binocular parallax triangulation. The binocular camera calculates the distance to obstacles using the parallax values ​​of the left and right images. The basic ranging formula is Do=(B×f) / d, where B is the baseline distance (120mm), f is the camera focal length, and d is the sub-pixel level parallax value after epipolar correction. However, when the tea garden management machine operates on slopes, the vehicle body experiences pitch tilting, and when traveling on bumpy terrain, it experiences lateral swaying. These dynamic disturbances directly lead to ranging errors.

[0029] Basic ranging compensation implementation method: In one basic embodiment of the present invention, to reduce the real-time computational load on the embedded controller, only the pitch angle α, which has the most significant impact on ranging accuracy, can be corrected using a first-order method. Specifically: The system uses a six-axis IMU attitude sensor to collect the vehicle's pitch angle α in real time. When α is greater than a preset angle (e.g., 3°), the parallax offset correction coefficient K is automatically activated. The corrected distance measurement formula is D=K×(B×f) / d, where K=1+K1×α. This correction coefficient is determined through the tilt angle-error fitting relationship obtained in field calibration tests on tea garden slopes and is applicable to tea garden operation slopes ranging from 0° to 12°. After attitude compensation, the three-dimensional coordinates (horizontal distance, vertical height, and lateral offset) of obstacles are accurately calculated, providing reliable distance input for subsequent hazard level determination.

[0030] In S6, the system combines the obstacle's category attributes, spatial location, and the current speed of the agricultural machinery to determine the obstacle and output corresponding obstacle avoidance commands according to the preset four-level hazard levels: First danger level (safe passage): The obstacle is 4.5m or more away from the machine body. Regardless of the type of obstacle, the agricultural machine should maintain its current speed (0-6km / h) and pass through. Second danger level (warning deceleration): When the obstacle is less than 4.5m and greater than or equal to 2.2m away from the machine, whether it is a hard obstacle or a living obstacle, the agricultural machinery must be forcibly decelerated to within 2km / h; Third danger level (detour planning): The obstacle is less than 2.2m and greater than or equal to 0.8m away from the machine body. Non-fixed stones, dead branches and other obstacles can be detoured. The system plans the local detour trajectory between the rows in combination with the machine body width (85cm) and the standard row spacing of the tea garden (1.2m±0.1m). Fourth Danger Level (Emergency Parking): When an obstacle is less than 0.8m away from the vehicle body, regardless of the type of obstacle, the system immediately sends an emergency parking command to the original vehicle's electronic control unit via the CAN bus to cut off the driving power and prioritize the safety of living personnel.

[0031] Obstacle avoidance control commands include constant speed passage commands, deceleration warning commands, detour planning commands, and emergency parking commands. These commands are sent to the original vehicle's electronic control unit via the CAN bus communication interface, with a communication delay of no more than 12ms. The success rate of emergency parking triggered by a living person at close range (less than 1m) is 100%, ensuring the safety of human-machine collaborative operations.

[0032] In S7, the system executes S2 to S6 in a closed-loop loop at a frame rate of 30fps. After each frame of image acquisition, preprocessing, recognition, ranging, and judgment are completed sequentially, and the position information of obstacles is dynamically updated. When the agricultural machinery is traveling at a speed of 0-6km / h, the pixel displacement of obstacles between adjacent frames is continuous and smooth. The system tracks the movement trajectory of the same obstacle through temporal correlation, reducing the jitter of obstacle avoidance actions caused by missed or false detections in a single frame.

[0033] In an optional embodiment, multimodal image data is fused to generate fused image data, including: Acquire the ambient illuminance value when acquiring multimodal image data; The fusion weights of visible light image data and infrared image data are determined based on the illuminance range in which the ambient illuminance value is located.

[0034] Furthermore, based on the illuminance range in which the ambient illuminance value falls, the fusion weights for visible light image data and infrared image data are determined, including: When the ambient illuminance value is greater than the high illuminance threshold, the fusion weight of the infrared image data is configured to be higher than that of the visible light image data. When the ambient illuminance value is less than the low illuminance threshold, the fusion weight of the visible light image data is configured to be higher than that of the infrared image data. Among them, the high illuminance threshold is greater than the low illuminance threshold.

[0035] It should be noted that the lighting conditions in the tea garden working environment vary drastically throughout the day. At midday, direct sunlight hits the tea tree canopy, resulting in strong specular reflection from the leaves and often causing large areas of overexposure in visible light images, obscuring the edges and textures of obstacles. Conversely, during dawn and dusk, oblique sunlight creates backlighting, making obstacle details difficult to discern in visible light images due to shadows. Therefore, this invention employs an adaptive lighting fusion strategy. Specifically, the system divides the ambient illuminance into three intervals: a strong illuminance interval (>800 lux), a normal illuminance interval (350-800 lux), and a weak backlight interval (<350 lux). In the strong illuminance interval, infrared images are unaffected by specular reflection from the leaves and can clearly reflect the contours and depth information of obstacles. Therefore, the fusion weight for infrared images is configured as 0.7, and the fusion weight for visible light images is configured as 0.3 to suppress leaf reflection noise. In low-light and backlight conditions, the infrared supplementary lighting is activated, resulting in a high signal-to-noise ratio for the infrared image, but insufficient texture detail. In this case, the fusion weight for the visible light image is configured as 0.7, and the fusion weight for the infrared image is configured as 0.3, utilizing the color and texture information retained in the visible light image to assist in category determination. In normal illumination conditions, both images exhibit relatively ideal quality, and a balanced fusion approach is adopted, with each image weighted at 0.5. Through this dynamic weight allocation strategy, the system can obtain high-quality fused images throughout all weather conditions.

[0036] In an optional embodiment, tea garden background stripping is performed based on fused image data to extract candidate obstacle regions, including: Based on a preset color space threshold segmentation, a fixed background area of ​​the tea garden is extracted; Pixels falling into a fixed background area in the fused image data are masked out, and the remaining image area is retained as an obstacle candidate area.

[0037] Furthermore, the preset color space is the HSV color space, and the fixed background area of ​​the tea garden includes the green tea tree ridge area and the brown ridge soil area.

[0038] It should be noted that the visual background between tea plantation ridges exhibits a high degree of regularity: the tea canopy is green, while the exposed soil on the ridges is brown, and the color distribution of both is relatively concentrated and stable in the HSV space. This invention utilizes this prior feature, employing a color thresholding method to achieve rapid background removal. In the HSV space, the hue (H) of the green tea plantation ridge area is between 35° and 85°, the saturation (S) is typically above 20%, and the brightness (V) is between 25% and 90%; the hue (H) of the brown soil on the ridge area is between 5° and 25°, the saturation (S) is typically below 40%, and the brightness (V) is between 15% and 60%. During system operation, each pixel of the fused image is traversed. If the HSV value of a pixel falls within either the aforementioned green or brown threshold range, the pixel is marked as a background pixel and set to a masked state, not participating in subsequent object detection network inference. The remaining non-background pixel areas are the obstacle candidate areas. In addition, considering the color shift of tea trees with the seasons (tender green in spring, dark green in summer, and a mixture of brown and green in winter), the system has four sets of HSV threshold parameters for spring, summer, autumn, and winter. Operators can manually select according to the season, or the system can automatically switch according to the date to ensure that the background stripping accuracy remains stable throughout the year.

[0039] In an optional embodiment, before inputting the obstacle candidate region into the target detection network for obstacle recognition, the method further includes: Dynamically crop the region of interest in the fused image data to remove pixel areas whose ground height exceeds a preset range.

[0040] It should be noted that when the tea garden management machine moves between rows, the camera's field of view includes not only obstacles on the ground but also invalid background elements such as the distant sky, distant trees, and the field ridges at the end of the tea rows. These areas are neither the targets that the agricultural machinery needs to avoid, and they also generate a large amount of useless convolution calculations, slowing down the inference speed. Therefore, this invention delineates an effective recognition region (i.e., a dynamic ROI) in the image space based on the standard row spacing of tea gardens (1.2m ± 0.1m) and camera installation parameters. Specifically, based on the camera's pitch angle (18°) and installation height (75cm), the pixel row range corresponding to a height of 5cm to 80cm above the ground in the image, and the pixel column range corresponding to the width of the tea row, are calculated. The system automatically crops out this effective recognition region in each frame of the image and only sends the pixels within this region to the subsequent background stripping and target detection modules. Taking a 1280×720 resolution image as an example, the effective pixel count after ROI cropping is approximately 68% of the original image, directly reducing the amount of invalid convolution calculations by about 32%, and reducing the inference time per frame by 6-8ms. At the same time, this preprocessing also fundamentally eliminates the possibility of background clutter such as distant birds or pedestrians being mistaken for obstacles.

[0041] In an optional embodiment, the obstacle candidate region is input into a target detection network for obstacle recognition, and the obstacle category and pixel location information are output, including: Acquire the ambient illuminance value when acquiring multimodal image data; The classification confidence threshold of the target detection network is determined based on the ambient illuminance value; Input the obstacle candidate region into the target detection network, and output the obstacle category, pixel location information and confidence score; Reject identification results with a confidence level lower than the classification confidence threshold.

[0042] Furthermore, the classification confidence threshold of the target detection network is determined based on the ambient illumination value, including: When the ambient illuminance value is greater than the high illuminance threshold, the classification confidence threshold is set to the first threshold. When the ambient illuminance value is less than the low illuminance threshold, the classification confidence threshold is set to the second threshold. When the ambient illuminance value is between the low illuminance threshold and the high illuminance threshold, the classification confidence threshold is set to the third threshold; Among them, the high illuminance threshold is greater than the low illuminance threshold, the first threshold is greater than the third threshold, and the third threshold is greater than the second threshold.

[0043] It should be noted that the classification confidence score output by the object detection network reflects the model's certainty about the recognition results. Under normal lighting conditions (350–800 lux), the model can obtain relatively stable recognition features. In this case, setting the confidence score threshold to 0.82 can achieve a good balance between false positives and false negatives.

[0044] Under strong reflective light conditions (illuminance greater than 800 lux), the specular reflection on the tea leaves generates a large number of specular noise points. These noise points exhibit obstacle-like properties in terms of texture and edge features, causing the model to mistakenly predict reflective areas on the leaves as obstacles and output abnormally high confidence scores, resulting in a large number of false targets. If a fixed threshold of 0.82 is still used in this situation, a large number of false targets will be included as valid detection results in subsequent obstacle avoidance decisions, leading to unnecessary false stops or detours. Therefore, when the illuminance is greater than 800 lux, the system raises the confidence threshold to 0.86, using a higher confidence threshold to filter out false bounding boxes generated by specular noise, ensuring that the detection results entering the obstacle avoidance decision have sufficiently reliable feature basis.

[0045] In low-light and backlight conditions (illuminance below 350 lux), with infrared illumination enabled, the texture features of the fused image are relatively clear, but the category color information is limited. The model's confidence in classifying some categories may be slightly low. In this case, the confidence threshold is lowered to 0.79 to maintain a sufficiently high recall rate and avoid missing high-priority obstacles such as live people. Through this dynamic threshold strategy, the system can obtain stable and reliable recognition output under different lighting conditions.

[0046] However, under strong reflective light conditions, leaf surface noise in visible light images increases, and some features are overexposed and masked, resulting in generally low confidence levels for the model output. If a fixed threshold of 0.82 is still used in this situation, a large number of obscured obstacles will be missed due to low confidence. Therefore, when the strong light intensity is greater than 800 lux, the system raises the confidence threshold to 0.86. Although the sensitivity decreases slightly, it effectively suppresses false targets caused by high light noise. When the weak backlight intensity is below 350 lux, infrared illumination is activated, and the texture features of the fused image are relatively clear, but the category color information is limited. The model's confidence in classifying some categories may be slightly low. In this case, the confidence threshold is lowered to 0.79 to maintain a sufficiently high recall rate and avoid missing high-priority obstacles such as live people. Through this dynamic threshold strategy, the system can obtain stable and reliable recognition output under different lighting conditions.

[0047] Preferred implementation method of multi-parameter fusion compensation: The basic implementation described above is suitable for tea gardens with relatively flat terrain. However, the actual tea garden working environment often involves lateral slopes, bumpy roads, and changes in vehicle speed, all of which can introduce nonlinear errors into binocular parallax ranging. Therefore, as a preferred implementation for further optimization of this basic scheme, the present invention also provides a multi-parameter fusion compensation scheme.

[0048] This preferred scheme, based on obtaining the pitch angle α, simultaneously... Obtain the pitch angle, yaw angle, vertical bump height, and travel speed of the self-propelled tea garden management machine; Calculate the distance D of the obstacle using the following formula: D=(1+K1×α+K2×β-K3×h+K4×v)×(B×f) / d; Where α is the pitch angle, β is the yaw angle, h is the vertical bump height, v is the travel speed, B is the baseline distance of the vision sensor, f is the focal length of the vision sensor, and d is the subpixel parallax value.

[0049] It should be noted that the multi-parameter formula in the preferred embodiment and the single-parameter formula in the basic embodiment are not mutually exclusive parallel solutions, but rather a progressive optimization relationship. When the tea garden working ground is flat (lateral deflection angle β≈0) and the driving is stable (vertical bumps h≈0 and the rate of change of speed v is extremely low), the three terms K2×β, -K3×h, and K4×v in the multi-parameter formula all approach zero. This preferred formula can naturally degenerate into the single-parameter correction formula D=(1+K1×α)×(B×f) / d in the basic embodiment. Thus, the two have an inherent unity in mathematical logic, ensuring both the simplicity and low computational power adaptability of the basic embodiment, while achieving high-precision ranging under complex working conditions through the preferred embodiment.

[0050] It should be noted that when tea garden management machines operate in mountainous tea gardens, their posture is not ideally horizontal and stationary, but is continuously affected by multiple disturbances such as terrain slope, uneven ridge surface, and engine vibration. Although a single tilt angle correction coefficient can compensate for the main ranging deviation caused by pitch tilt, it cannot fully cover the additional errors caused by lateral deflection (slope tilt), vertical bumps (ridge surface undulation), and changes in travel speed (inertia effect). Therefore, based on a large amount of field measurement data in tea gardens, this invention establishes a four-element coupled dynamic ranging correction model through multivariate nonlinear regression analysis. This model simultaneously introduces four dimensions of parameters: pitch angle α, lateral deflection angle β, vertical bump height h, and travel speed v. Wherein, β is the lateral rotation angle of the vehicle body around the forward direction, which is collected in real time by the roll angle channel of the IMU. During slope operations, β can reach 3°-8°; h is the instantaneous vertical displacement amplitude of the vehicle's center of gravity, reflecting the severity of the bumps, obtained by the second integral of the vertical acceleration of the IMU; v is the travel speed, read in real time by the vehicle speed pulse acquisition unit. The coefficients before each variable (0.017, 0.012, -0.004, 0.008) were obtained through cross-calibration tests with multiple slopes, speeds, and bump levels in the tea garden. Example: D=(1+0.017α+0.012β-0.004h+0.008v)×(B×f) / d. In actual operation, the system synchronously collects four-dimensional disturbance data at a frequency of 100Hz, calculates correction coefficients in real time, substitutes them into the distance measurement formula, and outputs the corrected distance D to the hazard level determination module.

[0051] Example 2: This invention provides a technical solution: an automatic obstacle visual recognition system for a self-propelled tea garden management machine, applicable to the aforementioned automatic obstacle visual recognition method for a self-propelled tea garden management machine, comprising: The hardware sensing module includes a binocular RGB-infrared fusion camera, which is used to simultaneously acquire visible light image data and infrared image data; An environmental adaptive control module is used to collect ambient illuminance values ​​and adaptively turn on or off supplementary lighting or filter operations based on the ambient illuminance values. The vehicle-mounted linkage correction module includes an IMU attitude sensor, which is used to collect real-time attitude parameters of the self-propelled tea garden management machine. The vehicle-mounted edge computing module has a built-in embedded industrial control motherboard. The industrial control motherboard is loaded with program instructions, and when the program instructions are executed, the identification method described in any of the above-described method embodiments is implemented. The agricultural machinery interactive execution module is used to electrically connect to the original vehicle electronic control unit of the self-propelled tea garden management machine, so as to output obstacle avoidance control commands to the original vehicle electronic control unit.

[0052] It should be noted that the connection relationships and data flow between the various modules of the system are as follows: The binocular camera of the hardware perception module is connected to the industrial control motherboard of the vehicle edge computing module via a USB 3.0 interface, transmitting visible light and infrared dual-channel image data in real time; the photosensitive chip of the environment adaptive control module is built into the binocular camera, and transmits data via I... 2 The C-bus communicates with the industrial control motherboard, reporting real-time illuminance values ​​and receiving supplemental lighting or filtering control commands. The IMU attitude sensor of the vehicle body linkage correction module is connected to the industrial control motherboard via the SPI interface, reporting three-axis acceleration, three-axis angular velocity, and three-axis tilt data at a frequency of 100Hz. The vehicle speed pulse acquisition unit is connected to the industrial control motherboard via a digital I / O port, reading the pulse frequency of the Hall sensor and converting it into travel speed. The vehicle edge computing module is an embedded industrial control motherboard with 4 TOPS computing power, equipped with an Intel Core i5-8365UE processor or an equivalent ARM architecture processor, configured with 8GB RAM and 64GB eMMC storage, running a Linux real-time operating system, and supporting inference acceleration of ONNX format lightweight models. The agricultural machinery interaction execution module is connected to the original vehicle travel electronic control unit via the CAN bus interface, supports the CAN2.0B communication protocol, sends obstacle avoidance commands at a baud rate of 250kbps, and the communication delay does not exceed 12ms. The entire system uses a 12V-24V wide-voltage vehicle power supply, adapting to voltage fluctuations in the tea garden management machine during start-up, acceleration, and braking. The front of the binocular RGB-infrared fusion camera is equipped with an arc-shaped transparent PC dustproof and scratch-proof protective cover. The surface of the protective cover is coated with a nano-hydrophobic anti-reflective coating, preventing water droplets from accumulating during operation after rain and protecting the lens from damage when scratched by branches and leaves.

[0053] In one specific system installation implementation, an 18-horsepower tracked self-propelled tea garden pruning machine is selected as the carrier. A binocular camera module is bolted to a mounting bracket at the center of the front end of the machine frame, at a height of 75cm from the ridge surface. The camera's tilt angle is fixed at 18°, ensuring that obstacles within a range of 0.3m to 8m in front of the machine are completely within the recognition field of view. The embedded industrial control motherboard is installed inside the machine's existing electrical control box, sharing power and grounding with the original vehicle controller. The IMU attitude sensor is rigidly fixed to the main beam of the vehicle body, ensuring that the relative position of the sensor and the vehicle body does not shift during operation. A CAN bus communication cable is led out from the industrial control motherboard and connected to the CAN bus port of the machine's travel controller via a standard DB9 interface. After the entire machine is installed, the operator performs one-click calibration through the host computer software: place the machine on a level ridge, the system automatically collects IMU zero-point data to complete attitude zeroing, automatically triggers the epipolar correction program of the binocular camera to complete parallax alignment, and the operator enters the machine parameters such as body width (85cm), minimum turning radius (1.2m), and braking response time (0.3s) through the interactive interface. After calibration, the system can be put into normal operation.

[0054] In one optional embodiment, the binocular RGB-infrared fusion camera has a baseline distance of 120mm, a camera pitch angle of 18°, a resolution of 1280×720, and a frame rate of 30fps. The environmental adaptive control module automatically activates infrared supplementary lighting when the ambient illuminance is below 350 lux and automatically enables a strong light filtering algorithm when the ambient illuminance is above 800 lux.

[0055] It should be noted that the above parameters are the result of multiple rounds of field measurements and optimizations based on the standard row spacing (1.2m±0.1m) and typical operating conditions in tea gardens. The 120mm baseline distance ensures sufficient parallax resolution within a 0.3m to 8m measurement range while keeping the camera module size within a reasonable range, facilitating installation in the confined space in front of the tea garden management machine. The fixed 18° pitch angle tilts the camera's optical axis forward and downward, ensuring that ground obstacles within the 0.3m-8m range are located in the middle and lower parts of the image, avoiding invalid pixels occupied by distant sky areas, and providing sufficient pixel height for classification and recognition of obstacles higher than 5cm. The 1280×720 resolution achieves a good balance between obstacle detail preservation and inference speed. The 30fps frame rate allows the system to track continuous positional changes of obstacles while traveling at 6km / h in real time, with the maximum displacement of obstacles between adjacent frames not exceeding 5% of the system's recognition field of view, ensuring the continuity of temporal tracking.

[0056] The system performs a quick self-check upon daily startup, including verifying the epipolar correction status of the binocular camera, checking the IMU zero-point drift (automatically zeroing if the drift exceeds ±0.5°), and performing a functional self-check of the supplementary lighting assembly. During daily operations, the system automatically collects approximately 200 background images of the tea garden each day, fine-tuning the HSV background segmentation threshold using a lightweight online learning algorithm to adapt to the gradual color changes of tea trees throughout their growth cycle. A complete system calibration is performed quarterly, including verifying and calibrating the focal length and distortion parameters of the binocular camera using a standard calibration board. The system storage media retains the operation logs and keyframe images from the past 30 days, facilitating post-operation review and analysis of failed identification cases, and continuous optimization of model performance. Through these operational and maintenance measures, the system maintains stable long-term recognition performance in continuous tea garden operations, requiring no frequent manual intervention or parameter adjustments.

[0057] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. An automatic obstacle visual recognition method for self-propelled tea garden management machines, characterized in that, include: Obtain the inherent parameters of the self-propelled tea garden management machine and complete the calibration of the vision sensor; The vision sensor synchronously acquires multimodal image data of the tea garden working environment, including visible light image data and infrared image data. The multimodal image data is fused to generate fused image data; Based on the fused image data, a background segmentation method based on color space thresholds is adopted to separate the fixed background region from the obstacle candidate region by utilizing the color prior information of the background between tea garden rows, and to extract the obstacle candidate region. The obstacle candidate region is input into the target detection network for obstacle recognition, and the category and pixel location information of the obstacle are output; the target detection network is an improved CFD-YOLOv8 network. Based on the principle of binocular parallax ranging, the ranging results are compensated by combining the real-time attitude parameters of the self-propelled tea garden management machine, and the spatial position of the obstacle relative to the self-propelled tea garden management machine is calculated. Based on the spatial location and the category, a preset hazard level is matched, and an obstacle avoidance control command corresponding to the hazard level is output.

2. The obstacle visual automatic recognition method for self-propelled tea garden management machines according to claim 1, characterized in that, The multimodal image data is fused to generate fused image data, including: Obtain the ambient illuminance value when the multimodal image data is acquired; Based on the illuminance range in which the ambient illuminance value falls, the fusion weights of the visible light image data and the infrared image data are determined; the specific process is as follows: When the ambient illuminance value is greater than the high illuminance threshold, the fusion weight of the infrared image data is configured to be higher than the fusion weight of the visible light image data; When the ambient illuminance value is less than the low illuminance threshold, the fusion weight of the visible light image data is configured to be higher than the fusion weight of the infrared image data; Wherein, the high illumination threshold is greater than the low illumination threshold.

3. The obstacle visual automatic recognition method for self-propelled tea garden management machines according to claim 1, characterized in that, Based on the fused image data, tea garden background stripping is performed to extract candidate obstacle regions, including: Based on a preset color space threshold segmentation, a fixed background area of ​​the tea garden is extracted; Pixels falling into the fixed background region in the fused image data are masked out, and the remaining image region is retained as the obstacle candidate region. The preset color space is the HSV color space, and the fixed background area of ​​the tea garden includes a green tea tree ridge area and a brown ridge soil area.

4. The obstacle visual automatic recognition method for self-propelled tea garden management machines according to claim 1, characterized in that, The distance measurement results are compensated by combining the real-time attitude parameters of the self-propelled tea garden management machine, and the spatial position of the obstacle relative to the self-propelled tea garden management machine is calculated, including: The pitch angle, lateral yaw angle, vertical bump height, and travel speed of the self-propelled tea garden management machine were obtained. The distance D of the obstacle is calculated using the following formula: D=(1+K1×α+K2×β-K3×h+K4×v)×(B×f) / d; Wherein, α is the pitch angle, β is the lateral deflection angle, h is the vertical turbulence height, v is the travel speed, B is the baseline distance of the visual sensor, f is the focal length of the visual sensor, d is the subpixel parallax value, and K1, K2, K3, and K4 are the tilt angle-error fitting coefficient, deflection-error fitting coefficient, height-error fitting coefficient, and speed-error fitting coefficient obtained through field calibration tests on tea garden slopes.

5. The obstacle visual automatic recognition method for self-propelled tea garden management machines according to claim 4, characterized in that, Before inputting the obstacle candidate region into the target detection network for obstacle recognition, the process also includes: The fused image data is dynamically cropped to a region of interest, and pixel regions in the fused image data that are outside a preset height range from the ground are removed. The backbone of the improved CFD-YOLOv8 network incorporates an RFCA receptive field attention convolutional module, which enhances the spatial feature extraction of small target obstacles. The ordinary convolutional layers in the neck network of the improved CFD-YOLOv8 network are replaced with Ghost lightweight convolutional layers. The detection head of the improved CFD-YOLOv8 network employs an occlusion adaptive loss function with a branch / leaf occlusion overlap weight factor.

6. The obstacle visual automatic recognition method for self-propelled tea garden management machines according to claim 5, characterized in that, The obstacle candidate region is input into the target detection network for obstacle recognition, and the obstacle category and pixel location information are output, including: Obtain the ambient illuminance value when the multimodal image data is acquired; The classification confidence threshold of the target detection network is determined based on the ambient illuminance value; The obstacle candidate region is input into the target detection network, which outputs the obstacle category, pixel location information, and confidence level. Identification results with a confidence level lower than the classification confidence threshold are removed.

7. The obstacle visual automatic recognition method for self-propelled tea garden management machines according to claim 6, characterized in that, Determining the classification confidence threshold of the target detection network based on the ambient illumination value includes: When the ambient illuminance value is greater than the high illuminance threshold, the classification confidence threshold is set to the first threshold. When the ambient illuminance value is less than the low illuminance threshold, the classification confidence threshold is set to the second threshold. When the ambient illuminance value is between the low illuminance threshold and the high illuminance threshold, the classification confidence threshold is set to the third threshold; Wherein, the first threshold is greater than the third threshold, and the third threshold is greater than the second threshold.

8. The obstacle visual automatic recognition method for self-propelled tea garden management machines according to claim 1, characterized in that, The preset danger levels include multiple levels, with different levels corresponding to different distance threshold ranges; Matching a preset hazard level based on the spatial location and the category includes: If the distance value at the spatial location is greater than or equal to a first distance threshold, a first hazard level is matched; If the distance value is less than the first distance threshold and greater than or equal to the second distance threshold, a second hazard level is matched; If the distance value is less than the second distance threshold and greater than or equal to the third distance threshold, a third hazard level is matched; If the distance value is less than the third distance threshold, a fourth hazard level is matched; Wherein, the first distance threshold is greater than the second distance threshold, and the second distance threshold is greater than the third distance threshold.

9. The obstacle visual automatic recognition method for self-propelled tea garden management machine according to claim 8, characterized in that, The obstacle avoidance control commands include constant speed passage commands, deceleration warning commands, detour planning commands, and emergency parking commands. The first hazard level corresponds to the constant speed passage command, the second hazard level corresponds to the deceleration warning command, the third hazard level corresponds to the detour planning command, and the fourth hazard level corresponds to the emergency parking command.

10. An automatic obstacle visual recognition system for a self-propelled tea garden management machine, the system being used to execute the automatic obstacle visual recognition method for a self-propelled tea garden management machine as described in any one of claims 1-9, characterized in that, include: The hardware sensing module includes a binocular RGB-infrared fusion camera, which is used to simultaneously acquire visible light image data and infrared image data; An environmental adaptive control module is used to collect ambient illuminance values ​​and adaptively turn on or off supplementary lighting or filter operations based on the ambient illuminance values. The vehicle body linkage correction module includes an IMU attitude sensor, which is used to collect the real-time attitude parameters of the self-propelled tea garden management machine; The vehicle-mounted edge computing module has a built-in embedded industrial control motherboard, which is loaded with program instructions. The agricultural machinery interactive execution module is used to electrically connect to the original vehicle electronic control unit of the self-propelled tea garden management machine, so as to output obstacle avoidance control commands to the original vehicle electronic control unit.

Images