Agricultural inspection robot autonomous navigation system and control method

Abstract

The application relates to the technical field of robot navigation control, and discloses an agricultural inspection robot autonomous navigation system and a control method, the system comprising a data acquisition module, a positioning module, a prediction module, a correction module and a control module; wherein: the data acquisition module is used for acquiring image data in an inspection path; the positioning module is used for acquiring a first positioning coordinate and a second positioning coordinate of the inspection robot; the prediction module is used for predicting an occlusion condition in the inspection path; the correction module is used for dynamically optimizing the acquisition of the second positioning coordinate; and the control module controls the movement of the inspection robot according to the first positioning coordinate and the second positioning coordinate, and compensates the relaxation degree of the inspection path according to the occlusion condition. The application improves the autonomous navigation capability of the agricultural inspection robot in a complex environment, guarantees the positioning precision and navigation accuracy, has good environmental adaptability, and is suitable for large-scale agricultural inspection scenes.

Description

Technical Field

[0001] This application relates to the technical field of robot navigation and control, specifically to an autonomous navigation system and control method for an agricultural inspection robot. Background Technology

[0002] One of the main challenges currently faced by agricultural inspection robots in complex farmland environments stems from the dynamic changes during crop growth. As the crop growth cycle progresses and environmental factors influence the environment, plant morphology, density, and spatial distribution constantly change. This time-varying characteristic places unique demands on the robot's perception system and navigation algorithms. Traditional navigation solutions typically treat the farmland environment as a static scene, failing to effectively address the environmental changes brought about by crop growth. This often leads to problems such as positioning drift and path failure in practical applications. Especially during the peak crop growth period, dense foliage not only obscures visual features but also generates dynamic interference due to natural swaying, all of which collectively affect the reliability and accuracy of inspection tasks.

[0003] For example, Chinese patent application CN104793616A discloses a night vision navigation system for agricultural harvesting robots, belonging to the field of agricultural robot navigation. This night vision navigation system includes a machine vision module, a GPS module, a data fusion module, and a motion control module. The machine vision module and GPS module process the collected information in real time and transmit it to the data fusion module. The data fusion module then fuses the received information and transmits it to the motion control module. The motion control module controls the movement of the agricultural harvesting robot through a steering system, thereby achieving autonomous navigation. This application uses a night vision imaging system combined with visual navigation and GPS navigation, effectively solving the navigation problem of agricultural harvesting robots in night vision environments and improving the reliability of navigation.

[0004] For example, Chinese patent application CN108227689A discloses a design method for autonomous navigation of agricultural mobile robots, including the following steps: analyzing the characteristics of various machine vision systems; designing an image acquisition and processing algorithm for crop row environment based on a panoramic vision system using the catadioptric principle; identifying navigation reference paths using Hough transform; designing a navigation controller for the agricultural mobile robot based on fuzzy control methods; and conducting simulated crop row navigation control experiments in a natural indoor environment without fixed light source illumination. This application's design method for autonomous navigation of agricultural mobile robots proposes applying a panoramic vision system based on the catadioptric principle to the autonomous navigation of agricultural vehicles or robots; using the catadioptric panoramic vision system as the core, and on existing hardware platforms, researching and designing navigation path recognition algorithms and navigation control algorithms for crop row environments.

[0005] All of the above technical solutions suffer from the problem described in this background: the navigation method cannot effectively cope with environmental changes.

[0006] The information disclosed in this background section is intended only to enhance the understanding of the overall background of this application and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0007] The technical problem to be solved by this application is to overcome the defects of the prior art and provide an autonomous navigation system and control method for agricultural inspection robots, so as to improve the autonomous navigation capability of agricultural inspection robots in complex environments and ensure positioning accuracy and navigation accuracy.

[0008] To solve the above-mentioned technical problems, this application provides the following technical solution:

[0009] On one hand, this application provides an autonomous navigation system for an agricultural inspection robot, including a data acquisition module, a positioning module, a prediction module, a correction module, and a control module; wherein:

[0010] The data acquisition module is used to collect image data along the inspection path;

[0011] The positioning module is used to perform real-time positioning of the inspection robot and obtain the first positioning coordinates; the positioning module also performs supplementary positioning of the inspection robot based on the image data and obtains the second positioning coordinates.

[0012] The prediction module predicts occlusion conditions in the inspection path based on the image data;

[0013] The correction module dynamically optimizes the acquisition of the second positioning coordinates based on the occlusion situation;

[0014] The control module controls the inspection robot to move along the inspection path based on the first and second positioning coordinates, and performs slack compensation on the inspection path based on the occlusion situation.

[0015] As a preferred embodiment of the autonomous navigation system for the agricultural inspection robot described in this application, the positioning module further includes a second positioning unit; the second positioning unit performs supplementary positioning of the inspection robot based on the image data to obtain second positioning coordinates, specifically including:

[0016] Acquire a reference frame and extract the reference coordinates and reference heading angle corresponding to the reference frame; the reference frame is a path image used as a supplementary positioning reference; the reference coordinates are the coordinates of the inspection robot in the global coordinate system when the reference frame is acquired; the reference heading angle is the heading angle of the inspection robot when the reference frame is acquired.

[0017] Acquire the target frame; the target frame is the target path image for calculating the second positioning coordinates;

[0018] Feature points are detected and matched between the reference frame and the target frame to obtain candidate target feature points in the target frame; the candidate target feature points are the feature points that are successfully matched; n target feature points are selected from the candidate target feature points; n is a positive integer;

[0019] The target displacement of the inspection robot corresponding to the target frame is calculated based on the position of the target feature point in the reference frame and the target frame, as well as the reference heading angle.

[0020] The second positioning coordinates of the inspection robot are calculated based on the displacement and reference coordinates; the target frame is marked as the new reference frame.

[0021] As a preferred embodiment of the autonomous navigation system for agricultural inspection robots described in this application, the prediction module is configured with an occlusion prediction model; the prediction module predicts the occlusion situation in the inspection path based on the occlusion prediction model; the occlusion situation includes the existence of fixed occlusion and dynamic occlusion.

[0022] The occlusion prediction model takes path images and infrared images as inputs and outputs the probability that there is fixed occlusion and the probability that there is dynamic occlusion on the inspection path.

[0023] The prediction module predicts occlusion in the inspection path based on an occlusion prediction model, specifically including:

[0024] The most recent consecutive n1 frames of path images and infrared images are input into the occlusion prediction model. The occlusion prediction model calculates and outputs the probability that there is fixed occlusion and dynamic occlusion at the location of the inspection path corresponding to the next consecutive n2 frames of path images; n1 and n2 are both positive integers.

[0025] The prediction module is also configured with a first threshold and a second threshold for occlusion probability; if the probability of fixed occlusion in the path image corresponding to the next n2 consecutive frames is greater than the first threshold, then the inspection path has fixed occlusion; if the average probability of dynamic occlusion in the path image corresponding to the next n2 consecutive frames is greater than the second threshold, then the inspection path has dynamic occlusion.

[0026] As a preferred embodiment of the autonomous navigation system for the agricultural inspection robot described in this application, the correction module is configured with a first correction strategy; if dynamic occlusion exists on the inspection path, the first correction strategy is triggered; the first correction strategy is used to dynamically optimize the selection of target frames in the supplementary positioning, specifically including:

[0027] Set a target time window, and extract the path image of each frame in the target time window as a candidate target frame;

[0028] For each candidate target frame, feature point detection and feature point matching with the reference frame are performed; the number of candidate target feature points in each candidate target frame is recorded; the correction module is also configured with a feature number threshold; candidate target frames with a number of candidate target feature points less than the feature number threshold are removed;

[0029] For any candidate target frame, calculate the offset of each candidate target feature point; the offset is the distance between the position of the candidate target feature point in the reference frame and its position in the candidate target frame; calculate the consistency of the offsets of all candidate target feature points; select the candidate target frame with the highest offset consistency as the target frame.

[0030] As a preferred embodiment of the autonomous navigation system for the agricultural inspection robot described in this application, the correction module is further configured with a second correction strategy; if dynamic occlusion exists on the inspection path, the second correction strategy is also triggered; the second correction strategy is used to dynamically optimize the selection of target feature points in the supplementary positioning, specifically including:

[0031] The outlier degree of each candidate target feature point in the target frame is calculated; the correction module is configured with an outlier degree threshold; candidate target feature points with an outlier degree less than the outlier degree threshold are selected as target feature points; the correction module is also configured with a minimum feature threshold, and if the number of target feature points is less than the minimum feature threshold, the supplementation of target feature points is triggered, specifically including:

[0032] Extract the path images of m consecutive frames preceding the target frame as supplementary detection frames; m is a positive integer; perform feature point detection and matching on each supplementary detection frame; mark the candidate feature points that can be detected in all supplementary detection frames as supplementary feature points; calculate the outlier of each supplementary feature point in each supplementary detection frame, and calculate the mean outlier of each supplementary feature point; select the r supplementary feature points with the smallest mean outlier to supplement the target feature point; r is the difference between the minimum feature threshold and the number of target feature points.

[0033] As a preferred embodiment of the autonomous navigation system for agricultural inspection robots described in this application, the positioning module includes a first positioning unit; the first positioning unit is used to perform real-time positioning of the inspection robot and obtain first positioning coordinates;

[0034] The first positioning unit is equipped with an RTK positioning device; the RTK positioning device is used to obtain the latitude and longitude coordinates of the inspection robot and convert the latitude and longitude coordinates into coordinates in the global coordinate system as the first positioning coordinates.

[0035] As a preferred embodiment of the autonomous navigation system for the agricultural inspection robot described in this application, the control module includes an inspection positioning unit; the inspection positioning unit controls the inspection robot to move along the inspection path based on the first positioning coordinates and the second positioning coordinates, as follows:

[0036] The real-time positioning coordinates of the inspection robot are determined based on the first positioning coordinates and the second positioning coordinates; the inspection positioning unit is configured with a real-time positioning strategy, which specifically includes: determining whether the first positioning coordinates are stable; if so, the real-time positioning coordinates are the first positioning coordinates; otherwise, the real-time positioning coordinates are the second positioning coordinates.

[0037] The real-time positioning strategy further includes: if the first positioning coordinates are stable, then calibrate the second positioning coordinates using the first positioning coordinates; specifically, when calculating the second positioning coordinates, obtain the first positioning coordinates that are time-synchronized with the reference frame and replace the reference coordinates corresponding to the reference frame.

[0038] The inspection and positioning unit is configured with a navigation route and a maximum deviation of the inspection path; the inspection and positioning unit controls the inspection robot to move along the inspection path based on the real-time positioning coordinates and the navigation route; the inspection and positioning unit also calculates the deviation of the real-time positioning coordinates from the navigation route, and if the deviation is greater than the maximum deviation, it triggers the inspection path correction and controls the inspection robot to move to the navigation route.

[0039] As a preferred embodiment of the autonomous navigation system for agricultural inspection robots described in this application, the control module further includes a path compensation unit; the path compensation unit is configured with a slack compensation strategy for slack compensation of the inspection path; the slack compensation strategy specifically includes: if there is dynamic occlusion on the inspection path, the speed of the inspection robot is reduced and the maximum deviation is increased; if there is fixed occlusion on the inspection path, the navigation route of the inspection path is partially reconstructed so that the navigation route bypasses the obstacle corresponding to the fixed occlusion.

[0040] As a preferred embodiment of the autonomous navigation system for the agricultural inspection robot described in this application, the image data includes path images and infrared images;

[0041] The data acquisition module includes a network camera unit and a thermal imaging unit; wherein, the network camera unit is used to acquire path images; the thermal imaging unit is used to acquire infrared images of the inspection path; the thermal imaging unit and the network camera unit acquire path images and infrared images synchronously through timestamp alignment.

[0042] Secondly, this application provides a control method for an agricultural inspection robot, comprising the following steps:

[0043] Collect image data along the inspection path;

[0044] The inspection robot is located in real time to obtain the first positioning coordinates; the inspection robot is further located based on the image data to obtain the second positioning coordinates.

[0045] Predict occlusion conditions in the inspection path based on the image data;

[0046] The acquisition of the second positioning coordinates is dynamically optimized based on the occlusion situation;

[0047] Based on the first and second positioning coordinates, the inspection robot is controlled to move along the inspection path, and the slack compensation of the inspection path is performed based on the occlusion situation.

[0048] Compared with the prior art, the beneficial effects achieved by this application are as follows:

[0049] This application achieves dual positioning assurance by combining real-time positioning and supplementary positioning based on image data, improving the accuracy and reliability of positioning and ensuring stable operation of the inspection robot in complex environments. It dynamically adjusts the robot's speed and path deviation tolerance range according to occlusion conditions, ensuring the smooth completion of inspection tasks, effectively preventing the robot from deviating from the predetermined route, and improving the stability and safety of inspections.

[0050] This application utilizes an occlusion prediction model to analyze path images and infrared images, enabling the prediction of both fixed and dynamic occlusion in advance, providing data support for path planning and correction. By dynamically optimizing the selection of target frames and target feature points, the interference of dynamic occlusion on localization is reduced, improving the robustness of the system's localization in occluded environments. Attached Figure Description

[0051] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0052] Figure 1 A schematic diagram of the autonomous navigation system for the agricultural inspection robot provided in this application;

[0053] Figure 2 A flowchart of the agricultural inspection robot control method provided in this application;

[0054] Figure 3 A schematic diagram of the target frame and its alternative target feature points provided in this application. Detailed Implementation

[0055] The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of this application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.

[0056] Example 1

[0057] This embodiment describes an autonomous navigation system for an agricultural inspection robot, referring to... Figure 1 The system includes a data acquisition module, a positioning module, a prediction module, a correction module, and a control module; among which:

[0058] The data acquisition module is used to collect image data along the inspection path;

[0059] The image data includes path images and infrared images;

[0060] The data acquisition module includes a network camera unit and a thermal imaging unit; wherein, the network camera unit is used to acquire path images; the thermal imaging unit is used to acquire infrared images of the inspection path; the thermal imaging unit and the network camera unit acquire path images and infrared images synchronously through timestamp alignment.

[0061] The path image is an optical image of the inspection path. This embodiment simultaneously acquires optical and infrared images, which facilitates the assessment of road conditions from different dimensions and improves the accuracy of identifying obstructions.

[0062] The positioning module is used to perform real-time positioning of the inspection robot and obtain the first positioning coordinates; the positioning module also performs supplementary positioning of the inspection robot based on the image data and obtains the second positioning coordinates.

[0063] The positioning module includes a first positioning unit and a second positioning unit;

[0064] The first positioning unit is used to perform real-time positioning of the inspection robot and obtain the first positioning coordinates; the first positioning unit is equipped with an RTK (real-time dynamic) positioning device; the RTK positioning device is used to obtain the latitude and longitude coordinates of the inspection robot and convert the latitude and longitude coordinates into coordinates in the global coordinate system as the first positioning coordinates.

[0065] The RTK positioning device receives satellite signals and differential data from ground base stations, and calculates the latitude and longitude of the inspection robot based on the satellite signals and the differential data, with an accuracy down to the centimeter level. This embodiment establishes a global coordinate system, representing both the first positioning coordinates obtained by the first positioning unit and the second positioning coordinates obtained by the second positioning unit in the global coordinate system, thereby achieving unified processing of positioning data from different sources.

[0066] The second positioning unit performs supplementary positioning of the inspection robot based on the image data to obtain second positioning coordinates, specifically including:

[0067] A reference frame is acquired, and the reference coordinates and reference heading angle corresponding to the reference frame are extracted. The reference frame is a path image used as a supplementary positioning reference. The reference coordinates are the coordinates of the inspection robot in the global coordinate system when the reference frame is acquired. The reference heading angle is the heading angle of the inspection robot when the reference frame is acquired. In this embodiment, the global coordinate system is preferably a Cartesian coordinate system, and the heading angle is further preferably the angle between the movement direction of the inspection robot and the coordinate axis of the global coordinate system.

[0068] The target frame is acquired; the target frame is the target path image for calculating the second positioning coordinates; that is, the second positioning coordinates are the coordinates of the inspection robot in the global coordinate system when the target frame is acquired. In this embodiment, the second positioning unit is set with a basic positioning cycle, and the path image corresponding to the start time of each basic positioning cycle is the target frame.

[0069] Feature points are detected and matched between the reference frame and the target frame to obtain candidate target feature points in the target frame; these candidate target feature points are those that have been successfully matched. n target feature points are then selected from the candidate target feature points, where n is a positive integer. This embodiment preferably uses the SIFT algorithm for feature point detection. Feature point matching involves identifying which feature points in the reference frame and the target frame are the same. For example, if the same leaf tip is detected in both the reference and target frames, this feature point has high similarity in the two frames and can be successfully matched; this feature point is then considered a candidate feature point. This embodiment also preferably uses FLANN matching for feature point matching. The second positioning unit incorporates multiple filtering strategies to automatically select target feature points from the successfully matched feature points, such as uniform sampling to avoid feature points clustering in local areas, ensuring a uniform global distribution of feature points and improving the stability of subsequent positioning calculations. The number of target feature points, n, is at least 4, and the larger n is, the higher the accuracy of the second positioning coordinate calculation.

[0070] The target displacement of the inspection robot corresponding to the target frame is calculated based on the position of the target feature point in the reference frame and the target frame, as well as the reference heading angle. In this embodiment, the method for calculating the target displacement of the inspection robot corresponding to the target frame is as follows:

[0071] The local displacement and heading angle change of the agricultural inspection robot from the corresponding time in the reference frame to the corresponding time in the target frame are calculated by using the positions of target feature points in the reference frame and the target frame. The local displacement is the displacement in the robot's built-in coordinate system, not the global coordinate system. This needs to be converted to a global coordinate system displacement using the target heading angle. The reference heading angle and the heading angle change are added together to obtain the target heading angle corresponding to the target frame. The displacement of the inspection robot in the global coordinate system is then calculated using the target heading angle and the local displacement. The target heading angle is the angle difference between the global coordinate system and the inspection robot's built-in coordinate system. Knowing the angle difference between the two coordinate systems, the local displacement in the built-in coordinate system can be converted to the displacement in the global coordinate system using trigonometric functions.

[0072] The second positioning coordinates of the inspection robot are calculated based on the displacement and reference coordinates; the target frame is marked as the new reference frame. In this embodiment, the second positioning coordinates are calculated as follows: let the reference coordinates include the horizontal coordinate x0 and the vertical coordinate y0, and decompose the displacement into the displacement Δx in the X-axis direction and the displacement Δy in the Y-axis direction; then the second positioning coordinates include the horizontal coordinate x0+Δx and the vertical coordinate y0+Δy. Supplementing positioning through the path image is an iterative calculation process. Each iteration requires calculating the robot's second positioning coordinates using the feature points of the reference frame and the target frame; at the end of each iteration, the target frame of this iteration is used as the reference frame for the next iteration, which is more suitable for scenarios where the inspection robot is continuously moving.

[0073] The prediction module predicts occlusion conditions in the inspection path based on the image data;

[0074] The prediction module is configured with an occlusion prediction model; the prediction module predicts the occlusion situation in the inspection path based on the occlusion prediction model; the occlusion situation includes the existence of fixed occlusion and dynamic occlusion.

[0075] The occlusion prediction model takes as input the path image and infrared image, and outputs the probability of fixed occlusion and dynamic occlusion on the inspection path. Fixed occlusion includes crop lodging, obstacle intrusion, and crop planting or growth deviation, etc., which are permanent occlusions that will not disappear without human intervention. They significantly affect the inspection robot's movement but have no impact on feature point detection and selection when calculating the second positioning coordinates. Dynamic occlusion includes crop branches and leaves dynamically intruding into the inspection path under wind force. It has a smaller impact on the inspection robot's movement, but due to the dynamic changes in features in the path image, it significantly affects the detection and selection of feature points in the target frame, increasing the calculation error of the second positioning coordinates.

[0076] The prediction module predicts occlusion in the inspection path based on an occlusion prediction model, specifically including:

[0077] The most recent consecutive n1 frames of path images and infrared images are input into the occlusion prediction model. The occlusion prediction model calculates and outputs the probability that there is fixed occlusion and the probability that there is dynamic occlusion at the location of the inspection path corresponding to the next consecutive n2 frames of path images. n1 and n2 are both positive integers. The specific values ​​of n1 and n2 are set by those skilled in the art based on actual needs.

[0078] The prediction module is also configured with a first threshold and a second threshold for occlusion probability; if the probability of fixed occlusion in the path image corresponding to the next n2 consecutive frames is greater than the first threshold, then the inspection path has fixed occlusion; if the average probability of dynamic occlusion in the path image corresponding to the next n2 consecutive frames is greater than the second threshold, then the inspection path has dynamic occlusion.

[0079] Path images can show whether there are crops or obstacles intruding along the inspection path. Continuous path images can reflect the dynamics of occlusion. The temperature distribution and its changes provided by infrared images can serve as a supplement and verification, improving the accuracy of occlusion prediction. The preferred occlusion prediction model of this application includes an input layer, a feature extraction layer, a spatiotemporal coding layer, a feature fusion layer, a prediction layer, and an output layer. The input layer receives a timestamp-aligned path image and an infrared image, extracts the pixel matrix, and normalizes it to provide standardized input for subsequent processing. The feature extraction layer uses a lightweight convolutional network and an attention mechanism to extract features of the inspection path region from the input image, used to subsequently determine whether the inspection path is occluded. The spatiotemporal coding layer uses a CNN module and an LSTM module to capture the time-varying features of dynamic occlusion and the persistent features of fixed occlusion, respectively. The feature fusion layer concatenates the features of the path image and the infrared image and performs weighted fusion using a spatiotemporal attention mechanism to establish a correlation between the two images and enhance the feature representation of key regions. The prediction layer calculates the probability of fixed and dynamic occlusion on the inspection path in future time periods using a fully connected layer. The output layer outputs the probability of dynamic and fixed occlusion in the next n2 frames through 2n2 probability output nodes.

[0080] In this embodiment, the prediction module is configured with an NX industrial computer as the hardware foundation for the occlusion prediction model to run. The occlusion prediction model, through its lightweight design and combined with the TensorRT inference acceleration of the NX industrial computer, can meet the real-time requirements and is especially suitable for edge AI scenarios such as farmland inspection.

[0081] The correction module dynamically optimizes the acquisition of the second positioning coordinates based on the occlusion situation;

[0082] The correction module is configured with a first correction strategy and a second correction strategy;

[0083] If dynamic occlusion exists along the inspection path, a first correction strategy is triggered; the first correction strategy is used to dynamically optimize the selection of target frames in the supplementary positioning, specifically including:

[0084] A target time window is set, and each frame of the path image within the target time window is extracted as a candidate target frame. In this embodiment, the start time of the basic positioning cycle is preferably taken as the midpoint of the target time window, that is, the target time window can cover the target frames under normal conditions (when the first correction strategy is not triggered). The length of the target time window can be set and adjusted according to actual needs.

[0085] For each candidate target frame, feature point detection and feature point matching with the reference frame are performed; the number of candidate target feature points in each candidate target frame is recorded; the correction module is also configured with a feature number threshold; candidate target frames with a number of candidate target feature points less than the feature number threshold are removed;

[0086] For any candidate target frame, calculate the offset of each candidate target feature point; the offset is the distance between the position of the candidate target feature point in the reference frame and its position in the candidate target frame; calculate the consistency of the offsets of all candidate target feature points;

[0087] The candidate target frame with the highest offset consistency is selected as the target frame. In this embodiment, the standard deviation of the offset of each candidate target feature point is preferred as the offset consistency. As the inspection robot moves, each feature point will experience a certain offset. When there is dynamic occlusion along the inspection path, such as branches swaying in the wind, the offset of feature points located on the swaying branches will change significantly. Offset consistency, such as the standard deviation of the offsets of all candidate target feature points, reflects the magnitude of the dynamic difference between the candidate target frame and the reference frame. The higher the offset consistency, the closer the branch swaying state of the candidate target frame and the reference frame is to that of the reference frame, the smaller the swaying error of the feature points, and the easier it is to extract target feature points with small calculation errors.

[0088] If dynamic occlusion exists along the inspection path, a second correction strategy is triggered; the second correction strategy is used to dynamically optimize the selection of target feature points in the supplementary positioning, specifically including:

[0089] Calculate the outlier of each candidate target feature point in the target frame; in this embodiment, the difference between the offset of any candidate target feature point and the mean offset of all candidate target feature points is preferred as the outlier between the corresponding candidate target feature points.

[0090] Reference Figure 3 , Figure 3 A schematic diagram of the target frame; Figure 3Each dashed triangle in the image marks a candidate target feature point in the target frame. When calculating the outlier of each candidate target feature point based on the second correction strategy provided in this embodiment, due to the swaying of branches and leaves, the position of candidate target feature points located at the tips of crop leaves and other locations changes significantly in different path images, resulting in a large outlier. The position of candidate target feature points located at the roots of crops, field ridges and other locations is more stable in different path images, resulting in a smaller outlier.

[0091] The correction module is configured with an outlier threshold; it selects candidate target feature points with an outlier value less than the outlier threshold as target feature points; the correction module is also configured with a minimum feature threshold, and if the number of target feature points is less than the minimum feature threshold, it triggers the replenishment of target feature points, specifically including:

[0092] Extract m consecutive frames of path images preceding the target frame as supplementary detection frames; m is a positive integer; the value of m is set by those skilled in the art based on actual needs; perform feature point detection and matching on each supplementary detection frame; mark the candidate feature points that can be detected in all supplementary detection frames as supplementary feature points; calculate the outlier of each supplementary feature point in each supplementary detection frame, and calculate the mean outlier of each supplementary feature point; select the r supplementary feature points with the smallest mean outlier to supplement the target feature point; r is the difference between the minimum feature threshold and the number of target feature points.

[0093] In this embodiment, the mean outlier of any supplementary feature point is the mean of its outliers across all supplementary detection frames and the target frame. When calculating the second positioning coordinates based on the reference frame and the target frame, it is necessary to ensure that the number of target feature points is not less than the minimum feature threshold to guarantee calculation accuracy. For example, if the minimum feature threshold is 8 and the number of target feature points is 5, then the three supplementary feature points with the smallest mean outlier are added as target feature points. The supplementary feature points with the smallest mean outlier have the following characteristics: they can be detected in continuous path images, and their overall outlier is relatively small, resulting in a smaller calculation error introduced when they participate in the calculation of the second positioning coordinates.

[0094] The control module controls the inspection robot to move along the inspection path based on the first and second positioning coordinates, and performs slack compensation on the inspection path based on the occlusion situation.

[0095] The control module includes an inspection and positioning unit and a path compensation unit;

[0096] The inspection positioning unit controls the inspection robot to move along the inspection path based on the first positioning coordinates and the second positioning coordinates, as follows:

[0097] The real-time positioning coordinates of the inspection robot are determined based on the first positioning coordinates and the second positioning coordinates; the inspection positioning unit is configured with a real-time positioning strategy, which specifically includes: determining whether the first positioning coordinates are stable; if so, the real-time positioning coordinates are the first positioning coordinates; otherwise, the real-time positioning coordinates are the second positioning coordinates.

[0098] The preferred method for determining whether the first positioning coordinates are stable in this embodiment is as follows: if the number of satellites locked by the RTK positioning device configured in the first positioning unit is less than 4, or the signal-to-noise ratio of the satellite carrier signal received by the RTK positioning device is less than 35dB, then the first positioning coordinates are unstable; otherwise, the first positioning coordinates are stable.

[0099] The real-time positioning strategy further includes: if the first positioning coordinates are stable, then calibrate the second positioning coordinates using the first positioning coordinates; specifically, when calculating the second positioning coordinates, obtain the first positioning coordinates that are time-synchronized with the reference frame and replace the reference coordinates corresponding to the reference frame.

[0100] By locating the inspection robot, real-time collected images and other sensor data need to be bound to geographic coordinates for later analysis, such as marking the location of pests and diseases. This ensures the robot moves along a predetermined inspection path, guaranteeing complete coverage and efficiency. While RTK positioning devices offer high accuracy, communication with satellites and ground base stations is easily obstructed in agricultural settings. When the number of locked satellites is insufficient or the signal-to-noise ratio of the satellite carrier signal is low, positioning accuracy drops significantly. Calculating positioning coordinates using path images is unaffected by communication quality, but iterative calculations introduce cumulative errors. This embodiment monitors the stability of the first positioning coordinate. When stable, it corrects the second positioning coordinate to reduce cumulative errors; when unstable, it uses the second coordinate to assist navigation, improving the robustness and accuracy of the inspection robot's positioning in agricultural environments such as greenhouses and orchards.

[0101] The inspection and positioning unit is configured with a navigation route and a maximum deviation of the inspection path; the inspection and positioning unit controls the inspection robot to move along the inspection path based on the real-time positioning coordinates and the navigation route; the inspection and positioning unit also calculates the deviation of the real-time positioning coordinates from the navigation route, and if the deviation is greater than the maximum deviation, it triggers the inspection path correction and controls the inspection robot to move to the navigation route.

[0102] In this embodiment, the maximum deviation is preferably 0.3 meters. The navigation route can be described by a curve equation in the global coordinate system. The distance between the real-time positioning coordinates and the navigation route is calculated using the curve equation. When the deviation is greater than 0.3 meters, the robot's movement direction is forcibly corrected, for example, by adjusting the robot's motor speed or steering angle, to gradually bring it back to the predetermined navigation route.

[0103] The path compensation unit is configured with a slack compensation strategy for slack compensation of the inspection path. The slack compensation strategy specifically includes: if there is dynamic occlusion on the inspection path, the speed of the inspection robot is reduced and the maximum deviation is increased; if there is fixed occlusion on the inspection path, the navigation route of the inspection path is partially reconstructed so that the navigation route bypasses the obstacle corresponding to the fixed occlusion.

[0104] In this application, "slack compensation" refers to allowing lateral offsets (such as detours) or longitudinal stretching (decelerated passage) in local path segments while maintaining the global path topology, and appropriately relaxing the limit on the maximum deviation. When there are dynamic obstructions on the inspection path, such as the dynamic swaying of crop branches and leaves, the speed of the inspection robot is reduced to decrease the risk of collision with crops and ensure safe passage. At the same time, the robot's positioning accuracy is affected by the dynamic obstruction, and slack compensation for inspection path correction is achieved by increasing the maximum deviation, for example, adjusting it from 0.3 meters to 0.5 meters, to reduce the number of forced corrections. When there are fixed obstructions, the original navigation route needs to be changed to avoid collisions, for example, by using LiDAR or binocular cameras to perceive the position of obstacles and generating obstacle avoidance paths using the lightweight path planning algorithm built into the NX industrial computer.

[0105] Example 2

[0106] This embodiment is the second embodiment of this application; it is based on the same inventive concept as Embodiment 1, and refers to... Figure 2 This embodiment introduces a control method for an agricultural inspection robot, including the following steps:

[0107] Collect image data along the inspection path; the image data includes optical and infrared images of the inspection path.

[0108] The inspection robot is located in real time to obtain the first positioning coordinates; the inspection robot is further located based on the image data to obtain the second positioning coordinates; wherein the first positioning coordinates are calculated based on the differential data of satellite signals received by the RTK positioning device and ground base station, and the second positioning coordinates are calculated through the position changes of feature points in the continuous path image.

[0109] Based on the image data, predict the occlusion situation in the inspection path; the occlusion situation includes fixed occlusion caused by crop lodging, obstacle intrusion, crop planting or growth deviation, as well as dynamic occlusion caused by crop branches and leaves intruding into the inspection path under the action of wind.

[0110] The acquisition of the second positioning coordinates is dynamically optimized based on the occlusion situation. When there is dynamic occlusion on the inspection path, it is easy to cause the failure of feature point detection and matching in the path image. This application ensures the calculation accuracy of the second positioning coordinates under dynamic occlusion by dynamically optimizing the selection of target frames and the filtering of target feature points.

[0111] The inspection robot is controlled to move along the inspection path based on the first and second positioning coordinates, and slack compensation is performed on the inspection path based on the occlusion situation. First, the stability of the first positioning coordinates is used to determine whether to use the first or second positioning coordinates to determine the real-time position of the inspection robot. Then, based on the real-time position of the inspection robot, it is controlled to move along the predetermined inspection route, and slack compensation is performed on the inspection path based on the occlusion situation to reduce the risk of collision and ensure that the inspection robot safely passes through the occluded area.

[0112] The specific functions of each module described above are explained in the relevant content of the autonomous navigation system for agricultural inspection robots described in Example 1, and will not be repeated here.

[0113] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of protection of this application, and these forms are all within the protection scope of this application.

Claims

1. An autonomous navigation system for an agricultural inspection robot, characterized by: The system comprises a data acquisition module, a positioning module, a prediction module, a correction module and a control module. The data acquisition module is configured to acquire image data in the inspection path. The positioning module is configured to perform real-time positioning on the inspection robot to obtain a first positioning coordinate, and to perform supplementary positioning on the inspection robot based on the image data to obtain a second positioning coordinate. The positioning module comprises a second positioning unit configured to perform supplementary positioning on the inspection robot based on the image data to obtain a second positioning coordinate. The second positioning unit is configured to obtain a reference frame, extract a reference coordinate and a reference heading angle corresponding to the reference frame, and calculate a target displacement of the inspection robot corresponding to the target frame based on the position of the target feature point in the reference frame and the target frame and the reference heading angle. The second positioning unit is configured to obtain a target frame, calculate a second positioning coordinate of the inspection robot based on the displacement and the reference coordinate, and mark the target frame as a new reference frame. The prediction module is configured to predict an occlusion condition in the inspection path based on the image data. The occlusion condition includes the existence of fixed occlusions and dynamic occlusions. The correction module is configured to dynamically optimize the acquisition of the second positioning coordinate based on the occlusion condition. The correction module is configured with a second correction strategy. If the inspection path has a dynamic occlusion, the second correction strategy is triggered. The second correction strategy is configured to dynamically optimize the selection of the target feature point in the supplementary positioning. The second correction strategy is configured to calculate the outlying degree of each candidate target feature point in the target frame, select a candidate target feature point with an outlying degree less than a threshold as a target feature point, and trigger supplementary selection of the target feature point if the number of target feature points is less than a minimum feature threshold. The second correction strategy is configured to extract m consecutive path images before the target frame as supplementary detection frames, perform feature point detection and matching on each supplementary detection frame, mark all candidate feature points that can be detected in all supplementary detection frames as supplementary feature points, calculate the outlying degree of each supplementary feature point in each supplementary detection frame, calculate the average outlying degree of each supplementary feature point, and select r supplementary feature points with the smallest average outlying degree to supplement the target feature point. The control module is configured to control the inspection robot to move along the inspection path based on the first positioning coordinate and the second positioning coordinate, and to compensate for the relaxation of the inspection path based on the occlusion condition.

2. The agricultural inspection robot autonomous navigation system of claim 1, wherein: The prediction module is configured with an occlusion prediction model; the prediction module predicts the occlusion condition in the inspection path based on the occlusion prediction model; The input of the occlusion prediction model includes path images and infrared images, and the output includes the probability of fixed occlusion and the probability of dynamic occlusion in the inspection path; The prediction module predicts the occlusion condition in the inspection path based on the occlusion prediction model, specifically including: input the latest continuous frame path image and the infrared image into the occlusion prediction model, and the occlusion prediction model calculates and outputs the probability of existence of the fixed occlusion and the probability of existence of the dynamic occlusion at the position of the inspection path corresponding to the future continuous frame path image; , are positive integers. The prediction module is further configured with a first threshold and a second threshold of the occlusion probability; if the future continuous If the probabilities that the frame path images correspond to exist fixed occlusion are all greater than the first threshold, the inspection path exists fixed occlusion; if the average of the probabilities that the future continuous If the average of the probabilities that the frame path images correspond to exist dynamic occlusion is greater than the second threshold, the inspection path exists dynamic occlusion.

3. The agricultural inspection robot autonomous navigation system of claim 2, wherein: The correction module is configured with a first correction strategy; If there is dynamic occlusion in the inspection path, the first correction strategy is triggered; The first correction strategy is used to dynamically optimize the selection of target frames in the supplementary positioning, specifically including: Set a target time window, and extract each frame of path image in the target time window as a candidate target frame; Detect feature points of each candidate target frame and match feature points between the candidate target frame and the reference frame; record the number of candidate target feature points of each candidate target frame; the correction module is also configured with a feature quantity threshold; eliminate the candidate target frame whose number of candidate target feature points is less than the feature quantity threshold; For any candidate target frame, calculate the offset of each candidate target feature point; the offset is the distance between the position of the candidate target feature point in the reference frame and the position of the candidate target feature point in the candidate target frame; calculate the offset consistency of all candidate target feature points; select the candidate target frame with the highest offset consistency as the target frame.

4. The agricultural inspection robot autonomous navigation system of claim 1, wherein: The positioning module includes a first positioning unit; the first positioning unit is used to position the inspection robot in real time to obtain the first positioning coordinates; The first positioning unit is configured with an RTK positioning device; the RTK positioning device is used to obtain the latitude and longitude coordinates of the inspection robot, and convert the latitude and longitude coordinates into coordinates in the global coordinate system as the first positioning coordinates.

5. The agricultural inspection robot autonomous navigation system of claim 4, wherein: The control module includes an inspection positioning unit; the inspection positioning unit controls the inspection robot to move along the inspection path based on the first positioning coordinates and the second positioning coordinates, as follows: Determine the real-time positioning coordinates of the inspection robot based on the first positioning coordinates and the second positioning coordinates; the inspection positioning unit is configured with a real-time positioning strategy, specifically including: judging whether the first positioning coordinates are stable, if yes, the real-time positioning coordinates are the first positioning coordinates; otherwise, the real-time positioning coordinates are the second positioning coordinates; The real-time positioning strategy further includes: if the first positioning coordinates are stable, calibrate the second positioning coordinates through the first positioning coordinates; specifically including: when calculating the second positioning coordinates, obtain the first positioning coordinates synchronized with the reference frame time, instead of the reference coordinates corresponding to the reference frame; The inspection positioning unit is configured with a navigation route of the inspection path and a maximum deviation; the inspection positioning unit controls the inspection robot to move along the inspection path based on the real-time positioning coordinates and the navigation route; the inspection positioning unit also calculates the deviation of the real-time positioning coordinates from the navigation route, if the deviation is greater than the maximum deviation, the inspection path correction is triggered, and the inspection robot is controlled to move to the navigation route.

6. The agricultural inspection robot autonomous navigation system of claim 5, wherein: The control module further comprises a path compensation unit; the path compensation unit is configured with a slack compensation strategy for slack compensation of the inspection path; the slack compensation strategy specifically comprises: if there is dynamic occlusion in the inspection path, the speed of the inspection robot is reduced and the maximum deviation is increased; if there is fixed occlusion in the inspection path, the navigation route of the inspection path is locally reconstructed to make the navigation route bypass the obstacle corresponding to the fixed occlusion.

7. The agricultural inspection robot autonomous navigation system of claim 1, wherein: The image data comprises path images and infrared images; The data acquisition module comprises a network camera unit and a thermal imaging unit; wherein the network camera unit is used to acquire path images; the thermal imaging unit is used to acquire infrared images of the inspection path; the thermal imaging unit and the network camera unit synchronously acquire path images and infrared images through timestamp alignment.

8. A method for controlling an agricultural inspection robot, based on the autonomous navigation system for agricultural inspection robot according to any one of claims 1-7, characterized in that: The method comprises the following steps: acquiring image data in the inspection path; real-time positioning of the inspection robot to obtain first positioning coordinates; supplementary positioning of the inspection robot based on the image data to obtain second positioning coordinates; predicting the occlusion situation in the inspection path based on the image data; dynamically optimizing the acquisition of the second positioning coordinates based on the occlusion situation; controlling the inspection robot to move along the inspection path based on the first positioning coordinates and the second positioning coordinates, and compensating the inspection path based on the occlusion situation.

Images