Autonomous navigation system of agricultural inspection robot and control method

Abstract

The invention relates to the technical field of robot navigation control, and discloses an agricultural inspection robot autonomous navigation system and a control method, the system comprises a data acquisition module, a positioning module, a prediction module, a correction module and a control module; wherein the data acquisition module is used for acquiring image data in an inspection path; the positioning module is used for acquiring a first positioning coordinate and a second positioning coordinate of the inspection robot; the prediction module is used for predicting the shielding condition in the inspection path; the correction module is used for dynamically optimizing the acquisition of the second positioning coordinates; and the control module controls the movement of the inspection robot according to the first positioning coordinate and the second positioning coordinate, and performs relaxation compensation on the inspection path according to the shielding condition. According to the invention, the autonomous navigation capability of the agricultural inspection robot in a complex environment is improved, the positioning precision and navigation accuracy are ensured, and the method has good environmental adaptability and is suitable for large-scale agricultural inspection scenes.

Description

Technical Field

[0001] The present application relates to the technical field of robot navigation control, and specifically to an autonomous navigation system and control method for an agricultural inspection robot. Background Art

[0002] One of the main challenges currently faced by agricultural inspection robots in complex farmland environments comes from the dynamic changes in crop growth. As the crop growth cycle progresses and environmental factors influence it, plant morphology, density, and spatial distribution constantly change. This time-varying nature places special demands on the robot's perception system and navigation algorithm. Traditional navigation solutions typically treat farmland environments as static scenes and are unable to effectively address the environmental changes brought about by crop growth. This often leads to positioning drift and path failure in practical applications. Especially during the peak crop season, dense branches and leaves not only obscure visual features but also create dynamic interference due to their natural swaying. These factors collectively affect the reliability and accuracy of inspection tasks.

[0003] For example, Chinese patent application publication number CN104793616A discloses a night vision navigation system for an agricultural harvesting robot, which belongs to the field of agricultural robot navigation. This night vision navigation system includes a machine vision module, a GPS module, a data fusion module, and a motion control module. The machine vision module and the GPS module process the collected information in real time and transmit it to the information fusion module. The information fusion module then fuses the received information and transmits it to the motion control module. The motion control module controls the movement of the agricultural harvesting robot by controlling the steering system, thereby achieving autonomous navigation. This application uses a night vision imaging system, combining visual navigation with GPS navigation, to effectively solve the navigation problem of the agricultural harvesting robot in night vision environments and improve navigation reliability.

[0004] For example, Chinese patent application publication number CN108227689A discloses a design method for autonomous navigation of an agricultural mobile robot. The method includes the following steps: analyzing the characteristics of various machine vision systems; designing an image acquisition and processing algorithm for crop row environments using a panoramic vision system based on the principle of catadioptric reflection; identifying navigation reference paths using the Hough transform; designing a navigation controller for the agricultural mobile robot based on fuzzy control methods; and conducting simulated crop row navigation control tests in an indoor, natural environment without fixed light sources. The design method for autonomous navigation of an agricultural mobile robot proposed in this application applies a panoramic vision system based on the principle of catadioptric reflection to the autonomous navigation of agricultural vehicles or agricultural robots. With the catadioptric panoramic vision system as the core, the method researches and designs navigation path recognition algorithms and navigation control algorithms for crop row environments on an existing hardware platform.

[0005] The above technical solutions all have the problem raised by this background technology: the navigation method cannot effectively cope with environmental changes.

[0006] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the application and should not be regarded as an admission or any form of suggestion that the information constitutes the prior art already known to ordinary technicians in this field. Summary of the Invention

[0007] The technical problem to be solved by this application is to overcome the defects of the existing technology, provide an agricultural inspection robot autonomous navigation system and control method, improve the autonomous navigation capability of the agricultural inspection robot in complex environments, and ensure positioning accuracy and navigation accuracy.

[0008] To solve the above technical problems, this application provides the following technical solutions:

[0009] On the one hand, the present application provides an autonomous navigation system for an agricultural inspection robot, comprising a data acquisition module, a positioning module, a prediction module, a correction module, and a control module; wherein:

[0010] The data acquisition module is used to collect image data along the inspection path;

[0011] The positioning module is used to perform real-time positioning of the inspection robot to obtain a first positioning coordinate; the positioning module also performs supplementary positioning of the inspection robot based on the image data to obtain a second positioning coordinate;

[0012] The prediction module predicts the occlusion situation in the inspection path based on the image data;

[0013] The correction module dynamically optimizes the acquisition of the second positioning coordinates based on the occlusion situation;

[0014] The control module controls the inspection robot to move along the inspection path based on the first positioning coordinates and the second positioning coordinates, and performs slack compensation on the inspection path based on the occlusion condition.

[0015] As a preferred solution of the autonomous navigation system for the agricultural inspection robot described in this application, the positioning module further includes a second positioning unit; the second positioning unit performs supplementary positioning of the inspection robot based on the image data to obtain a second positioning coordinate, specifically including:

[0016] Acquire a reference frame and extract reference coordinates and a reference heading angle corresponding to the reference frame; the reference frame is a path image used as a supplementary positioning reference; the reference coordinates are the coordinates of the inspection robot in the global coordinate system when the reference frame is acquired; the reference heading angle is the heading angle of the inspection robot when the reference frame is acquired;

[0017] Acquire a target frame; the target frame is a target path image for calculating a second positioning coordinate;

[0018] Detecting and matching feature points of the reference frame and the target frame to obtain candidate target feature points in the target frame; the candidate target feature points are feature points that have been successfully matched; selecting n target feature points from the candidate target feature points; n is a positive integer;

[0019] Calculating the target displacement of the inspection robot corresponding to the target frame based on the position of the target feature point in the reference frame and the target frame and the reference heading angle;

[0020] The second positioning coordinates of the inspection robot are calculated based on the displacement and the reference coordinates; and the target frame is marked as a new reference frame.

[0021] As a preferred solution of the autonomous navigation system for the agricultural inspection robot described in the present application, wherein: the prediction module is configured with an occlusion prediction model; the prediction module predicts the occlusion situation in the inspection path based on the occlusion prediction model; the occlusion situation includes the existence of fixed occlusion and dynamic occlusion;

[0022] The input of the occlusion prediction model includes the path image and the infrared image, and the output includes the probability of fixed occlusion and the probability of dynamic occlusion on the inspection path;

[0023] The prediction module predicts the occlusion situation in the inspection path based on the occlusion prediction model, specifically including:

[0024] The most recent n1 consecutive frame path images and infrared images are input into the occlusion prediction model, which calculates and outputs the probability of a fixed occlusion and the probability of a dynamic occlusion at the position of the inspection path corresponding to the next n2 consecutive frame path images; n1 and n2 are both positive integers;

[0025] The prediction module is also configured with a first threshold and a second threshold of the occlusion probability; if the probability of fixed occlusion corresponding to the path images of n2 consecutive frames in the future is greater than the first threshold, then there is fixed occlusion on the inspection path; if the average value of the probability of dynamic occlusion corresponding to the path images of n2 consecutive frames in the future is greater than the second threshold, then there is dynamic occlusion on the inspection path.

[0026] As a preferred solution of the autonomous navigation system for the agricultural inspection robot described in this application, the correction module is configured with a first correction strategy; if there is dynamic occlusion on the inspection path, the first correction strategy is triggered; the first correction strategy is used to dynamically optimize the selection of the target frame in the supplementary positioning, specifically including:

[0027] Setting a target time window, and extracting each frame of the path image in the target time window as a candidate target frame;

[0028] Perform feature point detection on each candidate target frame and match the feature points with the reference frame; record the number of candidate target feature points of each candidate target frame; the correction module is also configured with a feature number threshold; eliminate candidate target frames whose number of candidate target feature points is less than the feature number threshold;

[0029] For any candidate target frame, calculate the offset of each candidate target feature point; the offset is the distance between the position of the candidate target feature point in the reference frame and the position in the candidate target frame; calculate the offset consistency of all candidate target feature points; select the candidate target frame with the highest offset consistency as the target frame.

[0030] As a preferred solution of the autonomous navigation system for the agricultural inspection robot described in this application, the correction module is further configured with a second correction strategy; if there is dynamic occlusion on the inspection path, the second correction strategy is also triggered; the second correction strategy is used to dynamically optimize the selection of target feature points in the supplementary positioning, specifically including:

[0031] Calculate the outlier of each candidate target feature point in the target frame; the correction module is configured with an outlier threshold; select the candidate target feature point whose outlier is less than the outlier threshold as the target feature point; the correction module is also configured with a minimum feature threshold, and if the number of target feature points is less than the minimum feature threshold, trigger the addition of target feature points, specifically including:

[0032] Extract m consecutive frames of path images before the target frame as supplementary detection frames; m is a positive integer; perform feature point detection and matching on each supplementary detection frame; mark the candidate feature points that can be detected in all supplementary detection frames as supplementary feature points; calculate the outlier degree of each supplementary feature point in each supplementary detection frame, and calculate the outlier degree mean of each supplementary feature point; select r supplementary feature points with the smallest outlier degree mean to supplement the target feature point; r is the difference between the minimum feature threshold and the number of target feature points.

[0033] As a preferred solution of the autonomous navigation system for the agricultural inspection robot described in this application, wherein: the positioning module includes a first positioning unit; the first positioning unit is used to perform real-time positioning of the inspection robot and obtain a first positioning coordinate;

[0034] The first positioning unit is equipped with an RTK positioning device; the RTK positioning device is used to obtain the longitude and latitude coordinates of the inspection robot and convert the longitude and latitude coordinates into coordinates in a global coordinate system as first positioning coordinates.

[0035] As a preferred solution of the autonomous navigation system of the agricultural inspection robot described in this application, wherein: the control module includes an inspection positioning unit; the inspection positioning unit controls the inspection robot to move along the inspection path based on the first positioning coordinates and the second positioning coordinates, and the method is as follows:

[0036] Determine the real-time positioning coordinates of the inspection robot based on the first positioning coordinates and the second positioning coordinates; the inspection positioning unit is configured with a real-time positioning strategy, specifically including: determining whether the first positioning coordinates are stable, and if so, the real-time positioning coordinates are the first positioning coordinates; otherwise, the real-time positioning coordinates are the second positioning coordinates;

[0037] The real-time positioning strategy further includes: if the first positioning coordinate is stable, calibrating the second positioning coordinate by using the first positioning coordinate; specifically including: when calculating the second positioning coordinate, obtaining the first positioning coordinate synchronized with the reference frame time, replacing the reference coordinate corresponding to the reference frame;

[0038] The inspection and positioning unit is configured with a navigation route and a maximum deviation of the inspection path; the inspection and positioning unit controls the inspection robot to move along the inspection path based on the real-time positioning coordinates and the navigation route; the inspection and positioning unit also calculates the deviation of the real-time positioning coordinates from the navigation route. If the deviation is greater than the maximum deviation, the inspection path correction is triggered, and the inspection robot is controlled to move to the navigation route.

[0039] As a preferred solution of the autonomous navigation system of the agricultural inspection robot described in the present application, wherein: the control module also includes a path compensation unit; the path compensation unit is configured with a slack compensation strategy for compensating for the slack of the inspection path; the slack compensation strategy specifically includes: if there is dynamic occlusion on the inspection path, reducing the speed of the inspection robot and increasing the maximum deviation; if there is fixed occlusion on the inspection path, locally reconstructing the navigation route of the inspection path so that the navigation route bypasses the obstacles corresponding to the fixed occlusion.

[0040] As a preferred solution of the autonomous navigation system for agricultural inspection robots described in this application, wherein: the image data includes a path image and an infrared image;

[0041] The data acquisition module includes a network camera unit and a thermal imaging unit; wherein the network camera unit is used to acquire path images; the thermal imaging unit is used to acquire infrared images of the inspection path; the thermal imaging unit and the network camera unit synchronize the acquisition of path images and infrared images through timestamp alignment.

[0042] In a second aspect, the present application provides a method for controlling an agricultural inspection robot, comprising the following steps:

[0043] Collect image data along the inspection route;

[0044] Performing real-time positioning of the inspection robot to obtain a first positioning coordinate; performing supplementary positioning of the inspection robot based on the image data to obtain a second positioning coordinate;

[0045] predicting occlusion conditions in the inspection path based on the image data;

[0046] Dynamically optimizing the acquisition of the second positioning coordinates based on the occlusion situation;

[0047] The inspection robot is controlled to move along an inspection path based on the first positioning coordinates and the second positioning coordinates, and slack compensation is performed on the inspection path based on the occlusion condition.

[0048] Compared with the prior art, the beneficial effects achieved by this application are as follows:

[0049] This application achieves dual positioning assurance by combining real-time positioning with supplementary positioning based on image data, improving positioning accuracy and reliability and ensuring the stable operation of the inspection robot in complex environments. Dynamically adjusting the inspection robot's speed and path deviation tolerance based on occlusion conditions ensures the smooth completion of inspection tasks, effectively preventing the robot from straying from its intended route, and improving the stability and safety of inspections.

[0050] This application uses an occlusion prediction model to analyze path images and infrared images, enabling early prediction of fixed and dynamic occlusions, providing data support for path planning and correction. By dynamically optimizing the selection of target frames and target feature points, the interference of dynamic occlusion on positioning is reduced, improving the robustness of the system in occluded environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be derived from these drawings without inventive effort. Among them:

[0052] Figure 1 Schematic diagram of the structure of the autonomous navigation system for agricultural inspection robots provided in this application;

[0053] Figure 2 Flowchart of the agricultural inspection robot control method provided in this application;

[0054] Figure 3 This is a schematic diagram of the target frame and its alternative target feature points provided by this application. DETAILED DESCRIPTION

[0055] The technical solution of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Unless there is a conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0056] Example 1

[0057] This embodiment introduces an autonomous navigation system for an agricultural inspection robot. Figure 1 The system includes a data acquisition module, a positioning module, a prediction module, a correction module, and a control module; wherein:

[0058] The data acquisition module is used to collect image data along the inspection path;

[0059] The image data includes a path image and an infrared image;

[0060] The data acquisition module includes a network camera unit and a thermal imaging unit; wherein the network camera unit is used to acquire path images; the thermal imaging unit is used to acquire infrared images of the inspection path; the thermal imaging unit and the network camera unit synchronize the acquisition of path images and infrared images through timestamp alignment.

[0061] The path image is an optical image of the inspection path. This embodiment simultaneously collects optical images and infrared images, making it easier to determine the road conditions of the inspection path from different dimensions and improving the accuracy of identifying obstructions.

[0062] The positioning module is used to perform real-time positioning of the inspection robot to obtain a first positioning coordinate; the positioning module also performs supplementary positioning of the inspection robot based on the image data to obtain a second positioning coordinate;

[0063] The positioning module includes a first positioning unit and a second positioning unit;

[0064] The first positioning unit is used to perform real-time positioning of the inspection robot and obtain the first positioning coordinates; the first positioning unit is configured with an RTK (real-time kinematic) positioning device; the RTK positioning device is used to obtain the longitude and latitude coordinates of the inspection robot and convert the longitude and latitude coordinates into coordinates in the global coordinate system as the first positioning coordinates.

[0065] The RTK positioning device receives differential data from satellite signals and ground base stations and calculates the inspection robot's latitude and longitude based on the satellite signals and this differential data, with centimeter-level accuracy. This embodiment establishes a global coordinate system, expressing the first positioning coordinates obtained by the first positioning unit and the second positioning coordinates obtained by the second positioning unit in terms of coordinates within the global coordinate system, enabling unified processing of positioning data from different sources.

[0066] The second positioning unit performs supplementary positioning on the inspection robot based on the image data to obtain a second positioning coordinate, specifically including:

[0067] Obtain a reference frame and extract the reference coordinates and reference heading angle corresponding to the reference frame; the reference frame is a path image used as a supplementary positioning reference; the reference coordinates are the coordinates of the inspection robot in the global coordinate system when the reference frame is collected; the reference heading angle is the heading angle of the inspection robot when the reference frame is collected; in this embodiment, the global coordinate system is preferably a plane rectangular coordinate system, and the heading angle is further preferably the angle between the moving direction of the inspection robot and the coordinate axis of the global coordinate system.

[0068] Acquire a target frame; the target frame is a target path image for calculating the second positioning coordinates; that is, the second positioning coordinates are the coordinates of the inspection robot in the global coordinate system when the target frame is acquired. In this embodiment, the second positioning unit is provided with a basic positioning cycle, and the path image corresponding to the start of each basic positioning cycle is the target frame.

[0069] Feature points are detected and matched between the reference frame and the target frame to obtain candidate target feature points in the target frame; the candidate target feature points are successfully matched feature points; n target feature points are selected from the candidate target feature points; n is a positive integer; in this embodiment, the SIFT algorithm is preferably used for feature point detection; feature point matching is to identify which feature points in the reference frame and the target frame are the same feature point. For example, if the tip of the same leaf is detected as a feature point in both the reference frame and the target frame, then the feature point has a high similarity in the two frames and can be successfully matched. This feature point is a candidate feature point. In this embodiment, FLANN matching is also preferably used for feature point matching. The second positioning unit has multiple built-in screening strategies to automatically select target feature points from the successfully matched feature points, such as uniform sampling to avoid the aggregation of feature points in a local area, ensure the uniform distribution of global feature points, and improve the stability of subsequent positioning calculations. The number of target feature points n is at least 4, and the larger n is, the higher the calculation accuracy of the second positioning coordinates.

[0070] The target displacement of the inspection robot corresponding to the target frame is calculated based on the position of the target feature point in the reference frame and the target frame and the reference heading angle. In this embodiment, the method for calculating the target displacement of the inspection robot corresponding to the target frame is as follows:

[0071] The local displacement and heading angle change of the agricultural inspection robot from the corresponding moment in the reference frame to the corresponding moment in the target frame are calculated based on the positions of the target feature points in the reference frame and the target frame. The local displacement is the displacement in the robot's built-in coordinate system, not the global coordinate system, and needs to be converted to the displacement in the global coordinate system using the target heading angle in the global coordinate system. The reference heading angle and the heading angle change are added to obtain the target heading angle corresponding to the target frame. The displacement of the inspection robot in the global coordinate system is calculated using the target heading angle and the local displacement. The target heading angle is the angular difference between the global coordinate system and the inspection robot's built-in coordinate system. Once the angular difference between the two coordinate systems is known, the local displacement of the built-in coordinate system can be converted to the displacement in the global coordinate system using trigonometric functions.

[0072] The second positioning coordinates of the inspection robot are calculated based on the displacement and the reference coordinates; the target frame is marked as a new reference frame. In this embodiment, the second positioning coordinates are calculated as follows: let the reference coordinates include the horizontal coordinate x0 and the vertical coordinate y0, and decompose the displacement into the displacement Δx in the X-axis direction and the displacement Δy in the Y-axis direction; then the second positioning coordinates include the horizontal coordinate x0+Δx and the vertical coordinate y0+Δy. Supplementary positioning through path images is an iterative calculation process. Each iteration requires the calculation of the robot's second positioning coordinates through the feature points of the reference frame and the target frame; at the end of each iteration, the target frame of the current iteration is used as the reference frame for the next iteration, which is more suitable for scenarios where the inspection robot is continuously moving.

[0073] The prediction module predicts the occlusion situation in the inspection path based on the image data;

[0074] The prediction module is configured with an occlusion prediction model; the prediction module predicts the occlusion situation in the inspection path based on the occlusion prediction model; the occlusion situation includes the existence of fixed occlusion and dynamic occlusion;

[0075] The input of the occlusion prediction model includes the path image and the infrared image, and the output includes the probability of fixed occlusion and the probability of dynamic occlusion on the inspection path. Fixed occlusions include crop lodging, obstacle intrusion, and skewed crop planting or growth. That is, fixed occlusions are permanent and will not disappear without human intervention. They have a significant impact on the inspection movement of the inspection robot, but have no effect on the detection and screening of feature points when calculating the second positioning coordinates. Dynamic occlusions include the dynamic intrusion of crop branches and leaves into the inspection path under the action of wind. They have less impact on the inspection movement of the inspection robot, but due to the dynamic changes in features in the path image, they have a significant impact on the detection and screening of feature points in the target frame, increasing the calculation error of the second positioning coordinates.

[0076] The prediction module predicts the occlusion situation in the inspection path based on the occlusion prediction model, specifically including:

[0077] The most recent n1 consecutive frame path images and infrared images are input into the occlusion prediction model, and the occlusion prediction model calculates and outputs the probability of fixed occlusion and the probability of dynamic occlusion at the position of the inspection path corresponding to the future n2 consecutive frame path images; n1 and n2 are both positive integers; the specific values ​​of n1 and n2 are set by technical personnel in this field based on actual needs.

[0078] The prediction module is also configured with a first threshold and a second threshold of the occlusion probability; if the probability of fixed occlusion corresponding to the path images of n2 consecutive frames in the future is greater than the first threshold, then there is fixed occlusion on the inspection path; if the average value of the probability of dynamic occlusion corresponding to the path images of n2 consecutive frames in the future is greater than the second threshold, then there is dynamic occlusion on the inspection path.

[0079] Path images can show whether there are crops or obstacles invading the inspection path. Continuous path images can reflect the dynamics of occlusion. The temperature distribution and its change information provided by infrared images can serve as a supplement and verification to improve the prediction accuracy of occlusion conditions. The preferred occlusion prediction model of the present application includes an input layer, a feature extraction layer, a spatiotemporal coding layer, a feature fusion layer, a prediction layer, and an output layer; wherein the input layer receives a path image and an infrared image with timestamp alignment, extracts a pixel matrix and normalizes it, and provides standardized input for subsequent processing; the feature extraction layer adopts a lightweight convolutional network, combined with an attention mechanism, to extract features of the inspection path area from the input image, which is used to subsequently distinguish whether the inspection path is blocked; the spatiotemporal coding layer captures the time-varying features of dynamic occlusion and the continuous features of fixed occlusion through CNN modules and LSTM modules respectively; the feature fusion layer splices the features of the path image and the infrared image and then performs weighted fusion through the spatiotemporal attention mechanism to establish an association between the two images and enhance the feature expression of key areas; the prediction layer calculates the probability of fixed occlusion and dynamic occlusion in the inspection path in the future time period through a fully connected layer; the output layer outputs the probabilities of dynamic occlusion and fixed occlusion corresponding to the future n2 frames through 2n2 probability output nodes.

[0080] In this embodiment, the prediction module configures an NX industrial computer as the hardware basis for the operation of the occlusion prediction model; the above-mentioned occlusion prediction model can meet real-time requirements through lightweight model design combined with the TensorRT inference acceleration of the NX industrial computer, and is particularly suitable for edge AI scenarios such as farmland inspections.

[0081] The correction module dynamically optimizes the acquisition of the second positioning coordinates based on the occlusion situation;

[0082] The correction module is configured with a first correction strategy and a second correction strategy;

[0083] If there is dynamic occlusion on the inspection path, a first correction strategy is triggered; the first correction strategy is used to dynamically optimize the selection of the target frame in the supplementary positioning, specifically including:

[0084] A target time window is set, and each path image frame in the target time window is extracted as a candidate target frame. In this embodiment, the start time of the basic positioning cycle is preferably used as the midpoint of the target time window, that is, the target time window can cover the target frame under normal circumstances (when the first correction strategy is not triggered). The length of the target time window can be set and adjusted according to actual needs.

[0085] Perform feature point detection on each candidate target frame and match the feature points with the reference frame; record the number of candidate target feature points of each candidate target frame; the correction module is also configured with a feature number threshold; eliminate candidate target frames whose number of candidate target feature points is less than the feature number threshold;

[0086] For any candidate target frame, calculate the offset of each candidate target feature point; the offset is the distance between the position of the candidate target feature point in the reference frame and the position in the candidate target frame; calculate the offset consistency of all candidate target feature points;

[0087] The alternative target frame with the highest offset consistency is selected as the target frame. This embodiment preferably uses the standard deviation of the offset of each alternative target feature point as the offset consistency. As the inspection robot moves, each feature point will have a certain offset. When there is dynamic occlusion on the inspection path, such as branches and leaves swaying in the wind, the offset of the feature points located on the swaying branches and leaves will change significantly. Offset consistency, such as the standard deviation of the offset of all alternative target feature points, reflects the size of the dynamic difference between the alternative target frame and the reference frame. The higher the offset consistency, the closer the branch and leaf swinging state of the alternative target frame is to the reference frame, the smaller the swing error of the feature point, and the easier it is to extract the target feature point with a small calculation error.

[0088] If there is dynamic occlusion on the inspection path, a second correction strategy is also triggered; the second correction strategy is used to dynamically optimize the selection of target feature points in the supplementary positioning, specifically including:

[0089] Calculate the outlier degree of each candidate target feature point in the target frame; in this embodiment, the difference between the offset of any candidate target feature point and the mean offset of all candidate target feature points is preferably used as the outlier degree between the corresponding candidate target feature points;

[0090] Reference Figure 3 , Figure 3 A schematic diagram of the target frame; Figure 3Each dotted triangle box in the image marks an alternative target feature point in the target frame; when calculating the outlier of each alternative target feature point based on the second correction strategy provided in this embodiment, due to the swinging of branches and leaves, the position of the alternative target feature points located at the tip of the crop leaves, etc. in different path images changes more obviously, and the outlier is larger; the position of the alternative target feature points located at the root of the crop, the ridge of the field, etc. in different path images is relatively stable, and the outlier is smaller.

[0091] The correction module is configured with an outlier threshold; candidate target feature points with an outlier less than the outlier threshold are selected as target feature points; the correction module is also configured with a minimum feature threshold, and if the number of target feature points is less than the minimum feature threshold, the target feature points are supplemented, specifically including:

[0092] Extract m consecutive frames of path images before the target frame as supplementary detection frames; m is a positive integer; the value of m is set by a person skilled in the art based on actual needs; perform feature point detection and matching on each supplementary detection frame; mark the candidate feature points that can be detected in all supplementary detection frames as supplementary feature points; calculate the outlier degree of each supplementary feature point in each supplementary detection frame, and calculate the outlier degree mean of each supplementary feature point; select r supplementary feature points with the smallest outlier degree mean to supplement the target feature point; r is the difference between the minimum feature threshold and the number of target feature points.

[0093] In this embodiment, the mean outlier degree of any supplementary feature point is the mean outlier degree of the supplementary feature point in all supplementary detection frames and the target frame. When calculating the second positioning coordinates based on the reference frame and the target frame, it is necessary to ensure that the number of target feature points is not less than the minimum feature threshold to ensure the calculation accuracy. For example, if the minimum feature threshold is 8 and the number of target feature points is 5, the three supplementary feature points with the smallest mean outlier degree are added as target feature points. The supplementary feature points with the smallest mean outlier degree have the following characteristics: they can be detected in continuous path images, their outlier degree is generally small, and the calculation error introduced by participating in the calculation of the second positioning coordinates is small.

[0094] The control module controls the inspection robot to move along the inspection path based on the first positioning coordinates and the second positioning coordinates, and performs slack compensation on the inspection path based on the occlusion condition.

[0095] The control module includes a patrol positioning unit and a path compensation unit;

[0096] The inspection positioning unit controls the inspection robot to move along the inspection path based on the first positioning coordinates and the second positioning coordinates, and the method is as follows:

[0097] Determine the real-time positioning coordinates of the inspection robot based on the first positioning coordinates and the second positioning coordinates; the inspection positioning unit is configured with a real-time positioning strategy, specifically including: determining whether the first positioning coordinates are stable, and if so, the real-time positioning coordinates are the first positioning coordinates; otherwise, the real-time positioning coordinates are the second positioning coordinates;

[0098] In this embodiment, the preferred method for determining whether the first positioning coordinates are stable is as follows: if the number of satellites locked by the RTK positioning device configured by the first positioning unit is less than 4, or the signal-to-noise ratio of the satellite carrier signal received by the RTK positioning device is lower than 35dB, then the first positioning coordinates are unstable; otherwise, the first positioning coordinates are stable.

[0099] The real-time positioning strategy also includes: if the first positioning coordinate is stable, calibrating the second positioning coordinate by the first positioning coordinate; specifically including: when calculating the second positioning coordinate, obtaining the first positioning coordinate synchronized with the reference frame time, replacing the reference coordinate corresponding to the reference frame.

[0100] By positioning the inspection robot, the real-time collected images and other sensor data need to be bound to geographic coordinates for later analysis, such as marking the location of pests and diseases, etc., and at the same time ensure that the inspection robot moves according to the predetermined inspection path to ensure the integrity of the inspection coverage and inspection efficiency. The positioning accuracy of RTK positioning equipment is high, but in farmland scenarios, communications with satellites and ground base stations are easily blocked. When the number of locked satellites is too small or the signal-to-noise ratio of the satellite carrier signal is too low, the positioning accuracy drops significantly; the positioning coordinates are calculated through path images, which are not affected by the communication quality, but there are cumulative errors in the iterative calculation. This embodiment improves the positioning robustness and accuracy of the inspection robot in farmland environments such as greenhouses and orchards by monitoring the stability of the first positioning coordinates, correcting the second positioning coordinates when they are stable, reducing the cumulative error, and using the second positioning coordinates to assist navigation when they are unstable.

[0101] The inspection and positioning unit is configured with a navigation route and a maximum deviation of the inspection path; the inspection and positioning unit controls the inspection robot to move along the inspection path based on the real-time positioning coordinates and the navigation route; the inspection and positioning unit also calculates the deviation of the real-time positioning coordinates from the navigation route. If the deviation is greater than the maximum deviation, the inspection path correction is triggered, and the inspection robot is controlled to move to the navigation route.

[0102] In this embodiment, the preferred maximum deviation is 0.3 meters. The navigation route can be described by a curve equation in the global coordinate system. The curve equation is used to calculate the distance between the real-time positioning coordinates and the navigation route. If the deviation exceeds 0.3 meters, the robot's movement direction is forcibly corrected, for example by adjusting the robot's motor speed or steering angle, to gradually return it to the predetermined navigation route.

[0103] The path compensation unit is configured with a slack compensation strategy for compensating for slack in the inspection path; the slack compensation strategy specifically includes: if there is dynamic occlusion in the inspection path, reducing the speed of the inspection robot and increasing the maximum deviation; if there is fixed occlusion in the inspection path, locally reconstructing the navigation route of the inspection path so that the navigation route bypasses the obstacles corresponding to the fixed occlusion.

[0104] In this application, "slack compensation" means that under the premise of keeping the global path topology unchanged, the local path segment is allowed to produce lateral offset (such as detour) or longitudinal expansion (speed reduction), and the maximum deviation limit is appropriately relaxed. When there is dynamic occlusion on the inspection path, such as the dynamic swing of crop branches and leaves, the speed of the inspection robot is reduced to reduce the risk of collision with crops and ensure safe passage; at the same time, the positioning accuracy of the robot is affected by dynamic occlusion and is reduced. The slack compensation of the inspection path correction is achieved by increasing the maximum deviation, for example, adjusting it from 0.3 meters to 0.5 meters to reduce the number of forced corrections. When there is a fixed occlusion, the original navigation route needs to be changed to avoid collision, such as using a lidar or binocular camera to perceive the position of the obstacle, and generating an obstacle avoidance path through the built-in lightweight path planning algorithm of the NX industrial computer.

[0105] Example 2

[0106] This embodiment is the second embodiment of the present application; it is based on the same inventive concept as embodiment 1, and Figure 2 This embodiment introduces a control method for an agricultural inspection robot, comprising the following steps:

[0107] Collect image data along the inspection path; the image data includes optical images and infrared images of the inspection path.

[0108] The inspection robot is positioned in real time to obtain a first positioning coordinate; the inspection robot is supplementarily positioned based on the image data to obtain a second positioning coordinate; wherein the first positioning coordinate is calculated based on the differential data between the satellite signal received by the RTK positioning device and the ground base station, and the second positioning coordinate is calculated by the position change of the feature point in the continuous path image.

[0109] The occlusion conditions in the inspection path are predicted based on the image data; the occlusion conditions include fixed occlusions caused by crop lodging, obstacle invasion, crop planting or growth deviation, and dynamic occlusions caused by crop branches and leaves and other parts dynamically invading the inspection path under the action of wind.

[0110] The acquisition of the second positioning coordinates is dynamically optimized based on the occlusion situation; when there is dynamic occlusion on the inspection path, it is easy to cause the detection and matching of feature points in the path image to fail. This application ensures the calculation accuracy of the second positioning coordinates under dynamic occlusion by dynamically optimizing the selection of target frames and the screening of target feature points.

[0111] The inspection robot is controlled to move along an inspection path based on the first and second positioning coordinates, and slack compensation is performed on the inspection path based on the occlusion condition. First, based on the stability of the first positioning coordinates, it is determined whether to use the first or second positioning coordinates to determine the real-time position of the inspection robot. Then, based on the real-time position of the inspection robot, it is controlled to move along a predetermined inspection route. Slack compensation is performed on the inspection path based on the occlusion condition to reduce collision risks and ensure that the inspection robot safely passes through obstructed areas.

[0112] The specific functions of the above modules are implemented by referring to the relevant contents of the autonomous navigation system for agricultural inspection robots described in Example 1 and will not be elaborated on here.

[0113] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0114] The above describes the embodiments of the present application in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose and scope of protection of this application, all of which are protected by this application.

Claims

1. The autonomous navigation system of the agricultural inspection robot is characterized by: It includes data acquisition module, positioning module, prediction module, correction module and control module; among which: The data acquisition module is used to collect image data along the inspection path; The positioning module is used to perform real-time positioning of the inspection robot to obtain a first positioning coordinate; the positioning module also performs supplementary positioning of the inspection robot based on the image data to obtain a second positioning coordinate; The prediction module predicts the occlusion situation in the inspection path based on the image data; The correction module dynamically optimizes the acquisition of the second positioning coordinates based on the occlusion situation; The control module controls the inspection robot to move along the inspection path based on the first positioning coordinates and the second positioning coordinates, and performs slack compensation on the inspection path based on the occlusion condition.

2. The autonomous navigation system for agricultural inspection robots according to claim 1, wherein: The positioning module further includes a second positioning unit; the second positioning unit performs supplementary positioning of the inspection robot based on the image data to obtain a second positioning coordinate, specifically including: Acquire a reference frame and extract reference coordinates and a reference heading angle corresponding to the reference frame; the reference frame is a path image used as a supplementary positioning reference; the reference coordinates are the coordinates of the inspection robot in the global coordinate system when the reference frame is acquired; the reference heading angle is the heading angle of the inspection robot when the reference frame is acquired; Acquire a target frame; the target frame is a target path image for calculating a second positioning coordinate; Detecting and matching feature points of the reference frame and the target frame to obtain candidate target feature points in the target frame; the candidate target feature points are feature points that have been successfully matched; selecting n target feature points from the candidate target feature points; n is a positive integer; Calculating the target displacement of the inspection robot corresponding to the target frame based on the position of the target feature point in the reference frame and the target frame and the reference heading angle; The second positioning coordinates of the inspection robot are calculated based on the displacement and the reference coordinates; and the target frame is marked as a new reference frame.

3. The autonomous navigation system for agricultural inspection robots according to claim 2, wherein: The prediction module is configured with an occlusion prediction model; the prediction module predicts the occlusion situation in the inspection path based on the occlusion prediction model; the occlusion situation includes the existence of fixed occlusion and dynamic occlusion; The input of the occlusion prediction model includes the path image and the infrared image, and the output includes the probability of fixed occlusion and the probability of dynamic occlusion on the inspection path; The prediction module predicts the occlusion situation in the inspection path based on the occlusion prediction model, specifically including: The most recent n1 consecutive frame path images and infrared images are input into the occlusion prediction model, which calculates and outputs the probability of a fixed occlusion and the probability of a dynamic occlusion at the position of the inspection path corresponding to the next n2 consecutive frame path images; n1 and n2 are both positive integers; The prediction module is also configured with a first threshold and a second threshold of the occlusion probability; if the probability of fixed occlusion corresponding to the path images of n2 consecutive frames in the future is greater than the first threshold, then there is fixed occlusion on the inspection path; if the average value of the probability of dynamic occlusion corresponding to the path images of n2 consecutive frames in the future is greater than the second threshold, then there is dynamic occlusion on the inspection path.

4. The autonomous navigation system for agricultural inspection robots according to claim 3, wherein: The correction module is configured with a first correction strategy; If there is dynamic occlusion on the inspection path, the first correction strategy is triggered; The first correction strategy is used to dynamically optimize the selection of the target frame in the supplementary positioning, specifically including: Setting a target time window, and extracting each frame of the path image in the target time window as a candidate target frame; Perform feature point detection on each candidate target frame and match the feature points with the reference frame; record the number of candidate target feature points of each candidate target frame; the correction module is also configured with a feature number threshold; eliminate candidate target frames whose number of candidate target feature points is less than the feature number threshold; For any candidate target frame, calculate the offset of each candidate target feature point; the offset is the distance between the position of the candidate target feature point in the reference frame and the position in the candidate target frame; calculate the offset consistency of all candidate target feature points; select the candidate target frame with the highest offset consistency as the target frame.

5. The autonomous navigation system for agricultural inspection robots according to claim 4, characterized in that: The correction module is further configured with a second correction strategy; if there is dynamic occlusion on the inspection path, the second correction strategy is also triggered; The second correction strategy is used to dynamically optimize the selection of target feature points in the supplementary positioning, specifically including: Calculate the outlier of each candidate target feature point in the target frame; the correction module is configured with an outlier threshold; select the candidate target feature point whose outlier is less than the outlier threshold as the target feature point; the correction module is also configured with a minimum feature threshold, and if the number of target feature points is less than the minimum feature threshold, trigger the addition of target feature points, specifically including: Extract m consecutive frames of path images before the target frame as supplementary detection frames; m is a positive integer; perform feature point detection and matching on each supplementary detection frame; mark the candidate feature points that can be detected in all supplementary detection frames as supplementary feature points; calculate the outlier degree of each supplementary feature point in each supplementary detection frame, and calculate the outlier degree mean of each supplementary feature point; select r supplementary feature points with the smallest outlier degree mean to supplement the target feature point; r is the difference between the minimum feature threshold and the number of target feature points.

6. The autonomous navigation system for agricultural inspection robots according to claim 1, wherein: The positioning module includes a first positioning unit; the first positioning unit is used to perform real-time positioning of the inspection robot and obtain a first positioning coordinate; The first positioning unit is equipped with an RTK positioning device; the RTK positioning device is used to obtain the longitude and latitude coordinates of the inspection robot and convert the longitude and latitude coordinates into coordinates in a global coordinate system as first positioning coordinates.

7. The autonomous navigation system for agricultural inspection robots according to claim 6, characterized in that: The control module includes an inspection positioning unit; the inspection positioning unit controls the inspection robot to move along the inspection path based on the first positioning coordinates and the second positioning coordinates, and the method is as follows: Determine the real-time positioning coordinates of the inspection robot based on the first positioning coordinates and the second positioning coordinates; the inspection positioning unit is configured with a real-time positioning strategy, specifically including: determining whether the first positioning coordinates are stable, and if so, the real-time positioning coordinates are the first positioning coordinates; otherwise, the real-time positioning coordinates are the second positioning coordinates; The real-time positioning strategy further includes: if the first positioning coordinate is stable, calibrating the second positioning coordinate by using the first positioning coordinate; specifically including: when calculating the second positioning coordinate, obtaining the first positioning coordinate synchronized with the reference frame time, replacing the reference coordinate corresponding to the reference frame; The inspection and positioning unit is configured with a navigation route and a maximum deviation of the inspection path; the inspection and positioning unit controls the inspection robot to move along the inspection path based on the real-time positioning coordinates and the navigation route; the inspection and positioning unit also calculates the deviation of the real-time positioning coordinates from the navigation route. If the deviation is greater than the maximum deviation, the inspection path correction is triggered, and the inspection robot is controlled to move to the navigation route.

8. The autonomous navigation system for agricultural inspection robots according to claim 7, characterized in that: The control module also includes a path compensation unit; the path compensation unit is configured with a slack compensation strategy for compensating for slack in the inspection path; the slack compensation strategy specifically includes: if there is a dynamic occlusion in the inspection path, reducing the speed of the inspection robot and increasing the maximum deviation; if there is a fixed occlusion in the inspection path, locally reconstructing the navigation route of the inspection path so that the navigation route bypasses the obstacles corresponding to the fixed occlusion.

9. The autonomous navigation system for agricultural inspection robots according to claim 1, wherein: The image data includes a path image and an infrared image; The data acquisition module includes a network camera unit and a thermal imaging unit; wherein the network camera unit is used to acquire path images; the thermal imaging unit is used to acquire infrared images of the inspection path; the thermal imaging unit and the network camera unit synchronize the acquisition of path images and infrared images through timestamp alignment.

10. A control method for an agricultural inspection robot, which is implemented based on the autonomous navigation system for an agricultural inspection robot according to any one of claims 1 to 9, and is characterized in that: The following steps are involved: Collect image data along the inspection route; Performing real-time positioning of the inspection robot to obtain a first positioning coordinate; performing supplementary positioning of the inspection robot based on the image data to obtain a second positioning coordinate; predicting occlusion conditions in the inspection path based on the image data; Dynamically optimizing the acquisition of the second positioning coordinates based on the occlusion situation; The inspection robot is controlled to move along an inspection path based on the first positioning coordinates and the second positioning coordinates, and slack compensation is performed on the inspection path based on the occlusion condition.

Images