Automatic driving weeding robot control method, device and system

Abstract

The invention discloses a control method, device and system for an automatic driving weeding robot. The control method comprises the following steps that a preset driving path is obtained; position information of the automatic driving vehicle and an RGB image, collected by a first image collection unit, in front of the automatic driving vehicle are obtained; calculating a first course deviation according to the position information of the autonomous vehicle and the first anchor point corresponding to the position information; determining a visual navigation line according to crops in the RGB image, and calculating a second course deviation according to the visual navigation line and a corresponding second anchor point; performing weighted summation on the first course deviation and the second course deviation to obtain a third course deviation; and determining a steering angle of the autonomous vehicle according to the third course deviation, and calculating control voltages of motors on two sides of the autonomous vehicle according to the steering angle so as to reduce the shortest distance between the autonomous vehicle and a predetermined driving path. The control accuracy of automatic driving can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of weeding, and in particular relates to a control method, device and system for an automatic driving weeding robot. Background Art

[0002] Automatic weeding using a lawnmower can greatly reduce labor. When using a lawnmower for automatic weeding, it is generally driven automatically according to a preset path or the path of a seeder. The existing automatic driving according to a preset path uses the "tractor automatic driving system" with application number CN201910281278.3, which dynamically detects and determines the deviation of the tractor's current driving path from the preset driving path of the tractor, and controls the steering of the tractor according to the deviation of the tractor's current driving path from the preset driving path of the tractor, so that the tractor can drive on the preset driving path.

[0003] This technology achieves control only by the deviation of the current driving path from the preset driving path of the tractor. Since the row spacing of crops is small and the accuracy of positioning data is limited, its control accuracy is limited and it is not suitable for occasions with small row spacing of crops. Summary of the invention

[0004] In order to solve the problem of low control accuracy of existing control methods, the present invention proposes a control method, device and system for an automatic driving weeding robot.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A first aspect of the present invention discloses an automatic driving control method, comprising the following steps:

[0007] Obtaining a planned driving route;

[0008] Acquire position information of the autonomous driving vehicle and an RGB image in front of the autonomous driving vehicle acquired by the first image acquisition unit;

[0009] A first heading deviation is calculated based on the position information of the autonomous driving vehicle and a first anchor point corresponding to the position information, wherein the first aiming point is a point on the predetermined driving path;

[0010] Determine a visual navigation line according to the crops in the RGB image and calculate a second heading deviation according to the visual navigation line and a corresponding second anchor point, where the second anchor point is a point on the visual navigation line;

[0011] Taking a weighted sum of the first heading deviation and the second heading deviation to obtain a third heading deviation;

[0012] The steering angle of the autonomous driving vehicle is determined based on the third heading deviation, and the control voltage of the motors on both sides of the autonomous driving vehicle is calculated based on the steering angle to reduce the shortest distance between the autonomous driving vehicle and the predetermined driving path.

[0013] A second aspect of the present invention discloses a weeder control method, including an automatic driving control method and a weeding control method, wherein the automatic driving control method is an automatic driving control method described in the first aspect, and the weeding control method includes the following steps:

[0014] Acquire the image to be recognized acquired by the second image acquisition unit;

[0015] Identify weeds according to the image to be identified and determine the three-dimensional coordinates of the weeds in the robot arm coordinate system;

[0016] The distance between the position of the weeds and the mechanical arm is calculated, and the mechanical arm is driven to move to the position of the weeds according to the distance, and then the weeding component is started to achieve weeding.

[0017] The third aspect of the present invention discloses a control device, comprising a memory and a controller which are communicatively connected in sequence, wherein a computer program is stored in the memory, and the controller is used to read the computer program and execute an automatic driving control method described in the first aspect or a lawn mower control method described in the second aspect.

[0018] A third aspect of the present invention discloses an automatic driving system, comprising:

[0019] A host computer, wherein the host computer is the control device described in the third aspect and a first information transceiver unit connected to the control device;

[0020] At least one autonomous driving vehicle, the autonomous driving vehicle comprising a vehicle body, a control unit arranged on the vehicle body, and a first image acquisition unit, a navigation receiver, and a second information transceiver unit, all of which are signal-connected to the control unit, wherein the first image acquisition unit is used to acquire image information in front of the vehicle body in the direction of travel, and the first information transceiver unit is used to exchange information with the second information transceiver unit.

[0021] The fourth aspect of the present invention discloses an automatic driving system, including a vehicle body, a control unit arranged on the vehicle body, a first image acquisition unit and a navigation receiver both of which are signal-connected to the control unit, wherein the first image acquisition unit is used to acquire image information in front of the vehicle body in the direction of travel; the control unit is the control device described in the third aspect.

[0022] The beneficial effects of the present invention are:

[0023] The present invention determines a first heading deviation and a second heading deviation respectively through navigation position data and a visual navigation line determined by a visual image, and then determines a final heading deviation based on the first heading deviation and the second heading deviation, thereby improving the control accuracy of automatic driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 The figure is a flow chart of the automatic driving control method of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0027] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0029] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0030] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the invention is usually placed when in use, or the positions or positional relationships commonly understood by those skilled in the art, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0031] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0032] The first aspect of the present invention discloses an automatic driving control method, which can be executed by, but is not limited to, an automatic driving control device. The automatic driving control device can be software, or a combination of software and hardware. The automatic driving control device can be integrated into smart devices such as smart mobile terminals, tablets, computers, clouds, and PC host computers. The automatic driving control device can be not only an automatic driving control device for a lawn mower, but also an automatic driving control device for a lumberjack, a harvester, etc. Figure 1 As shown, the method includes steps S11 to S17. It should be noted that the step identifiers in this solution are only for the convenience of explaining the method and do not constitute a limitation on the order of precedence. The order of each step is based on its language description and the order of each signal.

[0033] Step S11, obtaining a planned driving route.

[0034] The predetermined driving path can be not only a specific driving path, but also a historical driving path of a certain vehicle. Taking the historical driving path of a certain vehicle as an example, when weeding, the predetermined driving path can be the driving path of the seeder during sowing; when harvesting, the predetermined driving path can be the driving path of the seeder during sowing. When felling trees, the predetermined driving path can be a driving path set manually.

[0035] For example, taking the scheduled driving path of a weeder as an example, when the seeder is sowing, a navigation receiver is installed on the frame of the seeder. In order to improve the positioning accuracy, the navigation receiver can be a dual-antenna Beidou-RTK navigation receiver, a satellite navigation receiver, etc. The navigation receiver outputs the precise latitude and longitude, heading, and speed information of the seeder to the PC host computer in real time every 1 second. When the seeder completes the field sowing operation, the PC host computer records the running path of the seeder and smoothes the path, which is used as the scheduled driving path of the subsequent weeder.

[0036] Step S12: Acquire the position information of the autonomous driving vehicle and the RGB image in front of the autonomous driving vehicle captured by the first image acquisition unit.

[0037] The autonomous driving vehicle is not only equipped with a navigation receiver to obtain real-time positioning information, but also has a first image acquisition unit installed in front of the autonomous driving vehicle to acquire RGB image information in front of the vehicle.

[0038] The first image acquisition unit may be a depth camera, such as a D435i.

[0039] In order to improve the accuracy of positioning information, the navigation receiver can also be a dual-antenna Beidou-RTK navigation receiver, a satellite navigation receiver, etc.

[0040] The position information can be the positioning information output by the navigation receiver. In order to further improve the accuracy of the positioning information, the Beidou navigation architecture uses the differential data of the network base station to make real-time corrections to the positioning information of the mobile station. The positioning information is transmitted to the PC host computer through the Modbus-RTU protocol. The PC host computer combines the positioning information with the feedback information of the left motor encoder and the right motor encoder, and uses the extended Kalman filter algorithm to eliminate the interference of environmental changes and information frequency on the positioning information, providing smooth position, speed and heading information for navigation. The left motor encoder and the right motor encoder collect the left motor speed and the right motor speed respectively.

[0041] Specifically, using the extended Kalman filter algorithm to eliminate interference of environmental changes and information frequency on positioning information includes steps 121 to 123.

[0042] Step 121, discretize the position information, heading information and speed information in the continuous motion equation to obtain the discretized motion equation:

[0043] x k =x k-1 +v*cos(yaw k-1 )*dt,y k =y k-1 +v*sin(yaw k-1 )*dt,

[0044] Determine the state quantity based on the discretized equation System input u k =[v r ,v l ] T , the observation matrix Among them, x, y are position information, yaw is heading information, l is the distance between the left and right motor positions, v r is the right motor speed, v l is the speed of the left motor. The speeds of the left and right motors are calculated based on the number of pulses given by the left and right brushless motor encoders, the number of single-turn encoder pulses and the rotation radius of the gear driven by the motor.

[0045] Step 122, according to the state quantity at the previous moment With the current system input u k For the current state The predicted prior estimation is performed to obtain the estimated value of the current state quantity, and the observation quantity is updated according to the positioning information provided by Beidou-RTK navigation. The specific calculation formula is as follows: Where f() represents the discretized motion equation, F represents the Jacobian matrix of f, represents the prior estimate of the covariance matrix at the current moment, Q is the navigation system noise error; P k-1 is the covariance matrix of the previous moment,

[0046] Step 123, obtain the a posteriori estimated state quantity according to the observed quantity and the a priori estimate of the state quantity And update the covariance matrix P k And the Kalman filter gain K, the specific calculation formula is as follows:

[0047] The posterior estimate of the state quantity It is used as a reference value for subsequent heading deviation calculations.

[0048] This step takes a lawn mower as an example. The navigation receiver can choose to update the lawn mower positioning information every 1 second and send it to the PC host computer through the RS232 port.

[0049] Step S13: Calculate a first heading deviation based on the position information of the autonomous driving vehicle and a first anchor point corresponding to the position information, where the first anchor point is a point on the driving path.

[0050] Specifically, the first anchor point is first determined based on the position information of the autonomous driving vehicle. The method for determining the first anchor point is: first calculate the position information of the autonomous driving vehicle and the target point of the predetermined driving path with the shortest distance. If the shortest distance target point is less than the preset anchor point distance, the autonomous driving vehicle calculates the distance between the current position of the autonomous driving vehicle and the target point on the preset path in the time sequence of the preset path target points in sequence, until the distance satisfies or exceeds the preset anchor point distance, and the point is used as the first anchor point. The anchor point distance is set according to the specific parameters of the autonomous driving vehicle. Preferably, the anchor point distance is greater than or equal to 1.3 times the axial width of the autonomous driving vehicle and less than or equal to 1.5 times the axial width of the autonomous driving vehicle; then the first heading deviation α is calculated based on the position information of the autonomous driving vehicle and the position of the first anchor point, wherein, (x c ,y c ),(x B ,y B ) are the position coordinates of the autonomous driving vehicle and the coordinates of the first anchor point, respectively.

[0051] Step S14: determining a visual navigation line according to the crops in the RGB image and calculating a second heading deviation according to the visual navigation line and a corresponding anchor point, wherein the second anchor point is a point on the visual navigation line.

[0052] Specifically, this step can be implemented by using any method from step S141 to step S146.

[0053] Step S141: convert the RGB image into a binary image, wherein the binary image includes a first pixel value and a second pixel value, the first pixel value represents a green area in the image, and the second pixel value represents a non-green area in the image.

[0054] Specifically, this step can directly convert the RGB image acquired by the first image acquisition unit into an HSV image, and then screen out the pixel points that meet the green condition according to the definition range of green and assign them to the first pixel value, which can be 0 or 255; the non-green area is assigned a second pixel value, and the corresponding second pixel value can be 255 or 0, thereby obtaining a first binary image, which can be directly used as the binary image finally obtained in this step.

[0055] In order to further improve the accuracy of green area processing, based on the conversion of the RGB image into the HSV image to obtain the first binary image, the EXG super green method can be used again to increase the proportion of the G component in the RGB image to obtain the EXG image; then compare the threshold with the pixel value of each pixel in the EXG image to convert the EXG image into a second binary image; and perform an AND operation on the pixels in the first binary image and the corresponding pixels in the second binary image to obtain a binary image.

[0056] Specifically, the ExG factor of each pixel is calculated first, ExG = 2G-BR; then the ExG factor is linearly mapped to the range of [0,255], and the calculation formula is: The threshold T is calculated using the Otus method by comparing ExG scaled With the threshold T, the ExG scaled Assign a value of 255 or 0, otherwise it will be less than T's ExG scaled Assigning a value of 0 or 255 can obtain the second binary image.

[0057] The use and operation can be judged by the response function F(p) expressed as:

[0058]

[0059] In this step, the interference of illumination changes on the image can be eliminated by processing the RGB image using the EXG super green method, and finally the small holes in the image are filled through corrosion operation.

[0060] Step S142: extracting feature points representing crops in the binary image.

[0061] Specifically, this step first uses a circular template of R pixels as the search area for the pixel with the first pixel value in the binary image, and compares the brightness of the pixel with that of each pixel in the search area; when the brightness of at least X consecutive pixels within the search radius of the pixel is greater than or less than the brightness of the pixel, the pixel is used as a candidate feature point in the candidate feature point set; for each candidate feature point in the candidate feature point set, the judgment window is used as the domain, and the pixel with the largest brightness in the judgment window is used as the feature point.

[0062] Among them, R can be 12, 14, 16, etc., X can be 7, 8, 9, etc., and the judgment window can be set to 4*4, 3*3, 2*2 or others.

[0063] For example, R is set to 12, X is set to 7, and the detection rule of taking pixel points as feature points can be expressed by the response function R(p):

[0064]

[0065] In the formula, I i It is represented by the brightness of the pixel (i=1,2,...,12) within the search radius; I p It represents the brightness of the pixel with the first pixel value in the binary image, that is, the brightness of the central pixel of the search radius; t represents the set threshold.

[0066] For each candidate feature point, a 3*3 judgment window is used as the judgment neighborhood. If there are multiple corner points in the local neighborhood, the brightness difference between the pixels in each neighborhood and the corner points is calculated, and only the corner points with the largest brightness difference in the 3*3 window neighborhood are retained. The brightness difference calculation formula is as follows: Among them I λ Represents the brightness of pixels in a 3*3 neighborhood; implements corner detection and redundant removal operations to reduce subsequent calculations.

[0067] Step S143: convert the feature points into a world coordinate system according to the internal parameters and depth information of the first image acquisition unit to obtain three-dimensional feature points.

[0068] Specifically, the conversion method is:

[0069] Where (x0, y0) is the pixel coordinate, f x ,f y is the focal length of the first image acquisition unit, (c x ,c y ) is the coordinate of the principal point, and (X, Y, d) are the horizontal, vertical and depth of the pixel in the world coordinate system respectively.

[0070] Step S144: clustering the three-dimensional feature points and fitting them using the least square method to obtain a navigation line.

[0071] For example, the depth camera field of view predetermines the clustering range to be [2, 3], and the sum of squared errors (SSE) of different clustering numbers k are calculated to select the clustering number, and the calculation formula is: Among them, C i is the i-th cluster, u i is the cluster center of the ith cluster, x is the sample feature point, and k is the number of populations.

[0072] Step S145: determine a second anchor point on the navigation line according to the intersection of the navigation line and a circle with a fixed point on the axis of the autonomous driving vehicle as the center and a threshold distance as the radius.

[0073] Specifically, the first image acquisition unit is installed in the center of the front of the autonomous driving vehicle, and its visual axis coincides with the axis of the autonomous driving vehicle. At this time, the intersection of the axis of the autonomous driving vehicle and the rear of the vehicle body can be selected as a fixed point. The threshold distance is determined according to the parameters of the autonomous driving vehicle, for example, it can be obtained by simulating the chassis width.

[0074] Step S146: Calculate a second heading deviation based on the coordinates of the second anchor point and the coordinates of a fixed point on the axis of the autonomous driving vehicle.

[0075] After the coordinates of the second anchor point and the fixed point are determined, the calculation method of the second heading deviation is consistent with the calculation method of the first heading deviation, which will not be described in detail here.

[0076] Step S15: weighted sum the first heading deviation and the second heading deviation to obtain a third heading deviation.

[0077] This solution can improve the accuracy of heading deviation calculation and the accuracy of the autonomous driving vehicle traveling along a preset driving path by calculating the first heading deviation and the second heading deviation and determining the third heading deviation based on the first heading deviation and the second heading deviation.

[0078] Step S16: determine the steering angle of the autonomous driving vehicle based on the third heading deviation, and calculate the control voltage of the motors on both sides of the autonomous driving vehicle based on the steering angle to reduce the shortest distance between the autonomous driving vehicle and the predetermined driving path.

[0079] Specifically, the third heading deviation is combined with the differential chassis operation model to calculate the steering angle, and a proportional control of the steering angle and the control voltage of the left and right brushless motor controllers of the differential chassis is established to pull the lawn mower along the planned route.

[0080] For example, take the depth camera D435i as an example, the depth field of view is 87°×58°, the height from the ground is 1m, and the horizontal tilt angle is 45°. According to the field of view and the installation position, calculate the horizontal field of view range The depth camera collects images of the front working area and transmits them to the PC host computer through the USB protocol. The PC host computer converts the image from the RGB color space to the HSV color space and defines the green range. By checking the pixel values, the target area is preliminarily screened out: the pixel values ​​that do not belong to the green range are set to 0, and the pixel values ​​that belong to the green range are set to 255, generating a binary image. At the same time, the program extracts the RGB color components, generates a grayscale image using the ExG calculation formula, and calculates the average value of all pixels in the grayscale image as the threshold. The pixel values ​​below the threshold are set to 0, and the pixel values ​​above the threshold are set to 255 to generate the second binary image. Finally, the two binary images are superimposed by the function F(p) to remove the influence of illumination changes. Then, the program performs fast corner point detection on the binary image, extracts candidate feature points, and performs redundancy removal based on the 3x3 neighborhood. Combined with the depth information output by the depth camera D435i and its internal parameter K, the redundant feature points are converted from the pixel coordinate system to the Beidou-RTK navigation coordinate system to complete the two-dimensional to three-dimensional operation. For compact planting of seedlings, the row spacing is 0.5-0.6 meters. The region of interest is extracted by adaptive K-means clustering, and the clustering range is set to 2-3 pixels. Adaptive clustering is performed on the three-dimensional feature points to extract the feature points of the region of interest. The navigation line is fitted based on the least squares method. The heading deviation α2 is obtained by calculating the deviation between the straight line connecting the preview point on the fitted navigation line and the rear axle of the chassis and the center axis of the vehicle body. The navigation program fuses the heading deviation calculated by Beidou-RTK navigation and visual navigation, establishes a proportional relationship between the heading deviation and the motor control voltage, and converts the heading deviation into the steering speed of the machine. The starting voltage of the 48V left and right motor controller is 1v, and the maximum voltage is 5v. Set a suitable turning voltage as the walking voltage of the lawn mower. If the fused heading deviation is greater than 0, it means that the machine needs to turn left. According to the steering speed converted from the heading deviation, the turning voltage of the left motor is reduced to achieve differential steering of the left and right motors. On the contrary, if the heading deviation is less than 0, the turning voltage of the right motor is reduced to achieve differential right steering. By updating the heading deviation in real time and adjusting the voltage of the left and right motors, precise navigation control can be achieved.

[0081] Based on the above-mentioned automatic driving method, the second aspect of the present invention discloses a lawn mower control method, including an automatic driving control method and a weeding control method. The automatic driving control method is the method disclosed in the first aspect, and the weeding control method includes steps S21 to S23.

[0082] Step S21: Acquire the image to be recognized acquired by the second image acquisition unit.

[0083] The second image acquisition unit may be arranged below the chassis to acquire images of the working area.

[0084] Step S22: Identify weeds according to the image to be identified and determine the three-dimensional coordinates of the weeds in the robot arm coordinate system.

[0085] The weeds in the image can be identified using a pre-trained weed detection model. The weed detection model can be any neural network model, such as the YOLOv8 S model.

[0086] During pre-training, we collected image information with weeds under different climate and lighting conditions, annotated the images using the LabelImg tool, and generated the corresponding label files. Then, we used the image processing tools provided by OpenCV to pre-process the data, including image denoising, normalization, cropping, and enhancement, to obtain the data to be divided. Specific data enhancement operations include image rotation, brightness adjustment, scaling, and flipping to increase the diversity of the data and the robustness of the model. Subsequently, the data were randomly divided into training and test sets in a ratio of 8:2. Based on the training set, the YOLOv8 S model was trained and optimized. The specific structure of the YOLOv8 S model includes an input layer, multiple convolutional layers and residual blocks, a feature pyramid network (FPN), and an output layer for multi-scale target detection. In order to improve the performance of the model in small target detection, the CBAM attention mechanism was added to the feature extraction part of the model to enhance the model's attention to important features, especially for small-sized weed targets. In addition, multi-scale feature fusion and enhanced CIOU loss function were introduced to further improve detection accuracy. The model is trained using the PyTorch framework, and the optimized YOLOv8 model is used to test the test set to obtain the test results. The model performance is evaluated through indicators such as precision, recall, and mean average precision (mAP) to complete the model training.

[0087] Step S23, calculating the distance between the position of the weeds and the mechanical arm, and driving the mechanical arm to move to the weed position according to the distance, and then starting the weeding component to achieve weeding.

[0088] Specifically, this step can first calculate the distance between the center of the weed identification frame and the robotic arm; then convert the distance into the number of pulses of the three-axis robotic arm servo motor to control the robotic arm to reach the center of the weed; finally, start the reel motor to achieve weeding.

[0089] Among them, the center coordinates of the detected weed object frame are converted into three-dimensional coordinates in the camera coordinate system through the visual navigation conversion relationship. According to the correspondence between the pre-calibrated robot coordinate system and the camera coordinate system, the three-dimensional coordinates of the weeds in the robot coordinate system are obtained. The three-dimensional coordinates are transmitted to the controller of the three-axis robot through the 485 communication protocol. After receiving the three-dimensional coordinates, the controller of the three-axis robot converts the distance information into the number of revolutions required by the three-axis servo motor. The servo motor drives the slider on the guide rail to the specified position through precise rotation feedback from its built-in encoder. After the slider reaches the specified position, the rolling motor on the slider is started, and the motor drives the roller to complete the weeding operation. As the detection model continuously provides new weed coordinates, the three-axis robot continuously adjusts the roller position to ensure the precise execution of the weeding operation.

[0090] For example, the weed detection model sends the center coordinates of the weed target frame to the three-axis robotic arm processor via the Modbus-RTU protocol, and performs pre-calibration based on the relative relationship between the camera installation position and the origin position of the three-axis robotic arm. The three-axis robotic arm moves to any nine known points within the working range, and records the three-dimensional coordinates P of each point in the robotic arm coordinate system. i =[X i ,Y i ,Z i ] T and the two-dimensional pixel coordinates p in the camera image i =[u i ,v i ] Where i = 1, 2…9, construct the perspective transformation relationship Where X i ,Y i ,Z i They represent the horizontal, vertical and horizontal coordinates of the pixel point in the world coordinate system, respectively. i ,v i They are the horizontal and vertical coordinates corresponding to the pixel points in the pixel coordinate system, and the following equation is established for each group of points:

[0091] Convert it into a linear equation system Ah = b and use the least squares method to solve the unknowns. i,j (i=1, 2, 3, j=1, 2, 3) are unknown numbers. Based on the perspective transformation matrix H, the center of the weed recognition frame is converted into a three-dimensional coordinate in the robotic arm coordinate system, the relationship between the distance and the motion of the three-axis robotic arm is established, the distance is converted into the number of pulses of the three-axis robotic arm servo motor, the robotic arm is controlled to reach the center of the weed, and the reel motor is started to complete the weeding operation.

[0092] The third aspect of the present invention discloses a control device, including a memory and a controller connected in sequence, wherein a computer program is stored on the memory, and the controller is used to read the computer program to execute an automatic driving control method described in the first aspect and any possible design thereof or a lawn mower control method described in the second aspect and any possible design thereof. Specifically, the memory may include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a flash memory (FlashMemory), a first-input first-output memory (FIFO), and / or a first-input last-output memory (FILO), etc.; the controller may not be limited to a microcontroller of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power supply unit, a display screen, and other necessary components.

[0093] Based on the different locations of the control devices, there are two ways to implement the autonomous driving system based on the above solution. One is that the control unit is a host computer to achieve remote control; the other is that the control unit is the control unit of the autonomous driving vehicle to achieve local control.

[0094] When the control unit is a host computer, the automatic driving system is characterized in that it includes a host computer and at least one automatic driving vehicle, the host computer is the control device described in the third aspect and a first information transceiver unit connected to the control device; the automatic driving vehicle includes a vehicle body, a control unit arranged on the vehicle body and a first image acquisition unit, a navigation receiver and a second information transceiver unit, all of which are signal-connected to the control unit, the first image acquisition unit is used to acquire image information in front of the vehicle body in the driving direction, and the first information transceiver unit is used to exchange information with the second information transceiver unit.

[0095] When the control unit is a control unit of an autonomous driving vehicle, the autonomous driving system includes a vehicle body, a control unit arranged on the vehicle body, a first image acquisition unit and a navigation receiver both of which are signal-connected to the control unit, wherein the first image acquisition unit is used to acquire image information in front of the vehicle body in the direction of travel; the control unit is the control device described in the third aspect.

[0096] Based on any of the above-mentioned autonomous driving systems, in order to realize automatic weeding, the above-mentioned autonomous driving system further includes a mechanical arm, a roller cutter, and a second image acquisition unit. The second image acquisition unit is arranged under the chassis of the autonomous driving vehicle and can be a depth camera.

[0097] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. An automatic driving control method, characterized in that: The following steps are involved: Obtaining a planned driving route; Acquire the position information of the autonomous driving vehicle and the RGB image in front of the autonomous driving vehicle acquired by the first image acquisition unit; calculate a first heading deviation according to the position information of the autonomous driving vehicle and a first anchor point corresponding to the position information, wherein the first aiming point is a point on the predetermined driving path; Determine a visual navigation line according to the crops in the RGB image and calculate a second heading deviation according to the visual navigation line and a corresponding second anchor point, where the second anchor point is a point on the visual navigation line; Taking a weighted sum of the first heading deviation and the second heading deviation to obtain a third heading deviation; The steering angle of the autonomous driving vehicle is determined based on the third heading deviation, and the control voltage of the motors on both sides of the autonomous driving vehicle is calculated based on the steering angle to reduce the shortest distance between the autonomous driving vehicle and the predetermined driving path.

2. The automatic driving control method according to claim 1, characterized in that: The obtaining of the position information of the autonomous driving vehicle includes: Acquire speed information of the motor encoder, positioning data and heading data of the navigation receiver, wherein the speed information includes the speed of the left motor and the speed of the right motor; Discretize the positioning data, heading information and speed information, where: In the formula, x k ,y k are the horizontal and vertical coordinates of the positioning data corresponding to the current moment k, yaw k is the heading information corresponding to the current moment k, l is the distance between the left and right motor positions, v r is the right motor speed, v l is the left motor speed; According to the state quantity at the previous moment With the current system input u k For the current state Perform a priori prediction to obtain the estimated value of the current state quantity and update the observed value, where u k =[v r ,v l ] T , f() represents the discretized motion equation, F represents the Jacobian matrix of f(), represents the prior estimate of the covariance matrix at the current moment, Q is the navigation system noise error; P k-1 is the covariance matrix of the previous moment. The posterior estimated value of the state quantity is obtained according to the observed value and the estimated value of the current state quantity. Update the covariance matrix P k and Kalman filter gain K, where 3. The automatic driving control method according to claim 1, characterized in that: The step of calculating the first heading deviation based on the position information of the autonomous driving vehicle and the first anchor point corresponding to the position information includes: Determine a first anchor point according to the position information of the autonomous driving vehicle; The first heading deviation α is calculated according to the position information of the autonomous driving vehicle and the position of the first anchor point, wherein: (x c ,y c ),(x B ,y B ) are the position coordinates of the autonomous driving vehicle and the coordinates of the first anchor point, respectively.

4. The automatic driving control method according to claim 1, characterized in that: The determining of the visual navigation line in the RGB image and calculating the second heading deviation according to the visual navigation line and the corresponding second anchor point includes: Converting the RGB image into a binary image, wherein the binary image includes a first pixel value and a second pixel value, the first pixel value represents a green area in the image, and the second pixel value represents a non-green area in the image; Extracting feature points representing crops in the binary image, The feature points are converted into a world coordinate system according to the internal parameters and depth information of the first image acquisition unit to obtain three-dimensional feature points; After clustering the three-dimensional feature points, a least square method is used to fit the navigation line; Determine a second anchor point on the navigation line according to an intersection of the navigation line and a circle with a fixed point on the axis of the autonomous driving vehicle as the center and a threshold distance as the radius; The second heading deviation is calculated based on the coordinates of the second anchor point and the coordinates of a fixed point on the axis of the autonomous driving vehicle.

5. The automatic driving control method according to claim 4, characterized in that: The step of converting the image into a binary image comprises: The RGB image is converted into an HSV image, and the pixel points meeting the green condition are screened out according to the definition range of green, and the pixel values ​​are assigned to the first pixel values, and the non-green areas are assigned to the second pixel values, so as to obtain a first binary image; The EXG super green method is used to increase the proportion of the G component in the RGB image to obtain an EXG image; Then compare the threshold with the pixel value of each pixel in the EXG image, and convert the EXG image into a second binary image; An AND operation is performed on the pixel points in the first binary image and the corresponding pixel points in the second binary image to obtain a binary image.

6. The automatic driving control method according to claim 4, characterized in that: The extracting of feature points representing crops in the binary image comprises: For a pixel point in the binary image whose pixel value is the first pixel value, a circular window of R pixels is used as a search area, and the brightness of the pixel point is compared with that of each pixel point in the search area; In response to at least X consecutive pixels within the search radius of the pixel having brightness greater than or less than the brightness of the pixel, the pixel is taken as a candidate feature point in a set of candidate feature points; For each candidate feature point in the candidate feature point concentration area, a rectangular window is used as the judgment area, and the pixel with the maximum brightness in the judgment neighborhood is taken as the feature point.

7. A weeding robot control method, comprising an automatic driving control method and a weeding control method, characterized in that: The automatic driving control method is an automatic driving control method according to any one of claims 1 to 6, and the weed control method comprises the following steps: Acquire the image to be recognized acquired by the second image acquisition unit; Identify weeds according to the image to be identified and determine the three-dimensional coordinates of the weeds in the robot arm coordinate system; The distance between the position of the weeds and the mechanical arm is calculated, and the mechanical arm is driven to move to the position of the weeds according to the distance, and then the weeding component is started to achieve weeding.

8. A weeding robot control device, comprising a memory and a controller which are sequentially connected in communication, wherein a computer program is stored in the memory, and wherein: The controller is used to read the computer program and execute an automatic driving control method as described in any one of claims 1 to 6 or a lawn mower control method as described in claim 7.

9. An automatic driving system, characterized in that: include: A host computer, wherein the host computer is the control device described in claim 8 and a first information transceiver unit connected to the control device; At least one autonomous driving vehicle, the autonomous driving vehicle comprising a vehicle body, a control unit arranged on the vehicle body, and a first image acquisition unit, a navigation receiver, and a second information transceiver unit, all of which are signal-connected to the control unit, wherein the first image acquisition unit is used to acquire image information in front of the vehicle body in the direction of travel, and the first information transceiver unit is used to exchange information with the second information transceiver unit.

10. An automatic driving system, characterized in that: It includes a vehicle body, a control unit arranged on the vehicle body, a first image acquisition unit and a navigation receiver, both of which are signal-connected to the control unit, wherein the first image acquisition unit is used to acquire image information in front of the vehicle body in the direction of travel; the control unit is the control device according to claim 8.

Images