An automatic driving weeding robot control method, device and system

Abstract

The application discloses a kind of automatic driving weeding robot machine control method, device and system, the control method includes the following steps: obtaining predetermined travel path;Obtain the position information of automatic driving vehicle and the RGB image that the first image acquisition unit of automatic driving vehicle front gathers;According to the position information of automatic driving vehicle and the first anchor point corresponding to the position information, first heading deviation is calculated;According to crop in the RGB image, visual navigation line is determined and second heading deviation is calculated according to the visual navigation line and corresponding second anchor point;The first heading deviation and second heading deviation are weighted and summed, and third heading deviation is obtained;According to the third heading deviation, the steering angle of automatic driving vehicle is determined, and the control voltage of motor on both sides of automatic driving vehicle is calculated according to the steering angle to reduce the shortest distance between automatic driving vehicle and predetermined travel path. It can improve the control accuracy of automatic driving.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of weeding, and particularly relates to an automatic driving weeding robot control method, device and system. BACKGROUND

[0002] Adopting a weeding machine to realize automatic weeding can greatly reduce labor. When automatic weeding is performed by using a weeding machine, automatic driving is generally performed according to a preset path or the path of a seeding machine. The existing automatic driving according to a preset path adopts the "tractor automatic driving system" with the application number CN201910281278.3, which detects and determines the deviation of the current driving path of the tractor relative to the preset driving path of the tractor, and controls the steering of the tractor according to the deviation of the current driving path of the tractor relative to the preset driving path of the tractor, so as to drive the tractor on the preset driving path.

[0003] By using the technology, the control is only realized by the deviation of the current driving path relative to the preset driving path of the tractor. Since the row distance of crops is small, the control accuracy is limited in the case of limited positioning data accuracy, and it is not suitable for occasions where the row distance of crops is small. SUMMARY

[0004] The application proposes an automatic driving weeding robot control method, device and system to solve the problem of low control accuracy of the existing control method.

[0005] The purpose of the application is realized by the following technical solutions:

[0006] The application discloses an automatic driving control method, which includes the following steps:

[0007] Obtaining a predetermined driving path;

[0008] Obtaining the position information of an automatic driving vehicle and the RGB image of the front of the automatic driving vehicle collected by a first image collection unit;

[0009] Calculating the first heading deviation according to the position information of the automatic driving vehicle and the first anchor point corresponding to the position information, wherein the first anchor point is a point on the predetermined driving path;

[0010] Determining the visual navigation line according to the crops in the RGB image and calculating the second heading deviation according to the visual navigation line and the corresponding second anchor point, wherein the second anchor point is a point on the visual navigation line;

[0011] Weighted sum of the first heading deviation and the second heading deviation to obtain the third heading deviation;

[0012] The steering angle of the autonomous vehicle is determined according to the third heading deviation, and the control voltages of the motors on both sides of the autonomous vehicle are calculated according to the steering angle to reduce the shortest distance between the autonomous vehicle and the predetermined driving path.

[0013] The second aspect of the present application discloses a weeding machine control method, including an autonomous driving control method and a weeding control method, the autonomous driving control method is the autonomous driving control method in the first aspect, and the weeding control method includes the following steps:

[0014] An image to be recognized is acquired by a second image acquisition unit;

[0015] Weeds are recognized according to the image to be recognized, and three-dimensional coordinates of the weeds in a mechanical arm coordinate system are determined.

[0016] The distance between the weeds and the mechanical arm is calculated, and the mechanical arm is driven to move to the position of the weeds according to the distance, and then the weeding component is started to realize weeding.

[0017] The third aspect of the present application discloses a control device, including a memory and a controller which are sequentially connected in communication, the memory stores a computer program, and the controller is used for reading the computer program and executing the autonomous driving control method in the first aspect or the weeding machine control method in the second aspect.

[0018] The third aspect of the present application discloses an autonomous driving system, including:

[0019] A host computer, the host computer is the control device in the third aspect and a first information transceiver unit connected to the control device;

[0020] At least one autonomous vehicle, the autonomous vehicle includes a vehicle body, a control unit arranged on the vehicle body, and a first image acquisition unit, a navigation receiver and a second information transceiver unit which are all signal connected to the control unit, the first image acquisition unit is used for acquiring image information in front of the driving direction of the vehicle body, and the first information transceiver unit is used for information interaction with the second information transceiver unit.

[0021] The fourth aspect of the present application discloses an autonomous driving system, including a vehicle body, a control unit arranged on the vehicle body, and a first image acquisition unit and a navigation receiver which are all signal connected to the control unit, the first image acquisition unit is used for acquiring image information in front of the driving direction of the vehicle body; and the control unit is the control device in the third aspect.

[0022] The present application has the following beneficial effects:

[0023] The first heading deviation and the second heading deviation are determined by the visual navigation line determined by the navigation position data and the visual image, and the final heading deviation is determined according to the first heading deviation and the second heading deviation, so that the control accuracy of automatic driving is improved. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only represent some of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0025] Figure 1 The flow chart of the automatic driving control method of the present application. DETAILED DESCRIPTION

[0026] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0027] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0028] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0029] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0030] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship commonly understood by those skilled in the art, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second" and the like are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0031] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "set", "mount", "connected", "connected" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be connected inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0032] The first aspect of the present application discloses an automatic driving control method, which can be executed by an automatic driving control device, which can be software or a combination of software and hardware, and can be integrated in intelligent mobile terminals, tablets, computers, cloud servers, PC host computers and other intelligent devices. The automatic driving control device can not only be the automatic driving control device of a weeding machine, but also the automatic driving control device of a logging machine, a harvester and the like. As shown in the figure, the method comprises steps S11 to S17. It should be noted that the step identification in the present scheme is only for the convenience of describing the method, and does not constitute a sequence limitation, and the sequence of each step is subject to the language description, and the sequence of each signal is subject to the connection. Figure 1

[0033] Step S11, obtaining a predetermined driving path.

[0034] The predetermined driving path can not only be a specific driving path, but also a historical driving path of a vehicle. Taking the historical driving path of a vehicle as an example, when weeding, the predetermined driving path can be the driving path of the seeder when sowing; when harvesting, the predetermined driving path can be the driving path of the seeder when sowing. When logging, the predetermined driving path can be a driving path set by a person.

[0035] ​For example, taking the predetermined driving path of the weeding machine as an example, the seeding machine is provided with a navigation receiver on the seeding machine frame during seeding. In order to improve the positioning accuracy, the navigation receiver can be a double-antenna Beidou-RTK navigation receiver, a satellite navigation receiver, etc. The navigation receiver outputs the accurate latitude and longitude, heading, and speed information of the seeding machine to the PC host computer every 1 s. After the seeding machine completes the field seeding operation, the PC host computer records the running path of the seeding machine and performs smoothing processing on the path, which is used as the predetermined driving path of the subsequent weeding machine.

[0036] In step S12, the position information of the autonomous vehicle and the RGB image of the front of the autonomous vehicle collected by the first image collection unit are obtained.

[0037] The autonomous vehicle is not only provided with a navigation receiver to obtain real-time positioning information, but also provided with a first image collection unit in front of the autonomous vehicle to collect the RGB image information in front of the vehicle.

[0038] The first image collection unit can be a depth camera, such as D435i.

[0039] In order to improve the accuracy of the positioning information, the navigation receiver can also be a double-antenna Beidou-RTK navigation receiver, a satellite navigation receiver, etc.

[0040] The position information can be the positioning information output by the navigation receiver. In order to further improve the accuracy of the positioning information, the Beidou navigation architecture corrects the positioning information of the mobile station in real time through the differential data of the network base station, and the positioning information is transmitted to the PC host computer through the Modbus-RTU protocol. The PC host computer combines the positioning information with the feedback information of the left motor encoder and the right motor encoder, and adopts an extended Kalman filtering algorithm to eliminate the interference of environmental changes and information frequency on the positioning information, so as to provide smooth position, speed and heading information for navigation. The left motor encoder and the right motor encoder collect the left motor speed and the right motor speed, respectively.

[0041] Specifically, the extended Kalman filtering algorithm for eliminating the interference of environmental changes and information frequency on the positioning information includes steps 121 to 123.

[0042] In step 121, the position information, heading information and speed information in the continuous motion equation are discretized to obtain a discretized motion equation:

[0043] x k =x k-1 +v*cos(yaw k-1 )*dt,y k =y k-1 +v*sin(yaw k-1 )*dt,

[0044] Determine the state quantity based on the discretization equation System input u k =[v r ,v l ] T , the observation matrix Among them, x, y are position information, yaw is heading information, l is the distance between the left and right motor positions, v r is the right motor speed, v l The speed of the left motor is calculated based on the number of pulses given by the left and right brushless motor encoders, the number of pulses of the single-turn encoder and the rotation radius of the gear driven by the motor.

[0045] Step 122, according to the state quantity at the previous moment With the current system input u k For the current state Perform a priori prediction to obtain the estimated value of the current state quantity, and update the observation quantity according to the positioning information provided by Beidou-RTK navigation. The specific calculation formula is as follows: Where f() represents the discretized motion equation, F represents the Jacobian matrix of f, represents the prior estimate of the covariance matrix at the current moment, Q is the navigation system noise error; P k-1 is the covariance matrix of the previous moment,

[0046] Step 123, obtain the posterior estimated state quantity according to the observation quantity and the prior estimate of the state quantity And update the covariance matrix P k And the Kalman filter gain K, the specific calculation formula is as follows:

[0047] The posterior estimate of the state quantity It is used as a reference value for subsequent heading deviation calculations.

[0048] This step takes a lawn mower as an example. The navigation receiver can choose to update the lawn mower positioning information every 1 second and send it to the PC host computer through the RS232 port.

[0049] Step S13: Calculate a first heading deviation based on the position information of the autonomous driving vehicle and a first anchor point corresponding to the position information, where the first anchor point is a point on the driving path.

[0050] Specifically, a first anchor point is determined according to the position information of the autonomous vehicle, and the first anchor point is determined by the following method: first, a target point with the shortest distance between the position information of the autonomous vehicle and a predetermined driving path is calculated; if the shortest distance target point is less than a preset anchor point distance, the autonomous vehicle calculates the distance between the current position of the autonomous vehicle and the target point on the preset path in the time sequence of the preset path target point in reverse order until the distance meets the preset anchor point distance, and the point is taken as the first anchor point. The anchor point distance is set according to the specific parameters of the autonomous vehicle. Preferably, the anchor point distance is greater than or equal to 1.3 times the axial width of the autonomous vehicle and less than or equal to 1.5 times the axial width of the autonomous vehicle; and a first heading deviation α is calculated according to the position information of the autonomous vehicle and the position of the first anchor point, wherein, (x c ,y c ),(x B ,y B ) are the position coordinates of the autonomous vehicle and the coordinates of the first anchor point, respectively.

[0051] In step S14, a visual navigation line is determined according to the crops in the RGB image, and a second heading deviation is calculated according to the visual navigation line and the corresponding anchor point. The second anchor point is a point on the visual navigation line.

[0052] Specifically, this step can be implemented by any method of steps S141 to S146.

[0053] In step S141, the RGB image is converted into a binary image, wherein the binary image includes a first pixel value and a second pixel value, the first pixel value represents a green region in the image, and the second pixel value represents a non-green region in the image.

[0054] Specifically, the RGB image collected by the first image acquisition unit can be directly converted into an HSV image, and then the pixel points meeting the green condition are filtered according to the definition range of green and assigned a first pixel value, which can be 0 or 255; the non-green region is assigned a second pixel value, which can be 255 or 0, so as to obtain a first binary image, which can be directly used as the binary image obtained in this step.

[0055] In order to further improve the accuracy of processing the green region, on the basis of the first binary image obtained by converting the RGB image into an HSV image, the EXG super green method can be used again to increase the proportion of the G component in the RGB image to obtain an EXG image; then the threshold value is compared with the pixel value of each pixel point in the EXG image, the EXG image is converted into a second binary image, and the AND operation is performed on the pixel points in the first binary image and the corresponding pixel points in the second binary image to obtain a binary image.

[0056] Specifically, the ExG factor of each pixel point is calculated first, ExG = 2G-B-R; then the ExG factor is linearly mapped to the range of [0, 255], and the calculation formula is: The threshold T is calculated by the Otus method, and the ExG scaled of the pixel point is compared with the size of the threshold T, the ExG scaled greater than T is assigned as 255 or 0, and vice versa, the ExG scaled less than T is assigned as 0 or 255, so that the second binary image is obtained.

[0057] The operation of AND can be determined by the response function F(p) as follows:

[0058]

[0059] This step can eliminate the interference of light changes on the image by processing the RGB image by the EXG super green method, and finally fill the small holes in the image by the erosion operation.

[0060] Step S142, extracting the feature points representing crops in the binary image.

[0061] Specifically, this step first takes the pixel point with the first pixel value in the binary image as the search area with a circular template of R pixel points, compares the brightness of the pixel point with the brightness of each pixel point in the search area; when there are at least X continuous pixel points in the search radius of the pixel point whose brightness is greater than or less than the brightness of the pixel point, the pixel point is taken as a candidate feature point in the candidate feature point set; for each candidate feature point in the candidate feature point set, take the judgment window as the field, and take the pixel with the maximum brightness in the judgment window as the feature point.

[0062] Wherein, R can be 12, 14, 16, etc., X can be 7, 8, 9, etc., and the judgment window can be set to 4*4, 3*3, 2*2 or other.

[0063] For example, R is set to 12 and X is set to 7, and the detection rule of taking the pixel point as the feature point can be represented by the response function R(p) as follows:

[0064]

[0065] In the formula, I i represents the brightness of the pixel point in the search radius (i = 1, 2,..., 12); I p represents the brightness of the pixel point with the first pixel value in the binary image, i.e. the brightness of the center pixel point in the search radius; t represents the set threshold.

[0066] For each candidate feature point, a 3*3 judgment window is taken as a judgment neighborhood, if there are multiple corner points in the local neighborhood, the brightness difference of each neighborhood pixel and the corner pixel is calculated, only the corner point with the largest brightness difference in the 3*3 window neighborhood is reserved, and the brightness difference calculation formula is as follows: Wherein I λ represents the pixel brightness in the 3*3 neighborhood; the corner point detection and redundancy removal operation is realized to reduce the subsequent calculation.

[0067] Step S143, converting the feature points into a world coordinate system according to the intrinsic parameters of the first image acquisition unit and the depth information, to obtain three-dimensional feature points.

[0068] Specifically, the conversion method is:

[0069] Where (x0, y0) is the pixel coordinate, f x ,f y is the focal length of the first image acquisition unit, (c x ,c y ) is the principal point coordinate, (X, Y, d) is the horizontal, vertical coordinate and depth of the pixel in the world coordinate system respectively.

[0070] Step S144, clustering the three-dimensional feature points and fitting to obtain a navigation line by using the least square method.

[0071] For example, the depth camera field of view range is determined in advance, the clustering range is approximately [2, 3], the total error sum of squares (SSE) of different clustering numbers k is calculated to select the clustering number, and the calculation formula is: Wherein C i is the i-th cluster group, u i is the cluster center of the i-th cluster group, x is the sample feature point, and k is the population number.

[0072] Step S145, determining a second anchor point on the navigation line according to the intersection of the navigation line and a fixed point on the axis of the autonomous vehicle with a threshold distance as the radius.

[0073] Specifically, the first image acquisition unit is installed in the front center of the autonomous vehicle, and the visual axis coincides with the axis of the autonomous vehicle. At this time, the intersection of the axis of the autonomous vehicle and the rear of the vehicle body can be selected as the fixed point. The threshold distance is determined according to the parameters of the autonomous vehicle, for example, it can be obtained by chassis width simulation.

[0074] Step S146, calculating the second heading deviation according to the coordinates of the second anchor point and the coordinates of a fixed point on the axis of the autonomous vehicle.

[0075] After the second anchor point coordinate and the fixed point coordinate are determined, the second heading deviation is calculated in the same way as the first heading deviation, and details are not repeated here.

[0076] Step S15, the first heading deviation and the second heading deviation are weighted and summed to obtain a third heading deviation.

[0077] The first heading deviation and the second heading deviation are calculated, and the third heading deviation is determined based on the first heading deviation and the second heading deviation, which can improve the accuracy of the heading deviation calculation and improve the accuracy of the autonomous vehicle driving along the preset driving path.

[0078] Step S16, according to the third heading deviation, the steering angle of the autonomous vehicle is determined, and the control voltage of the motors on both sides of the autonomous vehicle is calculated according to the steering angle to reduce the shortest distance between the autonomous vehicle and the predetermined driving path.

[0079] Specifically, the third heading deviation is combined with the differential chassis operation model to calculate the steering angle, and the proportional control of the steering angle and the control voltage of the differential chassis left and right brushless motor controllers is established, and the mower drives according to the planned route.

[0080] For example, taking the depth camera D435i as an example, the depth field of view angle is 87°×58°, the height from the ground is 1m, and the horizontal inclination angle is 45°. According to the field of view angle and the installation position, the transverse field of view range is calculated as The depth camera collects images of the front work area and transmits them to the PC host computer through the USB protocol. The PC host computer converts the images from the RGB color space to the HSV color space and defines the green color range. By checking the pixel values, the target area is preliminarily screened out: the pixel values outside the green color range are set to 0, and the pixel values within the green color range are set to 255, generating a binary image. At the same time, the program extracts the RGB color components, generates a grayscale image using the ExG calculation formula, and calculates the average value of all pixels in the grayscale image as the threshold. The pixel values below the threshold are set to 0, and the pixel values above the threshold are set to 255, generating a second binary image. Finally, the two binary images are superimposed through the function F(p) to remove the influence of light changes. Then, the program performs fast corner detection on the binary image, extracts candidate feature points, and performs de-redundancy processing based on a 3x3 neighborhood. Combined with the depth information output by the depth camera D435i and its intrinsic parameters K, the de-redundant feature points are converted from the pixel coordinate system to the Beidou-RTK navigation coordinate system, completing the two-dimensional to three-dimensional operation. For compact planting of seedlings with a row spacing of 0.5-0.6 meters, the region of interest is extracted through adaptive K-means clustering, and the clustering range is set to 2-3 pixels. The three-dimensional feature points are adaptively clustered to extract the feature points of the region of interest, and the navigation line is fitted based on the least squares method. The deviation of the preview point on the fitted navigation line from the straight line connecting the rear axle of the chassis and the center axis of the vehicle body is calculated to obtain the heading deviation α2. The navigation program fuses the heading deviations calculated by the Beidou-RTK navigation and the visual navigation, establishes a proportional relationship between the heading deviation and the motor control voltage, and converts the heading deviation into the steering speed of the machine. The starting voltage of the 48V motor controller is 1v, and the maximum voltage is 5v. Set the appropriate handle voltage as the walking voltage of the weeding machine. If the fused heading deviation is greater than 0, it means that the machine needs to turn left. According to the steering speed converted from the heading deviation, the handle voltage of the left motor is reduced to realize differential steering of the left and right motors. Conversely, if the heading deviation is less than 0, the handle voltage of the right motor is reduced to realize differential right steering. By updating the heading deviation in real time, the handle voltage of the left and right motors is adjusted to realize precise navigation control.

[0081] Based on the above automatic driving method, the second aspect of the present application discloses a weeding machine control method, which includes the automatic driving control method disclosed in the first aspect and a weeding control method. The automatic driving control method is the method disclosed in the first aspect, and the weeding control method includes steps S21 to S23.

[0082] Step S21, acquiring the image to be recognized collected by the second image acquisition unit.

[0083] The second image acquisition unit can be arranged below the chassis to collect images of the work area.

[0084] Step S22, weeds are identified according to the image to be identified, and three-dimensional coordinates of the weeds in the mechanical arm coordinate system are determined.

[0085] The identification of weeds in the image can adopt a pre-trained weed detection model. The weed detection model can be any neural network model, such as a YOLOv8 S model.

[0086] During pre-training, image information with weeds under different climates and light conditions is collected, and the images are labeled using the LabelImg tool to generate corresponding label files. Then, the image processing tool provided by OpenCV is used to pre-process the data, including image denoising, normalization, cropping, and enhancement operations, to obtain the data to be divided. Specific data enhancement operations include image rotation, brightness adjustment, scaling, and flipping to increase the diversity of the data and the robustness of the model. Subsequently, these data are randomly divided into a training set and a test set in a ratio of 8:2. Based on the training set, the YOLOv8 S model is trained and optimized. The specific structure of the YOLOv8 S model includes an input layer, multiple convolutional layers and residual blocks, a feature pyramid network (FPN), and an output layer for multi-scale target detection. To improve the performance of the model in small target detection, a CBAM attention mechanism is added to the feature extraction part of the model to enhance the model's attention to important features, especially for small-sized weed targets. In addition, a CIOU loss function for multi-scale feature fusion and enhancement is introduced to further improve the detection accuracy. The training of the model is carried out using the PyTorch framework, and the optimized YOLOv8 model is used to detect the test set to obtain the detection results. The performance of the model is evaluated by indicators such as precision, recall, and mean average precision (mAP), and the model training is completed.

[0087] Step S23, the distance between the position of the weeds and the mechanical arm is calculated, and the weed removal component is started to achieve weed removal after the mechanical arm is driven to move to the position of the weeds according to the distance.

[0088] Specifically, this step can first calculate the distance between the center of the identified box of the weeds and the mechanical arm; then convert the distance into the pulse number of the three-axis mechanical arm servo motor to control the mechanical arm to reach the center of the weeds; and finally start the hob motor to achieve weed removal.

[0089] The detected weed object frame center coordinates are converted into three-dimensional coordinates in the camera coordinate system through the visual navigation conversion relationship. According to the correspondence relationship between the mechanical arm coordinate system and the camera coordinate system, which is pre-calibrated, the three-dimensional coordinates of the weed in the mechanical arm coordinate system are obtained. The three-dimensional coordinates are transmitted to the controller of the three-axis mechanical arm through the 485 communication protocol. After receiving the three-dimensional coordinates, the controller of the three-axis mechanical arm converts the distance information into the number of rotations required by the three-axis servo motor. The servo motor drives the slider on the guide rail to move to the specified position through the feedback of the built-in encoder. After the slider reaches the specified position, the rolling motor on the slider is started, and the motor drives the rolling cutter to complete the weeding work. As the detection model continuously provides new weed coordinates, the three-axis mechanical arm continuously adjusts the position of the rolling cutter to ensure the accurate execution of the weeding work.

[0090] For example, the weed detection model sends the weed target frame center coordinates to the three-axis mechanical arm processor through the Modbus-RTU protocol, and pre-calibrates based on the relative relationship between the camera installation position and the three-axis mechanical arm origin position. By moving the three-axis mechanical arm to any 9 known points within the working range, the three-dimensional coordinates P i = [X i , Y i , Z i ] T of each point in the mechanical arm coordinate system are recorded, and the two-dimensional pixel coordinates p i = [u i , v i ] in the camera imaging are recorded, where i = 1, 2, …, 9, and a perspective transformation relationship is constructed. where X i , Y i , Z i represent the horizontal coordinate, vertical coordinate, and vertical coordinate of the pixel point in the world coordinate system, respectively, u i , v i are the horizontal coordinate and vertical coordinate of the pixel point in the pixel coordinate system, respectively, and the following equation is established for each group of points:

[0091] which is converted into a linear equation system Ah = b, and the unknowns are solved by the least squares method. H i,j (i = 1, 2, 3, j = 1, 2, 3) are unknowns. Based on the perspective transformation matrix H, the weed recognition frame center is converted into three-dimensional coordinates in the mechanical arm coordinate system, the distance and three-axis mechanical arm motion relationship is established, the distance is converted into the pulse number of the three-axis mechanical arm servo motor, the mechanical arm is controlled to reach the center of the weed, and the rolling cutter motor is started to complete the weeding work.

[0092] The third aspect of the present application discloses a control device, comprising a memory and a controller connected in sequence, the memory stores a computer program, and the controller is used for reading the computer program and executing the automatic driving control method in the first aspect and any possible design thereof or the weeding machine control method in the second aspect and any possible design thereof. Specifically, the memory can include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a flash memory, a first-in first-out memory (FIFO) and / or a first-in last-out memory (FILO), etc.; the controller can be a microcontroller with a model number of STM32F105 series. In addition, the computer device can further include, but is not limited to, a power unit, a display screen and other necessary components.

[0093] Based on the different positions of the control device, there are two implementation manners of the automatic driving system based on the above-mentioned scheme, one is that the control unit is a host computer to realize remote control, and the other is that the control unit is the control unit of the automatic driving vehicle to realize local control.

[0094] When the control unit is the host computer, the automatic driving system comprises a host computer and at least one automatic driving vehicle, the host computer is the control device in the third aspect and a first information transceiver unit connected to the control device; the automatic driving vehicle comprises a vehicle body, a control unit arranged on the vehicle body, and a first image acquisition unit, a navigation receiver and a second information transceiver unit all signal-connected to the control unit, the first image acquisition unit is used for acquiring image information in front of the driving direction of the vehicle body, and the first information transceiver unit is used for information interaction with the second information transceiver unit.

[0095] When the control unit is the control unit of the automatic driving vehicle, the automatic driving system comprises a vehicle body, a control unit arranged on the vehicle body, and a first image acquisition unit and a navigation receiver all signal-connected to the control unit, the first image acquisition unit is used for acquiring image information in front of the driving direction of the vehicle body; and the control unit is the control device in the third aspect.

[0096] On the basis of any of the above-mentioned automatic driving systems, in order to realize automatic weeding, the automatic driving system further comprises a mechanical arm, a rolling cutter and a second image acquisition unit. The second image acquisition unit is arranged below the chassis of the automatic driving vehicle and can be a depth camera.

[0097] Those skilled in the art will appreciate that the embodiments described herein are presented for purposes of illustration and that the inventive principles are not limited to these particular embodiments. Other variations and modifications can be made to the embodiments without departing from the spirit and scope of the inventive principles.

Claims

1. An automatic driving control method characterized by comprising: The method comprises the following steps: acquiring a predetermined driving path; acquiring position information of the autonomous vehicle and an RGB image of the front of the autonomous vehicle collected by a first image collection unit; calculating a first heading deviation according to the position information of the autonomous vehicle and a first anchor point corresponding to the position information, the first anchor point being a point on the predetermined driving path; determining a visual navigation line according to crops in the RGB image and calculating a second heading deviation according to the visual navigation line and a second anchor point on the visual navigation line; weighting and summing the first heading deviation and the second heading deviation to obtain a third heading deviation; determining a steering angle of the autonomous vehicle according to the third heading deviation and calculating control voltages of motors on both sides of the autonomous vehicle according to the steering angle to reduce the shortest distance between the autonomous vehicle and the predetermined driving path, determining a visual navigation line in the RGB image and calculating a second heading deviation according to the visual navigation line and a second anchor point, comprising: converting the RGB image into a binary image, wherein the binary image comprises a first pixel value and a second pixel value, the first pixel value representing a green region in the image and the second pixel value representing a non-green region in the image; extracting feature points representing crops in the binary image, converting the feature points into a world coordinate system according to the intrinsic parameters of the first image collection unit and depth information to obtain three-dimensional feature points; fitting the three-dimensional feature points to obtain a navigation line by using a least squares method after clustering the three-dimensional feature points; determining a second anchor point on the navigation line according to the intersection of the navigation line and a fixed point on the axis of the autonomous vehicle with a threshold distance as the radius; calculating a second heading deviation according to the coordinates of the second anchor point and the coordinates of the fixed point on the axis of the autonomous vehicle.

2. The automatic driving control method according to claim 1, characterized in that, The method comprises the following steps: acquiring speed information of the motor encoder, positioning data and heading data of the navigation receiver, the speed information comprising left motor speed and right motor speed; performing discretization processing on the positioning data, heading information and speed information, wherein, ， In the formula, x k , y k are the horizontal coordinate and the vertical coordinate in the positioning data corresponding to the current time k respectively, yaw k is the heading information corresponding to the current time k, l is the distance between the left and right motor positions, v r is the right motor speed, and v l is the left motor speed. According to the state quantity of the last time and the current system input u k k r l T The estimated value of the current state quantity is obtained by predicting the prior estimate, and the observation value is updated, wherein u k = [v r ,v l ] T , f() represents a discretized motion equation, F represents the Jacobian matrix of f(), and represents the prior estimate of the covariance matrix at the current time, Q is the navigation system noise error; P k-1 is the covariance matrix of the last time obtaining a state quantity posterior estimate value from the observation value and an estimated value of a current state quantity updating the covariance matrix P k and a Kalman filter gain K, wherein 3. The automatic driving control method of claim 1, wherein the method comprises the following steps: determining the first anchor point according to the position information of the autonomous vehicle; A first heading deviation a is calculated according to the position information of the autonomous driving vehicle and the position of the first anchor point, wherein (x c ,y c ),(x B ,y B ) are the coordinates of the position of the autonomous driving vehicle and the coordinates of the first anchor point, respectively.

4. The automatic driving control method of claim 1, wherein the method comprises the following steps: converting the RGB image into an HSV image, filtering out pixel points meeting the green condition according to the definition range of green and assigning the first pixel value to the pixel points to obtain a first binary image; increasing the proportion of the G component in the RGB image by using an EXG super green method to obtain an EXG image; and comparing a threshold value with the pixel value of each pixel point in the EXG image to convert the EXG image into a second binary image; performing an AND operation on the pixel points in the first binary image and the corresponding pixel points in the second binary image to obtain a binary image.

5. The automatic driving control method of claim 1, wherein the method comprises the following steps: The pixel point with the first pixel value in the binary image is searched in a circular window with R pixel points as a search region, and the brightness of the pixel point and each pixel point in the search region is compared; In response to the brightness of at least X continuous pixel points in the search radius of the pixel point being greater than or less than the brightness of the pixel point, the pixel point is taken as a candidate feature point in the candidate feature point set; Each candidate feature point in the candidate feature point set is taken as a judgment domain with a rectangular window, and the pixel with the maximum brightness in the judgment neighborhood is taken as a feature point.

6. A weeding robot control method including an automatic driving control method and a weeding control method, characterized by, The automatic driving control method is any one of claims 1-5, and the weeding control method comprises the following steps: Obtaining the to-be-identified image collected by the second image acquisition unit; According to the to-be-identified image, the weeds are identified and the three-dimensional coordinates of the weeds in the mechanical arm coordinate system are determined; the distance between the position of the weeds and the mechanical arm is calculated, and the mechanical arm is driven to move to the position of the weeds according to the distance, and then the weeding component is started to realize weeding.

7. A weeding robot control device comprising a memory and a controller connected in sequence in communication, wherein the memory has stored thereon a computer program, characterized in that: The controller is used to read the computer program, execute any one of claims 1-5 or the weeding robot control method of claim 6.

8. An autonomous driving system, characterized by, It comprises: A host computer, the host computer is the controller of claim 7 and the first information transceiver unit connected to the controller; At least one autonomous vehicle, the autonomous vehicle comprises a vehicle body, a control unit arranged on the vehicle body, and a first image acquisition unit, a navigation receiver and a second information transceiver unit all signal connected to the control unit, the first image acquisition unit is used to collect image information in front of the vehicle body driving direction, and the first information transceiver unit is used for information interaction with the second information transceiver unit.

9. An autonomous driving system, characterized by, It comprises a vehicle body, a control unit arranged on the vehicle body, and a first image acquisition unit, a navigation receiver, all signal connected to the control unit, the first image acquisition unit is used to collect image information in front of the vehicle body driving direction; the control unit is the controller of claim 7.

Images