Extensible unmanned agricultural operation platform

Abstract

The invention relates to an extensible unmanned agricultural operation platform. The platform comprises a loading platform and an automatic driving operation system for controlling the loading platform to advance. The loading platform is provided with drivable wheels and an extensible structure, the extensible structure is used for carrying devices and equipment for different tasks, and the automatic traveling operation system comprises a position information obtaining device, a visual image obtaining device and a main control device. The position information acquisition device is used for acquiring position information of the unmanned agricultural operation platform and sending the position information to the main control device, and the visual image acquisition device is used for acquiring front real-time visual image information of the unmanned agricultural operation platform and sending the received real-time visual image information to the main control device. And the main control device is used for performing operation path planning according to the position information to obtain operation path information, determining furrow position information according to the front real-time visual image information, and controlling the movement of the loading platform according to the path information and the furrow position information. According to the invention, by processing the real-time operation picture, furrow-level navigation is realized, and the damage of the platform to crops is greatly reduced.

Description

Technical Field

[0001] The present invention relates to the field of agricultural operation automation, and in particular to an expandable unmanned agricultural operation platform. Background Art

[0002] With the development of the times, smart agriculture has become an important direction for the development of modern agriculture, with the advantages of high efficiency, refinement, and intelligence. At present, there are many types of agricultural machinery products used to replace some traditional agricultural operations.

[0003] At present, many researchers at home and abroad are conducting research and development of agricultural machinery. Although the current equipment has achieved a certain degree of automation, it still has great limitations in adapting to complex terrain, meeting various operational requirements and achieving precise operation.

[0004] For example, the Chinese invention patent application with publication number CN113156943A discloses a high-gap inter-ridge multifunctional mobile platform structure and its control method, wherein the main structure of the platform is provided with multiple servos, ultrasonic modules, laser radar modules, control system integration modules, visual modules, reduction motors, etc. The ultrasonic modules are symmetrically fixed around the platform to avoid obstacles when the platform moves and to provide accurate distance data for the platform posture control; the laser radar module is fixed on the top of the platform to establish a surrounding environment model so that it can achieve autonomous navigation in unknown and complex inter-ridge environments; the visual modules are symmetrically fixed on both sides of the platform to detect the growth status and degree of pests and diseases of crops on the ridges.

[0005] However, the prior art cannot determine the precise position of the furrows and ridges, and therefore cannot achieve accurate navigation and obstacle avoidance at the ridge level, and the crops may still be crushed and damaged during the operation. Summary of the invention

[0006] The present invention aims to solve the problems that the existing unmanned agricultural operation platform has a low degree of unmanned operation, cannot achieve accurate navigation at the ridge and furrow level, and causes great damage to crops.

[0007] To solve the above technical problems, the present invention proposes an expandable unmanned agricultural operation platform, including a loading platform and an autonomous driving operation system for controlling the movement of the loading platform. The loading platform has drivable wheels, and the loading platform has an expandable structure for installing various expansion devices. The autonomous driving operation system includes a position information acquisition device, a visual image acquisition device, and a main control device, where: the position information acquisition device is used to acquire the position information of the unmanned agricultural operation platform and send it to the main control device; the visual image acquisition device is used to acquire the real-time visual image information in front of the unmanned agricultural operation platform and send the received real-time visual image information to the main control device; the main control device is used to plan an operation path based on the position information to obtain operation path information, determine the ridge position information based on the real-time visual image information in front, and control the movement of the loading platform according to the path information and the ridge position information.

[0008] According to a preferred embodiment of the present invention, the autonomous driving operation system further includes a lidar device. The lidar device is used to emit laser pulses and receive the reflected signals of the laser pulses, and send the received reflection information to the main control device; the main control device is further used to judge whether there are obstacles on the operation path according to the reflection information, and control the loading platform to perform an obstacle avoidance operation when it is judged that there are obstacles.

[0009] According to a preferred embodiment of the present invention, the obstacle avoidance operation includes: updating the operation path to bypass the obstacle.

[0010] According to a preferred embodiment of the present invention, the loading platform includes a steering motor for controlling the angle of the wheels to deflect; the step of the main control device controlling the movement of the loading platform according to the operation path information and the ridge position information includes: controlling the steering motor according to the ridge position information to drive the angle of the wheels to deflect so that the traveling direction of the loading platform is directly opposite to the ridge.

[0011] According to a preferred embodiment of the present invention, the main control device determines the ridge position information according to the following ridge recognition algorithm:

[0012] S1.1. Calculate the segmentation threshold of the foreground and background for the real-time visual image information in front, and segment the real-time visual image in front into a foreground part and a background part according to the segmentation threshold;

[0013] S1.2. Detect the contour of the image detection area for the segmented image, and search for a predetermined cone in the contour line of the area to obtain the coordinates of the upper vertex, left vertex, and right vertex of the cone;

[0014] S1.3. Determine whether the cone is a ridge and furrow area. If it is determined to be a ridge and furrow area, fill the ridge and furrow area to achieve connectivity of the ridge and furrow area;

[0015] S1.4. Set upper and lower boundary thresholds, extract all connected regions, and screen the positions of the through-connected regions, where the through-connected regions are the connected regions that contact the top and bottom of the image; generate a binary mask, and extract the contour boundaries of the through-connected regions; at the same time, segment the through-connected regions, and extract the center points of each truncated through-connected region, which are called navigation points;

[0016] S1.5. Cluster the navigation points, divide the navigation points into different through-connected regions, and perform linear fitting on the navigation points in each through-connected region to generate ridge and furrow navigation lines.

[0017] According to a preferred embodiment of the present invention, in step S1.3, a search model is used to perform vertical filling processing on the processed image to obtain a more fragmented and dispersed image. At the same time, the image is cropped, and an area threshold of the region is set to cover the foreground part of the small region as the background, so as to highlight the ridge and furrow area.

[0018] According to a preferred embodiment of the present invention, in step S1.4, the method for determining the center point of the through-connected region is to generate a minimum fitting rectangle for the segmented image, and further extract the centroid of the rectangular bounding region in the image to obtain the navigation line of each image band. After iterative processing, a complete set of navigation points can be obtained.

[0019] According to a preferred embodiment of the present invention, the visual image acquisition device is further configured to acquire real-time visual image information on both sides of the unmanned agricultural operation platform, and send the received real-time visual image information on both sides to the main control device; the main control device is further configured to determine whether the traveling direction of the loading platform deviates from the operation path according to the real-time visual image information on both sides, and control the steering motor to change the traveling direction of the loading platform when it is determined that there is a deviation, so as to correct the deviation amount between the loading platform and the operation path.

[0020] According to a preferred embodiment of the present invention, the main control device determines whether the traveling direction of the loading platform deviates from the operation path according to the following angle correction algorithm, and controls the steering motor to change the traveling direction of the loading platform when it is determined that there is a deviation:

[0021] S2.1. Establish a boundary detection neural network parameter model, and the model is used to calculate ridge and furrow boundary information according to the visual image information on both sides of the ridge and furrow;

[0022] S2.2. Use the boundary detection neural network parameter model to calculate the ridge and furrow boundary position information in real time according to the real-time visual image information on both sides, and calculate the ridge and furrow center line information according to the ridge and furrow boundary position information;

[0023] S2.3. Use the real-time visual image information on both sides to calculate and generate a vector median line in real time;

[0024] S2.4. Calculate the angle between the vector median line and the ridge groove median line, determine whether the row direction of the loading platform deviates from the operation path according to this angle, and use this angle as the deviation angle when deviation occurs.

[0025] According to a preferred embodiment of the present invention, the expandable structure includes an internally mounted expansion platform and an externally mounted expansion platform. The internally mounted expansion platform is used to fix in-machine expansion devices, and the externally mounted expansion platform is used to fix out-of-machine expansion devices.

[0026] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0027] 1. Through the ridge groove recognition algorithm, the present invention realizes the ridge groove-level navigation of the unmanned agricultural operation platform, effectively reducing the damage to crops during the operation process.

[0028] 2. The present invention proposes the concept of a through domain in the ridge groove recognition algorithm, further improving the performance of the ridge groove recognition algorithm, greatly improving the efficiency, reducing the operation cost, and effectively enhancing the economic benefits.

[0029] 3. The ridge groove recognition algorithm of the present invention is applicable to fragmented farmland terrains, and the longitudinal filling model applied therein has wide applicability.

[0030] 4. The present invention uses the real-time visual images on both sides to further correct the traveling angle of the platform, reducing the risk of navigation errors and improving the accuracy of navigation.

[0031] 5. The present invention designs an internally mounted expansion platform and an externally mounted expansion platform. By adding or combining different devices, the problem that the current related operation devices have a single function and need to be frequently replaced can be effectively solved. Description of the Drawings

[0032] Figure 1 is a schematic three-dimensional structure diagram of an unmanned agricultural operation platform according to an embodiment of the present invention.

[0033] Figure 2 is a schematic front structure diagram of an unmanned agricultural operation platform according to an embodiment of the present invention.

[0034] Figure 3 is a module architecture diagram of an automatic driving operation system of an unmanned agricultural operation platform according to an embodiment of the present invention.

[0035] Figure 4 is a flowchart of a ridge groove recognition algorithm according to an embodiment of the present invention.

[0036] Figure 5 It is the trivial scenario binary image processed by the step S1.1 of the ridge and furrow recognition algorithm according to an embodiment of the present invention.

[0037] Figure 6 It is the schematic diagram processed by the step S1.2 of the ridge and furrow recognition algorithm according to an embodiment of the present invention.

[0038] Figure 7 It is the schematic diagram of the longitudinal filling model in the step S1.2 of the ridge and furrow recognition algorithm according to an embodiment of the present invention.

[0039] Figure 8 It is the schematic diagram of the filling process in the step S1.3 of the ridge and furrow recognition algorithm according to an embodiment of the present invention.

[0040] Figure 9 It is the schematic diagram of the central point extraction process in the step S1.4 of the ridge and furrow recognition algorithm according to an embodiment of the present invention.

[0041] Figure 10 It is the schematic diagram of the ridge and furrow navigation line generated by fitting in the step S1.5 of the ridge and furrow recognition algorithm according to an embodiment of the present invention.

[0042] Figure 11 It is the flowchart of the angle correction algorithm according to an embodiment of the present invention. Detailed implementation manners

[0043] In view of the above problems, the present invention provides an extensible unmanned agricultural operation platform, including a loading platform and an autonomous driving operation system for controlling the travel of the loading platform.

[0044] As a preferred implementation manner, the loading platform is used for installing various functional components. More preferably, the loading platform has an extensible structure, and the extensible structure can have multiple extension platforms for installing various extension devices. For example, it includes an internal-mounted extension platform and an external-mounted extension platform. The internal-mounted extension platform is used for fixing the in-machine extension devices. For example, it is composed of four high-strength square steels symmetrically placed and fixed by bolts. One end of it is welded to the loading platform, and the other end is provided with extensible holes. The external-mounted extension platform is used for fixing the out-of-machine extension devices. For example, it is composed of four high-strength steel plates with extension spaces, which are respectively covered on the side of the equipment loading platform and connected by welding.

[0045] In addition, the loading platform also has drivable wheels. The wheels can be electrically driven by a drive motor, and preferably four-wheel drive.

[0046] As a part of the functional components, the automatic driving operation system at least includes a position information acquisition device, a visual image acquisition device, and a main control device.

[0047] The position information acquisition device is used to acquire the geographical position information of the unmanned agricultural operation platform and send it to the main control device. For example, it is a GPS component for acquiring GPS geographical position information. The position information acquisition device may also include an inertial navigation module for acquiring the three-dimensional position information of the platform in real time.

[0048] The visual image acquisition device is mainly used to acquire the real-time visual image information in front of the loading platform and send the received real-time visual image information to the main control device. Preferably, the visual image acquisition device can also be used to acquire the real-time visual image information on both sides of the loading platform. The visual image acquisition device can be, for example, a camera capable of acquiring RGB image information and is configured at the front of the loading platform, including three cameras on the left, middle, and right.

[0049] The core component for navigation calculation in the main control device plans the operation path according to the position information to obtain the operation path information, determines the ridge position information according to the real-time visual image information in front, and controls the movement of the loading platform according to the path information and the ridge position information.

[0050] As a more preferred method, the visual image acquisition device of the present invention is also used to acquire the real-time visual image information on both sides of the unmanned agricultural operation platform and send the received real-time visual image information on both sides to the main control device; the main control device also judges whether the traveling direction of the loading platform deviates from the operation path according to the real-time visual image information on both sides, and controls the steering motor to change the traveling direction of the loading platform when it is judged that there is a deviation, so as to correct the deviation amount between the loading platform and the operation path.

[0051] The working method of the present invention for the unmanned agricultural operation platform is as follows:

[0052] Step S1: The main control device of the platform receives the operation of the user and acquires the position information of the platform through the position information acquisition device;

[0053] Step S2: The main control device of the platform performs operation path planning through a path planning algorithm;

[0054] Step S3: The main control device of the platform performs navigation and controls the loading platform of the platform to move forward; at this time, the main control device preprocesses the real-time visual image information in front of the unmanned agricultural operation platform acquired by the visual image acquisition device and scales the picture to a fixed size;

[0055] Step S4. The main control device of the platform processes the picture of the ridge and furrow area through the ridge and furrow recognition algorithm to determine the position of the ridge and furrow for platform operation and the ridge and furrow navigation line.

[0056] Optionally, the automatic driving operation system further includes a lidar device, which detects in real time in front of the vehicle to determine whether there are obstacles, and transmits the detection information to the main control device in real time for obstacle avoidance operation.

[0057] Step S5. The main control device controls the platform to adjust its direction so that the traveling direction of the platform is directly facing the ridge and furrow.

[0058] As a preferred implementation, the cameras on both sides of the loading platform acquire the real-time visual images on both sides, use the traveling angle correction algorithm to further identify the specific ridge and furrow path, and transmit the angle of the navigation line back to the main control device.

[0059] Further, the ridge and furrow recognition algorithm is as follows:

[0060] S1.1. Calculate the segmentation threshold between the foreground and the background for the real-time visual image information in front, and segment the real-time visual image in front into a foreground part and a background part according to the segmentation threshold.

[0061] Specifically, the real-time operation picture captured by the RGB camera can be processed by converting from the RGB color space to the 2CgCrCb color space and OTSU threshold segmentation to obtain the binary segmentation image of the foreground and the background.

[0062] Preferably, the generation of the 2CgCrCb color space first converts the real-time RGB operation picture from the RGB color space to the YCrCb color space, and then doubles the greenness information in the YCrCb color space.

[0063] Preferably, the OTUS threshold segmentation first calculates the between-class variance and finds a threshold that maximizes the between-class variance between the foreground and the background. This threshold will be used to segment the image. According to the calculated threshold, the image is segmented into a foreground part and a background part; in the binary image, the foreground pixels are set to a specific value, while the background pixels are set to another value.

[0064] S1.2. Detect the contour of the area in the segmented image, and search for a predetermined cone in the contour line of the area to obtain the coordinates of the upper vertex, the left vertex and the right vertex of the cone.

[0065] Specifically, the segmented image can be processed by combining dilation and erosion in morphology to eliminate the noise after segmentation to the greatest extent, and detect the area contour. At the same time, the contour points are traversed in a discontinuous point jumping manner to generate the search model position.

[0066] S1.3. Determine whether the cone is a ditch area. If it is determined to be a ditch area, fill the ditch area to achieve connectivity of the ditch area.

[0067] Specifically, the search model can be used to vertically fill the processed image to obtain a more fragmented and dispersed image; at the same time, the image is cropped, the area threshold is set, and the foreground of the small area is covered as the background to highlight the ridge area.

[0068] Preferably, the vertical filling model is set as a triangle, in which the highest point is called the upper vertex, and the vertices on the left and right sides are called the left vertex and the right vertex. When there is a detection area and a ditch area for the upper vertex and the left vertex or the right vertex, the area is filled, and the filled area is the current position of the vertical filling model. After filling, the areas that cannot be directly connected in the vertical direction can be connected to form a complete area.

[0069] S1.4 sets upper and lower boundary thresholds, extracts all connected domains and screens the positions of connected domains, wherein the connected domains are connected domains that are in contact with the top and bottom of the image; generates a binary mask and extracts the contour boundary of the connected domain; at the same time, the connected domain is segmented and the center point of each truncated connected domain is extracted, which is called a navigation point;

[0070] Preferably, the method for determining the center point of the through-domain generates a minimum fitting rectangle for the segmented image, and further extracts the center of gravity of the rectangular enclosed area in the image to obtain the navigation line of each image band. After iterative processing, a complete set of ridge navigation points can be obtained.

[0071] S1.5. Connected domain matching is used to cluster the navigation points, and the least squares method is used for straight line fitting to obtain the ridge navigation line and transmit it back.

[0072] Further, the travel angle correction algorithm method is as follows:

[0073] S2.1. Establish a boundary detection neural network parameter model, which is used to calculate the ditch boundary information based on the visual image information on both sides of the ditch.

[0074] Specifically, remote agricultural IoT devices can be used to collect color image data of ridges and ditches, clean the collected data, and remove data that does not conform to the task scenario; perform data enhancement operations such as translation and noise processing on the collected data; add the enhanced data to the data set and divide it into training set, test set, and validation set in a ratio of 8:1:1;

[0075] Preferably, the boundary detection neural network consists of a feature extraction module, a multi-scale information fusion module and a classification module. The deep learning network is trained with the collected training set to obtain a trained boundary detection neural network parameter model.

[0076] Preferably, the feature extraction module is used to extract features of the ridge furrow boundary. It consists of an input module and multiple stage modules, and is used to extract deep features from images. Each stage contains multiple residual network blocks, and each residual block consists of convolutional units containing skip connections, which are used for feature extraction and information transmission of the network. The result output by the last residual block of each stage is represented as {O 1 ,O 2 ,O 3 ,O 4}.

[0077] Preferably, the multi-scale information fusion module performs two-way feature enhancement on the outputs of each stage of the feature extraction module, and is divided into two structures: top-down and bottom-up. The multi-scale information fusion module is mainly composed of convolutional layers. In the top-down structure, bilinear interpolation is used to perform upsampling operation on the image. The output result of the top-down module is {P 1 ,P 2 ,P 3 ,P 4}, and the bottom-up module outputs {N 1 ,N 2 ,N 3 ,N 4} respectively.

[0078] Preferably, the classification module consists of a fully connected layer and a classification layer. The multi-scale information fusion module inputs the last output N 4 of the bottom-up module into the classification module to complete the identification work of the ridge furrow boundary.

[0079] Preferably, the boundary detection neural network estimates the optimal boundary detection neural network parameter model by minimizing the loss function for classification prediction.

[0080] S2.2. The main control device uses the boundary detection neural network parameter model to calculate the ridge furrow boundary position information in real time according to the real-time visual image information on both sides, and calculates the ridge furrow center line information according to the ridge furrow boundary position information.

[0081] S2.3. The main control device uses the real-time visual image information on both sides to calculate and generate a vector center line in real time;

[0082] S2.4. Calculate the included angle between the vector center line and the ridge furrow center line, judge whether the traveling direction of the loading platform deviates from the operation path according to the included angle, and use the included angle as the deviation angle when deviation occurs.

[0083] To make the objectives, technical solutions, and advantages of the invention embodiments clearer, the following will clearly and completely describe the technical solutions in the invention embodiments in conjunction with the accompanying drawings in the invention embodiments. Obviously, the described embodiments are some, but not all, of the invention embodiments. Generally, the components of the invention embodiments described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0084] Figure 1 It is a schematic three-dimensional structure diagram of an unmanned agricultural operation platform according to an embodiment of the present invention. Figure 2 It is a schematic front structure diagram of the unmanned agricultural operation platform of this embodiment. As Figure 1 and Figure 2 shown, the unmanned agricultural operation platform of this embodiment includes a loading platform 1 and an autonomous driving operation system. The loading platform 1 has four drivable wheels 4, as well as a driving motor for controlling the forward movement of the wheels 4 and a steering motor for deflecting the angle. In addition, the loading platform 1 also has an expandable structure 3, including an internally mounted expansion platform 3-1 and an externally mounted expansion platform 3-2. The internally mounted expansion platform is used to fix in-machine expansion devices, and the externally mounted expansion platform is used to fix out-of-machine expansion devices. The internally mounted expansion platform 3-1 is symmetrically placed by four high-strength square steels, one end is welded to the loading platform 1, and the other end is provided with expandable holes for fixing to the expansion device with bolts. The externally mounted expansion platform 3-2 is composed of four high-strength steel plates with expansion spaces, which are respectively covered on the sides of the loading platform 1 and connected by welding.

[0085] The autonomous driving operation system includes a GPS component 5, an inertial navigation module 6, a lidar 7, an RGB camera 8, and a mobile platform controller 9. Among them, the GPS component 5 and the inertial navigation module 6 are used as position information acquisition devices to acquire the position information of the unmanned agricultural operation platform. The RGB camera 8 is used as a visual image acquisition device to acquire the real-time visual image information in front of the unmanned agricultural operation platform. The mobile platform controller 9 is used as the main control device.

[0086] In this embodiment, there are two GPS components 5, which are respectively fixed on both sides of the front part of the top of the loading platform 1 in a magnetic adsorption connection manner. The inertial navigation module 6 and the mobile platform controller 9 are arranged inside the equipment loading platform 1 and are linearly connected to each other. The lidar 7 is fixed at the front end of the loading platform 1. There are three RGB cameras 8, namely a left RGB camera 81, a middle RGB camera 82, and a right RGB camera 83, which are respectively arranged at the left, middle, and right positions at the front end of the loading platform 1.

[0087] Figure 3It is a module architecture diagram of the automatic driving operation system of an unmanned agricultural operation platform according to an embodiment of the present invention.

[0088] As Figure 3 shown, the mobile platform controller 9 receives the GPS component 5, the inertial navigation module 6, and the lidar 7. The three RGB cameras 81, 82, and 83 are all electrically connected to the mobile control platform controller 9 to transmit corresponding signals to the mobile control platform controller 9. The platform controller 9 then generates control information for controlling the drive motor and the steering motor, and sends the control information to the drive motor and the steering motor. Since the drive motor has little relevance to the present invention, it is not shown in Figure 2 the figure.

[0089] During the implementation of the above embodiment, the cloud control personnel set the starting and ending points to the mobile platform controller 9 of the acquisition platform. The GPS component 5 obtains the current platform longitude and latitude coordinates and sends them to the mobile platform controller 9. The mobile platform controller 9 performs automatic operation path planning and transmits real-time control electrical signals to the drive motor and the steering motor, so that the platform travels to the working starting point at the edge of the farmland according to the planned route. When automatically operating in the farmland, the real-time image captured by the RGB camera 8 in the middle will be transmitted to the mobile platform controller 9 and scaled to a size of 640*640. At the same time, the position of the ridge ditch and the navigation line in the image are determined. The lidar 7 scans the front 3-meter range in real time and transmits the scan result to the mobile platform controller 9 in real time. When encountering an obstacle, the mobile platform controller 9 updates the operation path to bypass the obstacle to achieve the automatic obstacle avoidance function. If the obstacle cannot be avoided, the drive motor is controlled to stop working. The mobile platform controller 9 transmits adjustment electrical signals to the steering motor according to the position of the ridge ditch and the navigation line, so that the traveling direction of the platform is directly facing the ridge ditch. When operating on the ridge ditch, the RGB cameras 8 on the left and right sides transmit the corresponding ridge ditch images back to the mobile platform controller 9, and the specific ridge ditch path is further identified through the angle correction algorithm, so that the platform will not deviate from the ridge ditch and damage the crops. Using the real-time corrected navigation angle information, the mobile platform controller 9 transmits adjustment electrical signals to the steering motor according to the position of the ridge ditch and the navigation line to control the movement angle trajectory of the platform.

[0090] Figure 4 shows a flowchart of the ridge ditch recognition algorithm according to an embodiment of the present invention. As Figure 4 shown, the ridge ditch recognition algorithm provided by the present invention is as follows:

[0091] S1.1. Calculate the segmentation threshold of the foreground and background for the forward real-time visual image information, and segment the forward real-time visual image into a foreground part and a background part according to the segmentation threshold.

[0092] In this embodiment, the real-time operation image captured by the RGB camera is subjected to a conversion process from the RGB color space to the 2CgCrCb color space. The conversion formula is as follows:

[0093]

[0094] Where Q is the luminance information, R, G, and B respectively represent the red, green, and blue components in the original image, and Cr, Cg, and Cb respectively represent the redness, greenness, and blueness information. Then, OTSU threshold segmentation processing is performed. All possible thresholds in the converted 2CgCrCb image are traversed to maximize the between-class variance between the foreground and the background. This threshold will be used to segment the image. According to the calculated threshold, the image is segmented into two parts: the foreground and the background.

[0095] Figure 5 is the binary image of the farmland plot with many trivial pictures after being processed by the step S1.1 of the present invention, where the white is the foreground image and the black is the background image.

[0096] S1.2. Detect the contour of the region of the segmented image, and search for a predetermined cone in the contour line of this region to obtain the coordinates of the upper vertex, left vertex, and right vertex of the cone.

[0097] In this step of this embodiment, the segmented image is eroded and then dilated using a 3×3 sized convolution kernel to eliminate the noise after segmentation to the greatest extent and detect the region contour. Figure 6 is for Figure 5 the schematic diagram after erosion and dilation processing.

[0098] Next, for the set C of region contour points 1 traverse the contour points, and set the upper vertex P in a discontinuous point jumping manner top and generate the corresponding position of the cone search model, where the height l of the cone search model height and the width l weight are set according to the ridge groove width. According to the upper vertex P top the height l height and the width l weight the horizontal and vertical coordinates of the left vertex and right vertex of the cone search model can be obtained as:

[0099]

[0100] Where H is the height of the image, W is the width of the image, y top is the vertical coordinate of the vertex P top and x top is the horizontal coordinate of the vertex P top Therefore, the left vertex P left of the cone search model and the right vertex Pright The coordinates are (x left , y left ), (x right , y right ). S1.3. Determine whether the cone is a ridge and furrow area. If it is determined to be a ridge and furrow area, fill the ridge and furrow area to achieve connectivity of the ridge and furrow area.

[0101] Figure 7 It is a schematic diagram of the longitudinal filling model. As Figure 7 shown, in this step of this embodiment, determine whether the pixel values at the positions of the left vertex P left and the right vertex P right are the same as the pixel value of the upper vertex P top . If the value of P top is the same as the value of P left or P right , set the values covered by the left half or right half of the model to the value of P top to achieve regional connectivity; if the two values are different, do not perform coverage reset; at the same time, crop the image boundary area to three-quarters of the original image height and width, set the regional area threshold to cover the foreground part of the small area as the background, so as to highlight the ridge and furrow area. Figure 8 Schematic diagram of the filling process of this step.

[0102] S1.4. Set the upper and lower boundary thresholds, extract all connected domains and screen the positions of the through-connected domains, where the through-connected domains are the connected domains in contact with the top and bottom of the image; generate a binary mask, extract the contour boundary of the through-connected domains; at the same time, segment the through-connected domains and extract the center point of each truncated through-connected domain, which is called the navigation point.

[0103] Figure 9 It is a schematic diagram of the center point extraction process. As Figure 9 shown, in this embodiment, set the upper and lower boundary thresholds to 20 pixels, extract all connected domains and screen the positions of the through-connected domains, where the through-connected domains are the connected domains in contact with the top and bottom of the image; generate a binary mask, extract the contour boundary of the through-connected domains; at the same time, horizontally divide the image into multiple slices, the height of each slice is Δh, extract the foreground pixel density distribution of each slice, and calculate its minimum circumscribed rectangle, and set the center point of the circumscribed rectangle as the navigation point.

[0104] S1.5. Cluster the navigation points, divide each navigation point into different through-connected domains, and perform linear fitting on the navigation points in each through-connected domain to generate a ridge and furrow navigation line.

[0105] In this embodiment, connected domain matching is used to cluster the navigation points, and they are divided into this contour according to the inclusion relationship between the navigation points and the contour boundaries R i of different through-connected domains. For each navigation point Pj Its clustering formula is as follows:

[0106]

[0107] Among them, inside(·) represents the relationship division operator, represents R i the set of navigation points in the through domain; The least squares method is used to perform a linear fitting in the form of Y = aX + b, and its formula is as follows:

[0108]

[0109] Among them, x i , y i , are the coordinates of the i-th navigation point, and N is the number of navigation points. After fitting, the ridgeline navigation line is obtained and sent back to the mobile platform controller 9.

[0110] Figure 10 is a schematic diagram of the ridgeline navigation line generated by fitting in this step. As Figure 10 shown, the red dotted line is the standard line, representing the navigation line manually marked by those with rich experience, which can be used as the gold standard; The yellow solid line is the fitting line, that is, the navigation line.

[0111] In this embodiment, the mobile platform controller 9 also judges whether the row direction of the loading platform deviates from the operation path according to the following angle correction algorithm, and controls the steering motor to change the traveling direction of the loading platform when it is judged that there is a deviation.

[0112] Next, refer to Figure 11 to illustrate the specific implementation manner of the angle correction algorithm of the present invention. As Figure 11 shown, the angle correction algorithm includes the following steps.

[0113] S2.1. Establish a boundary detection neural network parameter model, which is used to calculate the ridge boundary information according to the visual image information on both sides of the ridge.

[0114] Use the remote agricultural Internet of Things device to collect the color image data of the ridge; Clean the collected data to eliminate the data that does not conform to the task scenario; Perform data enhancement operations such as translation operations and noise processing on the collected data; Add the enhanced data to the data set and divide it into a training set, a test set, and a validation set according to a ratio of 8:1:1;

[0115] Construct a boundary detection neural network for ridge boundary recognition on both sides, which consists of a feature extraction module, a multi-scale information fusion module, and a classification module.

[0116] Among them, the feature extraction module consists of an input module and multiple stage modules, and is used to extract deep features from images. The input module consists of a convolutional layer and a max pooling layer. The data in the training set undergoes a convolutional operation with a convolutional kernel size of 7×7, an output channel of 64, and a stride of 2, and then enters a 3×3 max pooling layer with a stride of 2. The output of the input module is the input of the stage module. The stage module consists of convolutional layers with skip connections, the convolutional kernel size is 3×3, and the output channel numbers of the convolutional layers are 64, 128, 256, and 512 respectively, that is, the channel numbers of the results {O 1 ,O 2 ,O 3 ,O 4} output by the last residual block of each stage module are 64, 128, 256, and 512.

[0117] The multi-scale information fusion module is divided into two structures: top-down and bottom-up. First, top-down information fusion is performed. For the {O 1 ,O 2 ,O 3 ,O 4} output by the feature extraction module, a horizontal connection operation is performed, that is, 1×1 convolutional resampling, to adjust the output channel number to 64 without changing the output image size. In the top-down direction, a horizontal connection is made to O 4 to obtain P 4 . Then, P 4 is upsampled by bilinear interpolation by a factor of two and added to the result of the horizontal connection with O 3 . After that, a convolutional operation with a convolutional kernel size of 3×3 is performed to obtain P 3 . Repeat this operation to obtain the output results {P 1 ,P 2 ,P 3 ,P 4}, completing the top-down feature fusion process. The relationship between the output of the top-down structure and the features output by the feature extraction module is as follows:

[0118]

[0119] Among them, conv(·) represents the convolution operator, up(·) represents the upsampling operator, that is, the bilinear interpolation upsampling operation, l(·) represents the horizontal connection operator, and P i represents the result after the horizontal connection operation of the i-th layer feature, and n represents the total number of stage modules.

[0120] Then, bottom-up information fusion is performed, which is similar to the top-down structure, but no convolutional operation is performed after merging. For the {P 1 ,P 2 ,P3 , P 4} performs a horizontal connection operation, i.e., 1×1 convolution resampling, adjusts the number of output channels to 64, and in the bottom-up direction, for P 1 performs a horizontal connection to obtain N 1 , and then for N 1 performs a two-fold downsampling and adds and merges it with the result of the horizontal connection with P 2 . Repeat this operation to obtain the output result {N 1 , N 2 , N 3 , N 4}, completing the top-down feature fusion process. The relationship between the output of the bottom-up structure and the features output by the feature extraction module is as follows:

[0121]

[0122] where down(·) represents the downsampling operator, i.e., the two-fold downsampling operation, and N i represents the result of the i-th layer of bottom-up information fusion in the i-th layer;

[0123] The classification module is mainly composed of a fully connected layer and a classification layer. The output N of the bottom-up structure 4 is input into the fully connected layer, and then it is deformed and divided into a grid-like image with h rows and w columns. Suppose there are C ridge lines to be detected, and ridge boundary classification prediction is performed on it. The prediction probability value formula for the lane line is as follows:

[0124] M c,j = f cj (X) (8)

[0125] where M c,j represents the probability value of the c-th ridge line in the (w + 1)-th column of the j-th row, and f cj (·) represents the classifier of the c-th ridge line in the j-th row, and X represents the feature value.

[0126] The loss function for classification prediction is divided into a classification loss function L cls and a similarity loss function L sim , and the total loss is the sum of the two. The formula for the classification loss function is as follows:

[0127]

[0128] where, L focal (·) is the focal loss function operator, and T c,j is the one-hot encoding corresponding to the ridge line label value. The formula for the similarity loss function is as follows:

[0129]

[0130] Among them, ‖·‖ l represents the L 1 norm of the classification on adjacent rows.

[0131] The deep learning network is trained using the collected training set, and the trained boundary detection neural network parameter model is obtained by minimizing the loss function for classification prediction.

[0132] S2.2. Using the boundary detection neural network parameter model, calculate the ridge groove boundary position information in real time according to the real-time visual image information on both sides, and calculate the ridge groove center line information according to the ridge groove boundary position information.

[0133] S2.3. Use the real-time visual image information on both sides to calculate and generate a vector center line in real time.

[0134] The vector center line refers to the center line of the real-time picture, which is vector because it extends.

[0135] S2.4. Calculate the included angle between the vector center line and the ridge groove center line, judge whether the row direction of the loading platform deviates from the operation path according to the included angle, and use the included angle as the deviation angle when deviation occurs.

[0136] Finally, the navigation angle that needs to be adjusted will be obtained and sent back to the mobile platform controller 9.

[0137] Through the description of the above embodiments, it can be seen that the present invention realizes the ridge groove level navigation of the unmanned agricultural operation platform through the ridge groove recognition algorithm, effectively reducing the damage to crops during the operation process. The present invention proposes the concept of a through domain in the ridge groove recognition algorithm, further improving the performance of the ridge groove recognition algorithm, greatly improving the efficiency, reducing the operation cost, and effectively enhancing the economic benefits. The ridge groove recognition algorithm of the present invention is applicable to fragmented farmland terrains, and the longitudinal filling model applied therein has wide applicability. The present invention further corrects the traveling angle of the platform using the real-time visual images on both sides, reducing the risk of navigation errors and improving the accuracy of navigation.

[0138] The above is only a preferred embodiment of the present invention, and does not impose any form of limitation on the present invention. Any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention are all within the scope of the technical solution of the present invention.

Claims

1. An expandable unmanned agricultural operation platform, comprising a loading platform and an automatic driving operation system for controlling the movement of the loading platform, wherein the loading platform has drivable wheels, and wherein: The loading platform has an expandable structure, which is used to install various expansion equipment; The automatic driving operation system includes a position information acquisition device, a visual image acquisition device and a main control device, wherein: The position information acquisition device is used to acquire the position information of the unmanned agricultural operation platform and send it to the main control device; The visual image acquisition device is used to acquire the front real-time visual image information in front of the unmanned agricultural operation platform, and send the received real-time visual image information to the main control device; The main control device is used to plan the operation path according to the position information, obtain the operation path information, determine the ditch position information according to the real-time visual image information in front, and control the movement of the loading platform according to the path information and the ditch position information.

2. The expandable unmanned agricultural operation platform according to claim 1, characterized in that: The automatic driving operation system also includes a laser radar device, The laser radar device is used to emit laser pulses and receive reflected signals of the laser pulses, and send the received reflected information to the main control device; The main control device is also used to determine whether there is an obstacle on the working path according to the reflection information, and control the loading platform to perform obstacle avoidance operation when it is determined that there is an obstacle.

3. The expandable unmanned agricultural operation platform according to claim 2, characterized in that: The obstacle avoidance operation includes: updating the operation path to bypass obstacles.

4. The expandable unmanned agricultural operation platform according to claim 1, characterized in that: The loading platform includes a steering motor for controlling the angle of the wheels to deflect; The step in which the main control device controls the movement of the loading platform according to the working path information and the ditch position information includes: controlling the steering motor according to the ditch position information to drive the wheel to deflect at an angle so that the moving direction of the loading platform faces the ditch.

5. The expandable unmanned agricultural operation platform according to claims 1 to 4, characterized in that: The main control device determines the ridge position information according to the following ridge recognition algorithm: S1.1, calculating the segmentation threshold of the foreground and the background for the real-time visual image information in front, and segmenting the real-time visual image in front into a foreground part and a background part according to the segmentation threshold; S1.2, detecting the area contour of the segmented image, searching for a predetermined cone in the area contour, and obtaining the coordinates of the upper vertex, the left vertex, and the right vertex of the cone; S1.3, determining whether the cone is a ditch area, and if it is determined to be a ditch area, filling the ditch area to achieve connectivity of the ditch area; S1.4, set upper and lower boundary thresholds, extract all connected domains and filter the positions of connected domains, wherein the connected domains are connected domains that are in contact with the top and bottom of the image; generate a binary mask and extract the contour boundary of the connected domain; at the same time, segment the connected domain and extract the center point of each truncated connected domain, which is called a navigation point; S1.

5. Clustering the navigation points, dividing each navigation point into different through-domains, and performing straight line fitting on the navigation points in each through-domain to generate a ridge navigation line.

6. The expandable unmanned agricultural operation platform according to claim 5, characterized in that: In step S1.3, the search model is used to vertically fill the processed image to obtain a more fragmented and dispersed image. At the same time, the image is cropped, and the area threshold is set to cover the small area foreground as the background to highlight the ridge area.

7. The expandable unmanned agricultural operation platform according to claim 5, characterized in that: In step S1.4, the method for determining the center point of the penetration domain is to generate a minimum fitting rectangle for the segmented image, and further extract the center of gravity of the rectangular enclosed area in the image to obtain the navigation line of each image band. After iterative processing, a complete set of navigation points can be obtained.

8. The expandable unmanned agricultural operation platform according to claim 4, characterized in that: The visual image acquisition device is also used to acquire real-time visual image information on both sides of the unmanned agricultural operation platform, and send the received real-time visual image information on both sides to the main control device; The main control device is also used to determine whether the movement direction of the loading platform is offset from the working path based on the real-time visual image information on both sides, and control the steering motor to change the movement direction of the loading platform when it is determined that a deviation occurs, so as to correct the offset between the loading platform and the working path.

9. The expandable unmanned agricultural operation platform according to claim 8, characterized in that: The main control device determines whether the travel direction of the loading platform deviates from the working path according to the following angle correction algorithm, and controls the steering motor to change the travel direction of the loading platform when it is determined that the deviation occurs: S2.1, establishing a boundary detection neural network parameter model, wherein the model is used to calculate ditch boundary information based on visual image information on both sides of the ditch; S2.2, using the boundary detection neural network parameter model, calculating the ditch boundary position information in real time according to the real-time visual image information on both sides, and calculating the ditch centerline information according to the ditch boundary position information; S2.3, using the real-time visual image information on both sides to calculate and generate a vector centerline in real time; S2.

4. Calculate the angle between the center line of the vector and the center line of the furrow, determine whether the row direction of the loading platform is offset from the working path based on the angle, and use the angle as the offset angle when offset occurs.

10. The expandable unmanned agricultural operation platform according to claim 1, characterized in that: The expandable structure includes an internal expansion platform and an external expansion platform. The internal expansion platform is used to fix the expansion equipment inside the machine, and the external expansion platform is used to fix the expansion equipment outside the machine.

Images