Control method and apparatus for agricultural robot, and device and storage medium

Abstract

Disclosed in the present invention are a control method and apparatus for an agricultural robot, and a device and a storage medium. The method comprises: acquiring farmland environment images collected by an image collection device, and basic attributes, a traveling speed and a traveling direction of an agricultural robot, wherein the basic attributes comprise a wheel track and the spatial position of the image collection device on the agricultural robot; inputting the farmland environment images into a pre-trained semantic segmentation model based on channel and spatial attention mechanisms, and outputting feature images of crop rows in the farmland environment images; on the basis of the wheel track and the spatial position, determining position information of crop rows closest to wheels of the agricultural robot; on the basis of the spatial position, converting the position information into a robot coordinate system; and on the basis of the position information converted into the robot coordinate system and the current traveling speed, determining a steering angular speed of the agricultural robot, and controlling the agricultural robot to control the heading angle thereof according to the steering angular speed. By using the method, the operational efficiency of an agricultural robot is significantly improved by means of the automatic identification of crop rows and control technology.

Description

A method, apparatus, equipment and storage medium for controlling an agricultural robot. Technical Field

[0001] The present invention relates to the field of automatic control technology, and in particular to an agricultural robot control method, device, equipment and storage medium. Background Technology

[0002] Currently, crop row identification and tracking control is a key technological aspect of agricultural automation. Crop rows are typically rows of plants planted in farmland, and maintaining a well-ordered arrangement of crop rows is fundamental to efficient agricultural management. To improve production efficiency and reduce costs, modern agriculture is increasingly introducing automated equipment such as unmanned agricultural machinery and field management robots. These devices rely on advanced computer vision and control technologies to accurately identify and track crop rows, thereby enabling precision agriculture operations such as automatic weeding, spraying, fertilization, and harvesting.

[0003] Satellite navigation systems play a dominant role in the navigation and positioning of autonomous agricultural robots, achieving positioning accuracy at the decimeter or centimeter level. This is particularly true for large-scale, centralized open farms where satellite signals are relatively stable and reliable, and satellite positioning technology is relatively mature. However, with changes in farmland environments, especially in small-scale, dispersed, and regionalized environments, satellite positioning and navigation cannot automatically identify crop rows and cannot plan paths, move, or perform corresponding field management operations based on the distance between the wheels and crop rows, thus failing to meet practical application needs. Summary of the Invention

[0004] This invention provides an agricultural robot control method, apparatus, device, and storage medium. By automatically identifying crop rows and using tracking control technology, the reliance on human labor can be reduced, significantly improving the operational efficiency of agricultural robots.

[0005] In a first aspect, embodiments of the present invention provide an agricultural robot control method, comprising:

[0006] The system acquires farmland environment images captured by an image acquisition device, as well as the basic attributes, speed, and direction of travel of an agricultural robot. The basic attributes include wheelbase and the spatial position of the image acquisition device on the agricultural robot. The image acquisition device acquires images at a fixed acquisition angle.

[0007] The farmland environment image is input into a pre-trained semantic segmentation model based on channel and spatial attention mechanisms, and the model outputs feature images of crop rows in the farmland environment image; wherein, the training dataset of the semantic segmentation model consists of original farmland environment images collected under different environments; the original farmland environment images are labeled with the crop rows and their locations;

[0008] The location information of the crop row closest to the wheels of the agricultural robot is determined based on the wheel track and the spatial position.

[0009] The position information is converted to the robot coordinate system based on the spatial location.

[0010] The turning angular velocity of the agricultural robot is determined based on the position information transformed into the robot coordinate system and the travel speed, and the agricultural robot is controlled to control its heading angle according to the turning angular velocity.

[0011] Secondly, embodiments of the present invention also provide an agricultural robot control device, the device comprising:

[0012] The information acquisition module is used to acquire farmland environment images collected by the image acquisition device, as well as the basic attributes, driving speed, and driving direction of the agricultural robot; the basic attributes include: wheelbase and the spatial position of the image acquisition device on the agricultural robot; the image acquisition device acquires images at a fixed acquisition angle;

[0013] The feature image output module is used to input the farmland environment image into a pre-trained semantic segmentation model based on channel and spatial attention mechanisms, and output feature images of crop rows in the farmland environment image; wherein, the training dataset of the semantic segmentation model is the original farmland environment images collected under different environments; the original farmland environment images are labeled with the crop rows and their positions;

[0014] The location information module is used to determine the location information of the crop row closest to the wheels of the agricultural robot based on the wheel track and the spatial position;

[0015] A coordinate system transformation module is used to transform the position information to the robot coordinate system based on the spatial location;

[0016] The steering control module is used to determine the steering angular velocity of the agricultural robot based on the position information transformed into the robot coordinate system and the travel speed, and to control the agricultural robot to control the heading angle of the agricultural robot according to the steering angular velocity.

[0017] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0018] One or more processors;

[0019] Storage device for storing one or more programs.

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the agricultural robot control method provided in the embodiments of this disclosure.

[0021] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to implement the agricultural robot control method provided in embodiments of this disclosure.

[0022] Fifthly, this disclosure provides a computer program product, which includes a computer program that, when executed by a processor, implements the agricultural robot control method provided in the first aspect of the embodiment.

[0023] This invention discloses an agricultural robot control method, device, equipment, and storage medium, comprising: acquiring farmland environment images captured by an image acquisition device, as well as the basic attributes, driving speed, and driving direction of the agricultural robot; the basic attributes including: wheelbase and the spatial position of the image acquisition device on the agricultural robot; the image acquisition device acquiring images at a fixed acquisition angle; inputting the farmland environment images into a pre-trained semantic segmentation model based on channel and spatial attention mechanisms, and outputting feature images of crop rows in the farmland environment images; wherein, the training dataset of the semantic segmentation model is original farmland environment images acquired under different environments; the original farmland environment images are labeled with the crop rows and their positions; determining the position information of the crop row closest to the wheels of the agricultural robot based on the wheelbase and the spatial position; converting the position information to the robot coordinate system based on the spatial position; determining the steering angular velocity of the agricultural robot based on the position information converted to the robot coordinate system and the driving speed, and controlling the agricultural robot to control its heading angle according to the steering angular velocity. Using this method, through automatic identification of crop rows and clusters and tracking control technology, reliance on human labor can be reduced, and the operational efficiency of agricultural robots can be significantly improved. Attached Figure Description

[0024] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0025] Figure 1 is a flowchart of an agricultural robot control method provided in an embodiment of this disclosure;

[0026] Figure 2 is a feature image example of an agricultural robot control method provided in an embodiment of this disclosure;

[0027] Figure 3 is a schematic diagram of the structure of an agricultural robot control device provided in an embodiment of this disclosure;

[0028] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0029] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0030] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0031] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0032] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0033] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0034] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0035] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0036] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0037] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0038] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0039] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0040] Figure 1 is a flowchart of an agricultural robot control method provided by an embodiment of this disclosure. This embodiment of the disclosure is applicable to situations where satellite positioning and navigation cannot automatically identify crop rows and cannot plan a path based on the distance between the wheels and the crop rows. The method can be executed by an agricultural robot control device, which can be implemented in the form of software and / or hardware, or optionally, through an electronic device, such as a mobile terminal, a PC, or a server.

[0041] As shown in Figure 1, an agricultural robot control method provided in this embodiment of the present disclosure may specifically include the following steps:

[0042] S110. Acquire images of the farmland environment collected by the image acquisition device, as well as the basic attributes, driving speed, and driving direction of the agricultural robot.

[0043] The basic attributes include: wheelbase and the spatial position of the image acquisition device on the agricultural robot; the image acquisition device acquires images at a fixed acquisition angle.

[0044] In this embodiment, the image acquisition device can be a camera or other image acquisition device mounted on the agricultural robot. The basic attributes can be attributes of the agricultural robot itself. The wheelbase can be the wheelbase of the vehicle or the distance between the left and right tracks of a tracked vehicle.

[0045] In this embodiment, images of the farmland environment captured by an image acquisition device, as well as the basic attributes, speed, and direction of travel of the agricultural robot, are acquired. The basic attributes include wheelbase and the spatial position of the image acquisition device on the agricultural robot. The image acquisition device acquires images at a fixed acquisition angle.

[0046] S120. Input the farmland environment image into a pre-trained semantic segmentation model based on channel and spatial attention mechanisms, and output the feature image of the crop rows in the farmland environment image.

[0047] The training dataset for the semantic segmentation model consists of original farmland environment images collected under different conditions. These images are labeled with crop rows and their locations. The semantic segmentation model comprises an encoder-decoder module and an output layer.

[0048] In this embodiment, the semantic segmentation model using channel and spatial attention mechanisms is the VC-UNet semantic segmentation model, an improvement upon the U-Net semantic segmentation model. The training dataset consists of a robot image acquisition platform moving across and along crop rows, with cameras continuously acquiring crop images. Over 3500 images were collected to handle various weather conditions, including strong sunlight, weak light, balanced illumination, horizontal and vertical crop furrows, and occlusion. The crop row dataset was divided into a 9:1 ratio for training and validation, with 3150 images in the training set and 350 images in the validation set. To ensure consistency between the segmented dataset and the original images, each crop row in the crop images was labeled. Based on the segmentation target to be extracted, line labels were used to mark the start and end points of the crop rows and connect the points to form lines.

[0049] Specifically,

[0050] Figure 2 is an example image of a feature image of an agricultural robot control method provided in an embodiment of this disclosure.

[0051] Based on the above embodiments, inputting farmland environment images into a pre-trained semantic segmentation model based on channel and spatial attention mechanisms, and outputting feature images of crop rows in the farmland environment images, may include the following steps:

[0052] a1) Input the farmland environment image into the encoder-decoder module, output the edge feature image of the crop rows in the farmland environment image, and input the edge feature image into the output layer; the size of the edge feature image is the same as the size of the farmland environment image.

[0053] b1) Compare the feature value in the edge feature image with a set threshold. When the feature value is greater than or equal to the set threshold, set the pixel of the farmland environment image corresponding to the feature value to the first set value. When the feature value is less than the set threshold, set the pixel of the farmland environment image corresponding to the feature value to the second set value, and output the feature image.

[0054] Specifically, the encoder-decoder module consists of an encoder and a decoder. The encoder's main function is to extract features from the input image through convolutional layers and reduce the size of the feature maps through pooling layers, thereby reducing computation and capturing higher-level semantic information. An encoder block typically contains two 3x3 convolutional layers, a ReLU activation function, and a 2x2 max-pooling layer to progressively reduce the spatial dimension of the feature maps and increase the number of channels. Each layer of the encoder doubles the number of channels in the feature map while halving the resolution until an intermediate feature map is reached. The intermediate feature map is the last layer of the encoder part, containing the highest-level feature representation extracted from the input image. These features will be used in the decoder part to reconstruct the structural information of the original image. The decoder, on the other hand, functions in the opposite way to the encoder. It progressively restores the size of the feature maps through upsampling operations and fuses multi-scale features from the encoder through skip connections to reconstruct the detailed information of the image. Each layer of the decoder halves the number of channels in the feature map while doubling the resolution until it reaches the same size as the input image. The skip connection mechanism allows the decoder to directly access the feature maps of the corresponding layers of the encoder during upsampling, which helps to preserve more detailed information and improve the accuracy of segmentation.

[0055] In this embodiment, the first setting value and the second setting value are preset values, which are set according to the actual situation. In this embodiment, no specific limitation is made. For example, the first setting value can be 0 and the second setting value can be 255.

[0056] Specifically, the output layer is typically a 1x1 convolutional layer used to map the feature map generated by the decoder to the final segmentation result. Feature values ​​in the edge feature image are compared with a set threshold. When the feature value is greater than or equal to the threshold, the pixel corresponding to that feature value in the farmland environment image is set to the first set threshold; when the feature value is less than the threshold, the pixel corresponding to that feature value in the farmland environment image is set to the second set threshold, and the feature image is output. The number of output channels of this convolutional layer is usually equal to the number of classes in the image, for example, 2 in a binary classification problem.

[0057] S130. Determine the position information of the crop row closest to the wheels of the agricultural robot based on the wheel track and the spatial position.

[0058] In this embodiment, the location information may be a function mapping relationship corresponding to crop rows that meet certain set conditions.

[0059] Specifically, the wheel position information in the feature image is determined based on the wheel track and the spatial position; a corresponding rectangular region is selected from the feature image as the region of interest according to preset coordinates; the preset coordinates are the coordinates of the four corners of the rectangular region; and the function mapping relationship corresponding to the crop rows in the region of interest is extracted.

[0060] Determine the distance between each function mapping relationship and the wheel position information; use the set number of function mapping relationships with the smallest distance as the position information of the corresponding crop row.

[0061] Based on the above embodiments, determining the location information of the two crop rows whose sum of distances from the center point of the feature image to the two crop rows is greater than the wheel spacing and whose sum of distances is the smallest may include the following steps:

[0062] a2) Select the corresponding rectangular region from the feature image as the region of interest according to the preset coordinates; the preset coordinates are the coordinates of the four corners of the rectangular region.

[0063] b2) Extract the function mapping relationship corresponding to the crop rows in the region of interest.

[0064] c2) Determine the distance between each function mapping relationship and the wheel position information.

[0065] d2) The function mapping relationship with the smallest set distance is used as the location information of the corresponding crop row.

[0066] In this embodiment, the function mapping relationship can be a straight line. The set quantity in this embodiment can be 1 or 2. It is determined based on the ratio between the distance between crop rows and the wheel spacing. When the ratio is 1, the set quantity is 1; when the ratio is greater than 1, the set quantity is 2.

[0067] Specifically, the function mapping relationships corresponding to crop rows in the region of interest are extracted using the Hough line transform, and the distance from each function mapping relationship to the wheel position information is determined. A predetermined number of function mapping relationships with the smallest distances are used as the position information for the corresponding crop rows. Here, a threshold is set to merge lines with similar distances based on the smallest distance between the function mapping relationships and the wheel position information.

[0068] Based on the above embodiments, the region of interest is automatically adjusted. The vehicle speed is obtained in real time in the vehicle coordinate system using a satellite positioning system, and the bounding box of the region of interest is updated by calculating the maximum and minimum values ​​of the mask image.

[0069] Based on this embodiment, the method further includes: sorting the merged i straight lines according to their absolute distance to the center coordinates of the ROI clipping map, with serial numbers 1, 2, ..., i. Extracting one or two straight line equations based on the ratio q between the distances between crop rows and the spacing between rows. If q is not an integer, it is rounded to the nearest integer. When q is 1, the straight line closest to the center coordinates is extracted, i.e., the line equation with serial number 1; when q is greater than 1, the line equations with serial numbers q and q-1 are read, and the median of these two lines is calculated, representing the center line of the crop strip. When the row spacing equals the strip spacing, q is 1.

[0070] S140. Convert the position information to the robot coordinate system based on the spatial location.

[0071] Based on the above embodiments, converting position information to the robot coordinate system according to spatial location may include the following steps:

[0072] a3) Determine the coordinate transformation matrix according to the spatial location and the known transformation formula from the robot coordinate system to the camera coordinate system.

[0073] b3) The result of multiplying the position information with the coordinate transformation matrix is ​​used as the position information in the robot coordinate system.

[0074] Specifically, the conversion matrix from pixels to physical units is determined based on the acquisition angle, installation height, and installation position of the acquisition device on the robot, and then the conversion is performed. Alternatively, it can be calculated based on the mapping relationship between the two-dimensional image and the three-dimensional strip model.

[0075] S150. Determine the turning angular velocity of the agricultural robot based on the position information converted to the robot coordinate system and the driving speed, and control the agricultural robot to control the heading angle of the agricultural robot according to the turning angular velocity.

[0076] Specifically, the centerline of the location information is used as the navigation centerline. The current motion state information of the agricultural robot, located by the positioning device, is obtained. The angular deviation between the navigation centerline and the travel direction, as well as the lateral deviation between the navigation centerline and the current motion state information, are determined. The steering angular velocity is determined based on the angular deviation, lateral deviation, and travel speed.

[0077] Based on the above embodiments, determining the turning angular velocity of the agricultural robot according to the position information, travel speed, and travel direction transformed into the robot coordinate system may include the following steps:

[0078] Agricultural robots also include positioning devices.

[0079] a4) Use the center line of the two location information as the navigation center line.

[0080] b4) Obtain the current motion status information of the agricultural robot located by the positioning device.

[0081] In this embodiment, the current motion status information includes the current real-time location, the measured vehicle speed, and the heading angle. If the positioning device has no signal, it uses the motion status information such as location, speed, and heading published by the chassis odometer.

[0082] c4) Determine the angular deviation between the navigation centerline and the current motion state information, as well as the lateral deviation between the navigation centerline and the current motion state information.

[0083] d4) Determine the steering angular velocity based on the angular deviation, lateral deviation, and driving speed.

[0084] Specifically, calculate the angular deviation ang in the image coordinate system. t ang t =arctan(k L )

[0085] Calculate the lateral deviation dis in the image coordinate system t

[0086] Where, k L b represents the slope of the extracted navigation centerline; L Indicates the intercept of the extracted navigation centerline, img h ω represents the pixel width of the image; ω represents the speed adjustment coefficient; v represents the travel speed of the agricultural robot. The angular and lateral deviations in the image coordinate system will be transformed to the robot coordinate system.

[0087] The formula for calculating the steering angular velocity based on angular deviation, lateral deviation, and driving speed is as follows:

[0088] Where δ represents the steering angular velocity; L represents the wheelbase; ang e Indicates the angular deviation in the robot's coordinate system; dis e η represents the lateral deviation in the robot coordinate system; γ represents the first gain coefficient of the controller; v represents the travel speed.

[0089] Based on the above embodiments, it also includes:

[0090] a5) Divide the feature image into multiple sub-block regions according to a sliding window of a set size; the sliding window processes the feature image sequentially from top to bottom;

[0091] b5) Determine the pixels and values ​​of each row within the sub-block region;

[0092] c5) When there is a pixel sum value greater than a set threshold, the position of the pixel sum value is the position of the end of the crop row;

[0093] d5) When the end of the crop row is within the set range, the agricultural robot will exit.

[0094] Specifically, the feature image is segmented according to a sliding window of a set size to obtain multiple sub-regions; the sliding window processes each sub-region sequentially from top to bottom; the pixel sum and value of each row within the sub-region where the sliding window is located are determined; when a pixel sum and value is greater than a set threshold, it is determined that the agricultural robot has reached the end of the crop row. When the position of the end of the crop row is within a set range, the robot decelerates and performs an exit action. Typically, when the crop row end position value returned by the EDR function is detected in a sub-region within the range of [2h, 3h], the robot will perform an exit action, where h is the height of the region of interest (ROI).

[0095] Based on the above embodiments, the method further includes: determining the current distance (EDR) of the agricultural robot from the end of the crop row.

[0096] I(x,Y) is the predicted value of the semantic segmentation model. Y represents the cumulative pixel value of the sub-block region; H represents the vertical coordinate of the sub-block region; and Y is the width of the sub-block region. After the first valid EDR value is detected, a mean filter is used to reduce noise in the line-end position EDR value.

[0097] Formula for adjusting the speed of agricultural robots:

[0098] v cmd The robot's target speed, i.e., the speed of the control commands, v max : Maximum permissible speed of the robot; EDR: Position of the agricultural robot from the end of the crop row; h: Maximum height of the sub-block area; δ: Current turning angular velocity of the robot, δ max : Maximum permissible angular velocity during turning; α: Turning deceleration coefficient, which determines the rate of speed reduction during turning; β: Controls the rate of distance decay, which determines the rate of speed reduction to the end of the crop row.

[0099] This invention discloses an agricultural robot control method, comprising: acquiring farmland environment images captured by an image acquisition device, as well as the basic attributes, driving speed, and driving direction of the agricultural robot; the basic attributes include: wheelbase and the spatial position of the image acquisition device on the agricultural robot; the image acquisition device acquires images at a fixed acquisition angle; inputting the farmland environment images into a pre-trained semantic segmentation model based on channel and spatial attention mechanisms, and outputting feature images of crop rows in the farmland environment images; wherein, the training dataset of the semantic segmentation model consists of original farmland environment images acquired under different environments; the original farmland environment images are labeled with crop rows and their positions; determining the position information of the crop row closest to the wheels of the agricultural robot based on the wheelbase and the spatial position; converting the position information to the robot coordinate system based on the spatial position; determining the steering angular velocity of the agricultural robot based on the position information converted to the robot coordinate system and the driving speed, and controlling the agricultural robot to control its heading angle according to the steering angular velocity. Using this method, by automatically identifying crop row clusters and employing tracking control technology, reliance on human labor can be reduced, significantly improving the operational efficiency of agricultural robots.

[0100] Figure 3 is a schematic diagram of the structure of an agricultural robot control device provided in an embodiment of the present invention. As shown in Figure 3, the device includes: an information acquisition module 210, a feature image output module 220, a position information module 230, a coordinate system transformation module 240, and a control steering module 250.

[0101] The information acquisition module 210 is used to acquire farmland environment images collected by the image acquisition device, as well as the basic attributes, driving speed, and driving direction of the agricultural robot; the basic attributes include: wheel track and the spatial position of the image acquisition device on the agricultural robot; the image acquisition device acquires images at a fixed acquisition angle;

[0102] The feature image output module 220 is used to input the farmland environment image into a pre-trained semantic segmentation model based on channel and spatial attention mechanisms, and output feature images of crop rows in the farmland environment image; wherein, the training dataset of the semantic segmentation model is the original farmland environment images collected under different environments; the original farmland environment images are labeled with the crop rows and their positions;

[0103] The location information module 230 is used to determine the location information of the crop row closest to the wheels of the agricultural robot based on the wheel track and the spatial position;

[0104] The coordinate system transformation module 240 is used to transform the position information to the robot coordinate system according to the spatial position;

[0105] The control steering module 250 is used to determine the steering angular velocity of the agricultural robot based on the position information converted to the robot coordinate system and the travel speed, and to control the agricultural robot to control the heading angle of the agricultural robot according to the steering angular velocity.

[0106] The technical solutions provided in this disclosure, through automatic crop row cluster identification and tracking control technology, can reduce reliance on human labor and significantly improve the operational efficiency of agricultural robots.

[0107] Furthermore, the feature image output module 220 can be used for:

[0108] The semantic segmentation model includes: an encoder-decoder module and an output layer;

[0109] The farmland environment image is input into the encoder-decoder module, which outputs edge feature images of crop rows in the farmland environment image and inputs these edge feature images into the output layer; the size of the edge feature images is the same as the size of the farmland environment image.

[0110] The feature value in the edge feature image is compared with a set threshold. When the feature value is greater than or equal to the set threshold, the pixel of the farmland environment image corresponding to the feature value is set to a first set value. When the feature value is less than the set threshold, the pixel of the farmland environment image corresponding to the feature value is set to a second set value, and the feature image is output.

[0111] Furthermore, the location information module 230 can be used for:

[0112] The wheel position information in the feature image is determined based on the wheel track and the spatial position.

[0113] A rectangular region is selected from the feature image as the region of interest according to preset coordinates; the preset coordinates are the coordinates of the four corners of the rectangular region.

[0114] Extract the function mapping relationship corresponding to the crop rows in the region of interest;

[0115] Determine the distance between each of the aforementioned function mapping relationships and the wheel position information;

[0116] The function mapping relationship with the minimum distance is used as the location information of the corresponding crop row.

[0117] Furthermore, the location determination module 240 can be used for:

[0118] Determine the coordinate transformation matrix according to the spatial location and the known transformation formula from the robot coordinate system to the camera coordinate system;

[0119] The result of multiplying the position information with the coordinate transformation matrix is ​​used as the position information in the robot coordinate system.

[0120] Furthermore, the steering control module 250 can be used for:

[0121] The agricultural robot also includes a positioning device; the center line of the two location information points is used as the navigation center line;

[0122] Obtain the current motion state information of the agricultural robot located by the positioning device;

[0123] Determine the angular deviation between the navigation centerline and the current motion state information, as well as the lateral deviation between the navigation centerline and the current motion state information;

[0124] The steering angular velocity is determined based on the angular deviation, the lateral deviation, and the driving speed.

[0125] Furthermore, the location determination module 240 can be used for:

[0126] The formula for determining the steering angular velocity based on the angular deviation, the lateral deviation, and the driving speed is as follows:

[0127] Where δ represents the steering angular velocity; L represents the wheelbase; ang e Indicates the angular deviation; dis e η represents the lateral deviation; γ represents the first gain coefficient of the controller; v represents the driving speed.

[0128] Furthermore, the device also includes:

[0129] The feature image is divided into multiple sub-block regions according to a sliding window of a set size; the sliding window processes the feature image sequentially from top to bottom;

[0130] Determine the pixels and values ​​of each row within the sub-block region;

[0131] When the sum of the pixels exceeds a set threshold, the position of the sum of the pixels is the position of the end of the crop row;

[0132] When the end of the crop row is within a set range, the agricultural robot will exit.

[0133] The above-described apparatus can execute the methods provided in all the foregoing embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the above methods. Technical details not described in detail in this embodiment can be found in the methods provided in all the foregoing embodiments of the present invention.

[0134] Figure 4 shows a schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0135] As shown in Figure 4, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0136] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0137] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as agricultural robot control methods.

[0138] In some embodiments, the agricultural robot control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the agricultural robot control method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the agricultural robot control method by any other suitable means (e.g., by means of firmware).

[0139] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0143] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0144] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0145] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0146] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An agricultural robot control method characterized by, include: Acquire images of the farmland environment captured by image acquisition equipment, as well as the basic attributes and driving speed of the agricultural robot; The basic attributes include: wheelbase and the spatial position of the image acquisition device on the agricultural robot; the image acquisition device acquires images at a fixed acquisition angle; The farmland environment image is input into a pre-trained semantic segmentation model based on channel and spatial attention mechanisms, and the model outputs feature images of crop rows in the farmland environment image; wherein, the training dataset of the semantic segmentation model consists of original farmland environment images collected under different environments; the original farmland environment images are labeled with the crop rows and their locations; The location information of the crop row closest to the wheels of the agricultural robot is determined based on the wheel track and the spatial position. The position information is converted to the robot coordinate system based on the spatial location. The turning angular velocity of the agricultural robot is determined based on the position information transformed into the robot coordinate system and the travel speed, and the agricultural robot is controlled to control its heading angle according to the turning angular velocity.

2. The method of claim 1, wherein, The semantic segmentation model includes an encoder-decoder module and an output layer; correspondingly, inputting the farmland environment image into a pre-trained semantic segmentation model based on channel and spatial attention mechanisms, and outputting feature images of crop rows in the farmland environment image, includes: The farmland environment image is input into the encoder-decoder module, which outputs edge feature images of crop rows in the farmland environment image and inputs these edge feature images into the output layer; the size of the edge feature images is the same as the size of the farmland environment image. The feature value in the edge feature image is compared with a set threshold. When the feature value is greater than or equal to the set threshold, the pixel of the farmland environment image corresponding to the feature value is set to a first set value. When the feature value is less than the set threshold, the pixel of the farmland environment image corresponding to the feature value is set to a second set value, and the feature image is output.

3. The method of claim 1, wherein, Determining the location information of the crop row closest to the wheels of the agricultural robot based on the wheel track and the spatial position includes: The wheel position information in the feature image is determined based on the wheel track and the spatial position. A rectangular region is selected from the feature image as the region of interest according to preset coordinates; the preset coordinates are the coordinates of the four corners of the rectangular region. Extract the function mapping relationship corresponding to the crop rows in the region of interest; Determine the distance between each of the aforementioned function mapping relationships and the wheel position information; The function mapping relationship with the minimum distance is used as the location information of the corresponding crop row.

4. The method of claim 1, wherein, The step of converting the position information to the robot coordinate system based on the spatial location includes: Determine the coordinate transformation matrix according to the spatial location and the known transformation formula from the robot coordinate system to the camera coordinate system; The result of multiplying the position information with the coordinate transformation matrix is ​​used as the position information in the robot coordinate system.

5. The method of claim 1, wherein, The agricultural robot also includes a positioning device, which determines the turning angular velocity of the agricultural robot based on the position information transformed into the robot coordinate system and the traveling speed, including: Use the centerline of the location information as the navigation centerline; Obtain the current motion state information of the agricultural robot located by the positioning device; Determine the angular deviation between the navigation centerline and the current motion state information, as well as the lateral deviation between the navigation centerline and the current motion state information; The steering angular velocity is determined based on the angular deviation, the lateral deviation, and the driving speed.

6. The method of claim 5, wherein, The calculation formula for determining the steering angular velocity according to the angular deviation, the lateral deviation and the running speed is as follows: wherein δ denotes the steering angular velocity; L denotes the wheel base; ang e denotes the angular deviation; dis e denotes the lateral deviation; η denotes a first gain coefficient of the controller; γ denotes a second gain coefficient of the controller; v denotes the running speed.

7. The method of claim 1, wherein, Also includes: The feature image is divided into multiple sub-block regions according to a sliding window of a set size; The sliding window processes the feature images sequentially from top to bottom; Determine the pixels and values ​​of each row within the sub-block region; When the sum of the pixels exceeds a set threshold, the position of the sum of the pixels is the position of the end of the crop row; When the end of the crop row is within a set range, the agricultural robot will exit.

8. An agricultural robot control device characterized by comprising: include: The information acquisition module is used to acquire images of the farmland environment collected by the image acquisition device, as well as the basic attributes, driving speed, and driving direction of the agricultural robot; The basic attributes include: wheelbase and the spatial position of the image acquisition device on the agricultural robot; the image acquisition device acquires images at a fixed acquisition angle; The feature image output module is used to input the farmland environment image into a pre-trained semantic segmentation model based on channel and spatial attention mechanisms, and output feature images of crop rows in the farmland environment image; wherein, the training dataset of the semantic segmentation model is the original farmland environment images collected under different environments; the original farmland environment images are labeled with the crop rows and their positions; The location information module is used to determine the location information of the crop row closest to the wheels of the agricultural robot based on the wheel track and the spatial position; A coordinate system transformation module is used to transform the position information to the robot coordinate system based on the spatial location; The steering control module is used to determine the steering angular velocity of the agricultural robot based on the position information transformed into the robot coordinate system and the travel speed, and to control the agricultural robot to control the heading angle of the agricultural robot according to the steering angular velocity.

9. An electronic device, comprising: The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the agricultural robot control method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the agricultural robot control method according to any one of claims 1-7.

Images