Intelligent weeding method, device and equipment based on visual recognition and motion control

Abstract

The application discloses a kind of intelligent weeding method, device and equipment based on visual identification and motion control.The method comprises: identifying the coordinates position of all weeds in a sub area to be weeded by visual identification algorithm;Control the mechanical hand to remove all weeds, and generate a removal completion signal after removal is completed;The removal completion signal is transmitted to the visual identification system, so that the visual identification system identifies the sub area to be weeded again, and generates a corresponding removal completion confirmation signal;Control the target weeding machine to move to the next sub area to be weeded, and return to the step of identifying the coordinates position of all weeds in a sub area to be weeded by visual identification algorithm until the removal of weeds in the total area to be weeded is completed.The embodiment of the present application solves the technical problems of low efficiency and high labor cost caused by manual weeding in the prior art, realizes simple and fast automatic weeding, improves weeding efficiency and reduces labor cost.

Description

Technical Field

[0001] This invention relates to the field of automatic control technology, and in particular to an intelligent weeding method, apparatus and equipment based on visual recognition and motion control. Background Technology

[0002] In recent years, the automatic control industry has developed rapidly, and many factories have already implemented or are in the process of implementing automated control and automated production. Under this environment, agricultural production is also developing towards mechanization and automation to eliminate human labor. Currently, there are already general-purpose large-scale agricultural automation equipment such as combine harvesters and automatic seeders. However, there is still no corresponding automation equipment to replace human labor for some other types of agricultural products, such as the weeding process in the planting of Chinese medicinal herbs.

[0003] Unlike ordinary agricultural products, Chinese medicinal herbs require constant weeding during cultivation to ensure their normal growth. Currently, this weeding work can only be done manually, day after day, which is tiring and inefficient. Summary of the Invention

[0004] This invention provides an intelligent weeding method, device, and equipment based on visual recognition and motion control to solve the technical problems of low efficiency and high labor costs caused by manual weeding in the prior art. It realizes simple and quick automated weeding, improves weeding efficiency, and reduces labor costs.

[0005] According to one aspect of the present invention, an intelligent weeding method based on visual recognition and motion control is provided, comprising:

[0006] The coordinates of all weeds in a sub-area to be weeded within the total actual coverage area corresponding to the target weeding machine are identified by a visual recognition algorithm; wherein, the sub-area to be weeded is the area covered by the target weeding machine in a single operation.

[0007] The robotic arm is controlled to remove all the weeds corresponding to the coordinate positions of the weeds, and a corresponding removal completion signal is generated after the removal is completed;

[0008] The clearing completion signal is transmitted to the visual recognition system, so that the visual recognition system can perform secondary recognition on the sub-area to be cleared and generate a corresponding clearing completion confirmation signal;

[0009] In response to the confirmation signal that the weeding is completed, the target weeding machine is controlled to move to the next sub-area to be weeded, and the process returns to the step of identifying the coordinates of all weeds in one of the sub-areas to be weeded within the total area to be covered by the target weeding machine through a visual recognition algorithm, until the weeds in the total area to be weeded are cleared.

[0010] According to another aspect of the present invention, an intelligent weeding device based on visual recognition and motion control is provided, comprising:

[0011] The first identification module is used to identify the coordinates of all weeds in one of the sub-areas to be weeded within the total actual coverage area corresponding to the target weeder through a visual recognition algorithm.

[0012] The control module is used to control the robotic arm to remove the weeds corresponding to all the weed coordinate positions, and to generate a corresponding removal completion signal after the removal is completed;

[0013] The second identification module is used to transmit the clearing completion signal to the visual recognition system, so that the visual recognition system can perform secondary identification on the sub-area to be cleared and generate a corresponding clearing completion confirmation signal.

[0014] The execution module is used to respond to the clearing completion confirmation signal, control the target weeder to move to the next sub-area to be cleared, and return to the step of identifying the coordinate positions of all weeds in one of the sub-areas to be cleared in the actual total area through a visual recognition algorithm, until the clearing of weeds in the total area to be cleared is completed.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the intelligent weeding method based on visual recognition and motion control as described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the intelligent weeding method based on visual recognition and motion control as described in any embodiment of the present invention.

[0020] The technical solution of this invention identifies the coordinates of all weeds in a sub-area to be weeded within the total coverage area using a visual recognition algorithm. Based on these coordinates, a robotic arm is controlled to remove the weeds from the sub-area. After removing the weeds from the sub-area, a completion signal is transmitted to the visual recognition system. This system then performs secondary recognition of the sub-area, generates a corresponding completion confirmation signal, and controls the target weeder to move to the next sub-area to be weeded, continuing the weed removal process until all weeds in the total area to be weeded are removed. This solves the problems of low efficiency and high labor costs associated with manual weeding in existing technologies, achieving simple and quick automated weeding, improving weeding efficiency, and reducing labor costs.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating an intelligent weeding method based on visual recognition and motion control, provided as an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of the structure of a weeding machine provided in an embodiment of the present invention;

[0025] Figure 3 A flowchart of another intelligent weeding method based on visual recognition and motion control provided in an embodiment of the present invention;

[0026] Figure 4 A flowchart illustrating yet another intelligent weeding method based on visual recognition and motion control provided in an embodiment of the present invention;

[0027] Figure 5 This is a schematic diagram showing the actual total coverage area provided in an embodiment of the present invention;

[0028] Figure 6 This is a schematic diagram showing another actual total coverage area provided by an embodiment of the present invention;

[0029] Figure 7This is another schematic diagram showing the actual total coverage area provided by an embodiment of the present invention;

[0030] Figure 8 A schematic diagram of the structure of an intelligent weeding device based on visual recognition and motion control provided in an embodiment of the present invention;

[0031] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "original," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] In one embodiment, Figure 1 This is a flowchart illustrating an intelligent weeding method based on visual recognition and motion control, provided as an embodiment of the present invention. This embodiment is applicable to situations requiring automatic weeding. The method can be executed by an intelligent weeding device based on visual recognition and motion control. This intelligent weeding device can be implemented in hardware and / or software, and can be configured within the electronic equipment of a weeding machine. In this embodiment, Figure 2 This is a schematic diagram of the structure of a weeding machine provided in an embodiment of the present invention, as shown below. Figure 2As shown, the lawnmower in this embodiment includes: a controller 210, a robotic arm 220, a camera 230, and a drone 240. The camera 230 is used to photograph the weeding area corresponding to the current location of the lawnmower; the robotic arm 220 is used to weed the sub-area according to control commands; the drone 240 is used to capture images of the total weeding area according to control commands; and the controller 210 is used to send control commands to the camera 230, robotic arm 220, and drone 240. The camera 230 corresponds to the visual recognition system. It can be understood that the camera 230 is a hardware device, while the visual recognition system is a software system.

[0035] like Figure 1 As shown, the method includes:

[0036] S110. Identify the coordinates of all weeds in one of the sub-areas to be weeded within the total actual coverage area corresponding to the target weeding machine using a visual recognition algorithm.

[0037] The sub-area to be weeded refers to the area covered by the target weeding machine in a single operation. In this embodiment, the total area to be weeded can be understood as the area that actually needs to be weeded; the total actual coverage area refers to the area that the target weeding machine needs to cover during the process of clearing weeds from the total area to be weeded. This can be understood as the total actual coverage area at least including the total area to be weeded, meaning the area occupied by the total actual coverage area is greater than or equal to the area occupied by the total area to be weeded. In other words, the total actual coverage area is designed to ensure that the target weeding machine can achieve complete coverage of the total area to be weeded.

[0038] In this context, the sub-area to be weeded refers to the area covered by the target weeding machine in a single operation. This can be understood as follows: the length of the robotic arm in the target weeding machine is finite. During actual weeding operations, the area that the robotic arm can reach without the target weeding machine moving is considered a sub-area to be weeded. In this embodiment, the ratio of the area occupied by the total area actually covered to the area occupied by a sub-area to be weeded is a positive integer. For example, assuming the area covered by the target weeding machine in a single operation is 2m * 2m, then the area occupied by the sub-area to be weeded is 4m². 2 It should be noted that the ratio between the actual total coverage area and the area of ​​the sub-area to be weeded is a positive integer. That is, the actual total coverage area includes multiple sub-areas to be weeded. The first sub-area to be weeded is determined by the target weeding machine, and the target weeding machine is controlled to move to the position corresponding to the first sub-area to be weeded, so that the coordinates of all weeds in the first sub-area to be weeded can be identified by the visual recognition algorithm.

[0039] In this embodiment, the camera in the target weeder can take a picture of the first sub-area to be weeded, and the plant areas and coordinates of each plant in the sub-area to be weeded can be identified by a visual recognition algorithm. Then, the plant areas can be filtered by pre-stored medicinal herb features to obtain the coordinates of the weeds in the sub-area to be weeded.

[0040] S120. Control the robotic arm to remove all weeds corresponding to their coordinate positions, and generate a corresponding removal completion signal after the removal is completed.

[0041] The completion signal indicates that the robotic arm has finished clearing all weeds in the designated sub-area. In this embodiment, after identifying the coordinates of all weeds in one sub-area to be cleared within the total coverage area, the controller sends a control command to the robotic arm in the target weeding machine. The robotic arm is then guided to move to the location of the weeds and clear them until all weeds in the sub-area are cleared, generating the corresponding completion signal.

[0042] S130. The clearing completion signal is transmitted to the visual recognition system so that the visual recognition system can perform secondary recognition of the sub-area to be cleared and generate a corresponding clearing completion confirmation signal.

[0043] After receiving the clearing completion signal from the robotic arm, the clearing completion signal is sent to the vision recognition system. In response to the clearing completion signal, the vision recognition system performs a secondary recognition of the area to be cleared to confirm the accuracy of the clearing completion signal. If it confirms that the clearing is complete, it generates a corresponding clearing completion confirmation signal.

[0044] S140. In response to the confirmation signal that the weeding is completed, control the target weeder to move to the next sub-area to be weeded, and return to the step of identifying the coordinate position of the weeds in one of the sub-areas to be weeded in the total area to be covered by the target weeder through a visual recognition algorithm, until the weeds in the total area to be weeded are cleared.

[0045] In this embodiment, after the target weeder completes the task of clearing all weeds in the first sub-area to be cleared, it moves to the second sub-area to be cleared. The coordinates of all weeds in the second sub-area to be cleared are identified by a visual recognition algorithm, and the robotic arm is controlled to perform the weed clearing operation. This process is repeated until all weeds in the total area to be cleared are cleared.

[0046] The technical solution of this embodiment uses a visual recognition algorithm to identify the coordinates of all weeds in one sub-area to be cleared within the total coverage area. Based on the coordinates of all weeds, the robot arm is controlled to clear the weeds in the sub-area. After clearing the weeds in the sub-area, a clearing completion signal is transmitted to the visual recognition system, which then performs secondary recognition of the sub-area, generates a corresponding clearing completion confirmation signal, and controls the target weeder to move to the next sub-area to be cleared and clear the weeds until the weeds in the total area to be cleared are cleared. This solves the technical problems of low efficiency and high labor costs caused by manual weeding in the prior art, and achieves simple and quick automated weeding, improving weeding efficiency and reducing labor costs.

[0047] In one embodiment, Figure 3 This is a flowchart illustrating another intelligent weeding method based on visual recognition and motion control provided by an embodiment of the present invention. This embodiment, based on the above embodiments, describes the process of determining the coordinates and shape type of each vertex in the total area to be weeded. Figure 3 As shown, the intelligent weeding method based on visual recognition and motion control in this embodiment includes the following steps:

[0048] S310. Determine the camera shooting height based on the pre-obtained total area, area shape, and camera shooting angle, so that the camera can shoot the total area to be weeded according to the camera shooting height and camera shooting angle, and obtain the corresponding shooting image.

[0049] In actual operation, the automatic lawnmower can begin mowing after receiving a power-on command. The power-on command is the instruction used to trigger the target lawnmower to perform power-on and initialization operations. It can be understood as the power-on command activating the target lawnmower itself, initializing the controller within the target lawnmower, and resetting the robotic arm within the target lawnmower. The robotic arm reset operation refers to restoring the robotic arm to its initial position within the target lawnmower. For example, assuming the initial position of the robotic arm is the lower left corner of the target lawnmower, and the current position of the robotic arm is in the center of the target lawnmower, then upon receiving the power-on command, the robotic arm will automatically move from the center position to the lower left corner position. Of course, the initial position of the robotic arm can be set according to actual conditions and is not limited thereto.

[0050] It should be noted that the shape of the total area to be weeded only represents the number of sides corresponding to the total area to be weeded, and does not accurately represent the specific outline of the total area to be weeded. In the embodiment, the shape of the area is related to the vertex coordinates of the total area to be weeded. For example, when there are three vertex coordinates, the shape of the area to be weeded is a triangle; when there are four vertex coordinates, the shape is a quadrilateral; when there are five vertex coordinates, the shape is a pentagon, and so on. In the embodiment, the total area refers to the size of the area occupied by the total area to be weeded.

[0051] In this embodiment, the maximum diagonal length can be determined based on the pre-obtained total area and shape of the region; the camera shooting height can be determined based on the maximum diagonal length and the camera shooting angle; a shooting command is sent to the drone so that the camera in the drone can shoot the total area to be weeded according to the camera shooting height and camera shooting angle to obtain the corresponding image; wherein, the shooting command carries the camera shooting height and camera shooting angle. This can be understood as follows: the maximum diagonal length is used to determine the maximum length that the camera in the drone of the target weeder needs to focus on; the camera shooting height is adjusted according to the maximum diagonal length and camera shooting angle so that the camera covers the entire total area to be weeded as much as possible; then the controller sends a shooting command carrying the camera shooting height and camera shooting angle to the drone so that when the drone receives the shooting command, it adjusts its target flight attitude and flight destination position according to the camera shooting height and camera shooting angle, and after reaching the flight destination position, it shoots the total area to be weeded to obtain the corresponding image.

[0052] S320. Determine the completeness of the total area to be weeded based on the area outline and shape in the captured image.

[0053] The region outline refers to the outline of the total area to be weeded in the captured image. It can be understood as a specific representation of the shape and structure of the total area to be weeded. For example, the region outline can be rectangular, trapezoidal, or triangular, etc. Capture completeness is used to characterize whether the captured image completely contains the total area to be weeded. In this embodiment, comparing the region outline and the region shape determines whether the captured image completely contains the total area to be weeded.

[0054] S330: When the shooting integrity is set to "regional shooting integrity", the aspect ratio of the captured image is determined based on the camera shooting height and camera shooting angle.

[0055] In this embodiment, when the image capture completeness is defined as "regional capture completeness," meaning the captured image fully encompasses the total area to be weeded, the ratio between the captured image and the actual geographical location can be determined based on the camera's shooting height and angle. For example, assuming an image ratio of 1:10, the size of the total area to be weeded in the captured image would be 45*45, or 450*450.

[0056] S340. Determine the coordinates and shape type of each vertex in the total area to be weeded based on the scale of the captured image and the area outline.

[0057] In this embodiment, to facilitate the determination of the coordinates of each vertex, a coordinate system can be constructed using a point in the captured image as the origin to determine the coordinates of each vertex in the total area to be weeded in the captured image. For example, a coordinate system can be constructed using a vertex of the total area to be weeded in the captured image as the origin; then, based on the position of the total area to be weeded in the captured image and the image scale, the actual geographical location of each vertex in the total area to be weeded can be determined, and the shape type of the total area to be weeded can be accurately determined based on the area outline and the coordinates of each vertex. Alternatively, a coordinate system can be directly established, and the captured image corresponding to the total area to be weeded can be placed directly in the coordinate system to determine the coordinates of each vertex corresponding to the total area to be weeded. Of course, this is not limited and can be set according to the actual situation.

[0058] S350. Determine the actual total coverage area based on the pre-determined vertex coordinates and shape type of the total area to be weeded, as well as the area occupied by the sub-areas to be weeded.

[0059] It should be noted that a region must have at least three vertex positions to be constructed; that is, the area to be weeded must contain at least three vertex coordinates. In practice, the shape type of the total area to be weeded is related to the coordinates of each vertex within that area. For example, the shape type of the total area to be weeded can include: regular shapes and irregular shapes. Regular shapes refer to shapes that do not require completion operations on the total area to be weeded; irregular shapes refer to shapes that require completion operations on the total area to be weeded.

[0060] After receiving the power-on command, the target weeder starts its main body, initializes the controller, and resets the robotic arm. Then, it acquires the vertex coordinates and shape type of the total area to be weeded, as well as the area occupied by a sub-area. Based on the vertex coordinates and shape type of the total area, it determines whether a completion operation is needed. If completion is required, it is performed to obtain a regularly shaped area. Then, it checks if the ratio between the area of ​​the completed regularly shaped area and the area occupied by a sub-area is an integer. If not, the completed regularly shaped area is further completed to completely divide the area occupied by a sub-area, and this area is taken as the actual total coverage area.

[0061] S360: Identify the coordinates of all weeds in one of the sub-areas to be weeded within the total actual coverage area corresponding to the target weeder using a visual recognition algorithm.

[0062] S370: Control the robotic arm to remove all weeds corresponding to their coordinate positions, and generate a corresponding removal completion signal after the removal is completed.

[0063] S380. The clearing completion signal is transmitted to the visual recognition system so that the visual recognition system can perform secondary recognition of the sub-area to be cleared and generate the corresponding clearing completion confirmation signal.

[0064] S390, In response to the confirmation signal that the weeding is completed, control the target weeder to move to the next sub-area to be weeded, and return to the step of identifying the coordinate position of the weeds in one of the sub-areas to be weeded in the total area to be covered by the target weeder through a visual recognition algorithm, until the weeds in the total area to be weeded are cleared.

[0065] The technical solution of this embodiment, based on the above embodiment, sends a shooting command carrying the camera shooting height and camera shooting angle to the drone in the target weeding machine, so that the camera in the drone can shoot the total area to be weeded, obtain the corresponding shooting image, and determine the position coordinates and shape type of each vertex in the total area to be weeded according to the ratio and area outline of the shooting image, ensuring the accuracy of obtaining the shape type and vertex position coordinates of the total area to be weeded, thereby ensuring the effectiveness of subsequent control of the target weeding machine to automatically weed the sub-areas.

[0066] In one embodiment, Figure 4This is a flowchart illustrating another intelligent weeding method based on visual recognition and motion control provided by an embodiment of the present invention. This embodiment, based on the above embodiments, further details the process of determining the total actual coverage area, the process of identifying the coordinates of all weeds within the total actual coverage area, and the process of automatically removing weeds within the total actual coverage area using a target weeding machine. Figure 4 As shown, the method includes:

[0067] S410. Determine the actual total coverage area based on the pre-determined vertex coordinates and shape type of the total area to be weeded, as well as the area occupied by the sub-areas to be weeded.

[0068] In one embodiment, when the shape of the total area to be weeded is irregular, step S410 includes: identifying and extracting the coordinates of the nearest and farthest points of the total area to be weeded; determining the target total coverage area based on the coordinates of the nearest and farthest points; and determining the actual total coverage area based on the target total coverage area and the area occupied by the sub-areas to be weeded. The nearest point coordinates refer to the coordinates of the point closest to the origin in the coordinate system; the farthest point coordinates refer to the coordinates of the point farthest from the origin in the coordinate system. Figure 5 This is a schematic diagram showing the actual total coverage area provided in an embodiment of the present invention. (See diagram below.) Figure 5 As shown, a coordinate system is established, and the captured image corresponding to the total area to be weeded is placed in this coordinate system; then, the coordinates of each vertex corresponding to the total area to be weeded in the captured image are extracted; based on the coordinates of each vertex, the coordinates of the nearest point and the farthest point are obtained with reference to the origin. For example, as... Figure 5 As shown, the coordinates of the bottom left vertex of the area to be weeded (i.e., the area composed of line 1) are (1, 1), the coordinates of the top left vertex are (1.5, 4.5), the coordinates of the bottom right vertex are (8, 1), and the coordinates of the top right vertex are (7.8, 5). Based on these four points, we can determine the coordinates of the nearest point (1, 1) and the coordinates of the farthest point (8, 5). By drawing a rectangle based on the coordinates of the nearest and farthest points, we can complete the weeding process, resulting in the area shown in the image. Figure 5 The target coverage area is shown (i.e., the area composed of line 2); then, coverage pathfinding is performed based on the target coverage area, and it is determined whether to complete the target coverage area again according to the ratio between the occupied area of ​​the target coverage area and the occupied area of ​​a sub-area to be weeded (i.e., an area composed of line 3). If the ratio between the occupied area of ​​the target coverage area and the occupied area of ​​the sub-area to be weeded is a positive integer, then the target coverage area is the actual coverage area, so as to complete the complete coverage of the entire area to be weeded.

[0069] Figure 6This is a schematic diagram showing another actual total coverage area provided in an embodiment of the present invention. Figure 6 Is Figure 5 Based on this, the relationship between the target total coverage area and the actual total coverage area is further explained. Figure 5 Based on the above, if the ratio between the area occupied by the target total coverage region and the area occupied by the sub-region to be weeded is not a positive integer, then the target total coverage region is supplemented again to obtain the actual total coverage region. For example... Figure 6 As shown, each time the target weeder is covered Figure 6 The target area is the region corresponding to a sub-region to be weeded, and then the target coverage area is continuously advanced to cover the entire area to be weeded. For example, the target weeding machine can travel in the y-axis direction and advance in the x-axis direction. First, determine the ratio between the y-axis length of the target coverage area and the y-axis length of the sub-region to be weeded to determine the number of times to travel in the y-axis direction each time. That is, if the ratio between the y-axis length of the target coverage area and the y-axis length of the sub-region to be weeded is a positive integer, then this ratio is the number of travels; if this ratio is not a positive integer, then the value of the ratio plus one is the number of travels. Figure 5 As shown, if the ratio between the y-axis length of the total target coverage area and the y-axis length of the sub-area to be weeded is not a positive integer, then the target weeder will move 6 times along the y-axis. After completing one y-axis coverage, it continues to advance along the set x-axis direction, for example, as... Figure 6 The arrows indicate the direction to complete the coverage of the entire area to be weeded.

[0070] Generally, the total area to be weeded is quadrilateral or a small part is triangular or pentagonal. The above embodiment can be used to develop a pathfinding algorithm for the area to be weeded that is mostly quadrilateral.

[0071] In one embodiment, when the shape of the total area to be weeded is irregular, step S410 includes: identifying and extracting the coordinates of at least one key point on at least one target edge in the total area to be weeded; and determining the actual total coverage area based on the key point coordinates, the vertex coordinates of the total area to be weeded, and the area occupied by the sub-area to be weeded. Here, the key point coordinates refer to the coordinates of the intersection point between a sub-area to be weeded and one of the edges in the total area to be weeded. Figure 7 This is a schematic diagram illustrating another actual total coverage area provided by an embodiment of the present invention. (See diagram below.) Figure 7As shown, the total area to be weeded is triangular in shape. The intersection of the first sub-area to be weeded in each row and one of the edges of the total area to be weeded is used as the key point. Assuming the target weed cutter starts at point A, then points A, B, C, and D are all key points. The coordinates of each key point are obtained, and an actual total coverage area is formed based on the key point coordinates, vertex position coordinates, and the area occupied by the sub-area to be weeded. For example... Figure 7 As shown, the total actual coverage area consists of 14 sub-regions to be weeded, in order to achieve complete coverage of the total area to be weeded.

[0072] It should be noted that when the total area to be weeded is triangular, directly using... Figure 7 The method shown for determining the actual total coverage area is not the same as the method used. Figure 5 The method shown for determining the actual total coverage area ensures complete coverage of the total area to be weeded while avoiding the movement and coverage of useless areas by the target weeding machine.

[0073] S420. Obtain the original plant image of one of the sub-areas to be weeded within the total actual coverage area corresponding to the target weeding machine.

[0074] Here, the original plant images refer to the original images of all plants contained within a specific sub-region to be weeded. It's important to note that after the target weeder moves and fixes itself to a specific sub-region, identifying all plants within that sub-region does not require identifying plant species; it only requires distinguishing between plant areas and non-plant areas. This can be understood as follows: the target weeder's camera captures images of the sub-region to be weeded, and a simple grayscale contrast analysis algorithm and a classic Bayesian matting algorithm are used to distinguish between plant and non-plant areas. The images corresponding to the non-plant areas are then segmented and filtered to obtain images of all plants within the sub-region to be weeded.

[0075] S430. A visual recognition algorithm is used to process the original plant image to identify the coordinates of all plants in the area to be weeded.

[0076] In one embodiment, S430 includes: performing distortion correction and normalization planar processing on the original plant image within the target weeding sub-region using a visual recognition algorithm to obtain the corresponding target plant image; and analyzing the target plant image using a pixel coordinate scaling algorithm to determine the coordinate positions of all plants. In this embodiment, after segmenting the original plant image into individual plants, a visual recognition algorithm is used to perform distortion correction and normalization planar processing on the original plant image, and the coordinate positions of the plants in the actual geographic location are determined by the coordinates of the plants in the captured image and the scale of the captured image.

[0077] In this embodiment, to achieve accurate plant identification, a more accurate identification algorithm can be used. Traditional plant identification algorithms require manual extraction and recording of plant root, stem, and leaf features, which is labor-intensive and results in low accuracy due to limited reference features. In recent years, convolutional neural network deep learning algorithms have developed rapidly, which can automatically extract image features without manual intervention, overcoming the shortcomings of traditional plant leaf identification that relies on manual feature extraction.

[0078] It should be noted that the basic functional requirements of each part of the visual recognition algorithm are relatively simple. If existing open source libraries are used, most of the device performance will be wasted and the recognition time will be increased. Therefore, it is advisable to choose to develop a dedicated algorithm.

[0079] Furthermore, for situations requiring high accuracy in recognition, if the time and manpower costs of developing a separate recognition algorithm are too high, mainstream visual recognition libraries can be used directly. The advantages and disadvantages of mainstream visual recognition libraries can be collected, compared, and the best one can be selected.

[0080] S440. Filter all plant coordinates according to the pre-stored medicinal characteristics to obtain the coordinates of all weeds.

[0081] In this embodiment, after determining the coordinates of all plants, the coordinates and feature data of all plants are stored in a target cache location for later retrieval. Then, all medicinal herb features in the target cache location are compared with all plants in the sub-area to be weeded, and the coordinates of plants that match the medicinal herb features are deleted, thus obtaining the coordinates of all weeds in the sub-area to be weeded.

[0082] S450. Determine the optimal movement path of the robot arm based on the coordinates of all weeds and the coordinates of the first weed.

[0083] The first weed coordinate position refers to the coordinate position of the first weed that needs to be removed within the sub-area to be cleared. In this embodiment, the robot's trajectory is planned based on the coordinate positions of all weeds to determine the optimal movement route of the robot within the sub-area to be cleared. The optimal movement route can be understood as the shortest distance required to clear all weeds within the sub-area to be cleared.

[0084] S460: Control the robotic arm to remove weeds in the area to be weeded according to the optimal motion path.

[0085] In this embodiment, the robotic arm in the target weeding machine is driven according to the optimal motion route to remove weeds in the sub-area to be weeded in turn until all weeds in the sub-area to be weeded are removed.

[0086] S470. The clearing completion signal is transmitted to the visual recognition system so that the visual recognition system can perform secondary recognition of the sub-area to be cleared and generate a corresponding clearing completion confirmation signal.

[0087] S480, in response to the confirmation signal that the weeding is completed, control the target weeder to move to the next sub-area to be weeded, and return to the step of identifying the coordinate position of the weeds in one of the sub-areas to be weeded in the total area to be covered by the target weeder through a visual recognition algorithm, until the weeds in the total area to be weeded are cleared.

[0088] The technical solution of this embodiment, based on the above embodiments, determines the actual total coverage area according to the shape of the total area to be weeded when the total area to be weeded is irregular in shape, and uses this method to determine the corresponding actual total coverage area. This achieves the selection of the most suitable method for determining the actual total coverage area based on the shape and shape type of the total area to be weeded, avoiding the target weeding machine from driving in useless areas, improving the weeding efficiency of the target weeding machine, and indirectly extending the service life of the target weeding machine.

[0089] In one embodiment, Figure 8 This is a schematic diagram of the structure of an intelligent weeding device based on visual recognition and motion control, provided as an embodiment of the present invention. Figure 8 As shown, the device includes: a first identification module 810, a control module 820, a second identification module 830, and an execution module 840.

[0090] The first identification module 810 is used to identify the coordinates of all weeds in one of the sub-areas to be weeded within the total actual coverage area corresponding to the target weeding machine through a visual recognition algorithm.

[0091] The control module 820 is used to control the robotic arm to remove the weeds corresponding to all the weed coordinate positions, and to generate a corresponding removal completion signal after the removal is completed;

[0092] The second identification module 830 is used to transmit the clearing completion signal to the visual identification system, so that the visual identification system can perform secondary identification on the sub-area to be cleared and generate a corresponding clearing completion confirmation signal.

[0093] The execution module 840 is used to respond to the clearing completion confirmation signal, control the target weeder to move to the next sub-area to be cleared, and return to the step of identifying the coordinate positions of all weeds in one of the sub-areas to be cleared in the actual total area through a visual recognition algorithm, until the clearing of weeds in the total area to be cleared is completed.

[0094] In one embodiment, before identifying the coordinates of all weeds in one sub-area to be weeded within the total actual coverage area corresponding to the target weeder using a visual recognition algorithm, the intelligent weeding device based on visual recognition and motion control further includes:

[0095] The first determining module is used to determine the actual total coverage area based on the pre-determined vertex position coordinates and shape type of the total area to be weeded, as well as the area occupied by the sub-areas to be weeded.

[0096] In one embodiment, before determining the actual total coverage area based on the vertex coordinates and shape type of the total area to be weeded and the area occupied by the sub-areas to be weeded, the intelligent weeding device based on visual recognition and motion control further includes:

[0097] The second determining module is used to determine the camera shooting height based on the pre-acquired total area, area shape and camera shooting angle, so that the camera can shoot the total area to be weeded according to the camera shooting height and camera shooting angle to obtain the corresponding shooting image;

[0098] The third determining module is used to determine the completeness of the total area to be weeded based on the area outline and shape in the captured image.

[0099] The fourth determining module is used to determine the proportion of the captured image based on the camera's shooting height and shooting angle when the shooting integrity is that the area is fully captured.

[0100] The fifth determination module is used to determine the coordinates and shape type of each vertex in the total area to be weeded based on the scale of the captured image and the area outline.

[0101] In one embodiment, when the shape of the total area to be weeded is irregular, the first determining module includes:

[0102] The first identification and extraction unit is used to identify and extract the coordinates of the nearest and farthest points of the total area to be weeded.

[0103] The first determining unit is used to determine the total target coverage area based on the coordinates of the nearest point and the farthest point.

[0104] The second determining unit is used to determine the actual total coverage area based on the total target coverage area and the area occupied by the sub-areas to be weeded.

[0105] In one embodiment, when the shape of the total area to be weeded is irregular, the first determining module includes:

[0106] The second identification and extraction unit is used to identify and extract the coordinates of at least one key point on at least one target edge in the total area to be weeded.

[0107] The three-determining unit is used to determine the actual total coverage area based on the coordinates of key points, the coordinates of the vertex positions of the total area to be weeded, and the area occupied by the sub-areas to be weeded.

[0108] In one embodiment, the first identification module 810 includes:

[0109] The acquisition unit is used to acquire the original plant image of one of the sub-areas to be weeded within the total actual coverage area corresponding to the target weeding machine;

[0110] The identification unit is used to process the original plant image using a visual recognition algorithm to identify the coordinate positions of all plants in the sub-region to be weeded.

[0111] The screening unit is used to screen all plant coordinates according to pre-stored medicinal herb characteristics to obtain the coordinates of all weeds.

[0112] In one embodiment, the identification unit includes:

[0113] The processing subunit is used to perform distortion removal and normalization plane processing on the original plant image in the sub-region to be weeded using a visual recognition algorithm to obtain the corresponding target plant image;

[0114] The analysis subunit is used to analyze the target plant image using a pixel coordinate scaling algorithm to determine the coordinate positions of all plants.

[0115] In one embodiment, the control module 820 includes:

[0116] The fourth determining unit is used to determine the optimal movement path of the robot arm based on the coordinate positions of all weeds and the coordinate position of the first weed.

[0117] The control unit is used to control the robotic arm to remove weeds in the designated area according to the optimal motion path.

[0118] The intelligent weeding device based on visual recognition and motion control provided in the embodiments of the present invention can execute the intelligent weeding method based on visual recognition and motion control provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0119] In one embodiment, Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 9The diagram illustrates a schematic representation of an electronic device 10 that can be used to implement embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0120] like Figure 9 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0121] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0122] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as intelligent weeding methods based on visual recognition and motion control.

[0123] In some embodiments, the intelligent weeding method based on vision recognition and motion control can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the intelligent weeding method based on vision recognition and motion control described above can be performed. Alternatively, in other embodiments, the processor 11 may be configured by any other suitable means (e.g., by means of firmware) to perform a smart weeding method based on vision recognition and motion control, including: identifying the coordinate positions of all weeds in a sub-area to be weeded within the total area actually covered by the target weeding machine using a vision recognition algorithm; wherein the sub-area to be weeded is the area covered by the target weeding machine in a single operation; controlling a robotic arm to remove the weeds corresponding to all weed coordinate positions, and generating a corresponding removal completion signal after removal is completed; transmitting the removal completion signal to a vision recognition system so that the vision recognition system performs secondary recognition of the sub-area to be weeded and generates a corresponding removal completion confirmation signal; in response to the removal completion confirmation signal, controlling the target weeding machine to move to the next sub-area to be weeded, and returning to the step of identifying the coordinate positions of all weeds in a sub-area to be weeded within the total area actually covered by the target weeding machine using a vision recognition algorithm, until the removal of weeds in the total area to be weeded is completed.

[0124] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0126] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0128] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0129] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0130] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0131] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A smart weeding method based on visual recognition and motion control, characterized in that, include: The actual total coverage area is determined based on the pre-determined vertex coordinates and shape type of the total area to be weeded, as well as the area occupied by the sub-areas to be weeded. The coordinates of all weeds in a sub-area to be weeded within the total actual coverage area corresponding to the target weeding machine are identified by a visual recognition algorithm; wherein, the sub-area to be weeded is the area covered by the target weeding machine in a single operation. The robotic arm is controlled to remove all the weeds corresponding to the coordinate positions of the weeds, and a corresponding removal completion signal is generated after the removal is completed; The clearing completion signal is transmitted to the visual recognition system, so that the visual recognition system can perform secondary recognition on the sub-area to be cleared and generate a corresponding clearing completion confirmation signal; In response to the confirmation signal that the weeding is completed, the target weeding machine is controlled to move to the next sub-area to be weeded, and the process returns to the step of identifying the coordinates of all weeds in one of the sub-areas to be weeded within the total area actually covered by the target weeding machine through a visual recognition algorithm, until the weeds in the total area to be weeded are cleared. Where the shape of the total area to be weeded is irregular, the step of determining the actual total coverage area based on the pre-determined vertex coordinates and shape type of the total area to be weeded, and the area occupied by the sub-areas to be weeded, includes: Identify and extract the coordinates of at least one key point on at least one target edge in the total area to be weeded; The actual total coverage area is determined based on the coordinates of the key points, the coordinates of the vertices of the total area to be weeded, and the area occupied by the sub-areas to be weeded. Here, the key point coordinates refer to the coordinates of the intersection point between a sub-region to be weeded and one of the edges in the total area to be weeded; the ratio between the actual total area occupied and the area occupied by the sub-region to be weeded is a positive integer.

2. The method according to claim 1, characterized in that, Before determining the actual total coverage area based on the predetermined vertex coordinates and shape type of the total area to be weeded, and the area occupied by the sub-areas to be weeded, the method further includes: The camera shooting height is determined based on the pre-obtained total area, area shape, and camera shooting angle, so that the camera can take pictures of the total area to be weeded according to the camera shooting height and camera shooting angle, and obtain the corresponding captured image; The completeness of the image of the total area to be weeded is determined based on the area outline and the area shape in the captured image. When the shooting integrity is that the area is fully captured, the proportion of the captured image is determined based on the camera shooting height and the camera shooting angle; The coordinates and shape type of each vertex in the total area to be weeded are determined based on the scale of the captured image and the outline of the area.

3. The method according to claim 2, characterized in that, When the shape of the total area to be weeded is irregular, determining the actual total coverage area based on the predetermined vertex coordinates and shape of the total area to be weeded, and the area occupied by the sub-areas to be weeded, includes: Identify and extract the coordinates of the nearest and farthest points in the total area to be weeded; The total target coverage area is determined based on the coordinates of the nearest point and the coordinates of the farthest point. The actual total coverage area is determined based on the area occupied by the target total coverage area and the area of ​​the sub-area to be weeded.

4. The method according to claim 1, characterized in that, The step of identifying the coordinates of all weeds in a sub-area to be weeded within the total actual coverage area corresponding to the target weeder using a visual recognition algorithm includes: Acquire the original plant image of one of the sub-areas to be weeded within the total actual coverage area corresponding to the target weeder; The original plant image is processed using a visual recognition algorithm to identify the coordinates of all plants within the area to be weeded. The coordinates of all plants are filtered according to the pre-stored medicinal characteristics to obtain the coordinates of all weeds.

5. The method according to claim 4, characterized in that, The step of processing the original plant image using a visual recognition algorithm to identify the coordinates of all plants within the area to be weeded includes: A visual recognition algorithm is used to perform distortion removal and normalization plane processing on the original plant image in the sub-region to be weeded, so as to obtain the corresponding target plant image; The target plant image is analyzed using a pixel coordinate ratio algorithm to determine the coordinate positions of all plants.

6. The method according to claim 1, characterized in that, The controlled robotic arm removes the weeds corresponding to all the weed coordinate positions, including: The optimal movement path of the robot arm is determined based on the coordinates of all the weeds and the coordinates of the first weed. The robotic arm is controlled to remove weeds in the area to be removed according to the optimal motion path.

7. An intelligent weeding device based on visual recognition and motion control, characterized in that, include: The first determining module is used to determine the actual total coverage area based on the pre-determined vertex position coordinates and shape type of the total area to be weeded, as well as the area occupied by the sub-areas to be weeded. The first identification module is used to identify the coordinates of all weeds in one of the sub-areas to be weeded within the total actual coverage area corresponding to the target weeder through a visual recognition algorithm. The control module is used to control the robotic arm to remove the weeds corresponding to all the weed coordinate positions, and to generate a corresponding removal completion signal after the removal is completed; The second identification module is used to transmit the clearing completion signal to the visual recognition system, so that the visual recognition system can perform secondary identification on the sub-area to be cleared and generate a corresponding clearing completion confirmation signal. The execution module is used to respond to the clearing completion confirmation signal, control the target weeder to move to the next sub-area to be cleared, and return to the step of identifying the coordinate positions of all weeds in one sub-area to be cleared in the actual total area through a visual recognition algorithm, until the clearing of weeds in the total area to be cleared is completed; The first determining module includes: The second identification and extraction unit is used to identify and extract the coordinates of at least one key point on at least one target edge in the total area to be weeded. The third determining unit is used to determine the actual total coverage area based on the coordinates of key points, the coordinates of the vertex positions of the total area to be weeded, and the area occupied by the sub-areas to be weeded. Here, the key point coordinates refer to the coordinates of the intersection point between a sub-region to be weeded and one of the edges in the total area to be weeded; the ratio between the actual total area occupied and the area occupied by the sub-region to be weeded is a positive integer.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the intelligent weeding method based on visual recognition and motion control as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the intelligent weeding method based on visual recognition and motion control as described in any one of claims 1-6.

Images