A radish detection and extraction method based on dynamic analysis

Abstract

The present invention provides a radish detection and extraction method based on dynamic image analysis, comprising the following steps: S1: constructing an image detection system to simulate the side shape of a radish; S2: collecting image information of the probes on the left and right sides by respectively setting visual sensors on the upper ends of the probes on the left and right sides; S3: binarizing the extracted image information, calculating its center, and obtaining a curve graph composed of center points; S4: using an active probe to traverse the radish structure from bottom to top, and determining the extraction point of the radish through the curve graph; S5: sending the determined extraction point to an actuator to execute the extraction action. The development cost is low; the stability is good and the method has practical value; the method is simple, the result can be calculated quickly, and a high real-time requirement can be achieved; a low-cost embedded computing platform can be applied to greatly reduce the equipment cost; and the front and rear offset distances relative to the detection system are calculated to calculate the position of the radish in the robot coordinate system.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent plant picking, and more specifically, to a radish detection and picking method based on dynamic image analysis. Background Art

[0002] Among the existing large-scale agricultural machinery, radish harvesting robots are mostly used for picking radishes. The existing radish harvesting robots generally use mechanical arms to pick radishes, and use infrared to scan and locate the radishes. The actuator pushes the mechanical arm to the radish position point determined by the scan to perform the picking action. The infrared scanning-based method has the following problems: the radish stems or leaves will block the line of sight above the radish, resulting in large errors in the scanning process, and the position of the radish cannot be accurately located, and the effective picking point cannot be determined. As a result, the number of failures in the picking process will increase. Summary of the invention

[0003] The present invention aims at the technical problems existing in the prior art and provides a radish detection and extraction method based on dynamic image analysis. The rule-based method does not require algorithm training, thereby, does not require the collection of training data sets, does not require the construction of a training system, and has low development cost; has good stability and practical value; the method is simple, can quickly calculate results, and can meet high real-time requirements; can apply a low-cost embedded computing platform, greatly reducing equipment costs; can also directly determine the center position of the radish and the front and rear offset distances relative to the detection system, thereby calculating the position of the radish in the robot coordinate system.

[0004] According to a first aspect of the present invention, there is provided a method for detecting and removing radishes based on dynamic image analysis, comprising the following steps:

[0005] S1: constructing an image detection system, including setting a number of movable probes on both sides of the radish collection mechanism to contact the radish, thereby simulating the side shape of the radish;

[0006] S2: collecting image information of the probes on the left and right sides by respectively setting visual sensors on the upper ends of the probes on the left and right sides;

[0007] S3: Binarize the extracted image information, search for each foreground sub-block in the binarized image, and calculate its center to obtain a curve graph composed of the center points;

[0008] S4: using a movable probe to traverse the radish structure from bottom to top, and taking a probe image on the left and right sides every time it rises to a set height, respectively obtaining multiple groups of curve graphs on the left and right sides, and determining the extraction point of the radish through the curve graphs;

[0009] S5: Send the determined extraction point to the actuator to execute the extraction action.

[0010] Based on the above technical solution, the present invention can also make the following improvements.

[0011] Optionally, the curve graph in step S4 includes the left image center curve L i and the center curve R of the right image i , where the center curve L of the left image is i The corresponding average area LS i =(d*h 1 +d*h 2 +……+d*h M ) / (d*M), record the center curve R of the right image i The corresponding average area RS i =(f*g 1 +f*g 2 +……+f*g N ) / (f*N), M is the number of probes on the left side of the image detection system, N is the number of probes on the right side of the image detection system, d is the lateral distance between two adjacent probes on the left side of the image detection system, f is the lateral distance between two adjacent probes on the right side of the image detection system, h is the distance that the probe on the left side of the image detection system deviates from the original position, and g is the distance that the probe on the right side of the image detection system deviates from the original position. Then the sum of the average areas of the radishes measured S is obtained. i =LS i +RS i , i is a positive integer; the sum of the average areas of radishes corresponding to each curve Si is used to calculate the approximate gradient dS i =S i –S i+1 , and obtain the gradient sequence dS 1 , dS 2 , ..., dS i-1 , determine the radish extraction point based on the gradient sequence.

[0012] Optionally, determine the extraction point of the radish according to the following conditions:

[0013] When the corresponding approximate gradient value in the gradient sequence changes from a positive number to a negative number, the position point corresponding to the positive gradient value before the negative gradient value is the radish picking point;

[0014] or

[0015] When the absolute value of the gradient value suddenly increases from a smaller value, the position point corresponding to the gradient value corresponding to the smaller value is the picking point of the radish.

[0016] Optionally, when the gradient value does not change from positive to negative and its absolute value does not suddenly change from a small value to a large value, it indicates that no extraction point is found, and the judgment is made in the following way:

[0017] If the sum Si of the average areas of the radishes measured is less than the set threshold, it means that the extraction point is below the detection starting point, and the detection starting point is moved down for re-detection;

[0018] If the sum Si of the measured average areas of the radishes is greater than the set threshold, it means that the extraction point is above the detection starting point, and the detection starting point is moved up for re-detection.

[0019] Optionally, the center is a geometric center or a centroid.

[0020] Optionally, the set height of step S4 is 5-20 mm.

[0021] Optionally, the number of traversals in step S4 is not less than 3 times.

[0022] The present invention provides a radish detection and extraction method based on dynamic image analysis, which has the following beneficial effects:

[0023] (1) Rule-based methods do not require algorithm training, and therefore do not require the collection of training data sets or the construction of training systems, resulting in lower development costs.

[0024] (2) The rule-based algorithm of the present invention has good stability and practical value;

[0025] (3) This algorithm is simple, can quickly calculate the results, and can meet high real-time requirements;

[0026] (4) Because the calculation is simple, low-cost embedded computing platforms can be used, greatly reducing equipment costs;

[0027] (5) The algorithm can also be used to directly calculate the center position of the radish and the front and rear offset distance relative to the detection system, thereby calculating the position of the radish in the robot coordinate system. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A flow chart of a radish detection and extraction method based on dynamic image analysis provided by an embodiment of the present invention;

[0029] Figure 2 A schematic diagram of the probe distribution structure of a radish detection and extraction method based on dynamic image analysis provided by an embodiment of the present invention;

[0030] Figure 3 The result of image binarization processing of a radish detection and extraction method based on dynamic image analysis provided by an embodiment of the present invention;

[0031] Figure 4A central curve diagram of one side of a radish detection and extraction method based on dynamic image analysis provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0033] like Figure 1-Figure 4 As shown:

[0034] Figure 1 A flow chart of a radish detection and extraction method based on dynamic image analysis provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0035] S1: constructing an image detection system, including setting a number of movable probes on both sides of the radish collection mechanism to contact the radish, thereby simulating the side shape of the radish;

[0036] S2: collecting image information of the probes on the left and right sides by respectively setting visual sensors on the upper ends of the probes on the left and right sides;

[0037] S3: Binarize the extracted image information, search for each foreground sub-block in the binarized image, and calculate its center to obtain a curve graph composed of the center points;

[0038] S4: using a movable probe to traverse the radish structure from bottom to top, and taking a probe image on the left and right sides every time it rises to a set height, respectively obtaining multiple groups of curve graphs on the left and right sides, and determining the extraction point of the radish through the curve graphs;

[0039] S5: Send the determined extraction point to the actuator to execute the extraction action.

[0040] It can be understood that, in this embodiment, an image detection system is constructed to collect data through a plurality of movable probes to simulate the shapes of the two sides of the radish, wherein the shapes of the radish on the two sides constitute the overall shape of the radish, such as Figure 2 As shown in the figure, after the probe touches the radish, it will slide outwards. The probes on both sides slide outwards respectively under the resistance of the radish, while the inner side of the probe is attached to the surface of the radish, thereby simulating the overall shape of the radish. At the same time, the visual sensor collects the structural shape of the outer side or the inner side of the probe. Here, since the structure of the inner side of the probe may be inaccurate due to the occlusion of the upper end or stems and leaves of the radish, the collection is based on the outer side of the probe. The collected image information is binarized to form a curve graph; by shooting multiple groups of curve graphs and further analyzing the curve graphs to obtain the precise extraction point of the radish, it is helpful to extract the radish smoothly. Then, according to the judgment result, it is output to the extraction actuator to perform the extraction action.

[0041] In the embodiment, the rule-based method does not require algorithm training, and thus, does not require the collection of training data sets, nor does it require the construction of a training system, and the development cost is low; it has good stability and practical value; the method is simple, can quickly calculate the results, and can meet high real-time requirements; an inexpensive embedded computing platform can be used to greatly reduce equipment costs; and the center position of the radish and the front and rear offset distances relative to the detection system can also be directly determined, thereby calculating the position of the radish in the robot coordinate system.

[0042] In a possible embodiment, the curve graph in step S4 includes the left image center curve L i and the center curve R of the right image i , where the center curve L of the left image is i The corresponding average area LS i =(d*h 1 +d*h 2 +……+d*h M ) / (d*M), record the center curve R of the right image i The corresponding average area RS i =(f*g 1 +f*g 2 +……+f*g N ) / (f*N), M is the number of probes on the left side of the image detection system, N is the number of probes on the right side of the image detection system, d is the lateral distance between two adjacent probes on the left side of the image detection system, f is the lateral distance between two adjacent probes on the right side of the image detection system, h is the distance that the probe on the left side of the image detection system deviates from the original position, and g is the distance that the probe on the right side of the image detection system deviates from the original position. Then the sum of the average areas of the radishes measured S is obtained. i =LS i +RS i , i is a positive integer; the sum of the average area of ​​radishes corresponding to each curve S i Calculate the approximate gradient dS i =S i –S i+1 , and obtain the gradient sequence dS 1 , dS 2 , ..., dS i-1 , determine the radish extraction point based on the gradient sequence.

[0043] It can be understood that in the embodiment, the area is analyzed by the curve graph composed of the images on both sides, and then the area sizes of the radish cross sections collected multiple times during the traversal process are determined, and then the area sizes of these cross sections are compared to determine the turning point of the change of the radish cross section during the traversal from bottom to top, and then the exact extraction position of the radish is determined.

[0044] In a possible implementation manner, the radish extraction point is determined according to the following conditions:

[0045] When the corresponding approximate gradient value in the gradient sequence changes from a positive number to a negative number, the position point corresponding to the positive gradient value before the negative gradient value is the radish picking point;

[0046] or

[0047] When the absolute value of the gradient value suddenly increases from a smaller value, the position point corresponding to the gradient value corresponding to the smaller value is the picking point of the radish.

[0048] It can be understood that when the approximate gradient value changes from a positive number to a negative number, since it is traversed from bottom to top, the size of the previous cross-sectional area of ​​the radish is smaller than the size of the next cross-sectional area. Therefore, the cross-sectional area of ​​the radish during the traversal process gradually increases from bottom to top and then gradually decreases. Therefore, pulling out the radish at the maximum point of its cross-sectional area, or the point before the maximum point, can ensure the smooth pulling out of the radish. If it is pulled out at the latter point, it may fall off because its upper end area is gradually decreasing.

[0049] On the contrary, under another judgment condition, when the absolute value of the gradient value suddenly increases from a smaller value, the corresponding point or surface with the largest cross-sectional area of ​​the radish can be judged, so as to be extracted.

[0050] In a possible embodiment, when the gradient value does not change from a positive number to a negative number and its absolute value does not suddenly change from a small value to a large value, it indicates that no extraction point is found, and the judgment is made in the following manner:

[0051] If the sum of the measured average areas of the radishes Si is less than the set threshold, it means that the extraction point is below the detection starting point, and the detection starting point is moved down for re-detection; the threshold here is: when the diameter of the radish is 4-6cm, the Si mean corresponding to the chest of the radish is detected.

[0052] If the sum Si of the measured average areas of the radishes is greater than the set threshold, it means that the extraction point is above the detection starting point, and the detection starting point is moved up for re-detection.

[0053] It is understandable that when the above two judgment conditions are not met, it means that a suitable extraction point may not be detected at present. Therefore, it means that the extraction point may be below the detection starting point or above the detection starting point. If it is missed, the sum of the measured average areas of the radishes can be compared to determine whether it is above or below the detection starting point.

[0054] In addition, it should be noted that, in this embodiment, the center is the geometric center or the centroid. The set height of step S4 is 5-20 mm. The number of traversals in step S4 is not less than 3 times.

[0055] It is understandable that the set height is the distance of each upward movement. Specifically, the embodiment requires at least three position changes. In principle, the more times, the more accurate it is. However, too many times of detection may cause the machine to take longer to respond, which is not conducive to large-scale operations.

[0056] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0057] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0058] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0059] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0060] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0061] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0062] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for detecting and extracting radishes based on dynamic image analysis, It is characterized in that It includes the following steps: S1: constructing an image detection system, including setting a number of movable probes on both sides of the radish collection mechanism to contact the radish, thereby simulating the side shape of the radish; S2: collecting image information of the probes on the left and right sides by respectively setting visual sensors on the upper ends of the probes on the left and right sides; S3: Binarize the extracted image information, search for each foreground sub-block in the binarized image, and calculate its center to obtain a curve graph composed of the center points; S4: using a movable probe to traverse the radish structure from bottom to top, and taking a probe image on the left and right sides every time it rises to a set height, respectively obtaining multiple groups of curve graphs on the left and right sides, and determining the extraction point of the radish through the curve graphs; S5: Send the determined extraction point to the actuator to perform the extraction action; the curve graph in step S4 includes the left image center curve L i and the center curve R of the right image i , where the center curve L of the left image is i The corresponding average area LS i =(d*h 1 +……+d*h i +……+d*h M ) / (d*M), record the center curve R of the right image i The corresponding average area RS i =(f*g 1 +……+f*g i +……+f*g N ) / (f*N), M is the number of probes on the left side of the image detection system, N is the number of probes on the right side of the image detection system, d is the lateral distance between two adjacent probes on the left side of the image detection system, f is the lateral distance between two adjacent probes on the right side of the image detection system, h is the distance that the probe on the left side of the image detection system deviates from the original position, and g is the distance that the probe on the right side of the image detection system deviates from the original position. Then the sum of the average areas of the radishes measured S is obtained. i =LS i +RS i , i is a positive integer; the sum of the average areas of radishes corresponding to each curve Si is used to calculate the approximate gradient dS i =S i –S i+1 , and obtain the gradient sequence dS 1 , dS 2 , ..., dS i-1 , determine the radish extraction point based on the gradient sequence.

2. A method for detecting and extracting radishes based on dynamic image analysis according to claim 1, It is characterized in that When determining the extraction point of the radish, confirm it according to the following conditions: When the corresponding approximate gradient value in the gradient sequence changes from a positive number to a negative number, the position point corresponding to the positive gradient value before the negative gradient value is the radish picking point; or When the absolute value of the gradient value suddenly increases from a smaller value, the position point corresponding to the gradient value corresponding to the smaller value is the picking point of the radish.

3. A method for detecting and extracting radishes based on dynamic image analysis according to claim 2, It is characterized in that When the gradient value does not change from positive to negative and its absolute value does not suddenly increase from a small value, it means that no extraction point is found. The following method is used to judge: If the sum Si of the average areas of the radishes measured is less than the set threshold, it means that the extraction point is below the detection starting point, and the detection starting point is moved down for re-detection; If the sum Si of the measured average areas of the radishes is greater than the set threshold, it means that the extraction point is above the detection starting point, and the detection starting point is moved up for re-detection.

4. The method for detecting and extracting radishes based on dynamic image analysis according to claim 1, It is characterized in that The center is the geometric center or the centroid.

5. The method for detecting and extracting radishes based on dynamic image analysis according to claim 1, It is characterized in that The setting height of step S4 is 5-20 mm.

6. The method for detecting and extracting radishes based on dynamic image analysis according to claim 1, It is characterized in that The number of traversals in step S4 is not less than 3 times.

Images