A method, device and storage medium for distinguishing crop root, stem, apical bud and lateral branch

Abstract

The present invention discloses a method, device and storage medium for distinguishing terminal buds and lateral branches of crop roots and stems, and relates to the field of printing technology; the method comprises S1, setting a color and depth integrated camera on a mobile frame to shoot an image of the top of cotton, S2, using a key point detection algorithm based on deep learning to detect the position of the terminal bud, obtaining the position T of the terminal bud in the color image, finding the depth D corresponding to the T position of the color image in the depth image, and then obtaining the position Tc of the terminal bud in the camera coordinate system, S3, using a rectangular frame detection method of the yolo series to detect the position of flower buds in the image, and marking the positions of the flower buds and the terminal buds, S4, comparing the detection position of the terminal bud in the image with the detection position of the flower buds, and S5, filtering within a range of 10 cm based on the position Tc of each remaining terminal bud in the camera coordinate system; the present invention realizes accurate detection of the terminal bud position by shooting with a color and depth integrated camera in combination with a deep learning algorithm.

Description

Technical Field

[0001] The present invention relates to the field of agricultural intelligent technology, and in particular to a method, a device and a storage medium for distinguishing root, stem, apical bud and lateral branch of crops. Background Art

[0002] Apical dominance refers to the phenomenon that the apical buds of plants grow preferentially while the lateral branches are inhibited. During the growth of some crops, the auxin produced by the apical buds is transported downward and accumulated in large quantities in the lateral buds, causing the auxin concentration in the lateral buds to be too high, thereby inhibiting the growth of the lateral buds. This is the apical dominance phenomenon. Based on this principle in agricultural production, crops such as cotton are pinched when they grow to a certain height so that the apical buds no longer produce auxin, reducing the auxin content in the lateral buds and promoting the lateral buds to develop into lateral branches, thereby achieving the purpose of increasing production.

[0003] When performing mechanized topping operations, it is necessary to accurately identify the top position of each crop to ensure accurate topping operations. However, due to factors such as the growth status of crops and leaf obstruction, the top position may not be easy to accurately identify, which leads to inaccuracy and instability in mechanized topping operations. Taking cotton as an example, the height difference between the cotton top and the cotton bud is very small, the cotton bud cannot be cut, and the growth height of each cotton plant is also different. This requires identifying the position of the top of each cotton plant, so as to use the robotic arm blade to perform the cotton topping operation. In this process, the side branches cannot be cut, otherwise it will affect the growth and yield of the side branches and reduce the overall yield of the crops. Therefore, how to accurately identify the top position, avoid accidental injury to the side branches, adapt to different growth conditions and improve operation efficiency are the main problems that need to be solved in mechanized topping operations. Summary of the invention

[0004] The object of the present invention is to provide a method, device and storage medium for distinguishing side branches from the terminal buds of crop rhizomes, so as to filter side branches during the detection of cotton terminal buds and reduce the erroneous cutting of side branches.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] The present application provides a method for distinguishing crop root apical buds and lateral branches, comprising the following steps:

[0007] S1. A color and depth integrated camera is set on a mobile frame, with the camera's downward viewing angle, and the mobile frame is driven in the field, passing over the tops of multiple cottons at a certain speed. The depth from the top bud of the same cotton to the camera remains unchanged, so as to capture images of the tops of the cottons;

[0008] S2. According to the image data of the cotton top obtained in step S1, including the color image and the depth image, the position of the terminal bud is detected using a key point detection algorithm based on deep learning, and the position T of the terminal bud in the color image is obtained. The position transformation relationship between the color image and the depth image calibrated in advance is used to find the depth D corresponding to the T position of the color image in the depth image. Based on the pre-calibrated color camera internal parameters, the inverse of the internal parameter matrix is ​​multiplied by the T coordinate and then multiplied by the depth D to obtain the position T_c of the terminal bud in the camera coordinate system;

[0009] S3, to prevent the side branches from being mistakenly detected and thus cut, the bud positions in the image are detected using the rectangular frame detection method of the YOLO series, and are marked with numbers, where "2" indicates the bud position and "1" indicates the terminal bud position, and then the final detection result image is obtained;

[0010] S4, comparing the detection position of the terminal bud with the detection position of the flower bud in the image, if the detection position of the terminal bud is located within the rectangular frame of the flower bud, that is, inside the flower bud position, then deleting the detection result of the terminal bud;

[0011] S5. Based on the position T_c of each remaining terminal bud in the camera coordinate system, filter within the adjustable threshold range of n centimeters, filter according to the distance of the terminal buds within the range of n centimeters relative to the camera, retain the terminal buds closest to the camera, and delete the others.

[0012] Furthermore, according to step S1, the color and depth integrated camera will simultaneously output a color image and a depth image. Each pixel value in the depth image represents the distance from each pixel to the camera. For each pixel, the corresponding depth value is obtained according to its coordinate position in the depth image. The obtained depth information is analyzed, and then the exact distance from each pixel to the camera is calculated.

[0013] Furthermore, the position of the terminal bud in the color image is detected by a key point detection algorithm based on deep learning, and the position T of the terminal bud in the color image is obtained.

[0014] The position of the terminal bud in the color image is set to (x_t, y_t), where x_t represents the horizontal position and y_t represents the vertical position. This position is directly used as the position information of the terminal bud in the color image, then T = (x_t, y_t), which represents the position of the terminal bud in the color image;

[0015] Convert the pixel coordinates of the cotton top in the color image to the corresponding pixel coordinates in the depth image to obtain the depth value corresponding to the pixel point in the pixel coordinates, and then convert it into the actual distance value through the pixel value in the depth image, which is specifically expressed as D = f * L / (D_raw*S); where D represents the distance from the pixel point to the camera, f represents the focal length of the camera, L represents the baseline distance of the camera, D_raw represents the original depth pixel value of the corresponding position in the depth image, and S represents the scaling factor of the depth value; according to the position transformation relationship between the color image and the depth map that has been calibrated in advance, find the depth value D corresponding to the position T in the color image in the depth map.

[0016] Furthermore, based on the previously calibrated color camera internal parameters, the camera coordinate system position is calculated, the inverse matrix of the internal parameter matrix is ​​taken, and the position T coordinate is multiplied by the depth D to obtain the position T_c of the terminal bud in the camera coordinate system;

[0017] The color camera intrinsic parameter matrix obtained by prior calibration is represented as K, including parameters such as focal length and principal point position. The intrinsic parameter matrix is ​​a 3x3 matrix, and then the inverse matrix K^-1 of the intrinsic parameter matrix K is calculated. The inverse matrix is ​​used to convert pixel coordinates into coordinates in the camera coordinate system;

[0018] Multiply the position coordinates of the apical bud in the color image T = (x_t, y_t) by the depth value D to obtain the coordinates of the apical bud in the camera coordinate system. The specific calculation formula is: T_c = D * K^-1 * (x_t, y_t, 1)^T Here, (x_t, y_t, 1)^T is the coordinates of the apical bud in the image coordinate system expressed by homogeneous coordinates, including the horizontal position (x_t), the vertical position (y_t) and the unit coordinate 1; T_c represents the coordinates of the apical bud in the camera coordinate system;

[0019] The camera coordinate system position T_c calculated by the inverse matrix and the depth value is then transformed and scaled to represent it as the actual position relative to the camera.

[0020] Furthermore, the target detection algorithm of the YOLO series is used to establish a YOLO model, and a cotton image dataset for training is prepared, including image samples with marked flower buds and terminal buds. The selected YOLO model is trained using the prepared dataset, and the trained model is used to infer new cotton images so that it can identify flower buds and terminal buds in the cotton images; in the detected image, according to the results output by the model, the positions of the flower buds and terminal buds are marked with rectangular frames and marked with numbers, with the flower bud position marked as "2" and the terminal bud position marked as "1" to distinguish the positions of the flower buds and terminal buds.

[0021] Furthermore, the detection position of the terminal bud in the image is compared with the detection position of the flower bud, and the terminal bud detection result is deleted. The detection position information of the terminal bud and the flower bud is obtained from the results output by the yolo model, including the coordinates and size of the rectangular box. For each terminal bud detection result, its position is compared with the detection position of the flower bud. When the rectangular box of the terminal bud is completely contained in the rectangular box of the flower bud, that is, the terminal bud position is inside the flower bud position, the detection result of the terminal bud is regarded as a false detection and needs to be deleted. When the rectangular box of the terminal bud does not overlap with the rectangular box of the flower bud or the overlapping part is small, the detection result of the terminal bud is retained, and finally the terminal bud detection result after comparison and processing is output.

[0022] Furthermore, in step S5, for each remaining terminal bud, its distance relative to the camera is calculated, that is, the distance from T_c to the camera, and the distance between the terminal buds is calculated according to the position information in the camera coordinate system. The n=10, and an adjustable threshold range of 10 cm is set to determine which terminal buds are within this range. For each terminal bud, check whether there are other detected terminal bud results within a range of 10 cm around it. When there are other terminal buds whose distances are within the threshold range, filtering is required. In the same plant, the terminal bud closest to the camera is retained, that is, the terminal bud with the shortest distance is retained, and the other terminal buds are deleted. Finally, the filtered terminal bud results are output, the nearest terminal bud is retained, and other terminal buds within 10 cm are deleted.

[0023] Further, from the final detection result image in step S3, find the areas marked as "1" and "2" representing the terminal bud position and the flower bud position respectively, and then determine the specific position of the terminal bud by the coordinate value of the terminal bud position T obtained in step S2 in the color image, compare the flower bud position and the terminal bud position detected in step S3 with the terminal bud position obtained in step S2, and judge whether the positions match by comparing the coordinate values ​​of the positions marked as "1" and "2" in the color image with the terminal bud position T;

[0024] When the flower bud position detected in step S3 matches the terminal bud position T in step S2, that is, the position "2" is consistent with the terminal bud position T, it is confirmed that the processing object is the terminal bud, ensuring that only the terminal bud is processed to avoid accidental damage to the side branches.

[0025] A crop apical bud tracking and matching device comprises a processor, a memory and computer program instructions stored in the memory. When the computer program instructions are executed by the processor, the steps of the above-mentioned crop root and stem apical bud and lateral branch distinguishing method are implemented.

[0026] A storage medium stores computer program instructions, which, when executed by a processor, implement the steps of the above-mentioned method for distinguishing the apical buds and lateral branches of crop roots and stems.

[0027] The beneficial effects of the present invention are:

[0028] By shooting with a color and depth integrated camera and combining it with a deep learning algorithm, the position of the terminal bud can be accurately detected to avoid being affected by factors such as leaf occlusion, thereby achieving accurate identification of the position of the terminal bud. Then, by detecting and comparing the positions of the flower buds and filtering based on the principle of being closest to the camera, it is possible to effectively avoid misdetection of side branches, ensuring that only the terminal buds are identified and processed, thereby avoiding the impact on the side branches.

[0029] By filtering the adjacent terminal buds, the operation efficiency and accuracy can be improved, ensuring that only accurately identified terminal buds will be processed, thereby avoiding interference with side branches, improving the accuracy of mechanized topping operations, and helping to achieve accurate identification and processing of terminal buds of crops, while avoiding accidental damage to side branches, improving operation efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] For better understanding and implementation, the technical solution of the present application is described in detail below with reference to the accompanying drawings.

[0031] Figure 1 A flowchart of the steps of a method for distinguishing the lateral branches of crop rhizomes and apical buds provided in Example 1 of the present application;

[0032] Figure 2 A schematic diagram of the location of the terminal bud of a crop root and stem according to a method for distinguishing terminal buds and side branches provided in Example 1 of the present application;

[0033] Figure 3 A schematic diagram of the location of the terminal buds of crop rhizomes from a top-down perspective for a method for distinguishing terminal buds and side branches provided in Example 1 of the present application;

[0034] Figure 4 A schematic diagram of the position of the terminal bud in a color image of a method for distinguishing the terminal bud and lateral branches of a crop root provided in Example 1 of the present application;

[0035] Figure 5 A diagram showing the positions of flower buds and terminal buds in an image for distinguishing terminal buds and side branches of crop roots and stems provided in Example 1 of the present application;

[0036] Figure 6 This is a schematic diagram of the rectangular frame detection results of a method for distinguishing the apical buds and lateral branches of crop roots and stems provided in Example 1 of the present application. DETAILED DESCRIPTION

[0037] In order to further explain the technical means and effects taken by the present invention to achieve the predetermined invention purpose, exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of methods and systems consistent with some aspects of the present application as detailed in the attached claims.

[0038] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0039] The specific implementation methods, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments. Example

[0040] See also Figure 1-Figure 6 The present embodiment provides a method, device and storage medium for distinguishing the lateral branches of the root and stem terminal buds of crops, which can filter the lateral branches during the cotton terminal bud detection process and reduce the accidental cutting of the lateral branches.

[0041] The present invention provides a method for distinguishing crop root stem terminal buds and lateral branches, comprising the following steps:

[0042] S1, Figure 2 It is the location of the terminal bud. Figure 3 It is the location of the top bud from a bird's-eye view. A color and depth integrated camera is set on a mobile frame, and the camera's bird's-eye view is facing downward. The mobile frame is driven in the field and passes over the tops of multiple cottons at a certain speed. The depth from the top bud of the same cotton to the camera remains unchanged, and is used to capture images of the tops of the cottons.

[0043] S2, according to the image data of the cotton top obtained in step S1, including color image and depth image, use the key point detection algorithm based on deep learning such as yolo-pose to detect the position of the top bud, and obtain the following Figure 4 The point in the color image is the position T of the terminal bud in the color image. The position and the position transformation relationship between the color image and the depth image that have been calibrated in advance are used to find the depth D corresponding to the position T in the color image in the depth image. Based on the pre-calibrated color camera intrinsic parameters, the inverse of the intrinsic parameter matrix is ​​multiplied by the T coordinate and then by the depth D to obtain the position T_c of the terminal bud in the camera coordinate system.

[0044] Through the equipment and settings in S1, the image data of the top of the cotton is obtained, including color images and depth images. Then, the deep learning-based algorithm described in S2 is used to detect the position of the top bud, and the position T in the color image is obtained. According to the position transformation relationship mentioned in S2 and the pre-calibrated information, the corresponding depth D is calculated. Finally, according to the color camera internal parameters, combined with the position T and depth D in the color image, the accurate position T_c of the top bud in the camera coordinate system is calculated. In this way, through the organic combination of S1 and S2, the accurate detection and positioning of the cotton top bud position is achieved.

[0045] S3: To prevent the side branches from being mistakenly detected and cut, use the rectangular frame detection method of the yolo series to detect Figure 5 The flower buds in the image are marked with numbers, with "2" indicating the flower bud position and "1" indicating the terminal bud position, and the final detection result image is obtained, as shown in Figure 6 As shown, it is ensured that only the terminal buds are treated without accidentally damaging the side branches, thus achieving accurate treatment of the top of the cotton;

[0046] The flower bud position was detected by a rectangular frame detection method to avoid misdetecting the flower bud position as the terminal bud position, ensuring that only the terminal bud was processed without accidentally damaging the side branches.

[0047] S4, comparing the detection position of the terminal bud with the detection position of the flower bud in the image, if the detection position of the terminal bud is located within the rectangular frame of the flower bud, that is, inside the position of the flower bud, then deleting the detection result of the terminal bud, because the position of the flower bud usually corresponds to the position of the side branch, not the position of the terminal bud;

[0048] By comparing the terminal bud position with the flower bud position, it is ensured that the terminal bud is not located inside the flower bud position to avoid misdetecting the terminal bud position as the flower bud position, thereby removing possible false detection results.

[0049] S5. Based on the position T_c of each remaining apical bud in the camera coordinate system, filtering is performed within an adjustable threshold range of 10 cm. Cotton plants are planted at intervals of 10 cm, so if there are other detected apical bud results within 10 cm around each detected apical bud, filtering is required; filtering is performed based on the distance of the apical buds within 10 cm relative to the camera, retaining the apical buds closest to the camera and deleting the others, because the apical buds of the same plant are often higher than the side branches.

[0050] The S4 performs filtering by comparing the flower bud position and the terminal bud position, while the S5 retains the terminal bud closest to the camera. The combination of the two steps can effectively screen out the correct terminal bud position for subsequent processing, thereby improving the accuracy and reliability of the processing.

[0051] Furthermore, according to step S1, the color and depth integrated camera will simultaneously output a color image and a depth image. Each pixel value in the depth image represents the distance from each pixel to the camera. For each pixel, the corresponding depth value is obtained according to its coordinate position in the depth image. The obtained depth information is analyzed, and then the exact distance from each pixel to the camera is calculated.

[0052] Furthermore, the position of the terminal bud in the color image is detected by a key point detection algorithm based on deep learning, and the position T of the terminal bud in the color image is obtained.

[0053] The position of the terminal bud in the color image is set to (x_t, y_t), where x_t represents the horizontal position and y_t represents the vertical position. This position is directly used as the position information of the terminal bud in the color image, then T = (x_t, y_t), which represents the position of the terminal bud in the color image;

[0054] Convert the pixel coordinates of the cotton top in the color image to the corresponding pixel coordinates in the depth image to obtain the depth value corresponding to the pixel point in the pixel coordinates, and then convert it into the actual distance value through the pixel value in the depth image, which is specifically expressed as D = f * L / (D_raw*S); where D represents the distance from the pixel point to the camera, f represents the focal length of the camera, L represents the baseline distance of the camera, D_raw represents the original depth pixel value of the corresponding position in the depth image, and S represents the scaling factor of the depth value; according to the position transformation relationship between the color image and the depth map that has been calibrated in advance, find the depth value D corresponding to the position T in the color image in the depth map.

[0055] Furthermore, based on the previously calibrated color camera internal parameters, the camera coordinate system position is calculated, the inverse matrix of the internal parameter matrix is ​​taken, and the position T coordinate is multiplied by the depth D to obtain the position T_c of the terminal bud in the camera coordinate system;

[0056] The color camera intrinsic parameter matrix obtained by prior calibration is represented as K, including parameters such as focal length and principal point position. The intrinsic parameter matrix is ​​a 3x3 matrix, and then the inverse matrix K^-1 of the intrinsic parameter matrix K is calculated. The inverse matrix is ​​used to convert pixel coordinates into coordinates in the camera coordinate system;

[0057] Multiply the position coordinates of the apical bud in the color image T = (x_t, y_t) by the depth value D to obtain the coordinates of the apical bud in the camera coordinate system. The specific calculation formula is: T_c = D * K^-1 * (x_t, y_t, 1)^T Here, (x_t, y_t, 1)^T is the coordinates of the apical bud in the image coordinate system expressed by homogeneous coordinates, including the horizontal position (x_t), the vertical position (y_t) and the unit coordinate 1; T_c represents the coordinates of the apical bud in the camera coordinate system;

[0058] The camera coordinate system position T_c calculated by the inverse matrix and the depth value is then transformed and scaled to represent it as the actual position relative to the camera.

[0059] According to the pre-calibrated color camera internal parameters, the accurate position T_c of the terminal bud in the camera coordinate system is calculated, realizing the position conversion and calculation from the image coordinate system to the camera coordinate system.

[0060] Furthermore, the target detection algorithm of the YOLO series is used to establish a YOLO model, and a cotton image dataset for training is prepared, including image samples with marked flower buds and terminal buds. The selected YOLO model is trained using the prepared dataset, and the trained model is used to infer new cotton images so that it can identify flower buds and terminal buds in the cotton images; in the detected image, according to the results output by the model, the positions of the flower buds and terminal buds are marked with rectangular frames and marked with numbers, with the flower bud position marked as "2" and the terminal bud position marked as "1" to distinguish the positions of the flower buds and terminal buds.

[0061] Using the YOLO series of target detection algorithms, the positions of flower buds and terminal buds are automatically detected in cotton images and marked with numbers to distinguish the two parts, thereby achieving accurate detection of the flower bud positions.

[0062] Furthermore, the detection position of the terminal bud in the image is compared with the detection position of the flower bud, and the terminal bud detection result is deleted. The detection position information of the terminal bud and the flower bud is obtained from the results output by the YOLO model, including the coordinates and size of the rectangular box. For each terminal bud detection result, its position is compared with the detection position of the flower bud. When the rectangular box of the terminal bud is completely contained in the rectangular box of the flower bud, that is, the terminal bud position is inside the flower bud position, the detection result of the terminal bud is regarded as a false detection and needs to be deleted. When the rectangular box of the terminal bud does not overlap with the rectangular box of the flower bud or the overlapping part is small, the detection result of the terminal bud is retained. Finally, the terminal bud detection result after comparison and processing is output to ensure that only the correct terminal bud position information is retained.

[0063] The detection positions of the flower buds and terminal buds were compared, and the terminal bud detection results located inside the flower bud position were deleted to ensure accurate processing of the cotton top.

[0064] Furthermore, for each remaining terminal bud, calculate its distance relative to the camera, that is, the distance from T_c to the camera, calculate the distance between the terminal buds according to the position information in the camera coordinate system, set an adjustable threshold range of 10 cm to determine which terminal buds are within this range, and for each terminal bud, check whether there are other detected terminal bud results within a range of 10 cm around it. When there are other terminal buds whose distances are within the threshold range, filtering is required. In the same plant, retain the terminal bud closest to the camera, that is, retain the terminal bud with the shortest distance, and delete the other terminal buds. Finally, output the filtered terminal bud results, retain the nearest terminal bud, and delete other terminal buds within 10 cm to ensure accurate processing of the cotton top.

[0065] According to the position and distance information of the terminal buds in the camera coordinate system, filtering is performed within a range of 10 cm, the nearest terminal buds are retained, and other terminal bud results are deleted to meet the requirements of plant spacing and terminal bud height.

[0066] Further, from the final detection result image in step S3, find the areas marked as "1" and "2" representing the terminal bud position and the flower bud position respectively, and then determine the specific position of the terminal bud by the coordinate value of the terminal bud position T obtained in step S2 in the color image, compare the flower bud position and the terminal bud position detected in step S3 with the terminal bud position obtained in step S2, and judge whether the positions match by comparing the coordinate values ​​of the positions marked as "1" and "2" in the color image with the terminal bud position T;

[0067] When the flower bud position detected in step S3 matches the terminal bud position T in step S2, that is, the position "2" is consistent with the terminal bud position T, it is confirmed that the processing object is the terminal bud, ensuring that only the terminal bud is processed to avoid accidental damage to the side branches.

[0068] Through comparison and verification, we can ensure that only the top bud position is processed to avoid accidental damage to the side branches, thereby achieving accurate processing of the top of the cotton. Such a comparison process can help ensure accurate identification and processing of the target and improve the accuracy and reliability of the processing.

[0069] A crop apical bud tracking and matching device comprises a processor, a memory and computer program instructions stored in the memory. When the computer program instructions are executed by the processor, the steps of the above-mentioned crop root and stem apical bud and lateral branch distinguishing method are implemented.

[0070] A storage medium stores computer program instructions, which, when executed by a processor, implement the steps of the above-mentioned method for distinguishing the apical buds and lateral branches of crop roots and stems.

[0071] The present invention uses a color and depth integrated camera on a mobile frame to shoot the top image of cotton, uses a deep learning algorithm to detect the position of the terminal bud, ensures that the depth remains unchanged, and realizes accurate identification of the position of the terminal bud; detects the position of the flower bud by a rectangular frame detection method to avoid misdetecting the position of the flower bud as the position of the terminal bud, and deletes the detection result of the terminal bud located inside the flower bud by comparing the position of the terminal bud and the flower bud to prevent accidental damage to the side branches and ensure that only the terminal bud is processed; according to the position of each terminal bud in the camera coordinate system, filtering is performed within a range of 10 cm, the terminal bud closest to the camera is retained, and other terminal buds are deleted to ensure that only the nearest terminal bud is processed to avoid affecting the side branches.

[0072] Through the steps of a method for distinguishing the apical buds and lateral branches of crop roots, the position of the cotton apical buds can be accurately detected and located, ensuring that only the correct apical bud position information is retained, improving work efficiency and accuracy, avoiding accidental damage to lateral branches, and helping to achieve accurate identification and processing of the apical buds of crops.

[0073] Specifically, through image acquisition, application of deep learning algorithms, position comparison and filtering, combined with the technical means of color and depth integrated cameras, accurate processing of the cotton top is achieved, avoiding accidental damage to side branches, improving work efficiency and accuracy, and thus achieving the effect of accurate identification and processing of the top buds of crops.

[0074] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for distinguishing the lateral branches of crop rhizomes and terminal buds, characterized in that: The method comprises the following steps: S1, setting a color and depth integrated camera on a mobile frame, with the camera's downward viewing angle, driving the mobile frame in the field, passing over the tops of multiple cottons at a certain speed, and keeping the depth from the top bud of the same cotton to the camera unchanged, so as to capture images of the tops of the cottons; S2. According to the image data of the cotton top obtained in step S1, including the color image and the depth image, the position of the terminal bud is detected using a key point detection algorithm based on deep learning, and the position T of the terminal bud in the color image is obtained. The position transformation relationship between the color image and the depth image calibrated in advance is used to find the depth D corresponding to the T position of the color image in the depth image. Based on the pre-calibrated color camera internal parameters, the inverse of the internal parameter matrix is ​​multiplied by the T coordinate and then multiplied by the depth D to obtain the position T_c of the terminal bud in the camera coordinate system; S3, to prevent the side branches from being mistakenly detected and thus cut, the bud positions in the image are detected using the rectangular frame detection method of the YOLO series, and are marked with numbers, where "2" indicates the bud position and "1" indicates the terminal bud position, and then the final detection result image is obtained; S4, comparing the detection position of the terminal bud obtained in step S3 with the detection position of the flower bud, if the detection position of the terminal bud is located within the rectangular frame of the flower bud, that is, inside the flower bud position, then deleting the detection result of the terminal bud; S5. Based on the position T_c of the remaining terminal buds in the camera coordinate system obtained in step S4, filtering is performed within a threshold range of n centimeters. The terminal buds within the range of n centimeters are filtered according to their distance from the camera, and the terminal buds closest to the camera are retained, and the others are deleted.

2. A method for distinguishing crop root stem terminal buds and lateral branches according to claim 1, characterized in that: According to step S1, the color and depth integrated camera will simultaneously output a color image and a depth image. Each pixel value in the depth image represents the distance from each pixel to the camera. For each pixel, the corresponding depth value is obtained according to its coordinate position in the depth image. The obtained depth information is analyzed, and then the accurate distance from each pixel to the camera is calculated.

3. A method for distinguishing crop root stem terminal buds and lateral branches according to claim 2, characterized in that: The position of the apical bud in the color image is detected by a key point detection algorithm based on deep learning, and the position T of the apical bud in the color image is obtained. The position of the terminal bud in the color image is set to (x_t, y_t), where x_t represents the horizontal position and y_t represents the vertical position. This position is directly used as the position information of the terminal bud in the color image, then T = (x_t, y_t), which represents the position of the terminal bud in the color image; Convert the pixel coordinates of the cotton top in the color image to the corresponding pixel coordinates in the depth image to obtain the depth value corresponding to the pixel point in the pixel coordinates, and then convert it into the actual distance value through the pixel value in the depth image, which is specifically expressed as D = f * L / (D_raw*S); where D represents the distance from the pixel point to the camera, f represents the focal length of the camera, L represents the baseline distance of the camera, D_raw represents the original depth pixel value of the corresponding position in the depth image, and S represents the scaling factor of the depth value; according to the position transformation relationship between the color image and the depth map that has been calibrated in advance, find the depth value D corresponding to the position T in the color image in the depth map.

4. A method for distinguishing crop root stem terminal buds and lateral branches according to claim 1, characterized in that: Based on the previously calibrated color camera internal parameters, calculate the camera coordinate system position, take the inverse matrix of the internal parameter matrix, multiply the position T coordinate by the depth D, and obtain the position T_c of the terminal bud in the camera coordinate system; The color camera intrinsic parameter matrix obtained by prior calibration is represented as K, including the focal length and principal point position parameters. The intrinsic parameter matrix is ​​a 3x3 matrix. Then, the inverse matrix K^-1 of the intrinsic parameter matrix K is calculated. The inverse matrix is ​​used to convert the pixel coordinates into the coordinates in the camera coordinate system. Multiply the position coordinates of the apical bud in the color image T = (x_t, y_t) by the depth value D to obtain the coordinates of the apical bud in the camera coordinate system. The specific calculation formula is: T_c = D * K^-1 * (x_t, y_t, 1)^T Here, (x_t, y_t,1)^T is the coordinates of the apical bud in the image coordinate system expressed by homogeneous coordinates, including the horizontal position (x_t), the vertical position (y_t) and the unit coordinate 1; T_c represents the coordinates of the apical bud in the camera coordinate system; The camera coordinate system position T_c calculated by the inverse matrix and the depth value is then transformed and scaled to represent it as the actual position relative to the camera.

5. A method for distinguishing crop root stem terminal buds and lateral branches according to claim 1, characterized in that: The YOLO model is established by using the target detection algorithm of the YOLO series, and a cotton image dataset for training is prepared, including image samples with marked flower buds and terminal buds. The selected YOLO model is trained using the prepared dataset, and the trained model is used to infer new cotton images so that the flower buds and terminal buds in the cotton images can be identified. In the detected images, according to the results output by the model, the positions of the flower buds and terminal buds are marked with rectangular frames and marked with numbers, with the flower bud position marked as "2" and the terminal bud position marked as "1" to distinguish the positions of the flower buds and terminal buds.

6. A method for distinguishing crop root stem terminal buds and lateral branches according to claim 5, characterized in that: According to the comparison between the detection position of the terminal bud and the detection position of the flower bud in the image, and the terminal bud detection result is deleted, the detection position information of the terminal bud and the flower bud is obtained from the results output by the Yolo model, including the coordinates and size of the rectangular frame. For each terminal bud detection result, its position is compared with the detection position of the flower bud. When the rectangular frame of the terminal bud is completely contained in the rectangular frame of the flower bud, that is, the terminal bud position is inside the flower bud position, the detection result of the terminal bud is regarded as a false detection and needs to be deleted. When the rectangular frame of the terminal bud does not overlap with the rectangular frame of the flower bud or the overlapping part is small, the detection result of the terminal bud is retained, and finally the terminal bud detection result after comparison and processing is output.

7. A method for distinguishing crop root stem terminal buds and lateral branches according to claim 1, characterized in that: In step S5, for each remaining terminal bud, its distance relative to the camera is calculated, that is, the distance from T_c to the camera, and the distance between the terminal buds is calculated according to the position information in the camera coordinate system. The n=10, and a threshold range of 10 cm is set to determine which terminal buds are within this range. For each terminal bud, check whether there are other detected terminal bud results within a range of 10 cm around it. When there are other terminal buds whose distances are within the threshold range, filtering is required. In the same plant, the terminal bud closest to the camera is retained, that is, the terminal bud with the shortest distance is retained, and the other terminal buds are deleted. Finally, the filtered terminal bud results are output, the nearest terminal bud is retained, and other terminal buds within 10 cm are deleted.

8. A method for distinguishing crop root stem terminal buds and lateral branches according to claim 1, characterized in that: From the final detection result image in step S3, find the areas marked as "1" and "2" representing the terminal bud position and the flower bud position respectively, and then from the terminal bud position T obtained in step S2, determine the specific position of the terminal bud by the coordinate value of the position in the color image, compare the flower bud position and the terminal bud position detected in step S3 with the terminal bud position obtained in step S2, and judge whether the positions match by comparing the coordinate values ​​of the positions marked as "1" and "2" in the color image with the terminal bud position T; When the flower bud position detected in step S3 matches the terminal bud position T in step S2, that is, the "2" position is consistent with the terminal bud position T, it is confirmed that the processing object is the terminal bud, ensuring that only the terminal bud is processed to avoid accidental damage to the side branches.

9. A crop terminal bud tracking and matching device, characterized in that: The method comprises a processor, a memory and computer program instructions stored in the memory. When the computer program instructions are executed by the processor, a method for distinguishing the lateral branches of the root and stem of a crop as described in any one of claims 1 to 8 is implemented.

10. A storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by the processor, a method for distinguishing the lateral branches of crop root stems and apical buds as described in any one of claims 1 to 8 is implemented.

Images