Sugarcane growth height measurement method and system

Abstract

The present invention provides a method and system for measuring the growth height of sugarcane, which can be applied to the fields of agricultural measurement and artificial intelligence. The measurement method comprises: inputting multiple images obtained by shooting target sugarcane with a single depth camera into a target detection model to obtain a sugarcane bottom image and a sugarcane top image of the target sugarcane, the target detection model is a deep learning model improved based on sugarcane features, the depth camera is mounted on a gimbal, and the gimbal is used to control the shooting angle of the depth camera; according to the sugarcane top image and the sugarcane bottom image, a first distance from the depth camera to the sugarcane top and a second distance from the depth camera to the sugarcane bottom are obtained from the depth camera; the target angle of the depth camera is obtained, and the target angle is determined according to the first shooting angle for shooting the sugarcane top image and the second shooting angle for shooting the sugarcane bottom image; according to the first distance, the second distance and the target angle, the target growth height of the target sugarcane is obtained.

Description

Technical Field

[0001] The present invention relates to the fields of agricultural measurement and artificial intelligence, and more specifically, to a method and system for measuring the growth height of sugarcane. Background Art

[0002] As an important sugar crop and energy crop in the world, the yield and quality of sugarcane have a significant impact on the development of the sugar industry and related industries. In sugarcane planting management, real-time monitoring and evaluation of sugarcane height parameters are of great significance for optimizing cultivation practices and improving yield and quality.

[0003] In the process of realizing the concept of the present invention, the inventors found that there are at least the following problems in the related art: usually, when measuring the height of a single sugarcane, multiple cameras are used for measurement, which involves coordinate conversion between multiple cameras, complex calculations, and low height measurement efficiency. Summary of the invention

[0004] In view of this, the present invention provides a method and system for measuring the growth height of sugarcane.

[0005] One aspect of the present invention provides a method for measuring the growth height of sugarcane, comprising: inputting a plurality of images obtained by photographing a target sugarcane with a single depth camera into a target detection model to obtain a sugarcane bottom image and a sugarcane top image of the target sugarcane, wherein the depth camera is mounted on a gimbal, the gimbal is used to control the shooting angle of the depth camera, and the target detection model is a deep learning model improved based on sugarcane features; according to the sugarcane top image and the sugarcane bottom image, obtaining a first distance from the depth camera to the sugarcane top and a second distance from the depth camera to the sugarcane bottom from the depth camera; obtaining a target angle of the depth camera from the gimbal, the target angle being determined according to the first shooting angle for photographing the sugarcane top image and the second shooting angle for photographing the sugarcane bottom image; obtaining a target growth height of the target sugarcane according to the first distance, the second distance and the target angle;

[0006] The loss function of the target detection model is as follows:

[0007] (1)

[0008] L EIOU is the loss function of the target detection model, L IOU is the IOU loss function, L dis represents the direction loss function, L asp represents the aspect ratio loss function, IOU is the intersection-over-union ratio, which is used to measure the overlap between the sugarcane prediction box and the sugarcane real box, b is the coordinate of the sugarcane prediction box, and b gtis the coordinate of the sugarcane real box, ρ represents the distance between the sugarcane prediction box and the sugarcane real box, c is the minimum closure area, ω represents the width of the sugarcane coverage prediction box, ω gt represents the width of the sugarcane real box, h is the height of the sugarcane coverage prediction box, and h gt is the height of the actual frame of the sugarcane, c ω and c h They represent the width and height of the minimum bounding box of the intersection area between the sugarcane coverage prediction box and the sugarcane ground-truth box.

[0009] Another aspect of the present invention provides a sugarcane growth height measurement system, comprising: a single depth camera, used to photograph the target sugarcane, and generate the distance from the depth camera to the target sugarcane; a gimbal, used to carry the depth camera and control the shooting angle of the depth camera; a server, connected to the depth camera and the gimbal, used to execute the above-mentioned sugarcane growth height measurement method.

[0010] Another aspect of the present invention provides a device for measuring the growth height of sugarcane, comprising: an input module, for inputting a plurality of images obtained by photographing a target sugarcane with a single depth camera into a target detection model, to obtain a sugarcane bottom image and a sugarcane top image of the target sugarcane, the depth camera is mounted on a gimbal, the gimbal is used to control the shooting angle of the depth camera, the target detection model is a deep learning model improved based on sugarcane features, and the loss function of the target detection model is shown in formula (1). A first acquisition module, for acquiring a first distance from the depth camera to the sugarcane top and a second distance from the depth camera to the sugarcane bottom from the depth camera according to the sugarcane top image and the sugarcane bottom image. A second acquisition module, for acquiring a target angle of the depth camera from the gimbal, the target angle being determined according to the first shooting angle for photographing the sugarcane top image and the second shooting angle for photographing the sugarcane bottom image. A determination module, for obtaining a target growth height of the target sugarcane according to the first distance, the second distance and the target angle.

[0011] Another aspect of the present invention provides an electronic device, comprising:

[0012] one or more processors;

[0013] a memory for storing one or more programs,

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the above method.

[0015] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0016] Another aspect of the present invention provides a computer program product, the computer program product comprising computer executable instructions, and the instructions are used to implement the above method when executed.

[0017] According to an embodiment of the present invention, by only using a single depth camera to shoot the target sugarcane, obtaining a first distance from the depth camera to the top of the sugarcane, a second distance from the depth camera to the bottom of the sugarcane, and a target angle of rotation of the depth camera, the target growth height of the target sugarcane can be detected without considering issues such as coordinate conversion, thereby greatly reducing the amount of calculation and improving the efficiency of height measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0019] Figure 1 A system architecture diagram showing a method for measuring the growth height of sugarcane according to an embodiment of the present invention is applicable;

[0020] Figure 2 A diagram showing an application scenario of a system for measuring the growth height of sugarcane according to an embodiment of the present invention;

[0021] Figure 3 A flow chart showing a method for measuring the growth height of sugarcane according to an embodiment of the present invention is shown;

[0022] Figure 4A A schematic diagram showing a sugarcane image segmentation result according to an embodiment of the present invention is shown;

[0023] Figure 4B A schematic diagram showing the detection results of sugarcane according to an embodiment of the present invention;

[0024] Figure 4C A schematic diagram showing the detection result of the bottom of sugarcane according to an embodiment of the present invention;

[0025] Figure 5 A schematic diagram showing a method for measuring the growth height of sugarcane according to an embodiment of the present invention;

[0026] Figure 6 A schematic diagram showing a method for measuring the growth height of sugarcane based on dual cameras;

[0027] Figure 7 A schematic diagram showing a method for measuring the growth height of sugarcane according to another embodiment of the present invention;

[0028] Figure 8 A schematic diagram showing a method for measuring the growth height of sugarcane according to another embodiment of the present invention;

[0029] Fig. 9 A schematic diagram showing the sugarcane ridge spacing according to an embodiment of the present invention is shown;

[0030] Fig.10 A schematic diagram showing a method for measuring the growth height of sugarcane according to another embodiment of the present invention;

[0031] Fig.11 A schematic diagram showing a method for measuring the growth height of sugarcane according to yet another embodiment of the present invention;

[0032] Fig.12 A block diagram showing a device for measuring the growth height of sugarcane according to an embodiment of the present invention;

[0033] Fig.13 A block diagram of an electronic device suitable for implementing a method for measuring the growth height of sugarcane according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0034] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0035] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0036] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0037] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0038] In the embodiments of the present invention, the collection, updating, analysis, processing, use, transmission, provision, invention, storage and other aspects of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0039] In the embodiment of the present invention, before obtaining or collecting the user's personal information, the user's authorization or consent is obtained.

[0040] An embodiment of the present invention provides a method for measuring the growth height of sugarcane, comprising: inputting multiple images obtained by photographing target sugarcane with a single depth camera into a target detection model to obtain a sugarcane bottom image and a sugarcane top image of the target sugarcane, wherein the depth camera is mounted on a gimbal, and the gimbal is used to control the shooting angle of the depth camera; according to the sugarcane top image and the sugarcane bottom image, obtaining a first distance from the depth camera to the sugarcane top and a second distance from the depth camera to the sugarcane bottom from the depth camera; obtaining a target angle of the depth camera from the gimbal, the target angle being determined according to the first shooting angle for photographing the sugarcane top image and the second shooting angle for photographing the sugarcane bottom image; and obtaining a target growth height of the target sugarcane according to the first distance, the second distance and the target angle.

[0041] Figure 1 A system architecture diagram is shown in which a method for measuring the growth height of sugarcane according to an embodiment of the present invention can be applied.

[0042] It should be noted that Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present invention may be applied, to help those skilled in the art understand the technical content of the present invention, but do not mean that the embodiments of the present invention may not be used in other devices, systems, environments or scenarios.

[0043] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a depth camera 101, a gimbal 102, a mobile robot 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the depth camera 101, the gimbal 102, the mobile robot 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, and the like.

[0044] The user can use the depth camera 101 , the gimbal 102 , and the mobile robot 103 to interact with the server 105 via the network 104 to receive or send messages, etc.

[0045] The depth camera 101 is used to photograph the target sugarcane and generate the distance from the depth camera to the target sugarcane.

[0046] The gimbal 102 is used to carry a depth camera and control the shooting angle of the depth camera.

[0047] The server 105 is connected to the depth camera and the gimbal, and is used to input multiple images obtained by photographing the target sugarcane with a single depth camera into a target detection model to obtain a sugarcane bottom image and a sugarcane top image of the target sugarcane. The depth camera is mounted on the gimbal, and the gimbal is used to control the shooting angle of the depth camera; according to the sugarcane top image and the sugarcane bottom image, a first distance from the depth camera to the sugarcane top and a second distance from the depth camera to the sugarcane bottom are obtained from the depth camera; a target angle of the depth camera is obtained from the gimbal, and the target angle is determined according to the first shooting angle for photographing the sugarcane top image and the second shooting angle for photographing the sugarcane bottom image; and a target growth height of the target sugarcane is obtained according to the first distance, the second distance and the target angle.

[0048] The mobile robot 103 is connected to the server 105 and is used to carry the pan-tilt platform 102 to move the position of the depth camera and continue to detect another sugarcane. For example, the mobile robot 103 can carry the pan-tilt platform 102 and the server 105.

[0049] It should be noted that the sugarcane growth height measurement method provided in the embodiment of the present invention can generally be performed by the server 105. Accordingly, the sugarcane growth height measurement system provided in the embodiment of the present invention can generally be set in the server 105. The sugarcane growth height measurement method provided in the embodiment of the present invention can also be performed by a server or server cluster that is different from the server 105 and can communicate with the depth camera 101, the pan-tilt 102, the mobile robot 103 and / or the server 105. Accordingly, the sugarcane growth height measurement system provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the depth camera 101, the pan-tilt 102, the mobile robot 103 and / or the server 105. Alternatively, the sugarcane growth height measurement method provided in the embodiment of the present invention can also be performed by the depth camera 101, the pan-tilt 102, the mobile robot 103, or by other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103. Correspondingly, the sugarcane growth height measurement system provided in the embodiment of the present invention can also be set in the depth camera 101, the gimbal 102, the mobile robot 103, or in other terminal devices different from the depth camera 101, the gimbal 102, and the mobile robot 103.

[0050] According to an embodiment of the present invention, a system for measuring the growth height of sugarcane includes: a single depth camera, used to photograph target sugarcane and generate the distance from the depth camera to the target sugarcane; a gimbal, used to carry the depth camera and control the shooting angle of the depth camera; a server, connected to the depth camera and the gimbal, used to execute the sugarcane growth height measurement method of the embodiment of the present invention.

[0051] Figure 2 A diagram showing an application scenario of a system for measuring the growth height of sugarcane according to an embodiment of the present invention is shown.

[0052] like Figure 2 As shown, the mobile robot 103 is equipped with a gimbal 102, a controller 1021 of the gimbal 102, and a server 105. The gimbal 102 is equipped with a depth camera 101. The controller 1021 of the gimbal 102 can adjust the height of the gimbal 102 and control the rotation angle of the depth camera 101. The mobile robot 103 is used to move the position of the depth camera 101 to capture the main body of the sugarcane, thereby determining the target sugarcane to be measured.

[0053] For example, the mobile robot is selected as the mobile platform due to its small size, long battery life, large load, high precision, simple control and high cost performance. In addition, a 2-DOF servo gimbal is installed on the mobile platform, which can be stably used as a tool for carrying the depth camera used for detection. The depth camera is used as a sensing device for the sugarcane height characterization detection experiment, providing a hardware foundation for the subsequent automatic sugarcane height characterization measurement. The data image acquisition of this measurement system mainly relies on the mobile platform to continuously approach the sugarcane, and the rotation of the depth camera controlled by the gimbal to detect and measure the growth height data of the sugarcane.

[0054] Figure 3 A flow chart of a method for measuring the growth height of sugarcane according to an embodiment of the present invention is shown.

[0055] like Figure 3 As shown, the method for measuring the growth height of sugarcane includes operations S310 to S340.

[0056] In operation S310, a plurality of images obtained by photographing the target sugarcane using a single depth camera are input into a target detection model to obtain a sugarcane bottom image and a sugarcane top image of the target sugarcane.

[0057] In operation S320, a first distance from the depth camera to the top of the sugarcane and a second distance from the depth camera to the bottom of the sugarcane are acquired from the depth camera according to the sugarcane top image and the sugarcane bottom image.

[0058] In operation S330, a target angle of the depth camera is acquired from the gimbal, where the target angle is determined according to a first shooting angle for shooting an image of a sugarcane top and a second shooting angle for shooting an image of a sugarcane bottom.

[0059] In operation S340, a target growth height of a target sugarcane is obtained according to the first distance, the second distance, and the target angle.

[0060] According to an embodiment of the present invention, the depth camera is mounted on a gimbal, and the gimbal is used to control the shooting angle of the depth camera.

[0061] For example, the camera angle range of the depth camera can be 0 degrees to 180 degrees, and the depth camera at a fixed height can capture the bottom and top of the sugarcane.

[0062] For example, the camera angle range of the depth camera can be 0 to 120 degrees. When the distance between sugarcane ridges is large, the depth camera at a fixed height can be used to capture the bottom and top of the sugarcane. When the distance between sugarcane ridges is small, the height of the depth camera can be adjusted to capture the bottom and top of the sugarcane.

[0063] According to an embodiment of the present invention, the target detection model is a deep learning model improved based on sugarcane features. For example, the target detection model can be any deep detection model (the present invention adopts an improved EIOU-Loss function neural network model based on YOLOv5). For example, the detection model is trained using the collected sugarcane images to obtain the target detection model. Through training, the target detection model can accurately identify and locate the stem, top and bottom of the target sugarcane, and then calculate the target growth height of the target sugarcane.

[0064] For example, in order to make the prediction accuracy of the target detection model higher, the collected sugarcane image can be processed to extract sugarcane features. Denoising is performed using a deep learning-based method, and histogram equalization is used to enhance the sugarcane image and the histogram correction in the grayscale correction method. Histogram equalization is an image processing technique used to enhance image contrast. In the histogram equalization process, the grayscale histogram of the original sugarcane image is first calculated, and then the pixel values ​​in the sugarcane image are adjusted according to the cumulative distribution function of the histogram so that the pixel values ​​are evenly distributed. The grayscale range of the image is fully utilized, which enhances the details and visual effects of the image. The process of histogram correction is to perform a nonlinear transformation on the pixels of the original image, so that the brightness and contrast of the sugarcane image can be more flexibly enhanced. Since the research environment is a real sugarcane planting shed, there are complex situations such as weeds and uneven lighting. Therefore, after image processing, the image needs to be segmented. PSPNet (Pyramid Scene Parsing Network) is used as the baseline model for the sugarcane trunk segmentation experiment. The model structure mainly consists of a backbone feature extraction network and a pyramid pooling module. PSPNet uses MobileNetV2 (a lightweight convolutional neural network architecture) as the backbone network of the model, which reduces the model size and improves the accuracy of the model. The segmented sugarcane image facilitates the determination of the main position of the sugarcane and reduces environmental noise. The segmented sugarcane image is used as training data for the detection model. Image denoising, image enhancement, and image correction are used to improve the clarity of the sugarcane image, reduce noise, and facilitate computer processing.

[0065] For example, in order to cope with the challenge of sugarcane target detection in complex background environments, the improved Focal-EIOU Loss is used as the loss function of the detection model to replace the traditional bounding box regression loss, in order to accelerate convergence and improve the accuracy of the detection results. EIOU optimizes the loss function of CIOU and improves the calculation method of its directional loss. This improvement solves the problem that CIOU in conventional YOLOv5 may encounter when dealing with bounding boxes with inconsistent aspect ratios, making the loss function more accurately reflect the overlap between the predicted box and the real box. The calculation formula of EIOU Loss is shown in formula (1).

[0066] (1)

[0067] L EIOU is the EIOU loss function, which is used to measure the difference between the sugarcane prediction box and the sugarcane real box. IOU It is the IOU loss function, which is used to measure the difference between the intersection and union ratio between the sugarcane prediction box and the sugarcane real box. disL represents the directional loss function, which is used to measure the directional difference between the sugarcane prediction box and the sugarcane real box. asp represents the aspect ratio loss function, which is used to measure the difference in aspect ratio between the sugarcane prediction box and the sugarcane real box. IOU is the intersection over union, which is used to measure the degree of overlap between the sugarcane prediction box and the sugarcane real box. b is the coordinate of the sugarcane prediction box. b gt are the coordinates of the sugarcane ground truth box. ρ represents the distance between the sugarcane prediction box and the sugarcane ground truth box. c is the minimum closure area. ω represents the width of the sugarcane coverage prediction box. ω gt represents the width of the sugarcane real box. h represents the height of the sugarcane coverage prediction box. gt is the height of the actual frame of the sugarcane. c ω and c h They represent the width and height of the minimum bounding box of the intersection area between the sugarcane coverage prediction box and the sugarcane ground-truth box.

[0068] Using the Focal-EIOU Loss function instead of the traditional loss function can speed up the learning process of high-confidence targets, effectively improve the regression speed and positioning accuracy of the detection model, and make the target detection model more efficient and accurate when performing detection tasks.

[0069] The sugarcane top and sugarcane bottom are detected based on the deep learning target detection model improved by EIOU. Then, a first distance from the depth camera to the sugarcane top and a second distance from the depth camera to the sugarcane bottom are obtained from the aligned depth image.

[0070] Before measuring the height of sugarcane, it is necessary to segment the sugarcane image into the main trunk and background, and then label each segmented sugarcane body from left to right, and then encode and store the center coordinate information of each sugarcane. Then, the gimbal is rotated to align with the first sugarcane in the label, so that the center of the depth camera's view is aligned with the sugarcane body, and automatic alignment with the sugarcane body is achieved.

[0071] Figure 4A A schematic diagram showing a sugarcane image segmentation result according to an embodiment of the present invention is shown.

[0072] Figure 4A Figure (a) is an image of the target sugarcane taken with a single depth camera, and Figure 4A (b) is the image segmentation result of the target sugarcane. The segmentation result can be used to calculate the inclination of the target sugarcane, the distance between the ridges, etc.

[0073] Figure 4B A schematic diagram showing the detection results of sugarcane according to an embodiment of the present invention.

[0074] Figure 4BFigure (a) is an image of the target sugarcane captured by a single depth camera. The image is input into the target detection model to obtain the detection result of the sugarcane, as shown in Figure 2. Figure 4B As shown in Figure (b). Among them, Figure 4B In the figure (b), "sugarcane 0.7", "sugarcane 0.8", and "sugarcane 0.9" represent the numbers of each sugarcane, and the red box is the detected position of the sugarcane.

[0075] Figure 4C A schematic diagram showing the detection result of the bottom of sugarcane according to an embodiment of the present invention.

[0076] Figure 4C Figure (a) in the figure is a sugarcane bottom image obtained by shooting the target sugarcane with a single depth camera. The sugarcane bottom image is input into the target detection model to obtain the detection result of the sugarcane bottom, as shown in Figure 4C As shown in Figure (b). Among them, Figure 4C In the figure (b), "sugarcane root 0.8" indicates the root of sugarcane numbered 0.8, and "sugarcane root 0.9" indicates the root of sugarcane numbered 0.9.

[0077] like Figure 4A , 4B and Figure 4C As shown in the figure, PSPNet is used as the baseline model for the sugarcane trunk segmentation experiment in the sugarcane image processing process. The baseline model structure mainly consists of a backbone feature extraction network and a pyramid pooling module. The former uses MobileNetV2 as the backbone network of the model, which reduces the model size and improves the accuracy of the model. CBAM is introduced in the backbone feature extraction network to suppress useless feature extraction and enhance the focus on important features. The segmented image is used as the input of the network model for training.

[0078] The identification of the top or growth point of sugarcane is achieved through deep learning image processing and analysis methods, especially the training and optimization of the collected sugarcane images using the improved YOLOv5 neural network model. The YOLOv5 neural network model introduces the Ghost-C3 module, which further reduces the number of parameters and computational complexity without reducing the number of features that affect the accuracy of model detection. Therefore, it significantly improves the reasoning speed of the model, thereby improving the work efficiency of sugarcane top and bottom detection. Through training, the model can accurately identify and locate the top and bottom of sugarcane, while the detection of sugarcane stems is based on the collected sugarcane image data, processed using the YOLOv5obb network, and determined by determining the horizontal coordinate and inclination information of the sugarcane body. This embodiment detects and segments sugarcane in a sugarcane plantation.

[0079] The height of sugarcane of different heights and varieties was measured manually and by non-contact sugarcane height measurement based on deep learning, and the errors between the two and the error rate of non-manual detection were calculated. A total of 24 experiments were conducted, divided into 8 groups, each of which was measured twice using a neural network-based method, and the distance from the measurement position to the sugarcane was different each time. The error rate of the non-contact sugarcane height measurement method based on deep learning was ≤6%, the detection error was small, and the effect was quite good. The use of machines to automatically measure plant height can greatly improve efficiency and accuracy compared to manual measurement. Advances in this technology are expected to revolutionize sugarcane height measurement methods and make an important contribution to increasing sugar yields. The ability to accurately measure plant height can help monitor plant growth patterns, optimize cultivation practices, and identify high-yield sugarcane varieties.

[0080] The image captured by the depth camera is input into the target detection model, and the detection result of the image can be output. The measurement method based on deep learning and image processing is objective and consistent, avoiding the influence of human factors in traditional measurement methods. By training and optimizing the improved deep learning model, the detection accuracy and stability of sugarcane stems and tops are improved, thereby ensuring the accuracy of plant height measurement. Through the automated measurement process, the measurement efficiency of sugarcane plant height is greatly improved, and the workload and time cost of manual measurement are reduced.

[0081] For example, the target angle may be an angle at which the depth camera rotates from a first shooting angle to a second shooting angle.

[0082] For example, the target angle may include an angle at which the depth camera is rotated from the first shooting angle to the level angle, and an angle at which the depth camera is rotated to the level angle.

[0083] For example, the target angle may be any angle value of the depth camera rotating from the first shooting angle to the second shooting angle.

[0084] According to an embodiment of the present invention, the first distance, the second distance, the target angle and the target sugarcane growth height may form at least one geometric figure, and the target growth height of the target sugarcane may be calculated based on the first distance, the second distance and the target angle based on relevant geometric principles.

[0085] According to an embodiment of the present invention, by only using a single depth camera to shoot the target sugarcane, and then obtaining a first distance from the depth camera to the top of the sugarcane, a second distance from the depth camera to the bottom of the sugarcane, and a target angle of the depth camera, the target growth height of the target sugarcane can be detected without considering issues such as coordinate conversion, thereby greatly reducing the amount of calculation and improving the efficiency of height measurement.

[0086] Figure 5A schematic diagram showing a method for measuring the growth height of sugarcane according to an embodiment of the present invention is shown.

[0087] When the sugarcane tilt angle is large, the depth camera automatically outputs the first distance d1 and the second distance d2. After automatically aiming at the target sugarcane body, the camera is rotated downward, and the camera is stopped after the bottom of the sugarcane is detected. The distance d2 from the bottom of the sugarcane to the camera is obtained through the depth camera. The camera is rotated upward again, and the camera is stopped when the top of the sugarcane appears in the depth camera. The distance d1 from the top of the sugarcane to the depth camera is obtained through the depth camera, and the gimbal automatically obtains the angle α from the bottom to the top.

[0088] like Figure 5 As shown, the first shooting angle is a1, the second shooting angle is a2, the target angle is the difference between the first shooting angle and the second shooting angle, and the target angle is α.

[0089] According to an embodiment of the present invention, a formula for obtaining a target growth height of a target sugarcane according to the first distance, the second distance and the target angle is as follows:

[0090] (2)

[0091] h represents the target growth height of the target sugarcane, d1 represents the first distance, d2 represents the second distance, and α represents the target angle.

[0092] When working in the field, the uneven road surface will cause the camera to tilt, increasing the error. Since the distance from the sugarcane growth point to the top of the sugarcane is essentially measured, it will not be affected by the tilt of the camera. This method can effectively reduce the error caused by the tilt of the camera due to the uneven field roads when the robot is working. Therefore, even if the sugarcane is tilted or the road surface is uneven, the target growth height of the target sugarcane can be accurately measured.

[0093] Figure 6 A schematic diagram showing a method for measuring sugarcane growth height based on dual cameras.

[0094] like Figure 6 As shown in the figure, the distance between the two cameras is h0, the distance between the upper camera and the top of the sugarcane is d1, and the distance between the lower camera and the bottom of the sugarcane is d2. The binocular camera needs to convert the distance according to the conversion ratio and calibrate the two cameras. When the inclination angle of the plant is unknown, the plant growth height is usually calculated as the projection height h', rather than the actual plant growth height h.

[0095] In the embodiment of the present invention, even if the sugarcane is tilted, only a single depth camera is used to obtain the distance from the depth camera to the bottom and the top of the sugarcane, and the angle of rotation of the depth camera in the process of obtaining the distance from the bottom of the sugarcane to the top of the sugarcane is obtained from the gimbal. Subsequently, the actual growth height of the sugarcane is directly calculated by formula (2), without involving the calibration and coordinate conversion problems between multiple cameras, and the amount of calculation is small, which can improve the measurement efficiency of the growth height of the sugarcane.

[0096] According to an embodiment of the present invention, when the inclination angle of the target sugarcane is less than a preset threshold, the target angle includes a preset shooting angle, which is an angle value between the first shooting angle and the second shooting angle.

[0097] According to an embodiment of the present invention, when the depth camera captures the main body of the sugarcane, the image of the main body of the sugarcane may be first segmented to determine the tilt angle of the main body of the sugarcane.

[0098] For example, deep learning-based methods are used for denoising, histogram equalization is used to enhance the image, and histogram correction is used in the grayscale correction method. PSPNet (Pyramid Scene Parsing Network) is used as the baseline model for the sugarcane trunk segmentation experiment. The model structure mainly consists of a backbone feature extraction network and a pyramid pooling module. PSPNet uses MobileNetV2 (a lightweight convolutional neural network architecture) as the backbone network of the model, which reduces the model size while improving the accuracy of the model. The segmented sugarcane image facilitates the determination of the main body position of the sugarcane and the tilt angle of the sugarcane body, and reduces the environmental noise in the sugarcane image. The segmented sugarcane image is used as the input of the model for training.

[0099] According to an embodiment of the present invention, the preset threshold value may be set according to actual conditions. For example, when the tilt angle is less than 2 degrees, the target angle includes a horizontal direction angle.

[0100] The camera angle of the depth camera can be 0 degrees to 180 degrees. The depth camera at a fixed height can capture the bottom and top of the sugarcane.

[0101] According to an embodiment of the present invention, the preset shooting angle represents an angle in the horizontal direction between the first shooting angle and the second shooting angle.

[0102] According to an embodiment of the present invention, obtaining a target growth height of a target sugarcane according to a first distance, a second distance and a target angle includes: calculating a first growth height of the target sugarcane according to the first distance and a third distance, the third distance being obtained from a depth camera according to a preset shooting angle; calculating a second growth height of the target sugarcane according to the second distance and the third distance; and determining the target growth height of the target sugarcane according to the first growth height and the second growth height.

[0103] After the depth camera is automatically aligned with the target sugarcane body in the horizontal direction, the third distance d3 from the target sugarcane body to the depth camera is obtained through the depth camera, the depth camera is rotated downward, and the rotation of the depth camera is stopped after the bottom of the sugarcane is detected. The second distance d2 from the bottom of the sugarcane to the depth camera is obtained through the depth camera, and the second growth height h2 from the depth camera to the bottom of the sugarcane can be calculated.

[0104] After rotating the depth camera to a horizontal position, continue to rotate the depth camera upward until the top of the sugarcane is detected. The first distance d1 from the top of the sugarcane to the depth camera is obtained through the depth camera, and the first growth height h1 from the depth camera to the bottom of the sugarcane can be calculated.

[0105] Figure 7 A schematic diagram showing a method for measuring the growth height of sugarcane according to another embodiment of the present invention.

[0106] like Figure 7 As shown, the preset shooting angle β is the horizontal angle between the first shooting angle and the second shooting angle. A third distance d3 corresponding to the preset shooting angle is acquired from the depth camera.

[0107] The formula for the first growth height h1 is as follows:

[0108] (3)

[0109] The formula for the second growth height h2 is as follows:

[0110] (4)

[0111] The target growth height h of the target sugarcane is the sum of the first growth height h1 and the second growth height h2.

[0112] According to the embodiment of the present invention, when the sugarcane is not tilted, the height of the sugarcane can be detected by only obtaining the distances from the depth camera to the main body of the sugarcane, the bottom of the sugarcane and the top of the sugarcane from a single depth camera, which not only does not involve the ratio conversion problem between multiple cameras, but also does not need to obtain angle information from the gimbal. In this embodiment, the height of the sugarcane can be detected by only obtaining three distance information from the depth camera, the equipment deployment is simple, the calculation amount is small, and the measurement efficiency of the height of the sugarcane is improved.

[0113] According to an embodiment of the present invention, the above method also includes: when the pixel value of the sugarcane bottom image does not meet the pixel threshold, the height of the depth camera from the ground is used as the second growth height, and the difference between the preset shooting angle and the first shooting angle is used as the target angle; based on the first distance, the third distance and the target angle, the first growth height of the target sugarcane is calculated; based on the first growth height and the second growth height, the target growth height of the target sugarcane is determined.

[0114] According to an embodiment of the present invention, the pixel threshold that is not met may be lower than 50. The lower the pixel value, the darker the light at the bottom of the sugarcane, making it difficult to detect the bottom of the sugarcane using the target detection model.

[0115] First, measure the height of the depth camera from the ground, which is the second growth height h2. After automatically aligning the main body of the sugarcane, the camera looks straight at the gimbal and rotates the depth camera upward. When the top of the sugarcane appears in the depth camera, stop rotating the camera. The gimbal can automatically obtain the angle α from the level view to the top. When the sugarcane is not tilted, the height h of the sugarcane is calculated by the formula h=d1sinα+h2.

[0116] Figure 8 A schematic diagram showing a method for measuring the growth height of sugarcane according to yet another embodiment of the present invention.

[0117] like Figure 8 As shown, the height of the depth camera from the ground is taken as the second growth height h2. The difference between the preset shooting angle and the first shooting angle is taken as the target angle α. A third distance d3 corresponding to the preset shooting angle is acquired from the depth camera.

[0118] In one embodiment, when the sugarcane is not tilted, the first growth height of the target sugarcane can be calculated using formula (3).

[0119] The target growth height h of the target sugarcane is the sum of the first growth height h1 and the second growth height h2.

[0120] It should be noted that, when the sugarcane is not tilted and the light at the bottom of the sugarcane is dark, the error in measuring the growth height of the sugarcane in this embodiment is very small.

[0121] According to an embodiment of the present invention, in order to minimize the measurement error of the sugarcane growth height, the height of the depth camera can be lowered, that is, the height of the depth camera from the ground is used as the second growth height, which produces a smaller height error.

[0122] In another embodiment, the difference between the preset shooting angle and the first shooting angle is used as the target angle α. A third distance d3 corresponding to the preset shooting angle is acquired from the depth camera. The first growth height h1 can be calculated using formula (5).

[0123] (5);

[0124] Then, based on the cosine theorem, the target angle γ is calculated according to the first growth height h1, the first distance d1 and the third distance d3. Based on the Pythagorean theorem, the second growth height h can be calculated according to the height h2 of the depth camera from the ground and the target angle γ. 21 The target growth height h of the target sugarcane is the first growth height h1 and the second growth height h 21 sum.

[0125] The method of this embodiment can calculate the actual growth height of the sugarcane according to the target angle, the first distance, the third distance and the height of the depth camera from the ground, even if the sugarcane is tilted and the light at the bottom of the sugarcane is dark, without considering lowering the height of the depth camera, thereby improving the measurement accuracy.

[0126] According to an embodiment of the present invention, the method further includes: in a case where the multiple images include any one of the sugarcane bottom image and the sugarcane top image, adjusting the height of the depth camera to obtain the sugarcane bottom image and the sugarcane top image; acquiring the adjusted height, the third shooting angle and the fourth shooting angle of the depth camera from the gimbal, the third shooting angle being the difference between the preset shooting angle and the first shooting angle, and the fourth shooting angle being the difference between the preset shooting angle and the second shooting angle; and calculating the target growth height of the target sugarcane according to the first distance, the second distance, the adjusted height, the third shooting angle and the fourth shooting angle.

[0127] According to an embodiment of the present invention, the following situations may cause any one of the sugarcane bottom image and the sugarcane top image to be included in multiple images. One is that the distance between sugarcane ridges is small and the rotatable angle of the depth camera is less than 180 degrees, and the other is that the light at the bottom of the sugarcane is dark.

[0128] Fig. 9 A schematic diagram showing the sugarcane ridge spacing according to an embodiment of the present invention is shown.

[0129] like Fig. 9 As shown, the distance between two rows of sugarcane is taken as the ridge distance. When the ridge distance is small and the rotation angle of the depth camera from the ground to the sky cannot reach 180 degrees, a single depth camera cannot capture the bottom and top of the sugarcane at a fixed height.

[0130] For example, the specific height of the sugarcane cannot be determined, and the depth camera can only measure an angle range of 120°. If the height of the sugarcane is too high, the measurement may not be completed.

[0131] Fig.10 A schematic diagram showing a method for measuring the growth height of sugarcane according to another embodiment of the present invention is shown.

[0132] like Fig.10 As shown, the third shooting angle a3, the fourth shooting angle a4, the first distance d1', the second distance d2' and the adjustment height l.

[0133] The fourth distance d4' corresponding to the third shooting angle a3 and the fifth distance d5' corresponding to the fourth shooting angle a4 are obtained from the depth camera. According to the third shooting angle a3, the first distance d1' and the fourth distance d4', the first growth height h1' of the target sugarcane is calculated, and the calculation formula is the same as formula (2).

[0134] According to the fourth shooting angle a4, the second distance d2' and the fifth distance d5', the second growth height h2' of the target sugarcane is calculated, and the calculation formula is the same as formula (2).

[0135] Based on the Pythagorean theorem, the third growth height h3' of the target sugarcane is calculated according to the distance difference between the fourth distance d4' and the fifth distance d5' and the adjustment height l. The target growth height of the target sugarcane is determined according to the first growth height h1', the second growth height h2' and the third growth height h3'.

[0136] For example, the target growth height of the target sugarcane is the sum of the first growth height h1 ′, the second growth height h2 ′, and the third growth height h3 ′.

[0137] According to an embodiment of the present invention, even when the distance between sugarcane ridges is small or the light at the bottom of the sugarcane is dark, so that a single depth camera cannot capture the bottom and top at a fixed height, the growth height of the tilted sugarcane can be accurately measured by adjusting the height of the depth camera.

[0138] According to an embodiment of the present invention, the above method also includes: when the inclination angle of the target sugarcane is less than a preset threshold and the multiple images include any one of the sugarcane bottom image and the sugarcane top image, obtaining a third distance corresponding to a preset shooting angle from the depth camera; calculating the first growth height of the target sugarcane based on the first distance and the third distance; calculating the second growth height of the target sugarcane based on the second distance and the third distance; and obtaining the target growth height of the target sugarcane based on the adjusted height, the first growth height, and the second growth height.

[0139] According to an embodiment of the present invention, the inclination angle of the target sugarcane being smaller than a preset threshold value indicates that the target sugarcane is not inclined.

[0140] Fig.11 A schematic diagram showing a method for measuring the growth height of sugarcane according to yet another embodiment of the present invention.

[0141] After the depth camera automatically aligns with the target sugarcane body, the distance d3' from the target sugarcane body to the camera is obtained through the depth camera.

[0142] After the bottom of the sugarcane is detected, the camera rotation stops, and the second distance d2' from the bottom of the sugarcane to the camera is obtained through the depth camera. The gimbal can automatically obtain the downward rotation angle of the depth camera. The camera is rotated toward the sky. When the end angle of the depth camera rotation is reached, the top of the sugarcane has not been detected, and the rotation angle of the depth camera is returned to the horizontal angle.

[0143] After the gimbal is raised by 1 meter, the camera rotates upward until the top of the sugarcane appears in the camera's field of view and stops rotating. The first distance d1' from the top of the sugarcane to the depth camera is obtained through the depth camera, and the gimbal can automatically obtain the upward rotation angle of the camera.

[0144] like Fig.11 As shown, the third distance d3' corresponding to the preset shooting angle, the first distance d1', the second distance d2' and the adjusted height l of the depth camera. Based on the Pythagorean theorem, the first growth height h1' of the target sugar cane is calculated according to the first distance d1' and the third distance d3'. Based on the Pythagorean theorem, the second growth height h2' of the target sugar cane is calculated according to the second distance d2' and the third distance d3'. The sum of the adjusted height l of the depth camera, the first growth height h1 and the second growth height h2' is taken as the target growth height of the target sugar cane.

[0145] According to the embodiments of the present invention, the growth height of sugarcane can be detected by only obtaining the distances from the depth camera to the main body of sugarcane, the bottom of sugarcane and the top of sugarcane and the adjusted height of the depth camera from a single depth camera, which not only does not involve the ratio conversion problem between multiple cameras, but also does not need to obtain angle information from the gimbal. Even when the ridge distance is small, fast and accurate sugarcane measurement can be achieved.

[0146] Fig.12 A block diagram showing a device for measuring the growth height of sugarcane according to an embodiment of the present invention is shown.

[0147] like Fig.12 As shown, the device 1200 for measuring the growth height of sugarcane includes an input module 1210 , a first acquisition module 1220 , a second acquisition module 1230 and a first determination module 1240 .

[0148] The input module 1210 is used to input multiple images obtained by shooting the target sugarcane with a single depth camera into the target detection model to obtain the sugarcane bottom image and the sugarcane top image of the target sugarcane. The depth camera is mounted on a gimbal, and the gimbal is used to control the shooting angle of the depth camera. In one embodiment, the input module 1210 can be used to perform the operation S310 described above, which will not be repeated here.

[0149] The first acquisition module 1220 is used to acquire a first distance from the depth camera to the top of the sugarcane and a second distance from the depth camera to the bottom of the sugarcane from the depth camera according to the image of the top of the sugarcane and the image of the bottom of the sugarcane. In one embodiment, the first acquisition module 1220 can be used to perform the operation S320 described above, which will not be repeated here.

[0150] The second acquisition module 1230 is used to acquire the target angle of the depth camera from the gimbal, and the target angle is determined according to the first shooting angle for shooting the image of the top of the sugarcane and the second shooting angle for shooting the image of the bottom of the sugarcane. In one embodiment, the second acquisition module 1230 can be used to perform the operation S330 described above, which will not be repeated here.

[0151] The first determination module 1240 is used to obtain the target growth height of the target sugarcane according to the first distance, the second distance and the target angle. In one embodiment, the first determination module 1240 can be used to perform the operation S340 described above, which will not be described in detail here.

[0152] According to an embodiment of the present invention, the target angle is the difference between the first shooting angle and the second shooting angle; based on the first distance, the second distance and the target angle, a formula for obtaining the target growth height of the target sugarcane is as shown in formula (2).

[0153] According to an embodiment of the present invention, when the inclination angle of the target sugarcane is less than a preset threshold, the target angle includes a preset shooting angle, which is an angle value between the first shooting angle and the second shooting angle.

[0154] According to an embodiment of the present invention, the determination module 1240 includes a first calculation submodule, a second calculation submodule and a first determination submodule. The first calculation submodule is used to calculate the first growth height of the target sugar cane according to the first distance and the third distance, and the third distance is obtained from the depth camera according to a preset shooting angle. The second calculation submodule is used to calculate the second growth height of the target sugar cane according to the second distance and the third distance. The first determination submodule is used to determine the target growth height of the target sugar cane according to the first growth height and the second growth height.

[0155] According to an embodiment of the present invention, the formula for the first growth height h1 is as shown in Formula (3). The formula for the second growth height h2 is as shown in Formula (4).

[0156] According to an embodiment of the present invention, the device further includes a second determination module, a third calculation module and a third determination module. The second determination module is used to use the height of the depth camera from the ground as the second growth height and the difference between the preset shooting angle and the first shooting angle as the target angle when the pixel value of the sugarcane bottom image does not meet the pixel threshold; the third calculation module is used to calculate the first growth height of the target sugarcane according to the first distance, the third distance and the target angle; the third determination module is used to determine the target growth height of the target sugarcane according to the first growth height and the second growth height.

[0157] According to an embodiment of the present invention, the above-mentioned device further includes an adjustment module, a third acquisition module and a fourth calculation module. The adjustment module is used to obtain the sugarcane bottom image and the sugarcane top image by adjusting the height of the depth camera when the multiple images include any one of the sugarcane bottom image and the sugarcane top image; the third acquisition module is used to obtain the adjusted height, the third shooting angle and the fourth shooting angle of the depth camera from the gimbal, the third shooting angle is the difference between the preset shooting angle and the first shooting angle, and the fourth shooting angle is the difference between the preset shooting angle and the second shooting angle; the fourth calculation module is used to calculate the target growth height of the target sugarcane according to the first distance, the second distance, the adjusted height, the third shooting angle and the fourth shooting angle.

[0158] According to an embodiment of the present invention, the fourth calculation module includes a first acquisition submodule, a third calculation submodule, a fourth calculation submodule, a fifth calculation submodule, and a second determination submodule. The first acquisition submodule is used to acquire a fourth distance corresponding to a third shooting angle and a fifth distance corresponding to a fourth shooting angle from a depth camera; the third calculation submodule is used to calculate a first growth height of a target sugar cane according to the third shooting angle, the first distance, and the fourth distance; the fourth calculation submodule is used to calculate a second growth height of a target sugar cane according to the fourth shooting angle, the second distance, and the fifth distance; the fifth calculation submodule is used to calculate a third growth height of a target sugar cane according to a distance difference between the fourth distance and the fifth distance and an adjusted height; the second determination submodule is used to determine a target growth height of a target sugar cane according to the first growth height, the second growth height, and the third growth height.

[0159] According to an embodiment of the present invention, the device further includes a fourth acquisition module, a fifth calculation module, a sixth calculation module and a fourth determination module. The fourth acquisition module is used to acquire a third distance corresponding to a preset shooting angle from the depth camera when the inclination angle of the target sugarcane is less than a preset threshold and the multiple images include any one of the sugarcane bottom image and the sugarcane top image; the fifth calculation module is used to calculate the first growth height of the target sugarcane according to the first distance and the third distance; the sixth calculation module is used to calculate the second growth height of the target sugarcane according to the second distance and the third distance; the fourth determination module is used to obtain the target growth height of the target sugarcane according to the adjustment height, the first growth height and the second growth height.

[0160] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a proper combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be executed.

[0161] For example, any multiple of the input module 1210, the first acquisition module 1220, the second acquisition module 1230, and the first determination module 1240 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present invention, at least one of the input module 1210, the first acquisition module 1220, the second acquisition module 1230, and the first determination module 1240 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the input module 1210 , the first acquisition module 1220 , the second acquisition module 1230 , and the first determination module 1240 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0162] It should be noted that the measuring device for the sugarcane growth height in the embodiment of the present invention corresponds to the measuring method for the sugarcane growth height in the embodiment of the present invention. The description of the measuring device for the sugarcane growth height refers to the measuring method for the sugarcane growth height, which will not be repeated here.

[0163] Fig.13 A block diagram of an electronic device suitable for implementing a method for measuring the growth height of sugarcane according to an embodiment of the present invention is shown.

[0164] Fig.13 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0165] like Fig.13As shown, the electronic device 1300 according to an embodiment of the present invention includes a processor 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage part 1308 to a random access memory (RAM) 1303. The processor 1301 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1301 may also include an onboard memory for caching purposes. The processor 1301 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0166] In RAM 1303, various programs and data required for the operation of electronic device 1300 are stored. Processor 1301, ROM 1302 and RAM 1303 are connected to each other via bus 1304. Processor 1301 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 1302 and / or RAM 1303. It should be noted that the program can also be stored in one or more memories other than ROM 1302 and RAM 1303. Processor 1301 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0167] According to an embodiment of the present invention, the electronic device 1300 may further include an input / output (I / O) interface 1305, which is also connected to the bus 1304. The electronic device 1300 may further include one or more of the following components connected to the input / output (I / O) interface 1305: an input portion 1306 including a keyboard, a mouse, etc.; an output portion 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1308 including a hard disk, etc.; and a communication portion 1309 including a network interface card such as a LAN card, a modem, etc. The communication portion 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the input / output (I / O) interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed, so that a computer program read therefrom is installed into the storage portion 1308 as needed.

[0168] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1309, and / or installed from the removable medium 1311. When the computer program is executed by the processor 1301, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0169] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0170] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0171] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1302 and / or the RAM 1303 described above and / or one or more memories other than the ROM 1302 and the RAM 1303 .

[0172] An embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method for measuring the growth height of sugarcane provided by the embodiment of the present invention.

[0173] When the computer program is executed by the processor 1301, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0174] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1309, and / or installed from the removable medium 1311. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0175] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0176] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All these combinations and / or combinations fall within the scope of the present invention.

[0177] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for measuring the growth height of sugarcane, characterized in that: The method comprises: A plurality of images obtained by shooting a target sugarcane with a single depth camera are input into a target detection model to obtain a sugarcane bottom image and a sugarcane top image of the target sugarcane, wherein the depth camera is mounted on a gimbal, and the gimbal is used to control the shooting angle of the depth camera, and the target detection model is based on a deep learning model improved according to the characteristics of sugarcane; According to the sugarcane top image and the sugarcane bottom image, acquiring from the depth camera a first distance from the depth camera to the sugarcane top and a second distance from the depth camera to the sugarcane bottom; Acquire a target angle of the depth camera from the pan / tilt, wherein the target angle is determined according to a first shooting angle for shooting the image of the sugarcane top and a second shooting angle for shooting the image of the sugarcane bottom; Obtaining a target growth height of the target sugarcane according to the first distance, the second distance and the target angle; When the pixel value of the sugarcane bottom image does not meet the pixel threshold, the height of the depth camera from the ground is used as the second growth height, and the difference between a preset shooting angle and the first shooting angle is used as the target angle, wherein the preset shooting angle is an angle value between the first shooting angle and the second shooting angle; Calculating a first growth height of the target sugarcane according to the first distance, the third distance and the target angle; determining the target growth height of the target sugarcane according to the first growth height and the second growth height; In a case where the plurality of images include any one of the sugarcane bottom image and the sugarcane top image, the sugarcane bottom image and the sugarcane top image are obtained by adjusting the height of the depth camera; Acquire an adjusted height, a third shooting angle, and a fourth shooting angle of the depth camera from the gimbal, wherein the third shooting angle is a difference between the preset shooting angle and the first shooting angle, and the fourth shooting angle is a difference between the preset shooting angle and the second shooting angle; Calculate the target growth height of the target sugarcane according to the first distance, the second distance, the adjusted height, the third shooting angle, and the fourth shooting angle; The loss function of the target detection model is as follows: （1） L EIOU is the loss function of the target detection model, L IOU is the IOU loss function, L dis represents the direction loss function, L asp represents the aspect ratio loss function, IOU is the intersection-over-union ratio, which is used to measure the overlap between the sugarcane prediction box and the sugarcane real box, b is the coordinate of the sugarcane prediction box, and b gt is the coordinate of the sugarcane real frame, ρ represents the distance between the sugarcane prediction frame and the sugarcane real frame, c is the minimum closure area, ω represents the width of the sugarcane coverage prediction frame, ω gt represents the width of the sugarcane real frame, h is the height of the sugarcane coverage prediction frame, and h gt is the height of the actual frame of the sugarcane, c ω and c h Respectively represent the width and height of the minimum bounding box of the sugarcane coverage prediction box and the actual intersection area of ​​the sugarcane.

2. The method according to claim 1, characterized in that: The target angle is the difference between the first shooting angle and the second shooting angle; The formula for obtaining the target growth height of the target sugarcane according to the first distance, the second distance and the target angle is as follows: （2） h represents the target growth height of the target sugarcane, d1 represents the first distance, d2 represents the second distance, and α represents the target angle.

3. The method according to claim 1, characterized in that: In a case where the tilt angle of the target sugarcane is less than a preset threshold, the target angle includes a preset shooting angle.

4. The method according to claim 3, characterized in that: The step of obtaining the target growth height of the target sugarcane according to the first distance, the second distance and the target angle comprises: A first growth height of the target sugarcane is calculated according to the first distance and the third distance, wherein the third distance is obtained from the depth camera according to the preset shooting angle; calculating a second growth height of the target sugarcane according to the second distance and the third distance; The target growth height of the target sugarcane is determined according to the first growth height and the second growth height.

5. The method according to claim 4, characterized in that The formula of the first growth height h1 is as follows: （3） The formula of the second growth height h2 is as follows: （4） d1 represents the first distance, d2 represents the second distance, and d3 represents the third distance.

6. The method according to claim 1, characterized in that The step of calculating the target growth height of the target sugarcane according to the first distance, the second distance, the adjusted height, the third shooting angle, and the fourth shooting angle includes: Acquire, from the depth camera, a fourth distance corresponding to the third shooting angle and a fifth distance corresponding to the fourth shooting angle; calculating a first growth height of the target sugarcane according to the third shooting angle, the first distance and the fourth distance; calculating a second growth height of the target sugarcane according to the fourth shooting angle, the second distance and the fifth distance; calculating a third growth height of the target sugarcane according to a distance difference between the fourth distance and the fifth distance and the adjusted height; The target growth height of the target sugarcane is determined according to the first growth height, the second growth height and the third growth height.

7. The method according to claim 1, characterized in that The method further comprises: When the inclination angle of the target sugarcane is less than a preset threshold and the multiple images include any one of the sugarcane bottom image and the sugarcane top image, acquiring a third distance corresponding to the preset shooting angle from the depth camera; Calculating a first growth height of the target sugarcane according to the first distance and the third distance; calculating a second growth height of the target sugarcane according to the second distance and the third distance; The target growth height of the target sugarcane is obtained according to the adjusted height, the first growth height and the second growth height.

8. A sugarcane growth height measurement system, characterized in that: The system comprises: A single depth camera, used for photographing target sugarcane and generating a distance from the depth camera to the target sugarcane; A gimbal, used to carry the depth camera and control the shooting angle of the depth camera; A server, connected to the depth camera and the gimbal, for executing the method according to any one of claims 1 to 7.

Images