A method for measuring corn ear height based on target detection

Abstract

The present invention discloses a method for measuring the height of corn ear based on target detection, constructing a corn ear detection model; collecting real-time images and position and posture information of sensors, inputting the pre-processed real-time images into the corn ear detection model, extracting the corn ear image features, performing target detection on the pre-processed real-time images through the corn ear image features to obtain the position of the corn ear in the image; establishing a three-dimensional point cloud data set of the environment based on the real-time image, respectively obtaining the three-dimensional coordinates of the ground and the ear, and calculating the corn ear height measurement information. The present invention realizes accurate identification of corn ears through a deep learning method, constructs a three-dimensional map in combination with SLAM technology, and realizes automated and intelligent measurement of corn ear positions.

Description

Technical Field

[0001] The invention relates to a plant phenotype measurement and processing technology, and in particular to a corn ear height measurement method based on target detection. Background Art

[0002] Appropriate maize ear height can improve the plant's nutrient and light energy utilization efficiency and lodging resistance, which is of great significance to increasing and stabilizing maize yields. The rapid development of Simultaneous Localization And Mapping (SLAM) and deep learning-based target detection technology has made intelligent maize ear spatial positioning and height measurement possible.

[0003] The measurement of corn ear height is mainly done manually with a ruler, which is slow and labor-intensive, and can no longer meet the needs of agricultural production and intelligent breeding. In recent years, some scholars have obtained the vertical point cloud distribution of the corn canopy based on the relationship between the vertical distribution of corn leaf area and ear height, and further estimated the vertical distribution of corn leaf area and ear height based on the unmanned aerial vehicle laser radar (LiDAR). However, the corn canopy is tall and closed after the tasseling period. To obtain high-quality point cloud vertical distribution information, a high-performance LiDAR sensor is required, which is expensive. For crop breeding, accurate phenotype acquisition is a prerequisite for gene mining. However, in the face of a large amount of corn germplasm resources, the reliability and stability of the relationship between the vertical distribution of corn leaf area and ear height are questionable. Therefore, a method for measuring the height of corn female ears based on target detection is needed. Summary of the invention

[0004] The purpose of the present invention is to provide a method for measuring the height of a corn ear based on target detection.

[0005] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0006] The present invention comprises the following steps:

[0007] A trains the target detection model weights and builds a corn ear detection model;

[0008] B collects real-time images and sensor position and posture information;

[0009] C. inputting the preprocessed real-time image into the corn ear detection model, extracting the corn ear image features, and performing target detection on the preprocessed real-time image through the corn ear image features to obtain the coordinates of the corn ear represented in the image pixel coordinate system;

[0010] D recognizes the real-time image, obtains the three-dimensional spatial coordinates of the ground and the female ear in the world coordinate system, and obtains the measurement information of the height of the corn female ear.

[0011] Furthermore, the preprocessing method includes rotating the real-time image and stretching the image size, stretching each input image to 640×640 pixels as the input image of the corn ear detection model. Furthermore, making a data set for building a corn ear detection model includes: pre-collecting corn ear images, manually marking the position of the corn ear in the image with a rectangular frame, and using the center point coordinates (x true ,y true )、Width(w true ) and high (h true ) represents the rectangular box, and the box category is ear. The corn ear image and its rectangular box together constitute the corn ear dataset.

[0012] Furthermore, the method for constructing a corn ear detection model comprises:

[0013] Use the YOLO V5s architecture as the corn ear detection model architecture, adding convolutional layers, pooling layers, and fully connected layers.

[0014] Use CIoU (Complete-IoU) as the positioning loss function for predicting the rectangular box (L box ), which is used to represent the difference between the predicted rectangle and the manually labeled rectangle:

[0015]

[0016]

[0017]

[0018] Among them, IoU is the intersection over union ratio of the predicted box and the real box, ρ(b,b g ) is the center point of the prediction box (x pred ,y pred ) and the center point of the ground truth box (x true ,y true ), c is the diagonal length of the minimum closure area, v is the aspect ratio consistency measure, and α is the weight coefficient.

[0019] Binary Cross-Entropy Loss (BCE Loss) is used to calculate the classification loss (L cls ) and confidence loss (L conf ):

[0020]

[0021] in, is the probability of the category c of the prediction box, p cls,c is the one-hot encoding vector of the true category c.

[0022]

[0023] in, is the confidence of the prediction box, p obj is the confidence of the ground-truth box (0 or 1).

[0024] The overall loss function of YOLO v5 is ( Loss ) is the weighted sum of the above three losses. The specific form can be expressed as:

[0025] Loss = λ box ·L box +λ conf ·L conf +λ cls ·L cls

[0026] Among them, λbox, λcls, and λconf are the weight coefficients of each loss function, which are set to 0.05, 0.5, and 1 respectively.

[0027] The corn ear dataset is input into the convolutional neural model for training, and the back-propagation algorithm is used to update the weights of the corn ear detection model. The value of the loss function Loss is reduced to the optimal level through continuous iteration to learn the characteristics of corn ear.

[0028] Furthermore, when collecting real-time image information and sensor position and posture information, feature points are extracted from continuous frame data and matched to obtain feature points. The relative position and posture of the sensor are estimated by the odometer, and the cumulative error is eliminated by posture graph optimization. The coordinates of the three-dimensional points are calculated by triangulation through the camera pose and the matched feature point pairs, and all the calculated three-dimensional points are gathered to form a sparse point cloud. A dense point cloud is generated using a parallax-based stereo matching method, which is continuously accumulated to construct a three-dimensional map represented by the point cloud in the camera coordinate system. The processed point cloud data is imported into the database through point cloud generation, point cloud data preprocessing, point cloud registration and point cloud segmentation for corn female ear detection and ear height measurement.

[0029] Furthermore, the method for measuring the height information of corn ear includes: in a three-dimensional map represented by a point cloud, plane detection is performed based on a random sampling consensus algorithm (RANSAC), and the detected ground is converted from a camera coordinate system to a world coordinate system to obtain a Z coordinate (z0) of the ground. The value of z0 is updated regularly to reduce the cumulative error caused by movement. Based on the constructed corn ear detection model, the position of the corn ear is detected in the real-time image, and (x pred ,y pred ,w pred ,h pred ) represents the predicted rectangular box of the corn ear in the image pixel coordinate system. Use the midpoint of the bottom of the prediction box (x pred ,y pred +0.5*h pred ) and corner point (x pred ±0.5*w pred ,y pred +0.5*h pred ) is used as the candidate position of the base of the corn ear, and the depth value z of the candidate position is obtained by combining the three-dimensional point cloud map. c The candidate position with the smallest depth value is considered to be the corn ear position. The corn ear position is converted to the camera coordinate system as (x c ,y c ,z cmin ), and further converted to the world coordinate system as (x w ,y w ,z w ). Maize ear height h = z w –z0.

[0030] A computer-readable storage medium stores one or more programs which, when executed by an electronic device including a plurality of application programs, cause the electronic device to perform the method.

[0031] The beneficial effects of the present invention are:

[0032] The present invention realizes accurate identification of corn ears through a deep learning method, and constructs a three-dimensional map in combination with SLAM technology to achieve automated and intelligent measurement of corn ear positions. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of the present invention;

[0034] Figure 2 Schematic diagram of application of a corn ear height measurement method based on target detection provided by an embodiment of the present invention in an intelligent handheld terminal;

[0035] Figure 3 This is a visualization result diagram of the position and height of a corn ear of the present invention;

[0036] Figure 4 It is a hardware schematic diagram of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. The illustrative embodiments and descriptions of the present invention are used to explain the present invention but are not intended to limit the present invention.

[0038] like Figure 1 As shown, the present invention comprises the following steps:

[0039] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0040] The present invention comprises the following steps:

[0041] A trains the target detection model weights and builds a corn ear detection model;

[0042] B collects real-time images and sensor position and posture information;

[0043] When collecting real-time images and sensor position and posture information, you can continuously capture and process image sequences through dynamic images (video streams), or capture images at specific moments through static images (single-frame images); dynamic images are usually achieved by continuously capturing video streams; video streams need to be preprocessed, such as denoising and contrast adjustment, to improve image quality.

[0044] Extract key frames from video streams or perform real-time processing such as motion detection and target tracking; synchronize the position and posture information of sensors to achieve accurate data association.

[0045] Static image acquisition is completed by capturing a single frame image, which is suitable for situations where detailed analysis of a specific scene is required. Single frame images need to be preprocessed, such as grayscale and filtering, to enhance image features. After that, the image can be analyzed by feature extraction, pattern recognition, etc. The position and posture information of the sensor is recorded while capturing a single frame image to ensure data synchronization.

[0046] C. inputting the preprocessed real-time image into the corn ear detection model, extracting the corn ear image features, and performing target detection on the preprocessed real-time image through the corn ear image features to obtain the coordinates of the corn ear represented in the image pixel coordinate system;

[0047] Use computer vision techniques to extract features from the preprocessed images. These features may include shape, texture, color, etc., which help to distinguish corn ears from other objects; select a suitable machine learning or deep learning model as the basis for the detection model. For example, a convolutional neural network (CNN) can be used to automatically learn image features.

[0048] D recognizes the real-time image, obtains the three-dimensional spatial coordinates of the ground and the female ear in the world coordinate system, and calculates the height of the corn female ear.

[0049] In this implementation example, the preprocessing method includes rotating the real-time image and stretching the image size, stretching each input image to 640×640 pixels as the input image of the corn ear detection model. In this implementation example, the method of training the target detection model with the training data set to obtain the corn ear detection model includes taking the input of the corn ear detection model as an image; and outputting the number of corn ears in the image, the position and size of the ear target, and the image with the corn ears marked.

[0050] The method for constructing a corn ear detection model in this implementation example includes:

[0051] The YOLO V5s architecture is used as the corn ear detection model architecture, and convolutional layers, pooling layers, and fully connected layers are added. The output of this model is the category of the detected target and its predicted rectangular box. The predicted rectangular box uses the center point coordinates (x pred ,y pred )、Width(w true ) and high (h true )express.

[0052] Use CIoU (Complete-IoU) as the positioning loss function for predicting the rectangular box (L box ), which is used to represent the difference between the predicted rectangle and the manually labeled rectangle:

[0053]

[0054]

[0055] Among them, IoU is the intersection over union ratio of the predicted box and the real box, ρ(b,b g ) is the center point of the prediction box (x pred ,y pred ) and the center point of the ground truth box (x true ,y true ), c is the diagonal length of the minimum closure area, v is the aspect ratio consistency measure, and α is the weight coefficient.

[0056] Binary Cross-Entropy Loss (BCE Loss) is used to calculate the classification loss (L cls ) and confidence loss (L conf ):

[0057]

[0058] in, is the probability of the category c of the prediction box, p cls,c is the one-hot encoding vector of the true category c.

[0059]

[0060] in, is the confidence of the prediction box, p obj is the confidence of the ground-truth box (0 or 1).

[0061] The overall loss function (Loss) of YOLO v5 is the weighted sum of the above three losses. The specific form can be expressed as:

[0062] Loss = λ box ·L box +λ conf ·L conf +λ cls ·L cls

[0063] Among them, λbox, λcls, and λconf are the weight coefficients of each loss function, which are set to 0.05, 0.5, and 1 respectively.

[0064] The corn ear dataset is input into the convolutional neural model for training, and the back-propagation algorithm is used to update the weights of the corn ear detection model. The value of the loss function Loss is reduced to the optimal level through continuous iteration to learn the characteristics of corn ear.

[0065] In this implementation example, when collecting real-time image information and sensor position and posture information, feature points are extracted from continuous frame data and matched to obtain feature points. The relative position and posture of the sensor are estimated by the odometer, and the cumulative error is eliminated by posture graph optimization. The coordinates of the three-dimensional points are calculated using the camera posture and the matched feature point pairs, and all the calculated three-dimensional points are gathered to form a sparse point cloud. A dense point cloud is generated using a parallax-based stereo matching method, which is continuously accumulated to construct a three-dimensional map represented by a point cloud in the camera coordinate system. The processed point cloud data is imported into the database for corn ear detection and ear height measurement through point cloud generation, point cloud data preprocessing, point cloud registration and point cloud segmentation.

[0066] In this embodiment, the method for measuring the height measurement information of corn ear includes: in a three-dimensional map represented by a point cloud, plane detection is performed based on a random sampling consensus algorithm (RANSAC), and the detected ground is converted from the camera coordinate system to the world coordinate system to obtain the Z coordinate (z0) of the ground. The value of z0 is updated regularly to reduce the cumulative error caused by movement. Based on the constructed corn ear detection model, the position of the corn ear is detected in the real-time image, and (x pred ,y pred ,w pred ,h pred ) represents the predicted rectangular box of the corn ear in the image pixel coordinate system. Use the midpoint of the bottom of the prediction box (x pred ,y pred +0.5*h pred ) and corner point (x pred ±0.5*w pred ,y pred +0.5*h pred ) is used as the candidate position of the base of the corn ear, and the depth value z of the candidate position is obtained by combining the three-dimensional point cloud map. c The candidate position with the smallest depth value is considered to be the corn ear position. The corn ear position is converted to the camera coordinate system as (x c ,y c ,z cmin ), and further converted to the world coordinate system as (x w ,y w ,z w ). Maize ear height h = z w –z0.

[0067] If the ground is not completely flat, you may need to use multiple reference points on the ground to determine an average height or use a three-dimensional model of the ground to more accurately calculate the height difference. If SLAM (Simultaneous Localization and Mapping) technology is used, the mobile device will update its position and map information in real time during movement. The three-dimensional coordinates of the site and the three-dimensional coordinates of the ground reference point are obtained from the SLAM system. Assuming that the three-dimensional coordinates of the site are (100, 200, 10) and the three-dimensional coordinates of the ground reference point are (100, 200, 5) (), then the height of the object to be measured is h = 10-5 = 5 units. Figure 2 As shown, in this embodiment, when collecting real-time image information and position information of corn, the processed point cloud data is imported into the database through point cloud data preprocessing, point cloud registration and point cloud segmentation to construct a corn ear detection model.

[0068] Data collection. Figure 3As shown in the figure, the experiment prepared cameras and mobile phones to collect images containing corn ears in the corn field, a total of 613 images, and the resolution was set to 4000×3000. A total of 1000 corn ear image data samples were collected in the experiment. After recognition and analysis, a total of 641 heights were detected. After preprocessing to remove interference noise, a total of 613 images containing corn ears were collected in the corn field using cameras and mobile phones. After screening, 604 usable images were obtained, and the corn ear image set was established;

[0069] Labeling of corn ears. Use target detection and annotation software (such as LabelImg software) to label the corn ears in the corn ear image set. The annotation is a rectangular box with the base and top of the corn ear as diagonal lines, and each annotation is named. In this example, the label of the corn ear is named ear. The file containing the annotation information is saved in .xml format. The corn ear image set is merged with .xml to obtain the corn ear annotation dataset;

[0070] Divide the data set. Randomly select 527 images from the corn ear annotation data set as the training data set, and select 77 images as the validation data set;

[0071] Training of corn ear detection model. The Yolo V5s model was trained using the training set. The computer used for training was Lenovo GeekPro2021, whose CPU was AMD R5 5600G, GPU was NVIDIA RTX 306012GB, and memory was 16GB. The deep learning framework for training and testing was PyTorch. The size of the resulting corn ear detection model was 13.7MB, and the model was stored as a .pt file.

[0072] Software and hardware used. The handheld device is an iPad Pro 11-inch (second generation), equipped with a laser radar rangefinder, an RGB digital camera, an inertial measurement unit (IMU), and data processing and storage capabilities. The system is iOS. The software development language is Swift, the integrated development environment is Xcode, and the development hardware is a MacBookPro 2017.

[0073] Compilation of the corn ear detection model. Using Python and the PyTorch deep learning framework, the corn ear detection model in .pt format can be compiled into a method that can be executed by Swift. This method is named ObjectDetection.

[0074] Detect the ground and obtain its 3D coordinates. Use the planeDetection method in Swift to detect the horizontal plane and then manually select the ground. Use the hitTest method in Swift to obtain the world 3D coordinates (x0, y0, z0) of the ground while selecting it.

[0075] Corn ear detection and acquisition of its 3D coordinates. Use the ObjectDetection method compiled from the corn ear detection model to detect the position and confidence of the corn ear in a frame of image acquired by the RGB camera. The position of the corn ear is marked by a rectangular box, and the 2D coordinates of the left end, right end, and midpoint of the bottom of the rectangular box are obtained. Use the hitTest method in Swift to obtain the corresponding 3D coordinates based on the 2D coordinates of the left end, right end, and midpoint, and select the 3D coordinate closest to the camera as the world 3D coordinate (x1, y1, z1) of the base of the corn ear.

[0076] Calculation of the height of the corn ear: According to the three-dimensional coordinates of the base of the corn ear (x1, y1, z1) and the three-dimensional coordinates of the ground (x0, y0, z0), the difference between the vertical coordinates can be calculated to obtain the height of the corn ear h = z1-z0h = z1-z0.

[0077] Removal of outliers in the height of corn ears. The three-dimensional coordinates of corn ears that are far from the camera horizontally and outside the area of ​​interest will be removed, and the detection results of corn ears with low confidence will be removed.

[0078] Visualization of maize ear height measurement results ( Figure 3 ). The identified corn ear is marked with a rectangular frame on the screen, the identified base of the corn ear is marked with a solid dot on the screen, the confidence of the corn ear detection is marked on the top of the rectangular frame, and the height of the corn ear is marked on the bottom of the rectangular frame.

[0079] Record and annotate the corn ear height measurement results. Use the buttons to annotate and mark the currently measured plot. The measurement results will be saved in a .txt file.

[0080] An electronic device comprises: a processor; and a memory arranged to store computer executable instructions, which when executed cause the processor to perform the method.

[0081] A computer-readable storage medium stores one or more programs which, when executed by an electronic device including a plurality of application programs, cause the electronic device to perform the method.

[0082] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0083] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0084] The memory is used to store programs. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory may include internal memory and non-volatile memory, and provide instructions and data to the processor.

[0085] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an information security risk management device at the logical level. The processor executes the program stored in the memory and is specifically used to execute any of the above-mentioned corn ear height measurement methods based on target detection.

[0086] The above application Figure 1The embodiment shown discloses a method for measuring the height of a corn ear based on target detection, which can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0087] The electronic device may also perform Figure 1 A method for measuring the height of corn ear based on target detection is proposed. Figure 3 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0088] An embodiment of the present application also proposes a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions. When the instructions are executed by an electronic device including multiple applications, any one of the aforementioned corn ear height measurement methods based on target detection is executed.

[0089] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0090] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0091] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0093] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0094] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0095] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0096] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0097] The technical solution of the present invention is not limited to the above-mentioned specific embodiments. All technical variations made according to the technical solution of the present invention fall within the protection scope of the present invention.

Claims

1. A method for measuring the height of a corn ear based on target detection, characterized in that: The steps include: A trains the target detection model weights and builds a corn ear detection model; B collects images and sensor position and posture information in real time; C. inputting the preprocessed real-time image into the corn ear detection model, extracting the corn ear image features, and performing target detection on the preprocessed real-time image through the corn ear image features to obtain the coordinates of the corn ear represented in the image pixel coordinate system; D recognizes the real-time image, obtains the three-dimensional spatial coordinates of the ground and the female ear in the world coordinate system, and calculates the height of the corn female ear; The method for constructing a corn ear detection model comprises: The YOLO V5s architecture is used as the architecture of the corn ear detection model, and a convolutional layer, a pooling layer, and a fully connected layer are added. The output of the corn ear detection model is the category of the detection target and its predicted rectangular box. The predicted rectangular box is represented by the center point coordinates, width, and height; Use CIoU as the positioning loss function L for the predicted rectangular box box , used to represent the difference between the predicted rectangle and the manually labeled rectangle: Among them, IoU is the intersection-over-union ratio of the predicted box and the real box. is the center point of the prediction box (x pred , y pred ) and the center point of the ground truth box (x true , y true ), is the diagonal length of the minimum enclosing region, v is the aspect ratio consistency measure, α is the weight coefficient, Binary cross entropy loss is used to calculate the classification loss L cls and confidence loss L conf : in, is the category of the prediction box c The probability of Is the real category c The one-hot encoded vector of ; in, is the confidence of the prediction box, is the confidence of the real box, the confidence is 0 or 1; The overall loss function of YOLO v5 is the weighted sum of the above three losses, which can be expressed as: in, λbox , λcls , λ conf is the weight coefficient of each loss function, which is set to 0.05, 0.5 and 1 respectively; The corn ear dataset is input into the convolutional neural model for training, and the back-propagation algorithm is used to update the weights of the corn ear detection model. The value of the loss function Loss is reduced to the optimal level through continuous iteration to learn the characteristics of corn ear.

2. The method for measuring corn ear height based on target detection according to claim 1, characterized in that: The preprocessing method includes rotating the real-time image in direction and stretching the image in size, stretching each input image to a size of 640×640 pixels as an input image for the corn ear detection model.

3. The method for measuring the height of corn ear based on target detection according to claim 1, characterized in that: A data set for building a corn ear detection model is prepared, including: pre-collecting corn ear images, manually marking the positions of corn ears in the images with rectangular frames, using center point coordinates, width and height to represent the rectangular frames, the category of the rectangular frames is ear, and the corn ear images and their rectangular frames together constitute a corn ear data set.

4. The method for measuring the height of a corn ear based on target detection according to claim 1, characterized in that: When collecting real-time image information and sensor position and posture information, feature points are extracted from continuous frame data and matched to obtain feature points. The relative posture of the sensor is estimated by the odometer, and the cumulative error is eliminated by posture graph optimization. The coordinates of the three-dimensional points are calculated by triangulation through the camera posture and the matched feature point pairs. All calculated three-dimensional points are gathered to form a sparse point cloud. The parallax-based stereo matching method is used to generate a dense point cloud, which is continuously accumulated to construct a three-dimensional map represented by the point cloud in the camera coordinate system. The processed point cloud data is imported into the database through point cloud generation, point cloud data preprocessing, point cloud registration and point cloud segmentation for corn female ear detection and ear height measurement.

5. The method for measuring the height of a corn ear based on target detection according to claim 1, characterized in that: The method for measuring the height information of corn ear comprises: in a three-dimensional map represented by a point cloud, plane detection is performed based on a random sampling consistency algorithm, the detected ground is converted from a camera coordinate system to a world coordinate system, the Z coordinate of the ground is obtained, the value of z0 is updated regularly to reduce the cumulative error caused by movement, and the position of the corn ear is detected in a real-time image based on a constructed corn ear detection model, and the (x pred , y pred , w pred , h pred ) represents the predicted rectangular box of the corn ear in the image pixel coordinate system, using the midpoint of the bottom of the prediction box (x pred , y pred +0.5* h pred ) and corner points (x pred ±0.5* w pred , y pred +0.5*h pred ) is used as the candidate position of the base of the corn ear, and the depth value z of the candidate position is obtained by combining the 3D point cloud map c , the candidate position with the smallest depth value is considered to be the corn ear position.

6. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute any one of the methods of claims 1 to 5.

Images