Corn crop plant density detection method, device, equipment, medium and product

Abstract

The invention discloses a corn crop plant density detection method and device, equipment, a medium and a product, and relates to the field of agricultural machinery intelligent control and computer vision, and the method comprises the steps: obtaining the initial point cloud data of corn crops collected by a plurality of depth cameras in real time at a set position and posture; performing time registration and space registration on the initial point cloud data, and fusing the initial point cloud data to obtain fused point cloud data; and completing point cloud clustering and central point extraction to obtain a crop cluster and a representative position of the crop cluster. According to the method, the number information of corns is collected and comprehensively captured in real time at the set position and posture through the multiple depth cameras, and fusion processing is performed on the collected initial point cloud data under multiple visual angles, so that the problems of inter-crop shielding and visual angle limitation in a traditional single-visual-angle image collection method are solved; the corn grain number information is comprehensively and efficiently captured and fused, subsequent accurate extraction of the grain number information of corn plants with different densities is facilitated, and data support is provided for intelligent regulation and control of the feeding amount.

Description

Technical Field

[0001] The present application relates to the fields of intelligent control of agricultural machinery and computer vision, and in particular to a method, device, equipment, medium and product for detecting the density of corn crop plants. Background Art

[0002] Corn is an important grain crop in my country, and its efficient harvesting directly impacts grain production efficiency and farmers' income. During corn harvesting, feed rate is a key factor in determining harvesting efficiency and quality. Proper control of feed rate can significantly reduce corn harvest loss and impurity rates. However, in practice, accurate feed rate measurement is difficult, and there is a lack of feasible and effective automated statistical methods for measuring corn plant density ahead of the harvester. Therefore, research on precise corn plant density measurement technology is crucial for optimizing harvesting parameters and improving harvesting quality.

[0003] Currently, technology for detecting the density of mature corn plants primarily relies on single-view image acquisition methods, but this approach has significant limitations in applications in mature corn fields. For one thing, mature corn is mostly yellowish-white in color, which offers low contrast with the field soil and background, leading to reduced recognition accuracy. Furthermore, the high density of mature corn plants creates significant occlusion between crops. Furthermore, field operations require a large detection area, making it difficult to fully capture kernel count information using a single viewpoint. Furthermore, complex lighting conditions and ambient noise in the field further exacerbate detection challenges. Consequently, traditional single-image detection methods are poorly suited for complex field environments, necessitating the urgent need for more efficient and adaptable detection technologies. Summary of the Invention

[0004] The purpose of this application is to provide a corn crop plant density detection method, device, equipment, medium and product, which is conducive to broadening the field of view and comprehensively capturing corn kernel number information, improving recognition accuracy, and achieving stronger applicability in complex field environments.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for detecting corn crop plant density, comprising:

[0007] Initial point cloud data of corn crops collected in real time by multiple depth cameras at set positions and postures are obtained; the initial point cloud data are temporally and spatially registered and fused to obtain fused point cloud data; regions of interest are extracted from the fused point cloud data to obtain regional point cloud data; the regional point cloud data are simplified to obtain simplified point cloud data; the simplified point cloud data are clustered using triangulation and a minimum cut algorithm to obtain crop clusters; the center points of the crop clusters are extracted by principal direction fitting and centroid calculation, and the obtained center points are used as representative positions of the crop clusters.

[0008] In the second aspect, the present application provides a corn crop plant density detection device, including: an acquisition module, a fusion module, a region of interest extraction module, a point cloud simplification module, a point cloud clustering module, and a center point extraction module.

[0009] The acquisition module is used to obtain the initial point cloud data of corn crops collected in real time by multiple depth cameras at set positions and postures; the fusion module is used to perform temporal alignment and spatial alignment on the initial point cloud data and fuse them to obtain fused point cloud data; the region of interest extraction module is used to extract the region of interest from the fused point cloud data to obtain regional point cloud data; the point cloud simplification module is used to perform point cloud simplification on the regional point cloud data to obtain simplified point cloud data; the point cloud clustering module is used to perform point cloud clustering on the simplified point cloud data in combination with triangulation and minimum cut algorithm to obtain crop clusters; the center point extraction module is used to extract the center point of the crop cluster by main direction fitting and centroid calculation, and the obtained center point is used as the representative position of the crop cluster.

[0010] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-described corn crop plant density detection methods.

[0011] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the above-mentioned corn crop plant density detection methods.

[0012] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned corn crop plant density detection methods.

[0013] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0014] The present application provides a method, device, equipment, medium and product for detecting the density of corn crop plants. The method uses multiple depth cameras to collect and capture comprehensive corn kernel count information in real time at set positions and postures, and fuses the collected initial point cloud data from multiple perspectives, thereby solving the problems of crop occlusion and perspective limitations in traditional single-perspective image acquisition methods, and achieving comprehensive and efficient capture and fusion of corn kernel count information, which is conducive to the subsequent accurate extraction of high-density and low-density corn crop plants to obtain kernel count information; simplified point cloud data is obtained by extracting regions of interest and simplifying point clouds, which can effectively avoid interference in extracting useful point cloud data information, and significantly improve the quality and computational efficiency of point cloud data, laying a good foundation for subsequent point cloud clustering analysis and kernel count detection. Point cloud clustering and center point extraction are performed on the simplified point cloud data to obtain crop cluster information, which improves the overall recognition accuracy, accurately obtains the corn kernel count to complete corn crop plant density detection, and achieves strong applicability in complex field environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 This is a schematic diagram of the installation of two depth cameras in one embodiment of the present application, wherein: Figure 1 Part a in the figure is the magnified state of the right depth camera. Figure 1 Part b in the figure is the magnified state of the left depth camera;

[0017] FIG2( a ) is a schematic diagram of the field of view angle of the depth camera when the present application is observed from the left side, and FIG2( b ) is a schematic diagram of the field of view angle of the depth camera when the present application is observed from the top side;

[0018] Figure 3 This is a diagram of an application environment of a corn crop plant density detection method in one embodiment of the present application;

[0019] Figure 4 A schematic flow chart of a corn crop plant density detection method provided in one embodiment of the present application;

[0020] FIG5(a) is a schematic diagram of initial point cloud data acquired by the left depth camera based on high-density corn crop plants, and FIG5(b) is a schematic diagram of initial point cloud data acquired by the right depth camera based on high-density corn crop plants;

[0021] FIG6(a) is a schematic diagram of initial point cloud data acquired by the left depth camera based on low-density corn crop plants, and FIG6(b) is a schematic diagram of initial point cloud data acquired by the right depth camera based on low-density corn crop plants;

[0022] Figure 7 Schematic diagram of Figure 5(a) and Figure 5(b) after temporal registration and spatial registration and fusion;

[0023] Figure 8 for Figure 7 Schematic diagram of gravity direction correction;

[0024] Figure 9 for Figure 8 Schematic diagram of region of interest extraction;

[0025] Figure 10(a) is a schematic diagram of the distance weighted surface, Figure 10(b) is a schematic diagram of the view weighted surface, and Figure 10(c) is a schematic diagram of the local curvature weighted surface;

[0026] Figure 11 for Figure 9 Schematic diagram of point cloud simplification;

[0027] Figure 12 for Figure 11 Schematic diagram after denoising;

[0028] Figure 13(a) is a clustering result diagram of high-density corn crop plants, and Figure 13(b) is a clustering result diagram of low-density corn crop plants;

[0029] Figure 14 A schematic diagram of a specific process of a corn crop plant density detection method provided in one embodiment of the present application;

[0030] Figure 15 A schematic diagram of the functional modules of a corn crop plant density detection device provided in another embodiment of the present application;

[0031] Figure 16 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0033] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0034] Point cloud technology provides a new solution for corn kernel count detection. Compared with traditional image methods, point cloud data can provide three-dimensional structural information of crops and can effectively solve the problems of occlusion and viewing angle limitations. However, point cloud data usually relies on LiDAR to obtain, and LiDAR equipment is relatively expensive, which is not conducive to its promotion in large-scale agricultural production. In contrast, depth cameras can generate point cloud data at a lower cost. Although the accuracy is slightly lower than that of LiDAR, it can meet the detection needs in agricultural scenarios. This application proposes a method, device, equipment, medium and product for detecting corn crop plant density. It fully utilizes the economic benefits and multi-view fusion technology of depth cameras. Through the collection, registration, preprocessing and cluster analysis of multi-view point cloud data, it can achieve high-precision detection of corn crop density and kernel count, thereby providing reliable data information support for the intelligent control of corn harvesters, solving the detection problems in field environments, and providing technical support for crop density monitoring and dynamic vehicle speed control, thereby optimizing feed control and improving corn harvesting efficiency and quality. The core of this application is to use a multi-view depth camera to obtain three-dimensional point cloud data of corn crops in the field, and through a systematic processing process, to achieve accurate monitoring of corn crop plant density, and solve the applicability problem of traditional detection methods in complex field environments.

[0035] This application uses two ZED depth cameras as data acquisition equipment as an example. The equipment was selected due to its good point cloud generation capability and high cost performance. The system relies on two ZED depth cameras to collect multi-view point cloud data. The two depth cameras are designed to be installed in symmetrical positions on both sides of the front of the corn harvester, specifically above the rearview mirror. They are installed through a strong magnetic bracket. The camera is connected to the bracket through a 1 / 4 threaded port, and the bracket is connected to the harvester body through a strong magnet. The choice of this installation position is intended to achieve the maximum range of crop coverage and multi-view point cloud information collection, while reducing the interference of corn plants on the line of sight.

[0036] The camera installation parameters used in this application have been carefully calculated and repeatedly debugged to ensure high accuracy of data acquisition and system stability.

[0037] The specific camera mounting locations vary for different harvester models. To maximize overlap and redundancy in front crop detection, the two cameras should be mounted as far apart as possible, typically above the harvester's rearview mirror. The cameras should remain level in the roll direction, ensuring a roll angle of 0° to facilitate subsequent image stacking and fusion processing. The pitch angle should be determined based on the effective detection range and mounting height of the selected cameras. The vertical distance between the camera mounting point and the ground can be used as one leg of a right triangle, with the camera's maximum effective detection range as the hypotenuse. The pitch angle, or the angle subtended by the triangle, can be calculated. Regarding the heading angle, to maximize the advantages of a multi-view camera system, a certain degree of field of view overlap should be maintained. Adjust the angle based on the spacing between the two depth cameras so that, after the roll angle is set, the optical overlap area of ​​the images is as close as possible in both the horizontal and pitch directions, thereby improving detection stability and accuracy.

[0038] Figure 1The installation positions of two ZED depth cameras are shown. Through proper placement, blind spots are avoided during crop detection, maximizing crop point cloud data capture. The installation parameters for the two depth cameras were precisely calculated to achieve optimal crop coverage and acquisition accuracy. The specific installation parameters are as follows. Specifically, in this example, the horizontal spacing between the two depth cameras is set at 2.66 meters, ensuring sufficient overlap between the cameras and improving the accuracy of depth information fusion. The installation height (i.e., the vertical distance between the cameras and the ground) is 3.55 meters. This height allows the cameras to cover a wide crop area and avoid interference with other harvester equipment. In the right-handed coordinate system, the left camera is configured with a pitch angle of 25°, a roll angle of 0°, and a heading angle of 10°, while the right camera is configured with a pitch angle of 25°, a roll angle of 0°, and a heading angle of -10°. The 25° pitch angle ensures the camera's field of view covers the tops and stalks of the corn plants, preventing the loss of critical details due to steep viewing angles and ensuring a comprehensive capture of the overall crop growth status. Furthermore, the two depth cameras are set with heading angles of ±10°, allowing for appropriate overlap and complementarity in the horizontal plane, reducing redundant data and improving the utilization of point cloud information. Maintaining a roll angle of 0° effectively avoids image rotational distortion and ensures data stability. Combined with the ZED camera's field of view parameters, geometric projection calculations reveal the camera's projection area and the coverage of the collected point cloud information. As shown in Figures 2(a) and 2(b), when viewed from the left, the projection point of the left camera's optical axis center is approximately 7.62 meters in front of the mounting point, while the intersection of the two depth camera centerlines is approximately 7.54 meters in front of the mounting surface. These calculation results demonstrate that the adopted mounting scheme enables accurate monitoring of corn crop density within a detection distance of approximately 8 meters, ensuring high-precision detection of crop grain counts. This installation solution enables accurate monitoring of corn crop density within the detection distance, fully meeting the actual needs in precision agriculture applications.

[0039] In order to achieve real-time and efficient data collection and processing, this application uses NVIDIA's Jetson Nano embedded AI computing platform as the core processing unit. The Jetson Nano platform has powerful edge computing capabilities, and its CUDA (Compute Unified Device Architecture) can support high-performance point cloud data processing, while achieving fast computing at low power consumption, and is suitable for mobile scenarios in field operations. The Jetson Nano platform is installed in the harvester cab and connected to two depth cameras via a USB signal amplifier cable. The use of the amplifier cable is intended to ensure the stability of signal transmission, avoid signal attenuation caused by long-distance transmission, and achieve stable transmission and processing of point cloud data. The processed point cloud information is further transmitted to the harvester's own intelligent control system through the USB interface, providing accurate real-time data support for dynamic adjustment of operating parameters.

[0040] During data acquisition, the camera captures 3D point cloud data of the corn crop at a fixed 15 frame rate and transmits the data in real time to the processor for storage and preprocessing. After installation and layout, the hardware system operates in dynamic field acquisition mode. The two depth cameras coordinate their capture, capturing point cloud data of the corn crop from different angles, creating a multi-perspective coverage effect. This hardware design ensures the integrity and accuracy of the crop point cloud data, providing a sound data foundation for subsequent registration and cluster analysis.

[0041] The corn crop plant density detection method provided in the embodiment of the present application can be applied to Figure 3In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the initial point cloud data to the server 104. The server 104 receives the initial point cloud data, and the server 104 performs temporal registration and spatial registration and fuses the initial point cloud data to obtain fused point cloud data; extracts the region of interest from the fused point cloud data to obtain regional point cloud data; simplifies the regional point cloud data to obtain simplified point cloud data; clusters the simplified point cloud data by combining triangulation and minimum cut algorithms to obtain crop clusters; extracts the center point of the crop cluster by main direction fitting and centroid calculation, and uses the obtained center point as the representative position of the crop cluster. The server 104 can feedback the obtained crop cluster and the obtained center point to the terminal 102. In addition, in some embodiments, the corn crop plant density detection method can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly perform time alignment and spatial alignment and fusion on the initial point cloud data to be processed, complete point cloud clustering and center point extraction, and obtain crop clusters and representative positions of crop clusters. The server 104 can also obtain the initial point cloud data from the data storage system for time alignment and spatial alignment and fusion, complete point cloud clustering and center point extraction, and obtain crop clusters and representative positions of crop clusters.

[0042] Terminal 102 may include, but is not limited to, various desktop computers, laptops, smartphones, tablet computers, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart car devices. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Server 104 may be implemented as a standalone server or a server cluster consisting of multiple servers, or may be a cloud server.

[0043] In an exemplary embodiment, Figure 4 As shown, a method for detecting the density of corn crop plants is provided. The method is executed by a computer device, specifically, it can be executed by a computer device such as a terminal or a server alone, or it can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 3 The server 104 in FIG. 1 is used as an example to illustrate the method, which includes the following steps 100 to 600. Among them:

[0044] Step 100: acquiring initial point cloud data of corn crops collected in real time by multiple depth cameras at set positions and postures;

[0045] Step 200 , performing temporal registration and spatial registration on the initial point cloud data and fusing them to obtain fused point cloud data;

[0046] Step 300: extracting the region of interest from the fused point cloud data to obtain regional point cloud data;

[0047] Step 400 , simplifying the regional point cloud data to obtain simplified point cloud data;

[0048] Step 500 , performing point cloud clustering on the simplified point cloud data by combining triangulation and minimum cut algorithm to obtain crop clusters;

[0049] Step 600 : extracting the center point of the crop cluster by principal direction fitting and centroid calculation, and using the obtained center point as the representative position of the crop cluster.

[0050] By implementing the above-mentioned steps 100 to 600, the present application uses multiple depth cameras to collect and comprehensively capture corn kernel count information in real time at set positions and postures, and fuses and processes the collected initial point cloud data from multiple perspectives, thereby solving the problems of crop occlusion and perspective limitations that occur in traditional single-perspective image acquisition methods. By extracting regions of interest and streamlining point clouds, streamlined point cloud data is obtained, which can effectively avoid interference in extracting useful point cloud data information. Point cloud clustering and center point extraction are performed on the streamlined point cloud data to obtain crop cluster information, thereby improving the overall recognition accuracy, accurately obtaining the number of corn kernels to complete corn crop plant density detection, and achieving strong applicability in complex field environments.

[0051] The initial point cloud data obtained by the left depth camera and the right depth camera based on the high-density corn crop plants are shown in Figures 5(a) and 5(b), and the initial point cloud data obtained by the left depth camera and the right depth camera based on the low-density corn crop plants are shown in Figures 6(a) and 6(b). In another exemplary embodiment of the present application, the initial point cloud data is temporally and spatially registered and fused to obtain fused point cloud data, and the above step 100 is replaced by the following steps 101 to 103:

[0052] In step 101 , if the timestamp difference between a frame of initial point cloud data respectively collected by multiple depth cameras is not greater than a time threshold, the frames of initial point cloud data respectively collected by the multiple depth cameras are determined to be initial point cloud data at the same time.

[0053] In one embodiment, two ZED depth cameras are used to extract point cloud data and timestamps in real time, and the point cloud data is temporally aligned based on the timestamps. The frequency of extracting the depth point cloud is 15 Hz, that is, 15 frames of point cloud data are collected per second. The criterion for determining whether the point cloud data collected by the two depth cameras belongs to the same time is the difference in the timestamps of the two frames of point cloud data, Δt:

[0054] |t 左 -t 右 |≤Δt;

[0055] Among them, t 左 is the timestamp of the point cloud data of the left camera; t 右 is the timestamp of the point cloud data of the right camera; Δt is the time threshold, which is set to 70ms in this system.

[0056] If the above conditions are met, the two frames of point cloud data are considered to belong to the same moment, and further spatial registration and fusion are performed.

[0057] Because this application involves the fusion of data from two depth cameras, it is first necessary to time-align the point cloud data collected by the two depth cameras. According to the working characteristics of the camera, the camera operating frequency is set to 15Hz. In order to ensure that the point cloud data acquired by the two depth cameras are fused at the same time, the system is synchronized based on the timestamp. The time error threshold is set to 70ms, which means that as long as the time error between the two frames is within 70ms, the point cloud data of the two frames will be regarded as being at the same time, reducing the time alignment error caused by frame loss caused by unstable factors in the camera during operation.

[0058] In step 102 , the initial point cloud data of multiple depth cameras at the same time are unified into the global coordinate system using a coordinate transformation matrix according to the position and posture of the depth cameras, thereby obtaining the point cloud data of each of the multiple depth cameras in the global coordinate system.

[0059] In one embodiment, the point cloud data collected by the left and right depth cameras are unified into a global coordinate system using a coordinate transformation matrix based on the installation position and posture of the two depth cameras. The point cloud data collection coordinate system of each camera is based on its optical center as the origin. Spatial registration includes two steps: rotation and translation. The unified formula is:

[0060] P 全局左 =R 左 ·P 左局部 +T 左 ;

[0061] P 全局右 =R 右 ·P 右局部 +T 右 ;

[0062] Among them, P 全局左 The point cloud coordinates of the left camera are converted to the point cloud coordinates of the global coordinate system; P 全局右 The point cloud coordinates of the right camera are converted to the point cloud coordinates of the global coordinate system; P 左局部 is the point cloud coordinate in the camera local coordinate system; P右局部 is the point cloud coordinate in the local coordinate system of the camera; R 左 is the rotation matrix, used to represent the installation posture of the left camera; R 右 is the rotation matrix, which is used to represent the installation posture of the right camera; T 左 is the translation vector, which is used to represent the position of the left camera in the global coordinate system; T 右 is the translation vector, which represents the position of the left camera in the global coordinate system.

[0063] Rotation matrix R 右 、R 左 Calculate the rotation matrix based on the camera's pitch, roll, and yaw angles. In this application, the pitch, roll, and yaw angles of the left camera are 25°, 0°, and 10°, respectively, and those of the right camera are 25°, 0°, and -10°, respectively. The rotation matrix can be calculated using the following formula:

[0064] R=R Yaw ·R Pitch ·R Roll ;

[0065]

[0066] Among them, R Yaw is the rotation matrix of the heading angle; R Pitch is the rotation matrix of the pitch angle; R Roll is the rotation matrix of the roll angle.

[0067] Substitute the posture parameters of the left and right cameras into the above formula respectively to calculate the corresponding rotation matrix R 左 and R 右 .

[0068] Translation vector T 左 、T 右 Represents the camera's position in the global coordinate system. Based on the camera's installation position, the translation vector for the left camera is [-1.33, 0, 0] meters, and the translation vector for the right camera is [1.33, 0, 0] meters. The origin of the fused global coordinate system is set midway between the left and right cameras, on the same horizontal plane.

[0069] Finally, the global coordinates P of the point cloud are calculated 全局 , complete the spatial registration of the point clouds of the two depth cameras, laying the foundation for subsequent point cloud data fusion and analysis.

[0070] Because the two cameras independently collect point cloud data, the data they generate is in two different coordinate systems and cannot be directly processed and inspected. Therefore, the point cloud data collected by the two depth cameras needs to be spatially registered and converted to a unified global coordinate system. The global coordinate system is set at the center point of the line connecting the origins of the two camera coordinate systems and is established according to the right-hand coordinate system rule, with the x-axis pointing to the right of the harvester and the y-axis pointing to the front of the harvester. The collected point cloud data is uniformly converted to the global coordinate system based on the coordinate transformation matrix.

[0071] After spatial registration, the data of the two depth cameras are merged into a unified global coordinate system, providing a basis for subsequent data fusion. The effect after fusion is as follows: Figure 7 shown.

[0072] Step 103 : stack and fuse the point cloud data of the multiple depth cameras in the global coordinate system to obtain fused point cloud data.

[0073] In one embodiment, since the converted P 全局左 , P 全局右 They are coordinates in the same coordinate system and describe the same target, so they can be directly stacked and fused to obtain point cloud information with redundancy.

[0074]

[0075] Among them, P 全局 is the point cloud coordinate after the left and right cameras are transformed and stacked. For stacking operations.

[0076] In implementing the above steps 101 to 103, the present application adopts the method of comparing the timestamp difference with the time threshold for time alignment, and uses the method of unifying the coordinate transformation matrix to the global coordinate system for spatial alignment, laying the foundation for subsequent point cloud data fusion and analysis. The stacked and fused point cloud data achieves unification in time and space, effectively avoiding the limitations of a single perspective, and is conducive to point cloud data analysis in complex field environments.

[0077] Because the two depth cameras are installed at a low angle, the z-axis direction of the point cloud data obtained is not consistent with the gravity direction in the real world, which is not conducive to subsequent data processing. In order to ensure that the z-axis direction of the point cloud data is aligned with the gravity direction, the point cloud data needs to be converted to the standard gravity direction. Optionally, gravity direction correction can be performed during spatial registration or after fusion. The correction formula for gravity direction correction after fusion is as follows:

[0078]

[0079] Where θ is the pitch angle of the depth camera; P矫正 The coordinates of the corrected point cloud are as follows: Figure 8 shown.

[0080] Then, the fused point cloud data is preprocessed, specifically including: extracting the region of interest from the fused point cloud data to obtain regional point cloud data; and simplifying the regional point cloud data to obtain simplified point cloud data.

[0081] In order to reduce the interference of irrelevant point cloud data, only the area of ​​interest in front of the harvester is retained. The range is limited to the area in front of the harvester's header, where the length is the width of the header and the length is the effective recognition distance of the camera.

[0082] In a specific embodiment, for example, the range is limited to 8m×12m. The definition formula of the region of interest is:

[0083]

[0084] Among them, P ROI are the coordinates of the point cloud of interest; x' is the horizontal coordinate of the point cloud, ranging from [-4, 4] meters; y' is the vertical coordinate of the point cloud, ranging from [1, 13] meters; z' is the height coordinate of the point cloud, which must satisfy z' ≥ 1.5.

[0085] Since the purpose of this application is to detect the density of crop plants in the working area in front of the harvester, only the point cloud data within a certain distance range in front of the harvester is needed. However, the point cloud data currently obtained contains a large amount of irrelevant point cloud data. In order to improve the calculation efficiency, only the point cloud within the range of 8m×12m in front of the harvester is retained. The point cloud data outside this range will be eliminated, and the effect obtained is as follows Figure 9 shown.

[0086] Because this application uses two depth cameras for detection, the number of points in the obtained point cloud data is large, and there is a certain amount of information redundancy. Although this can make the collected information more complete and comprehensive, the computing power of the embedded AI platform is limited. In order to ensure the real-time data processing and enable the detected information to guide the operation in real time, it is necessary to reduce computational redundancy while retaining key structural information. This application requires point cloud simplification.

[0087] In order to reduce the redundancy of point cloud data while maintaining the spatial structure information of the point cloud, this application proposes to use an octree-based weighted downsampling method for simplification, specifically including:

[0088] Step 401, perform octree partitioning on the regional point cloud data to obtain the point cloud data after octree partitioning. In a specific embodiment, the point cloud is partitioned by octree:

[0089] P sample =Octree(P ROI ,Res);

[0090] Among them, Octree is the octree partitioning operation, P sample is the point cloud data after octree division; Res is the octree resolution, which indicates the minimum unit size of the point cloud block. For corn plants, it is set to 0.05m.

[0091] Step 402: Calculate the comprehensive weight according to the distance weight, the viewing angle weight, and the local curvature weight.

[0092] In one embodiment, various weighted surfaces are designed: Distance weights are designed based on a chi-square distribution, constructed and translated in two dimensions: closer point clouds receive higher weights, while those significantly decrease beyond 12 meters. View weights are depicted in the camera coordinate system using a normal distribution, with the central axis as the high-weight region and gradually transitioning toward the edges. Local curvature weights determine significant geometric transitions by determining the change in normal vectors within the subnode range, assigning higher weights to preserving edges and transition details, while reducing weights on smooth areas.

[0093]

[0094]

[0095] where w d is the distance weight; is the standard chi-square distribution; y peak is the coordinate of the highest weight point on the y-axis; Vy and Vz are the translation amounts; w θ is the perspective weight; w cure is the local curvature weight; V k is the kth octet leaf node; min is the minimum eigenvalue of the local covariance; p i is the coordinate of the i-th point in the leaf node; trace is the trace of the local covariance matrix;

[0096] After completing the design of all weight surfaces, visualization is performed, and the three weight surfaces are shown in Figure 10(a), Figure 10(b), and Figure 10(c).

[0097] This application believes that if the performance in any dimension is poor, the overall availability of that point will be greatly reduced. Therefore, multiplication is used to reflect the "shortboard effect". At the same time, in order to prevent the weight factor from being raised to the fourth power, resulting in excessive weight attenuation, a balancing hyperparameter λ is introduced to adjust the degree of confidence attenuation to an appropriate level. The value of λ is generally 1.5-2 and can be modified according to actual conditions.

[0098] Wi =(c i ) λ ×[w d w θ w curv ];

[0099] Among them, W i is the comprehensive weight, λ is the balance hyperparameter, and c is the confidence output by the depth camera itself, which is used to consider the overall lighting factors.

[0100] Step 403: traverse the point cloud data divided by the octree according to the comprehensive weight to obtain simplified point cloud data.

[0101] In one embodiment, in each leaf node, the comprehensive weight w is used i Calculate the weighted position:

[0102]

[0103] Among them, P leaf is the retention point in the cotyledon, p i To traverse all the points in this subleaf.

[0104] Finally, the retained points of each leaf node are aggregated to obtain the streamlined data:

[0105] P 精简 =(P sample , P leaf );

[0106] Where: P 精简 is the point cloud coordinate after weighted downsampling.

[0107] In order to remove isolated noise points in point cloud data, the SOR (Statistical Outlier Removal) method is used for processing. Statistical filtering is based on the mean distance d between each point and its neighboring points. mean To determine whether a point is a noise point. The formula is as follows:

[0108]

[0109] If d mean Exceeding the set threshold d threshold , then the point is considered a noise point. The noise judgment conditions are:

[0110] d mean >d threshold ;

[0111] Where: k is the number of neighborhood points set; P is the coordinate of the current point; P i is the coordinate of the i-th neighboring point of the current point; dthreshold The distance threshold for noise filtering is set to 1.5 times the standard deviation of the neighborhood point distance based on experimental experience.

[0112] Through this judgment, we can 精简 The final point cloud coordinates are obtained by separation:

[0113]

[0114] Among them: P final The final point cloud coordinates are obtained by preprocessing.

[0115] After downsampling, due to the interference of the complex field environment and the accuracy of the depth camera when collecting point cloud information, there are still some useless cluster points in the point cloud data. These points will interfere with the subsequent data processing. Therefore, this application uses statistical filtering to remove isolated noise points.

[0116] Through the above preprocessing steps, the quality and computational efficiency of point cloud data are significantly improved, laying a good foundation for subsequent point cloud clustering analysis and particle count detection. Figure 11 、 Figure 12 As shown in the figure, it can basically reflect the characteristics of crop rows, but the characteristics of individual crop plants are not obvious. Therefore, further processing is needed to highlight the characteristics of individual crops, so point cloud clustering and center point extraction are continued.

[0117] Point cloud clustering is one of the most important steps in this application. The purpose is to effectively group the point cloud data collected by the depth camera and correspond each cluster to a corn plant (or a corn crop), so as to achieve accurate statistics of the number of corn kernels. In order to more efficiently cluster the point cloud of field crops and improve the accuracy of point cloud data clustering, a new method combining triangulation and minimum cut algorithm is proposed. This method is comprehensively designed and optimized for the complexity of field plant distribution and the accuracy of clustering, which helps to improve the accuracy and efficiency of clustering results, especially in complex crop field environments, and can effectively avoid misjudgment and missed judgments.

[0118] In another exemplary embodiment of the present application, triangulation and minimum cut algorithms are combined to perform point cloud clustering on the simplified point cloud data to obtain crop clusters. The above step 500 is replaced by the following steps 501 to 502:

[0119] Step 501 : By establishing triangle connections between simplified point cloud data, edge weights between point cloud data are determined, and a point cloud connection graph is constructed.

[0120] First, to address the large volume of field point cloud data and the complex distribution of plants, Delaunay triangulation is used to rapidly construct a spatial connectivity graph for the point cloud. Triangulation ensures reasonable connections between adjacent points while avoiding meaningless long-distance connections. Through this process, each point is connected to its neighbors based on geometric relationships, providing accurate spatial connectivity information for subsequent point cloud segmentation.

[0121] In one embodiment, in order to quickly and accurately cluster the point clouds of the same corn plant (or corn) into a cluster, the output P final Perform Delaunay triangulation on the point cloud data and construct a point cloud connection graph G = (V, E). Each point is called a vertex v in the graph. i ∈V, if two points can be connected as an edge in the triangulation, then the edge is recorded as e=(v i ,e i )∈E, in the connection graph, the edge weight can be defined as the Euclidean distance, and the specific calculation formula is:

[0122] w e =||p i -p j ||;

[0123] Among them, w e Represents the connection point p i and p j The edge weight of , ||·|| represents the Euclidean distance.

[0124] In this application, Delaunay triangulation is used to construct a point cloud connection graph, that is, by establishing triangle connections between point cloud data and defining edge weights between point clouds.

[0125] Step 502 , cutting the point cloud connection graph, calculating the weight sum of all edge weights of each edge, identifying and removing the edge with the smallest weight sum connecting the point cloud clusters, and obtaining the crop clusters.

[0126] In point cloud clustering, the minimum cut algorithm calculates the weight sum of each edge, identifies the edges with the smallest weight sum connecting the point cloud clusters, and achieves the purpose of effective clustering by removing these edges with the smallest weight sum.

[0127] In one embodiment, the minimum cut algorithm is used to cut the connection graph and cluster the point cloud by removing the edges with the smallest weight. The goal of cutting is to minimize the total weight of the weight and the smallest edge. The specific objective function is:

[0128]

[0129] in is the set of edges that have been "cut off"; these edges with the smallest sum of weights often correspond to gaps between different plants or occlusion "breaks." After the minimum cut, the graph is decomposed into several subgraphs, each of which corresponds to a cluster in the original point cloud, that is, "one plant (or one corn)" corresponds to one cluster.

[0130] The clustering results can be obtained from this:

[0131] C={C1,C2,…,C M};

[0132] Where M is the number of clusters obtained by segmentation. Each C i All points in belong to the same cluster.

[0133] This clustering approach has the following advantages:

[0134] Efficient processing of complex point clouds: The triangulation graph can quickly construct the spatial connection relationship of the point cloud, which is suitable for the point cloud distribution of dense plants in the field.

[0135] Adaptive segmentation: The minimum cut algorithm is based on edge weight cutting and can adaptively separate occluded and overlapping corn plants.

[0136] Reduce misclassification: Compared with the traditional density clustering method, the method combining triangulation and minimum cut can more accurately divide the field point cloud, especially suitable for unevenly distributed plant environments.

[0137] After completing the construction of the connection graph, the minimum cut algorithm is used to segment the point cloud graph. The minimum cut algorithm evaluates the weights of the edges in the graph and adaptively removes the weights and the smallest edges to achieve point cloud clustering. Its core lies in using the distance between points as edge weights to adaptively cut complex point cloud structures, effectively avoiding inter-cluster interference caused by plant occlusion or overlap. Compared with traditional density clustering methods, the advantage of Delaunay triangulation combined with the minimum cut algorithm is that it can accurately control the segmentation accuracy according to the spatial layout of the point cloud, thereby improving the accuracy and stability of the clustering results, better adapting to the complex point cloud environment in the field, and significantly improving the accuracy of segmentation.

[0138] Furthermore, the distribution of field crop point clouds often alternates between dense and sparse locations. Traditional methods, such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise), are prone to misclassification when dealing with this uneven distribution. However, a connectivity graph constructed using triangulation accurately describes the local geometric structure of the point cloud. Combined with a minimum-cut algorithm that allows for flexible segmentation based on edge weights, this effectively reduces misclassification and is particularly suitable for unevenly distributed point cloud scenarios, thereby improving the robustness and reliability of segmentation.

[0139] After clustering, to further characterize the core location of each cluster (crop cluster) and accurately determine the representative point of each cluster (i.e., the location of the corn crop), this application uses two methods: PCA (Principal Component Analysis) principal direction fitting and centroid calculation. These two methods work together to extract the center point of each cluster (crop cluster), which serves as the basis for subsequent grain count calculations.

[0140] In another exemplary embodiment of the present application, the center point of the crop cluster is extracted by principal direction fitting and centroid calculation, and the obtained center point is used as the representative position of the crop cluster. The above step 600 is replaced by the following steps 601 to 602:

[0141] Step 601 : Perform main direction fitting on the crop cluster to obtain the main direction vector of the crop cluster.

[0142] Main direction fitting: PCA can be used to extract the main direction of the data. In order to more accurately locate the center of each cluster or the main axis direction of the plant, the cluster (crop cluster) C i Perform principal component analysis (PCA). Assume C i There are N i points, denoted as n1,n2,…,n N , first calculate the average value of the cluster points (initial centroid), the formula is as follows:

[0143]

[0144] in, is the coordinate average, N i Cluster C i All points within, n k is the kth point;

[0145] Then construct the covariance matrix:

[0146]

[0147] Among them, S i is the covariance matrix;

[0148] To S i Perform eigenvalue decomposition:

[0149] S i v = λv;

[0150] Where λ is the eigenvalue and v is the corresponding eigenvector.

[0151] λ max =max(λ1,λ2,λ3),

[0152] Then with λ max The corresponding eigenvector is denoted as v max , which is the main direction vector of the cluster, is used to describe the axial information of the corn plant.

[0153] Combined with the principal direction analysis (PCA) method, the center point is extracted from each cluster (crop cluster) as the representative point, providing an accurate basis for the final grain count statistics.

[0154] In step 602, the points of the crop cluster are projected onto the main direction vector, a weight function is determined based on the vertical distance of each point to the main direction axis, and the center coordinates are calculated based on the weight function to obtain the center point of each crop cluster. The obtained center point is used as the representative position of the crop cluster.

[0155] Centroid calculation: Calculate the weighted centroid under the guidance of the main direction, and further obtain the center point (which ultimately represents the spatial position of "a plant (or a corn)"). The centroid is the "center of gravity" of a cluster (crop cluster), usually calculated by weighted average of the coordinates of all points in the cluster. Let v max Cluster C i The main direction unit vector of the cluster. k , which can be projected onto v max On, we get:

[0156]

[0157] where d || represents the “scalar position” of the point along the main direction, d ⊥ Indicates the vertical distance from the point to the main direction axis (also known as the radial distance).

[0158] Use Gaussian attenuation form, with d ⊥ The size of is used to define a weight function:

[0159]

[0160] Among them, w(n k ) is point n k The vertical weight value (scalar position), exp is the exponential function symbol, σ is the control bandwidth, which means that we believe that "points within σ from the main axis are valid", and the weight decays rapidly after exceeding the limit.

[0161] If the value of σ is large, the points far from the main axis will also be given a larger weight, and the aggregation will be more "dispersed".

[0162] If σ is small, only points near the trunk are retained, which can effectively suppress noise.

[0163] Let the points in the cluster follow w(nk ) is the weight, then the final centroid coordinate is calculated as follows:

[0164]

[0165] in, Cluster C i The final center of mass coordinates.

[0166] This method allows us to extract the final center point of each cluster, which is the representative location of the cluster and usually corresponds to the location of the corn plant. The results of detecting corn plants at high and low densities are shown in Figures 13(a) and 13(b).

[0167] This can reduce the impact of noise or extreme points on the center position and ensure more robust center extraction.

[0168] By fitting the principal directions and extracting the centroid, we can accurately characterize the center of the corn plant, avoiding deviations caused by occlusion. This method adapts to the columnar or conical distribution characteristics of corn plants and effectively reduces noise interference in point cloud data.

[0169] After completing point cloud clustering and center point extraction, this application further accurately counts the number of crop particles to support the intelligent control system of the harvester.

[0170] Through point cloud clustering, each cluster (crop cluster) represents a corn plant (or a corn plant). After clustering, each cluster C i For example, a point cloud density or minimum point count threshold is often set to perform secondary filtering to avoid misidentifying small clusters as valid plants.

[0171] Therefore, in order to ensure the accuracy of particle count, this application also introduces a density verification step. Figure 14 As shown, this is a schematic diagram of the specific process of the corn crop plant density detection method of this application. By calculating the point cloud density of each cluster (crop cluster), clusters with lower density can be screened out. These clusters may be noise points or misdetected clusters. Therefore, clusters (crop clusters) with density lower than the set threshold will be eliminated. The formula for density verification is as follows:

[0172] First calculate the cluster bounding box volume V i :

[0173] V i =(x max -x min )·(y max -y min )·(z max -zmin );

[0174] Among them, V i is the bounding box volume of the i-th cluster, x max ,x min is the maximum / minimum horizontal coordinate of the points within the cluster; y max ,y min is the maximum / minimum longitudinal coordinate; z max ,z min is the maximum / minimum height coordinate;

[0175] Then use point N i Calculate the "point cloud density":

[0176]

[0177] Among them, ρ i is the point cloud density of the i-th cluster; N i is the number of points in the i-th cluster; V i is the bounding box volume of the i-th cluster;

[0178] Then construct the density histogram:

[0179]

[0180] Among them, p k is the normalized frequency of the kth density interval; n k is the number of clusters falling into the kth density interval; n j is the number of clusters falling into the jth density interval; K is the total number of density intervals (usually 256);

[0181] The inter-class variance is then calculated to determine the best adaptive segmentation value:

[0182]

[0183] in, is the inter-class variance when the kth interval is used as the boundary; ω k (1-ω k ) is the weight product of the two categories; (μ T ·ω k -μ k ) 2 is the degree of deviation between the means of two categories; p j is the proportion of clusters traversed that are smaller than k; ρ j is the density value that is less than k; ω k is the total weight of the “low density class” when the kth interval is used as the threshold; μ k is the average density value of the “low density class”; μ T is the average density of all clusters.

[0184] Based on the calculation of the between-class variance, the interval index with the largest value is selected as the optimal dividing point:

[0185]

[0186] Among them, k * The interval index that maximizes the inter-class variance is considered the optimal dividing point; argmax is the function that returns the maximum value; ρ thresh The minimum effective density threshold selected adaptively is used as the basis for cluster screening.

[0187] Finally, the number of plants after filtering all clusters with the adaptive cutoff point is:

[0188] M valid =|{C i |ρ i ≥ρ thresh}|;

[0189] Among them, M valid The number of plant clusters that are finally effectively retained corresponds to the number of corn kernels detected.

[0190] Finally, the system outputs the corn kernel count detection results, providing accurate data support for the intelligent control of the harvester.

[0191] This application uses triangulation graphs and minimum cut algorithms to accurately cluster corn plants in point clouds so that each cluster corresponds to one (or one) corn plant. At the same time, the center point of each cluster is extracted through PCA and centroid calculation to further improve the accuracy and stability of single plant identification. Finally, cluster statistics and density verification are combined to ensure the accuracy and reliability of the grain number detection results. These test results will provide data support for the intelligent control system of the harvester, helping it to dynamically adjust operating parameters according to crop density, optimize harvesting operations, improve harvesting efficiency and reduce losses. This method provides an efficient and robust technical solution for automatic detection of corn grain numbers in complex field environments.

[0192] Based on the same inventive concept, embodiments of the present application also provide a device for implementing the aforementioned corn crop plant density detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the corn crop plant density detection device provided below can be found in the above-described limitations of the corn crop plant density detection method and will not be further elaborated here.

[0193] In an exemplary embodiment, Figure 15As shown, a corn crop plant density detection device is provided, which includes: an acquisition module 1, a fusion module 2, a region of interest extraction module 3, a point cloud simplification module 4, a point cloud clustering module 5, and a center point extraction module 6.

[0194] The acquisition module 1 is used to obtain the initial point cloud data of corn crops collected in real time by multiple depth cameras at set positions and postures; the fusion module 2 is used to perform temporal alignment and spatial alignment on the initial point cloud data and fuse them to obtain fused point cloud data; the region of interest extraction module 3 is used to extract the region of interest from the fused point cloud data to obtain regional point cloud data; the point cloud simplification module 4 is used to perform point cloud simplification on the regional point cloud data to obtain simplified point cloud data; the point cloud clustering module 5 is used to perform point cloud clustering on the simplified point cloud data in combination with triangulation and minimum cut algorithm to obtain crop clusters; the center point extraction module 6 is used to extract the center point of the crop cluster by main direction fitting and centroid calculation, and use the obtained center point as the representative position of the crop cluster.

[0195] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 16 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store corn crop plant density detection processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the corn crop plant density detection method is implemented.

[0196] Those skilled in the art will understand that Figure 16 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0197] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0198] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0199] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0200] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0201] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0202] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0203] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0204] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for detecting plant density of corn crops, characterized in that: include: Obtain initial point cloud data of corn crops collected in real time by multiple depth cameras at set positions and postures; Perform temporal registration and spatial registration on the initial point cloud data and fuse them to obtain fused point cloud data; Extracting a region of interest from the fused point cloud data to obtain regional point cloud data; Simplify the regional point cloud data to obtain simplified point cloud data; Combining triangulation and minimum cut algorithm to cluster the simplified point cloud data to obtain crop clusters; The center point of the crop cluster is extracted by main direction fitting and centroid calculation, and the obtained center point is used as the representative position of the crop cluster.

2. The corn crop plant density detection method according to claim 1, characterized in that: The initial point cloud data is temporally registered and spatially registered and fused to obtain fused point cloud data, specifically including: If the timestamp difference between the frames of initial point cloud data collected by the multiple depth cameras is not greater than the time threshold, the frames of initial point cloud data collected by the multiple depth cameras are determined to be the initial point cloud data at the same time; According to the position and posture of the depth camera, the initial point cloud data of multiple depth cameras at the same time are unified into the global coordinate system using the coordinate transformation matrix to obtain the point cloud data of multiple depth cameras in the global coordinate system; The point cloud data of multiple depth cameras in the global coordinate system are stacked and fused to obtain fused point cloud data.

3. The corn crop plant density detection method according to claim 1, characterized in that: Perform gravity correction during spatial registration or after fusion.

4. The corn crop plant density detection method according to claim 1, characterized in that: The point cloud simplification is performed by using an octree-based weighted downsampling method, which specifically includes: Perform octree division on the regional point cloud data to obtain the point cloud data after octree division; Calculate the comprehensive weight based on the distance weight, viewing angle weight, and local curvature weight; The point cloud data after octree division is traversed by combining comprehensive weights to obtain simplified point cloud data.

5. The corn crop plant density detection method according to claim 1, characterized in that: The simplified point cloud data is clustered using triangulation and the minimum cut algorithm to obtain crop clusters, including: By establishing triangle connections between simplified point cloud data, edge weights between point cloud data are determined, and a point cloud connection graph is constructed; The point cloud connection graph is cut, and the weight sum of all edge weights of each edge is calculated to identify and remove the edge with the smallest weight sum connecting the point cloud clusters to obtain the crop clusters.

6. The corn crop plant density detection method according to claim 1, characterized in that: The center point of the crop cluster is extracted by fitting the main direction and calculating the centroid. The obtained center point is used as the representative position of the crop cluster. Specifically, the following steps are performed: Perform main direction fitting on the crop cluster to obtain the main direction vector of the crop cluster; The points of the crop cluster are projected onto the main direction vector, and the weight function is determined according to the vertical distance of each point to the main direction axis. The centroid coordinates are calculated according to the weight function to obtain the center point of each crop cluster, and the obtained center point is used as the representative position of the crop cluster.

7. A corn crop plant density detection device, characterized in that: The corn crop plant density detection device comprises: An acquisition module is used to obtain the initial point cloud data of corn crops collected in real time by multiple depth cameras at set positions and postures; The fusion module is used to perform temporal and spatial registration and fusion on the initial point cloud data to obtain fused point cloud data; An area of ​​interest extraction module is used to extract an area of ​​interest from the fused point cloud data to obtain regional point cloud data; Point cloud simplification module, used to simplify regional point cloud data to obtain simplified point cloud data; Point cloud clustering module, which combines triangulation and minimum cut algorithm to cluster the simplified point cloud data into crop clusters; The center point extraction module is used to extract the center point of the crop cluster through main direction fitting and centroid calculation, and use the obtained center point as the representative position of the crop cluster.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the corn crop plant density detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the corn crop plant density detection method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the corn crop plant density detection method according to any one of claims 1 to 6 is implemented.

Images