A method and system for detecting peanut seed skips

Abstract

This invention relates to a method and system for detecting missed planting of peanut seeds. The method calculates the dynamic frame rate of a multi-view camera based on the real-time angular velocity of the seed metering reel, thereby activating the multi-view camera to acquire multi-view images of peanut seeds on the seed metering reel. Visual features of the peanut seeds are extracted from the multi-view images. Scene-based feature enhancement is performed using a first register for preliminary identification of seed adhesion regions in two-dimensional space and for single-seed contour separation of the identified adhesion regions, and a second register for visual feature completion by associating visual feature sets from multiple perspectives. This generates seed point cloud maps and seed depth maps corresponding to the peanut seeds, obtaining all single-seed segmentation results and the detection area corresponding to each seed metering hole. Matching these two results achieves missed planting detection. Therefore, this invention is both resistant to adhesion and occlusion and adaptable to dynamic operating scenarios, improving the accuracy of missed planting detection.

Description

Technical Field

[0001] This invention relates to the field of agricultural automation technology, and in particular to a method and system for detecting missed sowing of peanut seeds. Background Technology

[0002] As an important oilseed crop and cash crop in my country, the quality of peanut planting directly affects subsequent yield and economic benefits, and missed planting is one of the most common quality problems in peanut planting.

[0003] Existing methods for detecting missed peanut seed planting mainly fall into two categories: 1. Contact detection using physical sensors installed in the seed box, furrow opener, seed dispensing holes, and seed rollers; 2. Non-contact detection using machine vision algorithms. However, due to the differences in size, shape, and color among peanut seeds, the above methods generally suffer from the following drawbacks: 1. The physical sensor has a demanding installation location, and when the seeder is operating at high speed, the signal jitter is obvious, which can easily lead to signal distortion; the image acquired by the visual sensor in dynamic environment is prone to motion blur, which affects subsequent detection. 2. Most existing methods are based on two-dimensional images for quantity statistics, which cannot obtain the three-dimensional spatial information of seeds. This results in poor ability to distinguish between repeated adhesion and occlusion, and is prone to missed or false judgments.

[0004] Therefore, there is an urgent need for a peanut seed under-seeding detection technology that is resistant to adhesion and shading and adaptable to dynamic operating scenarios, in order to improve the accuracy of under-seeding detection. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: the present invention provides a method and system for detecting missed sowing of peanut seeds, which can resist adhesion and shading in the detection of missed sowing, and can adapt to dynamic operation scenarios, thereby improving the accuracy of missed sowing detection.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a method for detecting missed sowing of peanut seeds, comprising: The real-time angular velocity of the seeding reel is collected in real time. The dynamic frame rate of the multi-view camera is calculated based on the real-time angular velocity. The multi-view camera is started based on the dynamic frame rate to collect multi-view images of peanut seeds on the seeding reel, thereby obtaining multi-view images of peanut seeds. All images from each viewpoint are input into a pre-trained encoder to extract visual features of peanut seeds, resulting in a visual feature set for each viewpoint. Based on the visual feature sets from all viewpoints, scene-based feature enhancement is performed on the visual feature sets from each viewpoint using N learnable registers, resulting in a scene-enhanced visual feature set for each viewpoint. The N learnable registers include a first register for initially identifying seed adhesion regions in two-dimensional space and performing single-seed contour separation on the identified seed adhesion regions, and a second register for associating visual feature sets from multiple viewpoints to complete the visual feature set from a single viewpoint. The layout information of each seeding hole on the seeding reel is obtained. Based on the layout information and the visual feature set enhanced by scene-based features of all perspectives, a seed point cloud map and a seed depth map corresponding to the peanut seed are generated. Motion compensation is performed on the seed point cloud map and the seed depth map based on the real-time angular velocity and the dynamic frame rate to obtain the motion-compensated seed point cloud map and the motion-compensated seed depth map. Each seeding hole is projected onto the motion-compensated seed depth map according to the layout information and the real-time angular velocity to obtain the detection area corresponding to each seeding hole. The motion-compensated seed point cloud map is geometrically segmented in three-dimensional space to obtain all individual seed segmentation results. All individual seed segmentation results are then projected onto the motion-compensated seed depth map and matched with the detection area corresponding to each seeding hole to obtain the seeding quantity corresponding to each seeding hole, thereby achieving missed seeding detection.

[0007] The beneficial effects of this invention are as follows: The dynamic frame rate of the multi-view camera is calculated based on the real-time angular velocity of the seed metering reel, which can adapt to the dynamic operation scenario of the seed metering reel. This ensures that the multi-view camera started at this dynamic frame rate can acquire clear and effective peanut seed images, avoiding image blurring and ghosting caused by the excessive rotation speed of the seed metering reel under traditional fixed frame rate acquisition, thus improving the quality of the acquired multi-view peanut seed images. Peanut seed visual features are extracted from all images under each viewpoint to ensure the comprehensiveness of the obtained visual feature set. The first register initially identifies seed adhesion areas in two-dimensional space and performs single-seed contour separation on the identified seed adhesion areas, effectively solving the problem of visual feature confusion and recognition deviation caused by peanut seeds easily sticking on the seed metering reel, ensuring the accuracy of the visual feature set after scene-enhanced features. The second register can associate the visual feature sets of multiple views to complete the features in the visual feature set under a single viewpoint, avoiding the problem of missing visual feature values ​​due to visual blind spots in a single viewpoint, thus improving the completeness of the visual feature set after scene-enhanced features. Based on the layout information of each seeding hole on the seeding reel and the visual feature set enhanced by scene-based features from all perspectives, seed point cloud maps and seed depth maps are generated. This method uses 3D point clouds to overcome the limitation that 2D images cannot accurately reflect the spatial distribution of peanut seeds. Furthermore, motion compensation is performed on the seed point cloud and seed depth map based on the real-time angular velocity of the seeding reel and the dynamic frame rate of the multi-view camera, further eliminating the feature drift problem caused by the actual rotation of the seeding reel. In addition, the layout information of each seeding hole on the seeding reel is combined to achieve accurate mapping of the detection area of ​​each seeding hole, thereby improving the accuracy of subsequent missed seed detection.

[0008] Optionally, the step of calculating the dynamic frame rate of the multi-view camera based on the real-time angular velocity, and starting the multi-view camera to acquire multi-view images of peanut seeds on the seed reel based on the dynamic frame rate, includes: The real-time phase of the seed metering reel is acquired in real time. This real-time phase is then input into a first theoretical formula to calculate the theoretical phase of each seeding hole on the seed metering reel. The real-time angular velocity is also input into a first dynamic formula to calculate the dynamic frame rate of the multi-view camera. Simultaneously, exposure constraints for the multi-view camera are generated based on the real-time angular velocity. The first theoretical formula is:

[0009] in, This represents the theoretical phase of the i-th seeding hole on the seeding reel. This indicates the initial calibration phase of the i-th seeding hole on the seeding reel. Indicates the real-time phase of the seeding reel; The first dynamic formula is: ; in, This indicates the dynamic frame rate, and N represents the preset minimum number of frames. This indicates the preset optimal imaging window. This indicates the real-time angular velocity of the seeding reel; The exposure constraint is as follows: ; ; in, Indicates the rate of peanut seed synthesis. Indicates the radius of the seeding reel. Indicates the maximum random bounce speed. Indicates the exposure time. This indicates the meta-size of a multi-view camera; Calculate the phase difference between the real-time phase and the theoretical phase of each seeding hole. When the phase difference meets the first condition, start the multi-view camera to acquire multi-view images of peanut seeds on the seeding reel based on the dynamic frame rate and the exposure constraint.

[0010] As described above, the phase difference between the real-time phase of the seed metering wheel and the theoretical phase of each seed metering hole is introduced as the starting mechanism for the multi-view camera. This ensures that the multi-view camera based on the dynamic frame rate can start acquisition in the optimal imaging window, improving the quality of the multi-view images. Furthermore, exposure constraints are introduced to further improve the clarity of the multi-view images, highlighting the contrast between peanut seeds and the background of the seed metering wheel in the multi-view images, which facilitates the accurate extraction of the visual features of peanut seeds in the subsequent process.

[0011] Optionally, the step of inputting all images from each viewpoint into a pre-trained encoder to extract visual features of peanut seeds yields a visual feature set corresponding to each viewpoint. Based on the visual feature sets from all viewpoints, scene-based feature enhancement is performed on the visual feature sets corresponding to each viewpoint using N learnable registers, resulting in a scene-enhanced visual feature set for each viewpoint, including: For each image from each viewpoint, connected component analysis is performed sequentially to obtain all foreground regions of each image. The area of ​​the bounding rectangle and the centroid coordinates of each foreground region are calculated, and the area of ​​the bounding matrix is ​​used as the projection size of the peanut seed in the corresponding foreground region. The standard deviation of the dimensions of each image is calculated based on the projected dimensions of peanut seeds in all foreground regions of each image, and the minimum average centroid distance of each image is calculated based on the centroid coordinates of all foreground regions of each image. Adaptive segmentation of the corresponding image is performed based on the minimum distance between the average centroids and the standard deviation of the size of each image to obtain all image blocks; All image patches from the same viewpoint are aggregated to obtain an image patch sequence for each viewpoint. The image patch sequence for each viewpoint is then input into a pre-trained encoder to extract visual features of peanut seeds, thus obtaining a visual feature set corresponding to each viewpoint. Based on the visual feature sets of all viewpoints, scene-based feature enhancement is performed on the visual feature sets corresponding to each viewpoint through N learnable registers, resulting in the scene-based feature enhanced visual feature sets for each viewpoint.

[0012] As described above, connected component analysis accurately separates the foreground and background regions of peanut seeds, avoiding interference from the background region. Based on the standard deviation of the size calculated from the area of ​​the circumscribed rectangle of the foreground region and the minimum distance of the average centroid, the image is adaptively divided into blocks to ensure that each image block contains only a single peanut seed or a local region of a single peanut seed. This avoids deviations in the extraction of visual features of peanut seeds caused by the overlapping of multiple peanut seeds, thereby improving the accuracy of the visual feature set.

[0013] Optionally, the visual feature set based on all viewpoints is further enhanced with scene-based features using N learnable registers to obtain the scene-enhanced visual feature set for each viewpoint, including: The first register groups the image blocks belonging to the same foreground region under each viewpoint to obtain the image block group corresponding to each foreground region, and constructs a local feature map based on the visual feature set of the image block group corresponding to each foreground region. The local feature map is input into a Gabor filter for peanut seed coat vein texture feature analysis to obtain the texture breakage ratio. At the same time, the local feature map is fitted with a least square ellipse to obtain the actual axis ratio, and the reference axis ratio of the image to which the local feature map belongs is obtained. The axis ratio anomaly is calculated based on the reference axis ratio and the actual axis ratio. The adhesion is calculated based on the axis ratio anomaly and the texture breakage ratio. If the adhesion exceeds the first adhesion threshold, the foreground region corresponding to the local feature map is the seed adhesion region. A main contour model with axis ratio constraint is used to perform single-seed contour separation in the seed adhesion region to obtain contour separation information. Based on the contour separation information, the visual feature set under the corresponding viewpoint is updated to output the first updated visual feature set. The second register establishes a correspondence between the same seeds from different viewpoints based on the contour separation information, and identifies the occluded image block from the first updated visual feature set according to the correspondence. For the occluded image block from the first updated visual feature set of other viewpoints, the register extracts the corresponding visual features as known features from the first updated visual feature set of other viewpoints. Based on the known features, the register performs feature completion on the visual features corresponding to the occluded image block to output the second updated visual feature set. The second updated visual feature set is used as the visual feature set after scene-based feature enhancement.

[0014] Optionally, the N learnable registers further include a third register for separating the seeding wheel background and a fourth register for feature alignment, wherein the step of using the second updated visual feature set as the scene-enhanced visual feature set includes: The third register extracts the corresponding local region from the images of adjacent frames according to the spatial position encoding of each image block in the second updated visual feature set. For each local region, the dynamic background features of the seeding wheel are extracted from the corresponding second updated visual feature set through the rotation blur kernel of the first twin network. The dynamic background features are then convolved with the corresponding local region to obtain the corresponding background feature response value. Simultaneously, the actual movement trajectory of peanut seeds is obtained from the corresponding second updated visual feature set through the second twin network, and the matching degree between the actual movement trajectory and the preset ideal movement trajectory is calculated. When the matching degree is lower than the first matching threshold and the corresponding background feature response value exceeds the first background threshold, the corresponding local area is marked as the seeding wheel background, and the image block corresponding to the local area marked as the seeding wheel background is subjected to feature suppression to achieve seeding wheel background separation, so as to output the third updated visual feature set, wherein the rotation speed of the rotation blur kernel is positively correlated with the real-time angular velocity; The fourth register establishes a local coordinate system for peanut seeds based on the ellipsoidal geometry prior of the peanut seeds. It extracts the pose features of each seed from the third updated visual feature set, calculates the pose deviation between the pose features of the same seed from different viewpoints based on the local coordinate system, and performs rotation compensation and position calibration on the visual features corresponding to the same seed from different viewpoints based on the pose deviation, so as to realize the alignment processing of the visual features of the same seed from different viewpoints, and outputs the visual feature set of each viewpoint after alignment processing. The visual feature set of each viewpoint after alignment processing is used as the visual feature set after scene-based feature enhancement.

[0015] As described above, the first register introduces aspect ratio anomaly based on the texture breakage ratio. This enables rapid identification of seed adhesion regions while improving the accuracy of identification from a morphological perspective. It also combines an active contour model with aspect ratio constraints for single-seed contour separation, forcing the separated single-seed contours to conform to standard shapes, thus improving the thoroughness of separation. The second register, during feature completion, first establishes a correspondence between the same seeds from different viewpoints based on contour separation information, avoiding confusion of seed visual features from different perspectives and improving the accuracy of feature completion. The third register extracts the dynamic background features of the seeding wheel using a rotational blur kernel positively correlated with the real-time angular velocity of the seeding wheel. This effectively distinguishes image blocks belonging to the seeding wheel background and achieves precise separation of the seeding wheel background through feature suppression, completely eliminating background interference. The fourth register establishes a local coordinate system for peanut seeds based on the ellipsoidal geometric prior of the peanut seeds, thereby aligning the visual features of the same seed from different viewpoints and improving the accuracy of alignment.

[0016] Optionally, generating the seed point cloud map and seed depth map corresponding to the peanut seed based on the layout information and the visual feature set enhanced with scene-based features from all viewpoints includes: The first multi-head attention network and the second multi-head attention network alternately perform intra-frame local attention weight calculation and inter-frame global attention weight calculation on the visual feature set after scene-based features of all viewpoints until the number of alternating executions reaches the preset number of loops, thereby generating a seed point cloud map and a seed depth map corresponding to the peanut seed. The process of calculating intra-frame local attention weights and inter-frame global attention weights by sequentially applying the enhanced visual feature sets of all viewpoints through the first and second multi-head attention networks includes: The first multi-head attention network identifies the image patch belonging to the seeding hole from the visual feature set enhanced by the scene-based features corresponding to each viewpoint based on the layout information, and uses an improved LBP operator to calculate the texture histogram corresponding to the image patch of the seeding hole, generates the corresponding texture entropy based on the texture histogram, and uses the Prewitt operator to calculate the gradient corresponding to the image patch of the seeding hole, and uses a clustering algorithm to cluster the direction of the gradient to obtain the category to which the image patch of the seeding hole belongs; Based on the category of the image patch of the seeding hole, the corresponding intra-frame local attention weight is generated, and the corresponding scene-enhanced visual feature set is enhanced with the intra-frame local attention weight to obtain the visual feature set enhanced with local features for each viewpoint. The visual feature sets enhanced with local features from all perspectives are merged to obtain the global visual feature set; The global visual feature set is used as input to the second multi-head attention network. The second multi-head attention network calculates inter-frame global attention weights under spatiotemporal and motion constraints. Based on these inter-frame global attention weights, global feature enhancement is performed on the global visual feature set to obtain the enhanced global visual feature set. The spatiotemporal constraints are: ; in, This represents the actual pixel position offset of the same seeding hole between adjacent frames. This indicates the real-time angular velocity of the seeding reel. Indicates the time interval between adjacent frames. This represents the pixel arc length corresponding to the central angle of adjacent seeding holes. This indicates the preset offset error threshold. Indicates the radius of the seed reel; The motion constraint conditions are as follows: ; ; in, The x-coordinate represents the pixel coordinates of the seeding hole from the main viewpoint. The x-coordinate represents the pixel coordinates of the seeding hole from the auxiliary viewpoint. The ordinate represents the pixel coordinates of the seeding hole from the main viewpoint. The ordinate represents the pixel coordinates of the seeding hole from the auxiliary viewpoint. This represents the preset horizontal axis error threshold. This indicates the preset vertical coordinate error threshold.

[0017] As described above, by alternately calculating intra-frame local attention weights and inter-frame global attention weights through two multi-head attention networks, the system focuses on the seeding holes and seed edge regions within a single frame image, addressing the problem of insufficient attention to small or occluded targets. Furthermore, through spatiotemporal and motion constraints, it achieves global fusion across multiple perspectives and frames, resolving the issues of information disconnect and feature misalignment in dynamic operations. This ensures that the generated seed point cloud and depth map accurately represent the three-dimensional spatial information of the seeds, providing a reliable foundation for subsequent geometric segmentation and improving the accuracy of missed seed detection.

[0018] Optionally, the layout information includes the radial distance, initial axial angle, and axial position of each seeding hole relative to the rotation axis of the seeding wheel. Motion compensation is performed on the seed point cloud map and the seed depth map based on the real-time angular velocity and the dynamic frame rate to obtain a motion-compensated seed point cloud map and a motion-compensated seed depth map. Each seeding hole is then projected onto the motion-compensated seed depth map according to the layout information and the real-time angular velocity to obtain the detection area corresponding to each seeding hole, including: The real-time angular velocity and the dynamic frame rate are respectively input into the first compensation formula and the second compensation formula for calculation to obtain the corresponding first compensation value and second compensation value. The first compensation formula is: ; in, This represents the first compensation value. This indicates the real-time angular velocity of the seeding reel. This indicates the distance from the peanut seed being photographed to the multi-view camera. This represents the angle between the direction of peanut seed movement and the X-axis of the multi-view camera. Indicates dynamic frame rate; The second compensation formula is: ; in, This represents the second compensation value. This indicates the real-time angular velocity of the seeding reel. This indicates the distance from the peanut seed being photographed to the multi-view camera. This represents the angle between the direction of peanut seed movement and the X-axis of the multi-view camera. Indicates dynamic frame rate; Motion compensation is performed on the seed depth map based on the first compensation value and the second compensation value. The horizontal coordinate of each pixel in the seed depth map is translated according to the first compensation value, and the vertical coordinate of each pixel in the seed depth map is translated according to the second compensation value to obtain the motion-compensated seed depth map. The intrinsic and extrinsic parameters of the multi-view camera are obtained. Based on the motion-compensated seed depth map, a new seed point cloud map is regenerated using the intrinsic and extrinsic parameters. The new seed point cloud map is used as the motion-compensated seed point cloud map. Based on the radial distance, initial axial angle, axial position, and real-time angular velocity of each seed metering hole relative to the rotation axis of the seed metering wheel, the real-time three-dimensional spatial coordinates of each seed metering hole on the seed metering wheel are calculated. Then, the real-time three-dimensional spatial coordinates of each seed metering hole on the seed metering wheel are projected onto the motion-compensated seed depth map using the intrinsic and extrinsic parameters to obtain the detection area corresponding to each seed metering hole.

[0019] As described above, the seed depth map is motion-compensated using the first compensation value and the second compensation value calculated by the first compensation formula and the second compensation formula. The motion-compensated seed depth map is then combined with the camera's intrinsic and extrinsic parameters to regenerate a point cloud map. This effectively eliminates the lag or misalignment of seed feature positions under dynamic operations, ensuring that the motion-compensated point cloud map and the motion-compensated depth map correspond precisely in spatial position. Based on the seed metering hole layout information, the detection area of ​​each seed metering hole is accurately located, avoiding errors in the sowing quantity statistics caused by the offset of the detection area.

[0020] Optionally, the step of performing three-dimensional geometric segmentation on the motion-compensated seed point cloud to obtain all individual seed segmentation results, and projecting all individual seed segmentation results onto the motion-compensated seed depth map and matching them with the detection area corresponding to each seeding hole to obtain the sowing quantity corresponding to each seeding hole, thereby realizing the missed sowing detection, includes: An ellipsoid fitting algorithm was used to fit the motion-compensated seed point cloud map based on the ellipsoidal geometric prior of peanut seeds to obtain all the initial seed point cloud clusters. From each initial seed point cloud cluster, a core point is selected as the current growth point, and all neighboring points corresponding to the current growth point are obtained. Using the current growth point as the growth center, a growth traversal is performed on all corresponding neighboring points. The depth difference between each neighboring point and the corresponding current growth point in the motion-compensated seed depth map, and the angle between their normal vectors in the motion-compensated point cloud map, are calculated. The angle between the normal vectors and the depth difference are input into a dynamic weight formula for calculation to obtain the corresponding dynamic weight value. The dynamic weight formula is as follows: ; in, This represents the dynamic weight value corresponding to the current growth point C and its neighboring point c. This represents the depth difference between the current growth point C and its neighboring point c in the motion-compensated seed depth map. This represents the angle between the normal vectors of the current growth point C and its neighboring point c in the motion-compensated point cloud map. This indicates the preset weighting coefficients; If the dynamic weight value is greater than the first weight threshold, the neighborhood point corresponding to the dynamic weight value is assigned to the initial seed point cloud cluster to which the current growth point belongs, and the neighborhood point newly assigned to the initial seed point cloud cluster is used as the new current growth point to perform growth traversal again until the final dynamic weight value is less than or equal to the first weight threshold, so as to obtain all the final seed point cloud clusters, and use all the final seed point cloud clusters as all the single seed segmentation results. All single-seed segmentation results are projected onto the motion-compensated seed depth map. The pixel area occupied by each single-seed segmentation result in the motion-compensated seed depth map is obtained. The pixel area is matched with the detection area corresponding to each seeding hole. The number of successfully matched single-seed segmentation results in the detection area corresponding to each seeding hole is counted. The number is used as the sowing quantity of the corresponding seeding hole to obtain the sowing quantity corresponding to each seeding hole, thereby realizing the detection of missed sowing.

[0021] As described above, ellipsoid fitting based on ellipsoidal geometric priors can quickly acquire initial seed point cloud clusters, laying the foundation for subsequent segmentation. Growth traversal based on dynamic weight values, combined with depth difference and normal vector angle, can achieve complete segmentation of single seeds, capturing all three-dimensional point clouds of a single seed to the greatest extent, avoiding incomplete segmentation caused by irregular peanut seed shape, uneven surface, or local point cloud dispersion, thereby improving the accuracy and reliability of missed seed detection.

[0022] Optionally, obtaining the seed quantity corresponding to each seed metering hole to achieve missed seed detection includes: When the number of seeds sown is less than the first quantity threshold, the seeding hole corresponding to the number of seeds sown is marked as a missed seeding. When the number of seeds sown is greater than or equal to the first quantity threshold, the KNN value of each successfully matched single seed segmentation result is calculated using the KNN algorithm, and the point cloud curvature of each successfully matched single seed segmentation result is calculated using the curvature change formula. If the point cloud curvature is greater than the first curvature threshold and the KNN value is greater than the first roughness threshold, then the corresponding successfully matched single seed segmentation result is marked as having surface defects, and the corresponding seeding hole is marked as having unqualified sowing quality. and / or When the number of seeds sown is greater than or equal to the first quantity threshold, the actual volume of each successfully matched single seed segmentation result is calculated, and the actual volume is input into the fullness calculation formula to calculate the fullness and obtain the corresponding fullness. If the fullness is lower than the first fullness threshold, the corresponding successfully matched single seed segmentation result is marked as an invalid seed, and the corresponding seeding hole is marked as unqualified in sowing quality.

[0023] As described above, the KNN algorithm can accurately mine the spatial neighborhood features of adjacent single-seed segmentation results, while point cloud curvature can accurately capture surface features. Compared with traditional surface defect detection, this improves the accuracy of surface defect detection and eliminates the need for manual judgment of saturation, thus improving efficiency and avoiding subjective human error. Combining the sowing quantity with sowing quality not only identifies missed sowing but also marks scenarios of substandard sowing quality, providing staff with more comprehensive feedback on sowing quality. This facilitates timely adjustment of sowing parameters, inspection of seed quality, and improvement of overall peanut sowing quality and yield.

[0024] In a second aspect, the present invention provides a peanut seed missing detection system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the peanut seed missing detection method described in the first aspect.

[0025] The technical effect of the peanut seed under-planting detection system provided in the second aspect is the same as that of the peanut seed under-planting detection method provided in the first aspect. Attached Figure Description

[0026] Figure 1 This is a flowchart of a method for detecting missed sowing of peanut seeds provided in this embodiment; Figure 2 This is a schematic diagram of the overall process of a method for detecting missed sowing of peanut seeds provided in this embodiment; Figure 3 This is a schematic diagram of the structure of a peanut seed missing detection system provided in this embodiment.

[0027] [Explanation of Labels in the Attached Image] 1. A detection system for missed sowing of peanut seeds; 2. Processor; 3. Memory. Detailed Implementation

[0028] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0029] Example 1 Please refer to Figures 1 to 2 This invention provides a method for detecting missed sowing of peanut seeds, comprising the following steps: S1. Real-time acquisition of the real-time angular velocity of the seed metering reel, calculation of the dynamic frame rate of the multi-view camera based on the real-time angular velocity, and activation of the multi-view camera to acquire multi-view images of peanut seeds on the seed metering reel based on the dynamic frame rate, thereby obtaining multi-view images of peanut seeds. In this embodiment, industrial-grade cameras, pre-calibrated spatially, are installed at angles directly above and to the side of the seeding reel. Sensors are installed at the shaft end of the seeding reel to detect its real-time angular velocity and phase. Figure 2 As shown, the dynamic frame rate of the multi-view camera is calculated based on the real-time angular velocity of the seeding reel, and the multi-view camera is started with this dynamic frame rate to acquire multi-view images of peanut seeds on the seeding reel, thus obtaining multi-view images of peanut seeds.

[0030] At this point, step S1, which involves calculating the dynamic frame rate of the multi-view camera based on the real-time angular velocity and then starting the multi-view camera to acquire multi-view images of the peanut seeds on the seed reel based on the dynamic frame rate, includes: S11. Real-time acquisition of the real-time phase of the seed metering reel; inputting the real-time phase into a first theoretical formula to calculate the theoretical phase of each seed metering hole on the seed metering reel; and inputting the real-time angular velocity into a first dynamic formula to calculate the dynamic frame rate of the multi-view camera. Simultaneously, based on the real-time angular velocity, exposure constraints for the multi-view camera are generated. The first theoretical formula is:

[0031] in, This represents the theoretical phase of the i-th seeding hole on the seeding reel. This indicates the initial calibration phase of the i-th seeding hole on the seeding reel. Indicates the real-time phase of the seeding reel; The first dynamic formula is: ; in, This indicates the dynamic frame rate, and N represents the preset minimum number of frames. This indicates the preset optimal imaging window. This indicates the real-time angular velocity of the seeding reel; The exposure constraint is as follows: ; ; in, Indicates the rate of peanut seed synthesis. Indicates the radius of the seeding reel. Indicates the maximum random bounce speed. Indicates the exposure time. This indicates the meta-size of a multi-view camera; S12. Calculate the phase difference between the real-time phase and the theoretical phase of each seeding hole. When the phase difference meets the first condition, start the multi-view camera to acquire multi-view images of peanut seeds on the seeding wheel based on the dynamic frame rate and the exposure constraint.

[0032] In this embodiment, as Figure 2 As shown, the theoretical phase of each seeding hole on the seeding reel is calculated based on the real-time phase of the seeding reel and the first theoretical formula. The dynamic frame rate of the multi-view camera is calculated based on the real-time angular velocity and the first dynamic formula. The exposure constraint condition of the multi-view camera is generated based on the real-time angular velocity. The phase difference is used as the activation mechanism. When the phase difference meets the first condition, the multi-view camera is activated to acquire multi-view images of the peanut rice dumplings on the seeding reel based on the dynamic frame rate and exposure constraint condition. The first condition is: phase difference ≤ .

[0033] S2. Input all images from each viewpoint into the pre-trained encoder to extract visual features of peanut seeds, and obtain the visual feature set corresponding to each viewpoint. Based on the visual feature set of all viewpoints, perform scene-based feature enhancement on the visual feature set corresponding to each viewpoint through N learnable registers to obtain the scene-enhanced visual feature set corresponding to each viewpoint. The N learnable registers include a first register for initially identifying seed adhesion regions in two-dimensional space and performing single-seed contour separation on the identified seed adhesion regions, and a second register for associating visual feature sets from multiple viewpoints to complete the visual feature set under a single viewpoint. In this embodiment, as Figure 2 As shown, all images from each viewpoint are input into a pre-trained encoder for peanut seed visual feature extraction, resulting in a visual feature set for each viewpoint. The visual feature set refers to the collection of peanut seed visual features from all images within the same viewpoint, including visual geometric features such as peanut seed edges, texture, and color. N learnable registers are used to perform scene-based feature enhancement on the visual feature set for each viewpoint, resulting in a scene-enhanced visual feature set for each viewpoint. N=4, and the four learnable registers are: a first register for initial identification of seed adhesion regions in two-dimensional space and single-seed contour separation of the identified seed adhesion regions; a second register for associating visual feature sets from multiple viewpoints to complete the visual feature set under a single viewpoint; a third register for separating the seed reel background; and a fourth register for feature alignment.

[0034] At this point, step S2 includes: S21. Perform connected component analysis on each image from each viewpoint to obtain all foreground regions of each image, calculate the area of ​​the bounding rectangle and the centroid coordinates of each foreground region, and use the area of ​​the bounding matrix as the projection size of the peanut seed in the corresponding foreground region. S22. Calculate the standard deviation of the dimensions of each image based on the projected dimensions of peanut seeds in all foreground regions of each image, and calculate the minimum average centroid distance of each image based on the centroid coordinates of all foreground regions of each image. S23. Based on the minimum distance between the average centroids and the standard deviation of the size of each image, adaptively divide the corresponding image into blocks to obtain all image blocks; S24. Collect all image patches under the same viewpoint to obtain the image patch sequence under each viewpoint. Input the image patch sequence under each viewpoint into the pre-trained encoder to extract the visual features of peanut seeds and obtain the visual feature set corresponding to each viewpoint. S25. Based on the visual feature sets of all viewpoints, scene-based feature enhancement is performed on the visual feature sets corresponding to each viewpoint through N learnable registers to obtain the scene-based feature enhancement visual feature sets corresponding to each viewpoint.

[0035] In this embodiment, as Figure 2 As shown, connected component analysis is performed sequentially on each image from each viewpoint to obtain all foreground regions of each image, which are the peanut seed regions. The area of ​​the bounding rectangle and the centroid coordinates of each foreground region are calculated, and the area of ​​the bounding matrix is ​​used as the projection size of the peanut seed in the corresponding foreground region. The size standard deviation of each image is calculated in this way. Based on the minimum distance between the average centroids and the size standard deviation of each image, the corresponding image is adaptively divided into blocks to obtain all image blocks. The adaptive block division rules are as follows: when the size standard deviation is less than the preset peanut seed size uniformity threshold and the corresponding minimum distance between the average centroids is greater than the preset peanut seed spacing threshold, a 10×10 block is used; otherwise, a 14×14 block is used. The preset peanut seed size uniformity threshold, the preset peanut seed spacing threshold, and the specific adaptive block division rules can all be adjusted according to the actual situation. Each image patch is assigned a unique position code, which incorporates the elliptical geometric priors of the peanut seed, such as the ratio of the major axis to the minor axis and the thickness. All image patches from the same viewpoint are aggregated to obtain the image patch sequence for each viewpoint. The image patch sequence for each viewpoint is then input into a pre-trained encoder to extract the visual features of the peanut seed, resulting in the visual feature set corresponding to each viewpoint.

[0036] At this point, step S25 includes: S251. The first register groups the image blocks belonging to the same foreground region under each viewpoint to obtain the image block group corresponding to each foreground region, and constructs a local feature map based on the visual feature set of the image block group corresponding to each foreground region. S252. Input the local feature map into a Gabor filter to perform peanut seed coat vein texture feature analysis to obtain the texture breakage ratio. At the same time, perform least squares ellipse fitting on the local feature map to obtain the actual axis ratio and obtain the reference axis ratio of the image to which the local feature map belongs. Calculate the axis ratio anomaly based on the reference axis ratio and the actual axis ratio. Calculate the adhesion based on the axis ratio anomaly and the texture breakage ratio. If the adhesion exceeds the first adhesion threshold, the foreground region corresponding to the local feature map is the seed adhesion region. S253. Use a main contour model with axis ratio constraint to perform single seed contour separation on the seed adhesion region to obtain contour separation information. Based on the contour separation information, update the visual feature set under the corresponding viewpoint and output the first updated visual feature set. S254. The second register establishes a correspondence between the same seeds in different viewpoints based on the contour separation information, and identifies the occluded image block in a single viewpoint from the first updated visual feature set according to the correspondence. For the occluded image block in a single viewpoint, the register extracts the visual features with corresponding relationships from the first updated visual feature set of other viewpoints as known features. Based on the known features, the register performs feature completion on the visual features corresponding to the occluded image block to output a second updated visual feature set. The second updated visual feature set is used as the visual feature set after scene-based feature enhancement.

[0037] At this point, step S254, which involves using the second updated visual feature set as the scene-enhanced visual feature set, includes: S2541. The third register extracts the corresponding local region from the images of adjacent frames according to the spatial position encoding of each image block in the second updated visual feature set. For each local region, the dynamic background features of the seeding wheel are extracted from the corresponding second updated visual feature set through the rotation blur kernel of the first twin network. The dynamic background features are convolved with the corresponding local region to obtain the corresponding background feature response value. S2542. Simultaneously, the actual movement trajectory of peanut seeds is obtained from the corresponding second updated visual feature set through the second twin network. The matching degree between the actual movement trajectory and the preset ideal movement trajectory is calculated. When the matching degree is lower than the first matching threshold and the corresponding background feature response value exceeds the first background threshold, the corresponding local area is marked as the seeding wheel background. Feature suppression is performed on the image block corresponding to the local area marked as the seeding wheel background to achieve seeding wheel background separation, so as to output the third updated visual feature set. The rotation speed of the rotation blur kernel is positively correlated with the real-time angular velocity. S2543. The fourth register establishes a local coordinate system for peanut seeds based on the ellipsoidal geometric prior of peanut seeds, extracts the pose features of each seed from the third updated visual feature set, calculates the pose deviation between the pose features of the same seed under different viewpoints based on the local coordinate system, and performs rotation compensation and position calibration on the visual features corresponding to the same seed under different viewpoints based on the pose deviation, so as to realize the alignment processing of the visual features of the same seed under different viewpoints, and outputs the visual feature set of each viewpoint after alignment processing, and uses the visual feature set of each viewpoint after alignment processing as the visual feature set after scene-based feature enhancement.

[0038] In this embodiment, the first register groups image blocks belonging to the same foreground region under each viewpoint to obtain image block groups corresponding to each foreground region. A local feature map is constructed based on the visual feature set of the image block groups corresponding to each foreground region. A Gabor filter is used to perform peanut seed coat vein texture feature analysis on the local feature map to obtain the texture breakage ratio. The local feature map is then fitted with a least-two-layer ellipse to obtain the actual aspect ratio. The actual aspect ratio and the reference aspect ratio of the image to which the local feature map belongs are input into the aspect ratio anomaly formula for calculation to obtain the aspect ratio anomaly degree. The aspect ratio anomaly formula is: ; The adhesion degree is calculated by inputting the axial ratio anomaly and the proportion of texture fracture into the adhesion formula. The adhesion degree formula is as follows: ; If the adhesion exceeds the first adhesion threshold, the foreground region corresponding to the local feature map is the seed adhesion region, where the first adhesion threshold is 0.35, which can be adjusted according to the actual situation. A principal contour model with aspect ratio constraints is used to separate the individual seed contours of the seed adhesion region, obtaining contour separation information. The principal contour model with aspect ratio constraints is represented as follows: ; in, Indicates contour smoothness, Indicates contour continuity, Indicates the edge degree of the contour. express , This represents the weighting coefficients that affect smoothness. Weighting coefficients that affect continuity Weighting coefficients that affect marginality Weighting coefficients affecting axis ratio constraints; The contour smoothness, contour continuity, and contour edge degree are calculated using existing technical methods, with the weighting coefficient affecting the axis ratio constraint being 0.8.

[0039] Based on contour separation information, the visual feature set under the corresponding viewpoint is updated, and a first updated visual feature set is output. The first updated visual feature set is used as the input of the second register. The second register establishes the correspondence between the same seeds under different viewpoints based on the contour separation information, and identifies the occluded image patch under a single viewpoint from the first updated visual feature set according to the correspondence. For the occluded image patch under a single viewpoint, the visual features with corresponding relationships are extracted from the first updated visual feature set of other viewpoints as known features. Based on the known features, feature completion is performed on the visual features corresponding to the occluded image patch. When performing feature completion, existing feature completion methods can be directly used, or the second register can be pre-trained so that the second register learns three biological features of peanut seeds: the distribution of seed coat vein texture nodes, the correlation of oil spot positions, and the geometric angle between the radicle and hypocotyl. When performing feature completion, it can identify which biological features are missing in the specific occluded image patch, and perform attention weight calculation to focus on the missing biological features for targeted feature completion. The second updated visual feature set output by the second register is used as the input of the third register.

[0040] The third register identifies image blocks belonging to the seeding wheel background in the visual feature set through a two-branch Siamese network and performs feature suppression on their corresponding features to achieve seeding wheel background separation. The two-branch Siamese network consists of a first Siamese network and a second Siamese network, both of which are pre-trained. Based on the spatial location encoding of each image block in the second updated visual feature set, corresponding local regions are extracted from images in adjacent frames. For each local region, the dynamic background features of the seeding wheel are extracted from the corresponding second updated visual feature set using the rotational blur kernel of the first Siamese network. The rotation speed of the rotational blur kernel of the first Siamese network is positively correlated with the real-time angular velocity of the seeding wheel. The expression for the rotational blur kernel is: ; in, Indicates the rotational fuzzy kernel. Indicates fuzzy standard deviation. Represents pixels The pixel x-coordinate, Represents pixels The pixel ordinate, Indicates the exposure time. represents the real-time angular velocity of the seed metering wheel, and s represents the linear velocity of the seed metering hole on the currently processed seed metering wheel; The extracted dynamic background features, i.e., the specific features obtained through the expression of the rotated blur kernel, are convolved with the corresponding local regions to obtain the corresponding background feature response values. Simultaneously, the actual movement trajectory of the peanut seed is obtained from the corresponding second updated visual feature set through a second Siamese network. The specific trajectory equation corresponding to the actual movement trajectory is: ; ; in, Indicates time as Time pixel The pixel x-coordinate, This represents the horizontal coordinate of the seed metering hole on the currently processed seed metering wheel. This indicates the rotation period of the seed metering holes on the currently processed seed metering wheel. This indicates the residence time of peanut seeds in the seed metering holes on the seed metering wheel in the current treatment. Indicates time as Time pixel The pixel ordinate, This represents the ordinate of the plane of the seed metering hole on the currently processed seed metering wheel. Indicates the first learnable parameter, This represents the second learnable parameter. This represents the third learnable parameter. This represents the fourth learnable parameter; The actual movement trajectory of peanut seeds is matched with a preset ideal movement trajectory, and the matching degree is calculated. If the matching degree is lower than a first matching threshold and the corresponding background feature response value exceeds a first background threshold, the corresponding local region is marked as the seeding wheel background. The first matching threshold is 0.3, and the first background threshold is 0.2, which can be adjusted according to the actual situation. Feature suppression is performed on the image patches corresponding to the local regions marked as seeding wheel background. Feature suppression methods include setting the visual features of the corresponding image patches to 0 or introducing a suppression coefficient. This achieves seeding wheel background separation, outputting a third updated visual feature set.

[0041] The third updated visual feature set is used as input to the fourth register. The fourth register establishes a local coordinate system for the peanut seed based on the ellipsoidal geometric prior of the peanut seed. The ellipsoidal geometric prior includes the major axis, minor axis, and the ratio between the major and minor axes. The local coordinate system is established with the major axis as the X-axis, the minor axis as the Y-axis, and the normal perpendicular to the ellipsoid as the Z-axis. The pose features of each seed are extracted from the third updated visual feature set. Based on the local coordinate system, the pose deviation between the pose features of the same seed from different viewpoints is calculated. Based on the pose deviation, rotation compensation and position calibration are performed on the visual features corresponding to the same seed from different viewpoints to align the visual features of the same seed from different viewpoints. The aligned visual feature set of each viewpoint is then used as the scene-enhanced visual feature set.

[0042] S3. Obtain the layout information of each seeding hole on the seeding reel. Based on the layout information and the visual feature set enhanced by scene-based features of all perspectives, generate a seed point cloud map and a seed depth map corresponding to the peanut seed. Perform motion compensation on the seed point cloud map and the seed depth map based on the real-time angular velocity and the dynamic frame rate to obtain the motion-compensated seed point cloud map and the motion-compensated seed depth map. Project each seeding hole onto the motion-compensated seed depth map according to the layout information and the real-time angular velocity to obtain the detection area corresponding to each seeding hole. At this point, step S3, which involves generating a seed point cloud map and a seed depth map corresponding to the peanut seed based on the layout information and the enhanced visual feature set of all viewpoint scene features, includes: S31. Through the first multi-head attention network and the second multi-head attention network, the visual feature set after the scene-based features of all viewpoints are enhanced, the intra-frame local attention weight calculation and the inter-frame global attention weight calculation are performed alternately until the number of alternating executions reaches the preset number of loops, and the seed point cloud map and seed depth map corresponding to the peanut seed are generated. At this point, the process described in S31, which involves sequentially performing intra-frame local attention weight calculation and inter-frame global attention weight calculation on the visual feature set enhanced with scene-based features from all viewpoints through the first multi-head attention network and the second multi-head attention network, includes: S311. The first multi-head attention network identifies the image patch belonging to the seeding hole from the visual feature set enhanced by the scene-based features corresponding to each viewpoint based on the layout information, and uses the improved LBP operator to calculate the texture histogram corresponding to the image patch of the seeding hole, generates the corresponding texture entropy based on the texture histogram, and uses the Prewitt operator to calculate the gradient corresponding to the image patch of the seeding hole, and uses a clustering algorithm to cluster the direction of the gradient to obtain the category to which the image patch of the seeding hole belongs. S312. Generate corresponding intra-frame local attention weights based on the category of the image block to which the seeding hole belongs, and perform local feature enhancement on the corresponding scene-enhanced visual feature set based on the intra-frame local attention weights to obtain the visual feature set after local feature enhancement for each viewpoint. S313. Merge the visual feature sets after enhancing the local features of all viewpoints to obtain the global visual feature set; S314. The global visual feature set is used as the input of the second multi-head attention network. The second multi-head attention network calculates the inter-frame global attention weights under spatiotemporal and motion constraints. Based on the inter-frame global attention weights, global feature enhancement is performed on the global visual feature set to obtain the global visual feature set after global feature enhancement. The spatiotemporal constraints are: ; in, This represents the actual pixel position offset of the same seeding hole between adjacent frames. This indicates the real-time angular velocity of the seeding reel. Indicates the time interval between adjacent frames. This represents the pixel arc length corresponding to the central angle of adjacent seeding holes. This indicates the preset offset error threshold. Indicates the radius of the seed reel; The motion constraint conditions are as follows: ; ; in, The x-coordinate represents the pixel coordinates of the seeding hole from the main viewpoint. The x-coordinate represents the pixel coordinates of the seeding hole from the auxiliary viewpoint. The ordinate represents the pixel coordinates of the seeding hole from the main viewpoint. The ordinate represents the pixel coordinates of the seeding hole from the auxiliary viewpoint. This represents the preset horizontal axis error threshold. This indicates the preset vertical coordinate error threshold.

[0043] In this embodiment, the layout information includes the radial distance, initial axial angle, and axial position of each seeding hole relative to the rotation axis of the seeding wheel. Based on the layout information of each seeding hole and the visual feature set enhanced by scene-based features obtained in step S2, the intra-frame local attention weight and inter-frame global attention weight are calculated alternately, and finally, a seed point cloud map and a seed depth map corresponding to the peanut seed are generated. When the first multi-head attention network calculates the intra-frame local attention weight, it first identifies the image block belonging to the seeding hole in each viewpoint from the visual feature set enhanced by scene-based features corresponding to each viewpoint based on the layout information. The method for judging the seeding hole is as follows: the third-order skewness of the pixel gray level is calculated for each image block, and according to the principle that the element size of the seeding hole changes dynamically with the local gray level variance, the connected region is extracted for each image block, and the seeding hole fit is calculated for each extracted connected region. The formula for calculating the fit is: ; in, Indicates compatibility. Indicates the first adaptation weight. This indicates the second adaptation weight. Represents the area of ​​the connected region. Indicates the standard area of ​​the seed metering hole. This represents the shortest distance from the center of the connected region to a node in the network of eyelets on the seed wheel. The pixel value representing the diameter of the seed metering hole; Among them, the nodes of the seeding array network are constructed based on the spacing parameters of each seeding hole on the seeding wheel.

[0044] It also calculates the matching degree between image patches and nodes in the orbital array network. The formula for calculating the matching degree is: ; in, Indicates the degree of matching; If the calculated pixel grayscale third-order skewness is greater than the first skewness value, and the fit is greater than or equal to the first fit value, and the matching degree is greater than or equal to the first matching value, then the image block is considered a seeding hole. An improved LBP operator is used to calculate the texture histogram for the identified seeding hole image blocks. This allows for further classification of the seeding hole image blocks. Images with a pixel grayscale third-order skewness greater than the first skewness value, a fit value between the first and second fit values, and a matching degree greater than or equal to the first matching value are marked as suspected seeding hole image blocks. Subsequent processing only applies to suspected seeding hole image blocks. The first skewness value is 0.2, the first fit value is 0.7, the first matching value is 0.8, and the second fit value is 0.5.

[0045] Texture entropy is generated based on the texture histogram. The Prewitt operator is used to calculate the gradient of the image patch corresponding to the seeding hole, and a clustering algorithm is used to cluster the gradient directions to obtain the category of the image patch belonging to the seeding hole. The categories include: seed edge region, adhesion region, and background region. The gradient directions of the seed edge region are concentrated in a few directions; for example, when the number of clusters is ≤3, it is considered a seed edge region. The adhesion region has a dispersed gradient direction due to the presence of multiple peanut seeds stacked together; when the number of clusters is between (3,7), it is considered an adhesion region. The gradient direction of the seeding wheel background is irregular; therefore, when the number of clusters is >7, it is considered the background of the seeding wheel.

[0046] Intra-frame local attention weights are generated based on the category of the image patch belonging to the seeding hole. For example, when the category is "seed edge region," a diagonal matrix with elements between 1.5 and 2.0 is generated; when the category is "adhesive region," a diagonal matrix with elements between 1.0 and 1.5 is generated; and when the category is "background region," a diagonal matrix with elements between 0.5 and 0.8 is generated. Each element of the diagonal matrix corresponds to the enhancement coefficient of an image patch, i.e., the corresponding intra-frame local attention weight. Based on the generated intra-frame local attention weights, local feature enhancement is performed on the corresponding scene-enhanced visual feature set to obtain the enhanced visual feature set for each viewpoint.

[0047] The visual feature sets enhanced from all local features of all perspectives are merged to obtain a global visual feature set, which serves as the input to the second multi-head attention network. The second multi-head attention network calculates inter-frame global attention weights under spatiotemporal and motion constraints. The spatiotemporal constraints essentially involve motion compensation of the global visual feature set, shifting the pixel positions of the corresponding seeding hole image blocks in each frame to spatially align the visual features of the same seeding hole across different frames. The motion constraints refer to searching for image blocks with similar visual features in auxiliary views (i.e., other perspectives) during global attention search, without performing a full-image search. Image blocks with visual feature similarity exceeding a preset similarity threshold are considered identical image blocks. The weights of identical image blocks are weighted and summed to obtain the inter-frame global attention weights for the image block corresponding to the seeding hole in the current main view. The initial weight value of the image block is obtained from the calculated similarity. Global feature enhancement is then performed on the global visual feature set based on the obtained inter-frame global attention weights, resulting in the enhanced global visual feature set. The intra-frame local attention weight calculation and inter-frame global attention weight calculation, alternately executed by the first and second multi-head attention mechanisms, can be performed in three stages. When the preset number of loops is 24, the first alternating execution stage (loops 1-8) has an intra-frame local attention weight to inter-frame global attention weight ratio of 6:2, focusing on extracting fine features within a single frame; the second alternating execution stage (loops 9-16) has an intra-frame local attention weight to inter-frame global attention weight ratio of 1:1, balancing intra-frame and inter-frame attention; and the third alternating execution stage (loops 17-24) has an intra-frame local attention weight to inter-frame global attention weight ratio of 2:6, focusing on extracting global features between frames. The final global feature-enhanced global visual feature set output after reaching the preset number of loops is then input into the decoding network for sequential decoding to generate the seed point cloud map and seed depth map corresponding to the peanut seed.

[0048] At this point, the layout information in step S3 includes the radial distance, initial axial angle, and axial position of each seeding hole relative to the rotation axis of the seeding wheel. Motion compensation is then performed on the seed point cloud map and the seed depth map based on the real-time angular velocity and the dynamic frame rate to obtain a motion-compensated seed point cloud map and a motion-compensated seed depth map. Each seeding hole is then projected onto the motion-compensated seed depth map according to the layout information and the real-time angular velocity to obtain the detection area corresponding to each seeding hole, including: S32. Input the real-time angular velocity and the dynamic frame rate into the first compensation formula and the second compensation formula respectively for calculation to obtain the corresponding first compensation value and second compensation value. The first compensation formula is: ; in, This represents the first compensation value. This indicates the real-time angular velocity of the seeding reel. This indicates the distance from the peanut seed being photographed to the multi-view camera. This represents the angle between the direction of peanut seed movement and the X-axis of the multi-view camera. Indicates dynamic frame rate; The second compensation formula is: ; in, This represents the second compensation value. This indicates the real-time angular velocity of the seeding reel. This indicates the distance from the peanut seed being photographed to the multi-view camera. This represents the angle between the direction of peanut seed movement and the X-axis of the multi-view camera. Indicates dynamic frame rate; S33. Motion compensation is performed on the seed depth map according to the first compensation value and the second compensation value. The horizontal coordinate of each pixel in the seed depth map is translated according to the first compensation value, and the vertical coordinate of each pixel in the seed depth map is translated according to the second compensation value to obtain the motion-compensated seed depth map. S34. Obtain the intrinsic and extrinsic parameters of the multi-view camera, and based on the motion-compensated seed depth map, regenerate a new seed point cloud map using the intrinsic and extrinsic parameters, and use the new seed point cloud map as the motion-compensated seed point cloud map. S35. Calculate the real-time three-dimensional spatial coordinates of each seed metering hole on the seed metering wheel based on the radial distance, initial axial angle, axial position, and real-time angular velocity of each seed metering hole relative to the rotation axis of the seed metering wheel. Then, project the real-time three-dimensional spatial coordinates of each seed metering hole on the seed metering wheel onto the motion-compensated seed depth map using the intrinsic and extrinsic parameters to obtain the detection area corresponding to each seed metering hole.

[0049] In this embodiment, the real-time angular velocity and dynamic frame rate are input into the first compensation formula and the second compensation formula for calculation. Motion compensation is then performed on the seed depth map based on the obtained first and second compensation values. The horizontal coordinate of each pixel in the seed depth map is shifted according to the first compensation value, and the vertical coordinate of each pixel in the seed depth map is simultaneously shifted according to the second compensation value, resulting in a motion-compensated seed depth map. For motion compensation of the seed point cloud map, a new seed point cloud map is generated by combining the motion-compensated seed depth map with the intrinsic and extrinsic parameters of the multi-view camera. The intrinsic parameter refers to the focal length; the extrinsic parameter refers to the rotation matrix. This ensures that the new seed point cloud map corresponds to the motion-compensated seed depth map. Based on the radial distance, initial axial angle, axial position, and real-time angular velocity of each seed metering hole relative to the rotation axis of the seed metering wheel, the real-time three-dimensional spatial coordinates of each seed metering hole on the seed metering wheel are calculated. Then, the real-time three-dimensional spatial coordinates of each seed metering hole on the seed metering wheel are transformed to the world coordinate system or camera coordinate system using extrinsic parameters. Finally, the world coordinate system or camera coordinate system is projected onto the motion-compensated seed depth map using intrinsic parameters. With the projection point as the center, a rectangular region is generated according to the expected imaging size of each seed metering hole in the motion-compensated seed depth map to obtain the detection area corresponding to each seed metering hole, that is, the rectangular region corresponds to the detection area.

[0050] S4. Perform three-dimensional geometric segmentation on the motion-compensated seed point cloud map to obtain all individual seed segmentation results. Project all individual seed segmentation results onto the motion-compensated seed depth map and match them with the detection area corresponding to each seeding hole to obtain the sowing quantity corresponding to each seeding hole, thereby realizing missed sowing detection.

[0051] At this point, step S4 includes: S41. An ellipsoid fitting algorithm is used to fit the motion-compensated seed point cloud map based on the ellipsoidal geometric prior of peanut seeds to obtain all the initial seed point cloud clusters. S42. Select a core point from each initial seed point cloud cluster as the current growth point, and obtain all neighboring points corresponding to the current growth point. Using the current growth point as the growth center, perform a growth traversal on all corresponding neighboring points. Calculate the depth difference between each neighboring point and the corresponding current growth point in the motion-compensated seed depth map, and the angle between their normal vectors in the motion-compensated point cloud map. Input the angle between their normal vectors and the depth difference into the dynamic weight formula to calculate the corresponding dynamic weight value. The dynamic weight formula is: ; in, This represents the dynamic weight value corresponding to the current growth point C and its neighboring point c. This represents the depth difference between the current growth point C and its neighboring point c in the motion-compensated seed depth map. This represents the angle between the normal vectors of the current growth point C and its neighboring point c in the motion-compensated point cloud map. This indicates the preset weighting coefficients; S43. If the dynamic weight value is greater than the first weight threshold, the neighborhood point corresponding to the dynamic weight value is assigned to the initial seed point cloud cluster to which the current growth point belongs, and the neighborhood point newly assigned to the initial seed point cloud cluster is used as the new current growth point to perform growth traversal again until the final dynamic weight value is less than or equal to the first weight threshold, so as to obtain all the final seed point cloud clusters, and use all the final seed point cloud clusters as all the single seed segmentation results. S44. Project all single seed segmentation results onto the motion-compensated seed depth map, obtain the pixel area occupied by each single seed segmentation result in the motion-compensated seed depth map, match the pixel area with the detection area corresponding to each seeding hole, count the number of successfully matched single seed segmentation results in the detection area corresponding to each seeding hole, and use the number as the sowing quantity of the corresponding seeding hole to obtain the sowing quantity corresponding to each seeding hole, thereby realizing the missed sowing detection.

[0052] In this embodiment, an ellipsoidal fitting algorithm is used to fit the motion-compensated seed point cloud map based on the ellipsoidal geometric prior of peanut seeds, obtaining all initial seed point cloud clusters. From each initial seed point cloud cluster, a core point is selected as the current growth point. The core point can be the geometric center or the point with the smallest curvature. All neighborhood points corresponding to the current growth point are obtained. With the current growth point as the growth center, growth traversal is performed on all corresponding neighborhood points. The depth difference between each neighborhood point and the corresponding current growth point in the motion-compensated seed depth map and the angle between the normal vectors in the motion-compensated point cloud map are calculated. Based on the dynamic weight value calculated by the angle between the normal vectors and the depth difference, it is determined whether the neighborhood point needs to be assigned to the initial seed point cloud cluster to which the current growth point belongs. In this way, all final seed point cloud clusters are obtained, that is, all single seed segmentation results. The first weight threshold is 0.6, which can be adjusted according to the actual situation.

[0053] All individual seed segmentation results are projected onto the motion-compensated seed depth map. The projection method is as follows: for each 3D point in the individual seed segmentation result, the intrinsic and extrinsic parameters of the multi-view camera are used for projection. The pixel area occupied by each individual seed segmentation result in the motion-compensated seed depth map is counted. The pixel area is matched with the detection area corresponding to each seeding hole. If there is an intersection, that is, a successful match, the number of successfully matched individual seed segmentation results in the detection area corresponding to each seeding hole is counted. The counted number is used as the sowing quantity for the corresponding seeding hole to obtain the sowing quantity corresponding to each seeding hole, thereby realizing the detection of missed sowing.

[0054] At this point, step S44, which involves obtaining the seed quantity corresponding to each seed metering hole to achieve missed seed detection, includes: S441. When the number of seeds sown is less than the first quantity threshold, the seeding hole corresponding to the number of seeds sown is marked as a missed seeding. S442. When the number of seeds sown is greater than or equal to the first quantity threshold, the KNN value of each successfully matched single seed segmentation result is calculated using the KNN algorithm, and the point cloud curvature of each successfully matched single seed segmentation result is calculated using the curvature change formula. If the point cloud curvature is greater than the first curvature threshold and the KNN value is greater than the first roughness threshold, then the corresponding successfully matched single seed segmentation result is marked as having surface defects, and the corresponding seeding hole is marked as having unqualified sowing quality. and / or When the number of seeds sown is greater than or equal to the first quantity threshold, the actual volume of each successfully matched single seed segmentation result is calculated, and the actual volume is input into the fullness calculation formula to calculate the fullness and obtain the corresponding fullness. If the fullness is lower than the first fullness threshold, the corresponding successfully matched single seed segmentation result is marked as an invalid seed, and the corresponding seeding hole is marked as unqualified in sowing quality.

[0055] In this embodiment, not only is missed sowing detected based on the sowing quantity, but sowing quality is also detected. If no missed sowing is found, the sowing quality is further detected. The KNN value of each successfully matched single seed segmentation result is calculated using the KNN algorithm, and the point cloud curvature of each successfully matched single seed segmentation result is calculated using the curvature change formula. If the point cloud curvature is greater than a first curvature threshold and the KNN value is greater than a first coarseness threshold, the single seed segmentation result is considered dense, indicating that the peanut seed has surface defects, and it is marked. At the same time, the corresponding seeding hole is marked as unqualified in sowing quality. The first curvature threshold is 0.8, and the first coarseness threshold is 2.5. The specific values ​​can be adjusted according to the actual situation.

[0056] In addition to detecting surface defects in peanut seeds, the system also calculates seed plumpness. It calculates the actual volume of each successfully matched single seed segmentation result, inputs the actual volume into the plumpness calculation formula, and obtains the corresponding plumpness. The plumpness formula is: ; If the saturation level is lower than the first saturation threshold, the corresponding successfully matched single seed segmentation result will be marked as an invalid seed. The first saturation threshold is 0.7, which can be adjusted according to the actual situation. The corresponding seeding hole will be marked as unqualified in terms of sowing quality.

[0057] For seeding holes marked as missed, a replanting device will be triggered to replant. For seeding holes marked as unqualified in sowing quality, you can choose to directly trigger the replanting device to replant, or you can choose to remove peanut seeds with surface defects / invalid seeds before triggering the replanting device to replant.

[0058] Example 2 Please refer to Figure 3 The present invention provides a peanut seed missing detection system 1, including a memory 3, a processor 2, and a computer program stored in the memory 3 and run on the processor 2. When the processor 2 executes the computer program, it implements the steps in Embodiment 1.

[0059] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0060] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0061] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0062] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0063] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0064] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0065] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. A method for detecting missed sowing of peanut seeds, characterized in that, include: The real-time angular velocity of the seeding reel is collected in real time. The dynamic frame rate of the multi-view camera is calculated based on the real-time angular velocity. The multi-view camera is started based on the dynamic frame rate to collect multi-view images of peanut seeds on the seeding reel, thereby obtaining multi-view images of peanut seeds. All images from each viewpoint are input into a pre-trained encoder to extract visual features of peanut seeds, resulting in a visual feature set for each viewpoint. Based on the visual feature sets from all viewpoints, scene-based feature enhancement is performed on the visual feature sets from each viewpoint using N learnable registers, resulting in a scene-enhanced visual feature set for each viewpoint. The N learnable registers include a first register for initially identifying seed adhesion regions in two-dimensional space and performing single-seed contour separation on the identified seed adhesion regions, and a second register for associating visual feature sets from multiple viewpoints to complete the visual feature set from a single viewpoint. The layout information of each seeding hole on the seeding reel is obtained. Based on the layout information and the visual feature set enhanced by scene-based features of all perspectives, a seed point cloud map and a seed depth map corresponding to the peanut seed are generated. Motion compensation is performed on the seed point cloud map and the seed depth map based on the real-time angular velocity and the dynamic frame rate to obtain the motion-compensated seed point cloud map and the motion-compensated seed depth map. Each seeding hole is projected onto the motion-compensated seed depth map according to the layout information and the real-time angular velocity to obtain the detection area corresponding to each seeding hole. The motion-compensated seed point cloud map is geometrically segmented in three-dimensional space to obtain all individual seed segmentation results. All individual seed segmentation results are then projected onto the motion-compensated seed depth map and matched with the detection area corresponding to each seeding hole to obtain the seeding quantity corresponding to each seeding hole, thereby achieving missed seeding detection.

2. The method for detecting missed sowing of peanut seeds as described in claim 1, characterized in that, The step of calculating the dynamic frame rate of the multi-view camera based on the real-time angular velocity, and then activating the multi-view camera to acquire multi-view images of peanut seeds on the seed reel based on the dynamic frame rate, includes: The real-time phase of the seed metering reel is acquired in real time. This real-time phase is then input into a first theoretical formula to calculate the theoretical phase of each seeding hole on the seed metering reel. The real-time angular velocity is also input into a first dynamic formula to calculate the dynamic frame rate of the multi-view camera. Simultaneously, exposure constraints for the multi-view camera are generated based on the real-time angular velocity. The first theoretical formula is: in, This represents the theoretical phase of the i-th seeding hole on the seeding reel. This indicates the initial calibration phase of the i-th seeding hole on the seeding reel. Indicates the real-time phase of the seeding reel; The first dynamic formula is: ； in, This indicates the dynamic frame rate, and N represents the preset minimum number of frames. This indicates the preset optimal imaging window. This indicates the real-time angular velocity of the seeding reel; The exposure constraint is as follows: ； ； in, Indicates the rate of peanut seed synthesis. Indicates the radius of the seeding reel. Indicates the maximum random bounce speed. Indicates the exposure time. This indicates the meta-size of a multi-view camera; Calculate the phase difference between the real-time phase and the theoretical phase of each seeding hole. When the phase difference meets the first condition, start the multi-view camera to acquire multi-view images of peanut seeds on the seeding reel based on the dynamic frame rate and the exposure constraint.

3. The method for detecting missed sowing of peanut seeds as described in claim 1, characterized in that, The process involves inputting all images from each viewpoint into a pre-trained encoder to extract visual features of peanut seeds, resulting in a visual feature set for each viewpoint. Based on these visual feature sets, scene-based feature enhancement is performed on each viewpoint's visual feature set using N learnable registers, yielding a scene-enhanced visual feature set for each viewpoint, including: For each image from each viewpoint, connected component analysis is performed sequentially to obtain all foreground regions of each image. The area of ​​the bounding rectangle and the centroid coordinates of each foreground region are calculated, and the area of ​​the bounding matrix is ​​used as the projection size of the peanut seed in the corresponding foreground region. The standard deviation of the dimensions of each image is calculated based on the projected dimensions of peanut seeds in all foreground regions of each image, and the minimum average centroid distance of each image is calculated based on the centroid coordinates of all foreground regions of each image. Adaptive segmentation of the corresponding image is performed based on the minimum distance between the average centroids and the standard deviation of the size of each image to obtain all image blocks; All image patches from the same viewpoint are aggregated to obtain an image patch sequence for each viewpoint. The image patch sequence for each viewpoint is then input into a pre-trained encoder to extract visual features of peanut seeds, thus obtaining a visual feature set corresponding to each viewpoint. Based on the visual feature sets of all viewpoints, scene-based feature enhancement is performed on the visual feature sets corresponding to each viewpoint through N learnable registers, resulting in the scene-based feature enhanced visual feature sets for each viewpoint.

4. The method for detecting missed sowing of peanut seeds as described in claim 3, characterized in that, The visual feature set based on all viewpoints is enhanced with scene-based features for each viewpoint using N learnable registers, resulting in the scene-enhanced visual feature set for each viewpoint, including: The first register groups the image blocks belonging to the same foreground region under each viewpoint to obtain the image block group corresponding to each foreground region, and constructs a local feature map based on the visual feature set of the image block group corresponding to each foreground region. The local feature map is input into a Gabor filter for peanut seed coat vein texture feature analysis to obtain the texture breakage ratio. At the same time, the local feature map is fitted with a least square ellipse to obtain the actual axis ratio, and the reference axis ratio of the image to which the local feature map belongs is obtained. The axis ratio anomaly is calculated based on the reference axis ratio and the actual axis ratio. The adhesion is calculated based on the axis ratio anomaly and the texture breakage ratio. If the adhesion exceeds the first adhesion threshold, the foreground region corresponding to the local feature map is the seed adhesion region. A main contour model with axis ratio constraint is used to perform single-seed contour separation in the seed adhesion region to obtain contour separation information. Based on the contour separation information, the visual feature set under the corresponding viewpoint is updated to output the first updated visual feature set. The second register establishes a correspondence between the same seeds from different viewpoints based on the contour separation information, and identifies the occluded image block from the first updated visual feature set according to the correspondence. For the occluded image block from the first updated visual feature set of other viewpoints, the register extracts the corresponding visual features as known features from the first updated visual feature set of other viewpoints. Based on the known features, the register performs feature completion on the visual features corresponding to the occluded image block to output the second updated visual feature set. The second updated visual feature set is used as the visual feature set after scene-based feature enhancement.

5. The method for detecting missed sowing of peanut seeds as described in claim 4, characterized in that, The N learnable registers also include a third register for separating the seeding wheel background and a fourth register for feature alignment, wherein the second updated visual feature set is used as the scene-enhanced visual feature set, which includes: The third register extracts the corresponding local region from the images of adjacent frames according to the spatial position encoding of each image block in the second updated visual feature set. For each local region, the dynamic background features of the seeding wheel are extracted from the corresponding second updated visual feature set through the rotation blur kernel of the first twin network. The dynamic background features are then convolved with the corresponding local region to obtain the corresponding background feature response value. Simultaneously, the actual movement trajectory of peanut seeds is obtained from the corresponding second updated visual feature set through the second twin network, and the matching degree between the actual movement trajectory and the preset ideal movement trajectory is calculated. When the matching degree is lower than the first matching threshold and the corresponding background feature response value exceeds the first background threshold, the corresponding local area is marked as the seeding wheel background, and the image block corresponding to the local area marked as the seeding wheel background is subjected to feature suppression to achieve seeding wheel background separation, so as to output the third updated visual feature set, wherein the rotation speed of the rotation blur kernel is positively correlated with the real-time angular velocity; The fourth register establishes a local coordinate system for peanut seeds based on the ellipsoidal geometry prior of the peanut seeds. It extracts the pose features of each seed from the third updated visual feature set, calculates the pose deviation between the pose features of the same seed from different viewpoints based on the local coordinate system, and performs rotation compensation and position calibration on the visual features corresponding to the same seed from different viewpoints based on the pose deviation, so as to realize the alignment processing of the visual features of the same seed from different viewpoints, and outputs the visual feature set of each viewpoint after alignment processing. The visual feature set of each viewpoint after alignment processing is used as the visual feature set after scene-based feature enhancement.

6. The method for detecting missed sowing of peanut seeds as described in claim 1, characterized in that, The step of generating a seed point cloud map and a seed depth map corresponding to the peanut seed based on the layout information and the enhanced visual feature set of all viewpoint scene features includes: The first multi-head attention network and the second multi-head attention network alternately perform intra-frame local attention weight calculation and inter-frame global attention weight calculation on the visual feature set after scene-based features of all viewpoints until the number of alternating executions reaches the preset number of loops, thereby generating a seed point cloud map and a seed depth map corresponding to the peanut seed. The process of calculating intra-frame local attention weights and inter-frame global attention weights by sequentially applying the enhanced visual feature sets of all viewpoints through the first and second multi-head attention networks includes: The first multi-head attention network identifies the image patch belonging to the seeding hole from the visual feature set enhanced by the scene-based features corresponding to each viewpoint based on the layout information, and uses an improved LBP operator to calculate the texture histogram corresponding to the image patch of the seeding hole, generates the corresponding texture entropy based on the texture histogram, and uses the Prewitt operator to calculate the gradient corresponding to the image patch of the seeding hole, and uses a clustering algorithm to cluster the direction of the gradient to obtain the category to which the image patch of the seeding hole belongs; Based on the category of the image patch of the seeding hole, the corresponding intra-frame local attention weight is generated, and the corresponding scene-enhanced visual feature set is enhanced with the intra-frame local attention weight to obtain the visual feature set enhanced with local features for each viewpoint. The visual feature sets enhanced with local features from all perspectives are merged to obtain the global visual feature set; The global visual feature set is used as input to the second multi-head attention network. The second multi-head attention network calculates inter-frame global attention weights under spatiotemporal and motion constraints. Based on these inter-frame global attention weights, global feature enhancement is performed on the global visual feature set to obtain the enhanced global visual feature set. The spatiotemporal constraints are: ； in, This represents the actual pixel position offset of the same seeding hole between adjacent frames. This indicates the real-time angular velocity of the seeding reel. Indicates the time interval between adjacent frames. This represents the pixel arc length corresponding to the central angle of adjacent seeding holes. This indicates the preset offset error threshold. Indicates the radius of the seed reel; The motion constraint conditions are as follows: ； ； in, The x-coordinate represents the pixel coordinates of the seeding hole from the main viewpoint. The x-coordinate represents the pixel coordinates of the seeding hole from the auxiliary viewpoint. The ordinate represents the pixel coordinates of the seeding hole from the main viewpoint. The ordinate represents the pixel coordinates of the seeding hole from the auxiliary viewpoint. This represents the preset horizontal axis error threshold. This indicates the preset vertical coordinate error threshold.

7. The method for detecting missed sowing of peanut seeds as described in claim 1, characterized in that, The layout information includes the radial distance, initial axial angle, and axial position of each seeding hole relative to the rotation axis of the seeding wheel. Motion compensation is performed on the seed point cloud map and the seed depth map based on the real-time angular velocity and the dynamic frame rate to obtain a motion-compensated seed point cloud map and a motion-compensated seed depth map. Each seeding hole is then projected onto the motion-compensated seed depth map according to the layout information and the real-time angular velocity to obtain the detection area corresponding to each seeding hole, including: The real-time angular velocity and the dynamic frame rate are respectively input into the first compensation formula and the second compensation formula for calculation to obtain the corresponding first compensation value and second compensation value. The first compensation formula is: ； in, This represents the first compensation value. This indicates the real-time angular velocity of the seeding reel. This indicates the distance from the peanut seed being photographed to the multi-view camera. This represents the angle between the direction of peanut seed movement and the X-axis of the multi-view camera. Indicates dynamic frame rate; The second compensation formula is: ； in, This represents the second compensation value. This indicates the real-time angular velocity of the seeding reel. This indicates the distance from the peanut seed being photographed to the multi-view camera. This represents the angle between the direction of peanut seed movement and the X-axis of the multi-view camera. Indicates dynamic frame rate; Motion compensation is performed on the seed depth map based on the first compensation value and the second compensation value. The horizontal coordinate of each pixel in the seed depth map is translated according to the first compensation value, and the vertical coordinate of each pixel in the seed depth map is translated according to the second compensation value to obtain the motion-compensated seed depth map. The intrinsic and extrinsic parameters of the multi-view camera are obtained. Based on the motion-compensated seed depth map, a new seed point cloud map is regenerated using the intrinsic and extrinsic parameters. The new seed point cloud map is used as the motion-compensated seed point cloud map. Based on the radial distance, initial axial angle, axial position, and real-time angular velocity of each seed metering hole relative to the rotation axis of the seed metering wheel, the real-time three-dimensional spatial coordinates of each seed metering hole on the seed metering wheel are calculated. Then, the real-time three-dimensional spatial coordinates of each seed metering hole on the seed metering wheel are projected onto the motion-compensated seed depth map using the intrinsic and extrinsic parameters to obtain the detection area corresponding to each seed metering hole.

8. The method for detecting missed sowing of peanut seeds as described in claim 1, characterized in that, The process involves geometrically segmenting the motion-compensated seed point cloud map in three-dimensional space to obtain all individual seed segmentation results. These individual seed segmentation results are then projected onto the motion-compensated seed depth map and matched with the detection area corresponding to each seeding hole to determine the sowing quantity for each seeding hole. This process enables missed seeding detection. An ellipsoid fitting algorithm was used to fit the motion-compensated seed point cloud map based on the ellipsoidal geometric prior of peanut seeds to obtain all the initial seed point cloud clusters. From each initial seed point cloud cluster, a core point is selected as the current growth point, and all neighboring points corresponding to the current growth point are obtained. Using the current growth point as the growth center, a growth traversal is performed on all corresponding neighboring points. The depth difference between each neighboring point and the corresponding current growth point in the motion-compensated seed depth map, and the angle between their normal vectors in the motion-compensated point cloud map, are calculated. The angle between the normal vectors and the depth difference are input into a dynamic weight formula for calculation to obtain the corresponding dynamic weight value. The dynamic weight formula is as follows: ； in, This represents the dynamic weight value corresponding to the current growth point C and its neighboring point c. This represents the depth difference between the current growth point C and its neighboring point c in the motion-compensated seed depth map. This represents the angle between the normal vectors of the current growth point C and its neighboring point c in the motion-compensated point cloud map. This indicates the preset weighting coefficients; If the dynamic weight value is greater than the first weight threshold, the neighborhood point corresponding to the dynamic weight value is assigned to the initial seed point cloud cluster to which the current growth point belongs, and the neighborhood point newly assigned to the initial seed point cloud cluster is used as the new current growth point to perform growth traversal again until the final dynamic weight value is less than or equal to the first weight threshold, so as to obtain all the final seed point cloud clusters, and use all the final seed point cloud clusters as all the single seed segmentation results. All single-seed segmentation results are projected onto the motion-compensated seed depth map. The pixel area occupied by each single-seed segmentation result in the motion-compensated seed depth map is obtained. The pixel area is matched with the detection area corresponding to each seeding hole. The number of successfully matched single-seed segmentation results in the detection area corresponding to each seeding hole is counted. The number is used as the sowing quantity of the corresponding seeding hole to obtain the sowing quantity corresponding to each seeding hole, thereby realizing the detection of missed sowing.

9. The method for detecting missed sowing of peanut seeds as described in claim 8, characterized in that, The process of obtaining the seed quantity corresponding to each seed metering hole and realizing missed seed detection includes: When the number of seeds sown is less than the first quantity threshold, the seeding hole corresponding to the number of seeds sown is marked as a missed seeding. When the number of seeds sown is greater than or equal to the first quantity threshold, the KNN value of each successfully matched single seed segmentation result is calculated using the KNN algorithm, and the point cloud curvature of each successfully matched single seed segmentation result is calculated using the curvature change formula. If the point cloud curvature is greater than the first curvature threshold and the KNN value is greater than the first roughness threshold, then the corresponding successfully matched single seed segmentation result is marked as having surface defects, and the corresponding seeding hole is marked as having unqualified sowing quality. and / or When the number of seeds sown is greater than or equal to the first quantity threshold, the actual volume of each successfully matched single seed segmentation result is calculated, and the actual volume is input into the fullness calculation formula to calculate the fullness and obtain the corresponding fullness. If the fullness is lower than the first fullness threshold, the corresponding successfully matched single seed segmentation result is marked as an invalid seed, and the corresponding seeding hole is marked as unqualified in sowing quality.

10. A detection system for missed peanut seed sowing, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 9.

Images