A Dual-Parameter Detection System and Method for Maize Kernels Based on Physical Prior and Multi-Source Fusion

Abstract

This invention discloses a dual-parameter detection system and method for corn kernels based on physical priors and multi-source fusion. The system includes a hopper, a kernel conveying mechanism, a low-light image acquisition module, a mechanical detection acquisition module, and an edge computing and control module. The low-light image acquisition module is located at a high-level observation window, the mechanical detection acquisition module is located below the kernel conveying mechanism, and the edge computing and control module simultaneously acquires corn kernel images and axial stress wave signals. The method performs low-light enhancement, defogging processing, and target segmentation on the images under physical prior constraints, extracts mechanical features from the stress wave signals, and outputs corn kernel feeding amount and moisture content detection parameters through a feature-level multi-source heterogeneous data fusion mapping model. This invention achieves the synchronous acquisition of multi-source signals required for corn kernel feeding amount and moisture content detection, and is suitable for real-time detection in dark dust environments within the threshing chamber.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection of agricultural machinery and online analysis of agricultural product quality, specifically to a dual-parameter detection system and method for corn kernel feeding amount and moisture content in the threshing chamber or kernel conveying mechanism of a corn combine harvester. Background Technology

[0002] When a corn combine harvester is threshing, the state of the material entering the threshing drum or grain conveying mechanism directly affects the threshing load, breakage rate, unthreshed rate, and subsequent cleaning quality. Dynamic feed rate characterizes the amount of grain material entering the detection area per unit time, while moisture content affects grain elasticity, impact damping, reflection characteristics, and the risk of threshing damage. Therefore, real-time acquisition of feed rate and moisture content during combine harvester operation is crucial for adaptive speed regulation and low-loss harvesting control of the threshing components.

[0003] Existing detection methods typically employ single visual inspection, single weighing inspection, near-infrared inspection, or piezoelectric impact inspection. While visual inspection can reflect grain area, distribution, and apparent condition, the threshing chamber of a combine harvester presents challenges due to its enclosed working space, insufficient lighting, and high dust concentration. Strong supplemental lighting can easily cause dust scattering and image fogging, and low-light images may exhibit motion blur, unclear boundaries, and target overlap. Conventional image enhancement or general segmentation models lack constraints on dust scattering and grain motion blur within the threshing chamber, easily leading to image feature distortion.

[0004] Piezoelectric impact or stress wave detection can reflect the impact intensity of grains and changes in material load, but a single mechanical signal is affected not only by the feed rate but also by factors such as grain moisture content, grain elasticity, installation preload, mechanical vibration, and temperature drift. If parameter estimation is based solely on a single mechanical signal, coupling errors between feed rate and moisture content can easily occur.

[0005] Therefore, there is a need for a system and method that can simultaneously acquire images and mechanical signals of corn kernels in a dark dust environment in the threshing chamber, and achieve dual parameter detection of feed amount and moisture content through physical prior constraints and multi-source fusion. Summary of the Invention

[0006] Purpose of the invention: The purpose of this invention is to overcome the shortcomings of the prior art and provide a dual-parameter detection system and method for corn kernels based on physical prior and multi-source fusion, which solves the problems of poor image acquisition quality, large coupling error of single-source detection parameters, and difficulty in synchronously fusing image features and mechanical features in the dark dust environment of the threshing chamber.

[0007] Technical Solution: The first aspect of this invention provides a dual-parameter detection system for corn kernels based on physical prior and multi-source fusion, including a hopper, a kernel conveying mechanism, an outer wall of a threshing chamber, a high-level observation window disposed on the outer wall of the threshing chamber, a low-illuminance image acquisition module, a mechanical detection acquisition module, and an edge computing and control module; the low-illuminance image acquisition module is disposed at the high-level observation window and is used to acquire images of corn kernels passing through the kernel conveying mechanism; the mechanical detection acquisition module is disposed below the kernel conveying mechanism and is used to acquire axial stress wave signals generated by the collision of corn kernels; the edge computing and control module is electrically connected to the low-illuminance image acquisition module and the mechanical detection acquisition module respectively, and the edge computing and control module is provided with a synchronization trigger interface for outputting a synchronization trigger signal, an image acquisition interface for receiving corn kernel images, a mechanical signal interface for receiving axial stress wave signals, and a detection parameter output interface for outputting detection parameters.

[0008] Furthermore, the low-light image acquisition module includes an observation window mounting flange, a portal-shaped mounting bracket, a starlight-level global shutter image sensor, and two sets of infrared stroboscopic light sources. The observation window mounting flange is located at the high-position observation window, the portal-shaped mounting bracket is fixed to the observation window mounting flange, the starlight-level global shutter image sensor is mounted on the support beam of the portal-shaped mounting bracket, and the two sets of infrared stroboscopic light sources are respectively located on both sides of the starlight-level global shutter image sensor.

[0009] Furthermore, a transparent dust cover is provided on the outer side of the lens of the starlight-level global shutter image sensor, and the transparent dust cover is arranged opposite to the high-position observation window; the transparent dust cover is made of high-transparency optical glass, and the outer surface of the transparent dust cover is provided with a nano-hydrophobic and anti-fouling coating.

[0010] Furthermore, the two sets of infrared stroboscopic light sources are symmetrically arranged relative to the starlight-level global shutter image sensor. The light-emitting surface of the infrared stroboscopic light source is tilted towards the inside of the shooting field of view of the starlight-level global shutter image sensor, and the tilt angle of the light-emitting surface of the infrared stroboscopic light source relative to the vertical direction is 20°±2°.

[0011] Furthermore, the mechanical detection and acquisition module includes, from top to bottom, a conveyor plate support flange, a spherical self-aligning pressure equalizing block, an upper insulating gasket, an annular piezoelectric ceramic sensor, a lower insulating gasket, and a frame fixing base. The conveyor plate support flange is rigidly connected to the bottom of the grain conveying mechanism.

[0012] Furthermore, the frame fixing base, lower insulating gasket, annular piezoelectric ceramic sensor, upper insulating gasket, and conveyor plate support flange are connected by a central pre-tightening bolt, which is used to maintain the annular piezoelectric ceramic sensor under static compressive prestress.

[0013] Furthermore, the spherical self-aligning pressure equalizing block is disposed between the conveyor plate support flange and the upper insulating gasket. The upper surface of the spherical self-aligning pressure equalizing block is a plane that mates with the bottom surface of the conveyor plate support flange, and the lower surface of the spherical self-aligning pressure equalizing block is a convex spherical surface that mates with the upper insulating gasket.

[0014] Furthermore, the mechanical detection and acquisition module also includes a high-frequency data acquisition card, an electrode lead-out groove, and a shielded signal line; the annular piezoelectric ceramic sensor is connected to the high-frequency data acquisition card through the shielded signal line, the shielded signal line is led out through the electrode lead-out groove, the electrode lead-out groove is filled with epoxy resin sealant, and a waterproof gland is provided at the outlet of the shielded signal line.

[0015] Furthermore, the edge computing and control module includes a processor, a memory, a synchronization trigger interface, an image acquisition interface, a light source driving interface, a mechanical acquisition interface, and a data output interface; the synchronization trigger interface is used to output synchronization trigger signals to the low-light image acquisition module and the mechanical detection acquisition module; the image acquisition interface is connected to the low-light image acquisition module; the mechanical acquisition interface is connected to the mechanical detection acquisition module; and the data output interface is used to output corn kernel feeding amount detection parameters and moisture content detection parameters.

[0016] Furthermore, the low-light image acquisition module and the mechanical detection acquisition module are arranged along the grain conveying path of the grain conveying mechanism, and the image acquisition area of ​​the low-light image acquisition module and the stress wave acquisition area of ​​the mechanical detection acquisition module correspond to the same grain conveying path.

[0017] A second aspect of this invention provides a method for dual-parameter detection of maize kernels based on physical prior and multi-source fusion, comprising: S1: Acquire images of corn kernels passing through the kernel transport mechanism and axial stress wave signals generated by the collision of corn kernels; S2: Perform low-light enhancement, dehazing, and target segmentation on corn kernel images based on physical priors to extract the visual features of corn kernels; S3: Perform time-domain and frequency-domain feature extraction on the axial stress wave signal to obtain mechanical characteristics; S4: Input visual and mechanical features into the feature-level multi-source heterogeneous data fusion mapping model, and output corn kernel feeding amount detection parameters and moisture content detection parameters.

[0018] Furthermore, the low-light enhancement, dehazing, and target segmentation of corn kernel images based on physical priors include the following steps: S21: Estimating motion vectors between adjacent frames of corn kernel images based on ORB feature point matching; S22: Input the motion vector as a priori parameter of motion blur into the low-light enhancement and dehazing coupled network; S23: Low-light enhancement and defogging coupled network reconstructs corn kernel images based on dust absorption-scattering coupled degradation model, and outputs reconstructed images; S24: Edge cropping and alignment of the reconstructed image based on motion vectors; S25: Perform boundary-aware target segmentation on the reconstructed image after edge cropping and alignment to obtain the corn kernel region and extract visual features.

[0019] Furthermore, the low-light enhancement and defogging coupling network adopts a single encoder-dual decoder structure. The single encoder is used to extract multi-scale shared features of the corn kernel image, the first decoder is used to perform illuminance adaptive mapping, and the second decoder is used to introduce a dust absorption-scattering coupled degradation model. The transmittance map obtained according to the degradation model is multiplied element-wise with the feature map in the second decoder to achieve feature modulation under physical prior constraints.

[0020] Furthermore, the visual features include kernel pixel area, near-infrared average reflectance, and gray-level co-occurrence matrix contrast, while the mechanical features include integral impulse, power spectrum centroid frequency, and 5kHz~15kHz band energy ratio. The feature-level multi-source heterogeneous data fusion mapping model takes a six-dimensional feature vector composed of visual and mechanical features as input, performs nonlinear compensation inversion of corn kernel feeding amount based on visual pixel area and mechanical integral impulse, and inverts corn kernel moisture content based on the six-dimensional feature vector.

[0021] Furthermore, the visual features include the seed pixel area (Area) and the near-infrared average reflectance (R). avg And the contrast ratio Con of the gray-level co-occurrence matrix, the mechanical features include integral impulse, power spectral centroid frequency F c The energy ratio E in the 5kHz to 15kHz frequency band r The feature-level multi-source heterogeneous data fusion mapping model takes a six-dimensional feature vector V composed of visual and mechanical features as input. ; The parameter m for detecting corn kernel feed rate is calculated using a nonlinear compensation formula: Where a, k, and b are the compensation coefficients obtained through calibration; when If the effective pixel area ratio of a single frame is less than 10%, switch to the pure mechanics estimation branch. The pure mechanics estimation branch is calculated according to the following formula: Where c and d are the mechanical branch compensation coefficients obtained through calibration; the water content detection parameters are obtained by multilayer perceptron inversion with a six-dimensional feature vector V as input.

[0022] Beneficial effects: Compared with the prior art, the present invention has at least the following beneficial effects: (1) The system of the present invention sets the low illumination image acquisition module at the high observation window on the outer wall of the threshing chamber and sets a transparent dust cover and two sets of inclined infrared strobe light sources, which can acquire corn kernel images in the dark dust environment of the threshing chamber and reduce the influence of dust adhesion and light source reflection on image acquisition. (2) The mechanical detection and acquisition module of the present invention is set on the lower side of the grain conveying mechanism and forms a pre-tightened detection structure through the conveying plate support flange, spherical automatic self-aligning pressure equalizing block, insulating gasket, annular piezoelectric ceramic sensor and frame fixing base, which can improve the stability of axial stress wave signal transmission and acquisition. (3) The system of the present invention sets an electrode lead-out groove, epoxy resin sealant, shielded signal line and waterproof gland at the signal lead-out position of the annular piezoelectric ceramic sensor, which can improve the protection capability of the mechanical detection acquisition module in the field dust, mud and water and vibration environment. (4) The system of the present invention sets up a synchronous trigger interface, an image acquisition interface, a light source driving interface, a mechanical acquisition interface and a data output interface through the edge computing and control module, so that the low illumination image acquisition module and the mechanical detection acquisition module can cooperate to collect along the same grain conveying path, which facilitates the output of corn grain feeding amount detection parameters and moisture content detection parameters; (5) The method of the present invention introduces motion blur prior and dust absorption-scattering coupled degradation model in the image processing process, and reconstructs corn kernel images in dark dust environment through low illumination enhancement and defogging coupled network, which can improve the reliability of subsequent target segmentation and visual feature extraction. (6) The method of the present invention simultaneously acquires corn kernel images and axial stress wave signals, and extracts visual and mechanical features. It outputs feeding amount detection parameters and moisture content detection parameters through a feature-level multi-source heterogeneous data fusion mapping model, which can reduce parameter coupling errors in single visual detection or single mechanical detection. (7) The method of the present invention uses edge computing and control module to perform synchronous triggering, image acquisition, mechanical signal acquisition and detection parameter output, which facilitates real-time detection of dual parameters of corn kernels during the operation of combine harvester. Attached Figure Description

[0023] Figure 1 This is a side view of the overall hardware installation layout inside the threshing chamber of the present invention; Figure 2 for Figure 1 Internal sectional view; Figure 3 for Figure 1 A magnified side view diagram of a portion of the image; Figure 4 The attached figure shows the outer casing of the detection system provided in the embodiment of the present invention; Labeling Explanation: 1-Hopper; 2-Grain Transfer Mechanism; 3-Starlight-level Global Shutter Image Sensor; 4-Infrared Strobe Source; 5-High-level Observation Window; 6-Spherical Automatic Self-aligning Pressure Equalizing Block; 7-High-frequency Data Acquisition Card; 8-Transparent Dust Cover; 9-Low-light Image Acquisition Module; 10-Upper Insulating Gasket; 11-Lower Insulating Gasket; 12-Outer Wall of Threshing Chamber; 13-Left Support Leg; 14-Right Support Leg; 15-Frame Fixing Base; 16-Observation Window Mounting Flange; 17-Support Beam; 18-Gate-shaped Mounting Bracket; 19-Annular Piezoelectric Ceramic Sensor; 20-Conveyor Plate Support Flange; 21-Electrode Lead-out Groove; 22-Shielded Signal Line; 23-Edge Computing and Control Module; 24-Mechanical Detection and Acquisition Module. Detailed Implementation

[0024] The technical solution of the present invention will be further described below with reference to the accompanying drawings. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Without departing from the concept of the present invention, those skilled in the art can make equivalent substitutions or modifications to the relevant structures, parameters, algorithm models, or data processing methods, and such substitutions or modifications should all fall within the scope of protection of the present invention.

[0025] Example 1

[0026] like Figures 1 to 4 As shown, this embodiment provides a dual-parameter detection system for corn kernels based on physical prior and multi-source fusion. It is applied to the threshing chamber and kernel conveying mechanism of a corn combine harvester. It is used to simultaneously acquire image information and mechanical signal information during the corn kernel conveying process, and output corn kernel feeding amount detection parameters and moisture content detection parameters based on the image information and mechanical signal information.

[0027] The corn kernel dual-parameter detection system of this embodiment includes a hopper 1, a kernel conveying mechanism 2, a threshing chamber outer wall 12, a high-level observation window 5, a low-light image acquisition module 9, a mechanical detection acquisition module 24, and an edge computing and control module 23. The hopper 1 is located at the feed end of the kernel conveying mechanism 2 and is used to guide corn kernels into the kernel conveying mechanism 2; the kernel conveying mechanism 2 is used to receive and transport corn kernels; the threshing chamber outer wall 12 is located outside the kernel conveying mechanism 2, and the high-level observation window 5 is located on the threshing chamber outer wall 12 to provide an image acquisition channel for the low-light image acquisition module 9. The low-light image acquisition module 9 and the mechanical detection acquisition module 24 are arranged along the same kernel conveying path, so that the image acquisition area and the stress wave acquisition area correspond to the corn kernels on the same conveying path.

[0028] The low-light image acquisition module 9 is positioned at the high-level observation window 5, with its acquisition direction facing the grain conveying path of the grain conveying mechanism 2. As corn kernels pass through the grain conveying mechanism 2, the low-light image acquisition module 9 acquires images of the corn kernels. The mechanical detection acquisition module 24 is positioned below the grain conveying mechanism 2. When corn kernels are conveyed on the grain conveying mechanism 2 and collisions occur, the mechanical detection acquisition module 24 acquires the corresponding axial stress wave signal. The low-light image acquisition module 9 and the mechanical detection acquisition module 24 are arranged along the same grain conveying path, ensuring that the image acquisition area and the stress wave acquisition area correspond to the corn kernels on the same conveying path.

[0029] The low-light image acquisition module 9 includes an observation window mounting flange 16, a portal-shaped mounting bracket 18, a starlight-level global shutter image sensor 3, and two sets of infrared stroboscopic light sources 4. The observation window mounting flange 16 is located at the high-level observation window 5, and the portal-shaped mounting bracket 18 is fixed to the observation window mounting flange 16. The portal-shaped mounting bracket 18 includes a support beam 17, on which the starlight-level global shutter image sensor 3 is mounted, facing the high-level observation window 5. The two sets of infrared stroboscopic light sources 4 are respectively located on both sides of the starlight-level global shutter image sensor 3 to provide infrared illumination for the image acquisition area. A transparent dust cover 8 is provided on the outer side of the lens of the starlight-level global shutter image sensor 3. The transparent dust cover 8 is arranged opposite to the high-level observation window 5 to prevent dust, debris, and moisture from the threshing chamber from entering the lens area. The transparent dust cover 8 is made of high-transparency optical glass, and its outer surface is coated with a nano-hydrophobic and anti-fouling coating, which can reduce the adhesion of dust and moisture on the outer surface of the transparent dust cover 8 and reduce the probability of the image acquisition channel being blocked during continuous operation. Two sets of infrared strobe sources 4 are symmetrically arranged with respect to the starlight-level global shutter image sensor 3. The emitting surface of the infrared strobe source 4 is tilted towards the inside of the shooting field of view of the starlight-level global shutter image sensor 3, and the tilt angle of the emitting surface of the infrared strobe source 4 relative to the vertical direction is 20°±2°. Through the tilted arrangement, the illumination range of the infrared strobe source 4 can cover the shooting field of view of the starlight-level global shutter image sensor 3, while reducing the possibility of light being directly reflected to the lens and forming light spots or halos.

[0030] The mechanical detection and acquisition module 24 includes a conveyor plate support flange 20, a spherical self-aligning pressure equalizing block 6, an upper insulating gasket 10, an annular piezoelectric ceramic sensor 19, a lower insulating gasket 11, and a frame fixing base 15. The conveyor plate support flange 20 is connected to the lower side of the grain conveying mechanism 2 and is used to transmit the impact of corn kernels on the grain conveying mechanism 2 to the detection structure below. The spherical self-aligning pressure equalizing block 6 is located below the conveyor plate support flange 20, the upper insulating gasket 10 is located below the spherical self-aligning pressure equalizing block 6, the annular piezoelectric ceramic sensor 19 is located below the upper insulating gasket 10, the lower insulating gasket 11 is located below the annular piezoelectric ceramic sensor 19, and the frame fixing base 15 is located below the lower insulating gasket 11.

[0031] The frame fixing base 15, the lower insulating gasket 11, the annular piezoelectric ceramic sensor 19, the upper insulating gasket 10, and the conveyor plate support flange 20 are connected by a central pre-tightening bolt. After the central pre-tightening bolt is tightened, the annular piezoelectric ceramic sensor 19 maintains static compressive prestress. The pre-tightening structure can reduce the axial gap in the detection structure, so that the axial stress wave generated by the collision of corn kernels can be transmitted to the annular piezoelectric ceramic sensor 19 through the kernel transmission mechanism 2 and the conveyor plate support flange 20.

[0032] A spherical self-aligning pressure equalizing block 6 is disposed between the conveyor plate support flange 20 and the upper insulating gasket 10. The upper surface of the spherical self-aligning pressure equalizing block 6 is a plane that mates with the bottom surface of the conveyor plate support flange 20, and the lower surface of the spherical self-aligning pressure equalizing block 6 is a convex spherical surface that mates with the upper insulating gasket 10. During assembly, the convex spherical surface can perform posture compensation under pressure, making the annular piezoelectric ceramic sensor 19 more uniformly stressed and reducing the risk of localized stress concentration due to installation plane errors.

[0033] The mechanical detection and acquisition module 24 also includes a high-frequency data acquisition card 7, an electrode lead-out groove 21, and a shielded signal line 22. The ring-shaped piezoelectric ceramic sensor 19 is connected to the high-frequency data acquisition card 7 via the shielded signal line 22. The shielded signal line 22 is led out through the electrode lead-out groove 21, which is filled with epoxy resin sealant. A waterproof gland is provided at the exit point of the shielded signal line 22. Through the cooperation of the electrode lead-out groove 21, epoxy resin sealant, and waterproof gland, the sensor signal lead-out part can be sealed and protected, reducing the impact of field dust, mud, and vibration on the stability of signal acquisition.

[0034] The edge computing and control module 23 includes a processor, a memory, a synchronization trigger interface, an image acquisition interface, a light source drive interface, a mechanical acquisition interface, and a data output interface. The synchronization trigger interface is connected to the low-light image acquisition module 9 and the mechanical detection acquisition module 24, respectively, and is used to output synchronization trigger signals. The image acquisition interface is connected to the starlight-level global shutter image sensor 3, and is used to receive corn kernel images. The light source drive interface is connected to the infrared strobe light source 4, and is used to drive the infrared strobe light source 4 to operate. The mechanical acquisition interface is connected to the high-frequency data acquisition card 7, and is used to receive axial stress wave signals. The data output interface is used to output corn kernel feeding amount detection parameters and moisture content detection parameters.

[0035] During system operation, corn kernels enter the kernel conveying mechanism 2 from the hopper 1 and move along the kernel conveying path. The edge computing and control module 23 outputs a synchronous trigger signal through the synchronous trigger interface, enabling the low-light image acquisition module 9 and the mechanical detection acquisition module 24 to synchronously acquire data on the corn kernels along the same kernel conveying path. The low-light image acquisition module 9 acquires corn kernel images through the high-level observation window 5, while the mechanical detection acquisition module 24 acquires axial stress wave signals generated by corn kernel collisions. After receiving the corn kernel images and axial stress wave signals, the edge computing and control module 23 processes the two types of data and outputs corn kernel feeding amount detection parameters and moisture content detection parameters.

[0036] In this embodiment, the low-light image acquisition module 9 and the mechanical detection acquisition module 24 are arranged along the same grain conveying path, so that the image information and axial stress wave signal correspond to the corn grains on the same working path, improving the spatial correspondence between visual and mechanical information. The low-light image acquisition module 9 adapts to the image acquisition requirements in the dark dust environment of the threshing chamber through the high-position observation window 5, the transparent dust cover 8, and the infrared strobe light source 4; the mechanical detection acquisition module 24 improves the stability of axial stress wave signal acquisition through the central pre-tightening, spherical self-aligning, and sealed lead wire structure; the edge computing and control module 23 realizes the synchronous detection of corn grain feeding amount and moisture content through synchronous triggering, image reception, mechanical signal reception, and data output.

[0037] Example 2

[0038] This embodiment, based on the corn kernel dual-parameter detection system described in Embodiment 1, further illustrates a dual-parameter detection method for corn kernel feeding amount and moisture content. The detection method includes simultaneous acquisition of dual-modal data, image feature extraction based on physical priors, mechanical feature extraction of axial stress wave signals, and multi-source fusion dual-parameter inversion.

[0039] S1: Synchronously acquire images of corn kernels passing through the kernel transfer mechanism 2 and axial stress wave signals generated by the collision of corn kernels.

[0040] Corn kernels enter the kernel conveying mechanism 2 through the hopper 1 and move along the kernel conveying path. The edge computing and control module 23 outputs a synchronous control signal through the synchronous trigger interface, causing the infrared strobe light source 4, the starlight-level global shutter image sensor 3, and the high-frequency data acquisition card 7 to operate according to the same acquisition sequence. The starlight-level global shutter image sensor 3 acquires images of corn kernels passing through the kernel conveying mechanism 2 through the high-position observation window 5; the annular piezoelectric ceramic sensor 19 outputs an axial stress wave signal after being impacted by the corn kernels, and the high-frequency data acquisition card 7 acquires the axial stress wave signal and transmits it to the edge computing and control module 23.

[0041] In a preferred embodiment, the synchronization control signal is a pulse width modulation (PWM) synchronization control signal. The edge computing and control module 23 simultaneously triggers the infrared strobe light source 4 to illuminate, the starlight-level global shutter image sensor 3 to expose, and the high-frequency data acquisition card 7 to sample via the PWM synchronization control signal, ensuring that the corn kernel image and the axial stress wave signal correspond to the same kernel conveying path and the same operating period. More preferably, the time synchronization error of the dual-modal data is no greater than a preset synchronization error threshold, which can be 10 μs. The acquisition frequency band of the axial stress wave signal can be 1 kHz to 20 kHz.

[0042] S2: Perform low-light enhancement, dehazing, and target segmentation on the corn kernel image based on physical priors to extract the visual features of the corn kernel.

[0043] After acquiring the corn kernel image, the edge computing and control module 23 first estimates the motion vector between adjacent corn kernel images. In a preferred embodiment, the edge computing and control module 23 uses the ORB feature point matching method to estimate the motion vector between adjacent images. Addressing the problem that dust particles in the high-dust environment of the threshing chamber easily generate false feature points, the edge computing and control module 23 first uses the grayscale difference between corn kernels and dust particles in the near-infrared image to set a preset grayscale threshold, pre-screening and removing false dust feature points.

[0044] In a preferred embodiment, the grayscale value of corn kernels in the near-infrared image is higher than that of dust particles. For example, the grayscale value of corn kernels can be distributed between 80 and 200, while the grayscale value of dust particles can be distributed between 20 and 60. A grayscale threshold is set accordingly to remove false feature points from dust particles. Subsequently, the edge calculation and control module 23 processes the remaining feature points using a random sampling consistency algorithm, fits the homography transformation matrix between adjacent frames, and removes mismatched points whose deviation from the homography transformation matrix is ​​greater than a preset pixel deviation threshold. In a preferred embodiment, the preset pixel deviation threshold is 3 pixels. After the above processing, the motion vector between adjacent frames of corn kernel images is obtained.

[0045] After completing the motion vector estimation, the edge computing and control module 23 uses the motion vector as a priori parameter for the motion blur kernel and inputs it into the low-light enhancement and dehazing coupling model. The low-light enhancement and dehazing coupling model includes a shared feature extraction unit, an illumination adaptive mapping unit, and a dust degradation constraint unit. The shared feature extraction unit is used to extract multi-scale shared features from the corn kernel image; the illumination adaptive mapping unit is used to improve the problem of insufficient image brightness in dark environments; and the dust degradation constraint unit is used to combine the absorption, scattering, and texture attenuation characteristics under dust environments to provide physical prior constraints on the image dehazing and detail restoration process.

[0046] In a preferred embodiment, the low-light enhancement and defogging coupling model employs a single encoder-dual decoder architecture. The encoder uses a lightweight convolutional neural network structure to extract shared features, such as the inverse residual structure of the first few layers of MobileNetV2; the first decoder is used to complete the illumination adaptive mapping; the second decoder embeds a closed-cavity dust absorption-scattering coupling degradation model based on radiative transfer theory to characterize the image brightness attenuation, scattering fogging, and texture blurring processes under dusty conditions in the threshing chamber.

[0047] During image reconstruction, the edge computing and control module 23 calculates a priori transmittance maps based on the dust absorption-scattering coupled degradation model, and inputs the priori transmittance maps into the low-light enhancement and dehazing coupled model to perform feature modulation on the intermediate feature maps. In a preferred embodiment, the priori transmittance maps are multiplied element-wise with the intermediate layer feature maps in the dust degradation constraint unit to achieve feature enhancement and dust degradation suppression under physical prior constraints, and output the reconstructed image.

[0048] The low-light enhancement and dehazing coupled model employs a dynamic weighted joint loss function during training. This dynamic weighted joint loss function includes light fidelity loss, physical constraint loss, and structural similarity loss, and its expression is: Among them, L total For dynamic weight joint loss function, To preserve the accuracy of illumination, For physical constraint loss, L ssim For structural similarity loss, , , These are the corresponding weighting coefficients.

[0049] The illuminance fidelity loss uses a pixel-level L1 norm to constrain the brightness consistency between the reconstructed image output by the network and the real sharp image. Its expression is: The physical constraint loss uses a pixel-level L1 norm to constrain the consistency between the network-predicted transmittance map and the prior transmittance map calculated based on the radiative transfer model. Its expression is: In the formula, , These are the height and width of the image, respectively. Output the reconstructed image to the network. For true and clear images, This is a network-predicted transmittance map. This is a priori transmittance map calculated based on the radiative transfer model.

[0050] The weighting logic is as follows: when the average brightness is less than 30 grayscale values, the brightness is increased in increments of 0.05. At the same time, lower When the global mean transmittance is <0.4, adjust the level in increments of 0.05. At the same time, lower ; , The value range is 0.2 to 0.6. The value range is 0.2 to 0.4, and it always satisfies... .

[0051] After target segmentation, the edge computing and control module 23 extracts the visual features of the corn kernels. These visual features include kernel pixel area, near-infrared average reflectance, and gray-level co-occurrence matrix contrast. The calculation parameters for the gray-level co-occurrence matrix contrast are: window size... Grayscale levels 16, pixel distance 1, angle , , , Take the average value.

[0052] S3: Perform time-domain and frequency-domain feature extraction on the axial stress wave signal to obtain mechanical features.

[0053] After being impacted by corn kernels, the annular piezoelectric ceramic sensor 19 outputs an axial stress wave signal. The high-frequency data acquisition card 7 acquires the axial stress wave signal and transmits it to the edge computing and control module 23. The edge computing and control module 23 preprocesses the axial stress wave signal, including at least one of temperature drift correction, low-frequency mechanical vibration noise filtering, and voltage fluctuation compensation.

[0054] In a preferred embodiment, the edge computing and control module 23 performs temperature drift correction on the output signal of the ring piezoelectric ceramic sensor 19 based on a temperature calibration lookup table. The temperature calibration lookup table is obtained by calibrating the ring piezoelectric ceramic sensor 19 under different temperature conditions and is used to correct the impact of ambient temperature changes on the sensor output amplitude during field operations.

[0055] In a preferred embodiment, the edge computing and control module 23 employs an adaptive Kalman filter to filter out low-frequency mechanical vibration noise. More preferably, the adaptive Kalman filter is used to suppress low-frequency mechanical vibration noise below 50Hz to reduce interference from combine harvester body vibration, grain conveying mechanism 2 vibration, and ground excitation on the axial stress wave signal.

[0056] In a preferred embodiment, the edge computing and control module 23 corrects the impact of power supply voltage fluctuations on the output stability and synchronous acquisition stability of the infrared stroboscopic light source 4 through voltage adaptive compensation. The voltage adaptive compensation method can be implemented using a PID control algorithm, adjusting the driving current or driving duty cycle of the infrared stroboscopic light source 4 within the range of agricultural machinery battery voltage fluctuations to maintain a stable output luminous flux.

[0057] After preprocessing, the edge computing and control module 23 performs time-domain and frequency-domain feature extraction on the axial stress wave signal to obtain mechanical features. These mechanical features include the integral impulse, the power spectrum centroid frequency Fc, and the target band energy ratio Er.

[0058] The integral impulse reflects the intensity of the mechanical response generated by the collision of corn kernels; the centroid frequency Fc of the power spectrum reflects the spectral distribution characteristics of the axial stress wave signal; and the target band energy ratio Er reflects the energy proportion of the corn kernel collision signal within the target band. In a preferred embodiment, the target band is 5kHz to 15kHz, and the axial stress wave signal acquisition frequency band is 1kHz to 20kHz.

[0059] S4: Input visual and mechanical features into the feature-level multi-source heterogeneous data fusion mapping model, and output corn kernel feeding amount detection parameters and moisture content detection parameters.

[0060] The edge computing and control module 23 concatenates visual and mechanical features to form a six-dimensional feature vector, which is: Where Area is the area of ​​the seed pixel, R avg Here, is the near-infrared average reflectance, Con is the gray-level co-occurrence matrix contrast, Impulse is the integral impulse, and F... c E is the centroid frequency of the power spectrum.r The energy ratio is in the 5kHz to 15kHz frequency band.

[0061] The six-dimensional feature vector is input into the feature-level multi-source heterogeneous data fusion mapping model. During the feed rate detection process, the feature-level multi-source heterogeneous data fusion mapping model performs a preliminary estimate based on the visual pixel area and combines it with the mechanical integral impulse for nonlinear compensation, outputting the corn kernel feed rate detection parameters. The feed rate inversion uses the following nonlinear compensation formula: Where m is the corn kernel feeding amount detection parameter, Area is the kernel pixel area, Impulse is the integral impulse, and a, k, and b are the compensation coefficients obtained through calibration.

[0062] When image quality is poor, dust obstruction is severe, lens contamination occurs, or seed region segmentation is unreliable, the edge computing and control module 23 automatically switches to the pure mechanical estimation branch. Specifically, when the proportion of effective seed pixels in a single frame is lower than a preset area threshold, or the image quality evaluation value is lower than a preset quality threshold, the edge computing and control module 23 automatically switches to the pure mechanical estimation branch. In a preferred embodiment, the preset area threshold is 10%.

[0063] The formula for calculating the pure mechanics estimation branch is: Where c and d are the mechanical branch compensation coefficients obtained through calibration.

[0064] In the moisture content detection process, a feature-level multi-source heterogeneous data fusion mapping model performs moisture content inversion based on a six-dimensional fused feature vector V, outputting the corn kernel moisture content detection parameter M. In a preferred embodiment, the moisture content inversion model adopts a multilayer perceptron structure, including an input layer, a hidden layer, and an output layer. The input layer receives the six-dimensional fused feature vector V, and the output layer outputs the corn kernel moisture content detection parameter M.

[0065] More preferably, the input layer is 6-dimensional, the hidden layer is 32-dimensional, and the output layer is 1-dimensional. To adapt to the computing resources of agricultural machinery edge computing, the low-light enhancement and defogging coupling model, the boundary-aware target segmentation model, and the water content inversion model can all undergo model compression processing. In a preferred embodiment, the model compression processing includes structured channel pruning and INT8 offline post-training quantization. Through model compression processing, the edge computing and control module 23 can complete real-time inference under limited computing resources. More preferably, the single-frame inference latency is no more than 30ms.

[0066] In this embodiment, the feature-level multi-source heterogeneous data fusion mapping model undergoes static bench joint inversion calibration before deployment. During calibration, standard samples of corn kernels with different moisture content gradients are prepared, and the true values ​​of moisture content and feed amount for each standard sample are obtained. In a preferred embodiment, the moisture content of the standard samples covers the range of 15% to 35%, and at least five moisture content gradients are set. The true value of moisture content is obtained by drying, and the true value of feed amount is obtained by electronic weighing.

[0067] During the calibration process, the synchronization accuracy of the low-light image acquisition module 9 and the mechanical detection acquisition module 24 is first calibrated to ensure that the synchronization error does not exceed the preset synchronization error threshold. Then, the low-light image acquisition module 9 and the mechanical detection acquisition module 24 are used to synchronously acquire each standard sample to obtain the corresponding corn kernel image and axial stress wave signal. Next, visual features are extracted from the corn kernel image, and mechanical features are extracted from the axial stress wave signal. The visual features and mechanical features are then spliced ​​together to form a six-dimensional fusion feature vector V.

[0068] Using the true values ​​of moisture content and feed amount as supervision labels, the feature-level multi-source heterogeneous data fusion mapping model is trained until it meets the preset detection accuracy requirements. In a preferred embodiment, the preset detection accuracy requirements are: the relative error of feed amount detection is no greater than 5%, and the absolute error of moisture content detection is no greater than 2%. After training, the feature-level multi-source heterogeneous data fusion mapping model is deployed to the edge computing and control module 23 for synchronous detection of corn kernel feed amount and moisture content during the actual operation of the combine harvester.

[0069] The detection method in this embodiment synchronously acquires image and mechanical information along the same grain conveying path through the low-light image acquisition module 9 and the mechanical detection acquisition module 24. It improves the reliability of visual features in dark dust environments by using low-light enhancement and defogging processing under physical prior constraints, and reduces parameter coupling errors in single detection methods by using feature-level fusion of visual and mechanical features, thereby achieving synchronous output of corn grain feeding amount detection parameters and moisture content detection parameters.

[0070] The content not described in detail in this embodiment belongs to conventional technical content that can be understood and implemented by those skilled in the art. The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions or improvements made within the concept and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A dual-parameter detection system for maize kernels based on physical priors and multi-source fusion, characterized in that, The system includes a hopper (1), a grain conveying mechanism (2), an outer wall of the threshing chamber (12), a high-level observation window (5) set on the outer wall of the threshing chamber (12), a low-light image acquisition module (9), a mechanical detection acquisition module (24), and an edge computing and control module (23). The low-light image acquisition module (9) is set at the high-level observation window (5) and is used to acquire images of corn grains passing through the grain conveying mechanism (2). The mechanical detection acquisition module (24) is set on the lower side of the grain conveying mechanism (2) and is used to acquire axial stress wave signals generated by the collision of corn grains. The edge computing and control module (23) is electrically connected to the low-light image acquisition module (9) and the mechanical detection acquisition module (24) respectively. The edge computing and control module (23) is provided with a synchronous trigger interface for outputting synchronous trigger signals, an image acquisition interface for receiving corn grain images, a mechanical signal interface for receiving axial stress wave signals, and a detection parameter output interface for outputting detection parameters.

2. The dual-parameter detection system for maize kernels based on physical prior and multi-source fusion as described in claim 1, characterized in that, The low-light image acquisition module (9) includes an observation window mounting flange (16), a gate-shaped mounting bracket (18), a starlight-level global shutter image sensor (3), and two sets of infrared strobe sources (4). The observation window mounting flange (16) is located at the high-position observation window (5), the gate-shaped mounting bracket (18) is fixed on the observation window mounting flange (16), the starlight-level global shutter image sensor (3) is mounted on the support beam (17) of the gate-shaped mounting bracket (18), and the two sets of infrared strobe sources (4) are respectively located on both sides of the starlight-level global shutter image sensor (3).

3. The dual-parameter detection system for maize kernels based on physical prior and multi-source fusion according to claim 2, characterized in that, The two sets of infrared strobe light sources (4) are symmetrically arranged relative to the star-level global shutter image sensor (3). The light-emitting surface of the infrared strobe light source (4) is tilted towards the inside of the shooting field of view of the star-level global shutter image sensor (3), and the tilt angle of the light-emitting surface of the infrared strobe light source (4) relative to the vertical direction is 20°±2°.

4. The dual-parameter detection system for maize kernels based on physical prior and multi-source fusion according to claim 1, characterized in that, The mechanical detection and acquisition module (24) includes, from top to bottom, a conveyor plate support flange (20), a spherical self-aligning pressure equalizing block (6), an upper insulating gasket (10), an annular piezoelectric ceramic sensor (19), a lower insulating gasket (11), and a frame fixing base (15). The conveyor plate support flange (20) is rigidly connected to the bottom of the grain conveying mechanism (2). The frame fixing base (15), the lower insulating gasket (11), the annular piezoelectric ceramic sensor (19), the upper insulating gasket (10), and the frame fixing base (15) are arranged sequentially from top to bottom. The conveyor plate support flange (20) is connected through a central pre-tightening bolt, which is used to keep the annular piezoelectric ceramic sensor (19) under static compression prestress. The spherical self-aligning pressure equalizing block (6) is disposed between the conveyor plate support flange (20) and the upper insulating gasket (10). The upper surface of the spherical self-aligning pressure equalizing block (6) is a plane that mates with the bottom surface of the conveyor plate support flange (20), and the lower surface of the spherical self-aligning pressure equalizing block (6) is a convex spherical surface that mates with the upper insulating gasket (10).

5. The dual-parameter detection system for maize kernels based on physical prior and multi-source fusion according to claim 1, characterized in that, The mechanical detection acquisition module (24) also includes a high-frequency data acquisition card (7), an electrode lead-out groove (21), and a shielded signal line (22); the annular piezoelectric ceramic sensor (19) is connected to the high-frequency data acquisition card (7) through the shielded signal line (22), the shielded signal line (22) is led out through the electrode lead-out groove (21), the electrode lead-out groove (21) is filled with epoxy resin sealant, and a waterproof gland is provided at the outlet of the shielded signal line (22).

6. The dual-parameter detection system for maize kernels based on physical prior and multi-source fusion according to claim 1, characterized in that, The edge computing and control module (23) includes a processor, a memory, a synchronous trigger interface, an image acquisition interface, a light source driving interface, a mechanical acquisition interface, and a data output interface. The synchronous trigger interface is used to output synchronous trigger signals to the low-light image acquisition module (9) and the mechanical detection acquisition module (24). The image acquisition interface is connected to the low-light image acquisition module (9), the mechanical acquisition interface is connected to the mechanical detection acquisition module (24), and the data output interface is used to output corn kernel feeding amount detection parameters and moisture content detection parameters.

7. A dual-parameter detection method for maize kernels based on physical prior and multi-source fusion, characterized in that, Using the corn kernel dual-parameter detection system as described in any one of claims 1-6, the following steps are performed: S1: Acquire images of corn kernels passing through the kernel transport mechanism and axial stress wave signals generated by the collision of corn kernels; S2: Perform low-light enhancement, dehazing, and target segmentation on corn kernel images based on physical priors to extract the visual features of corn kernels; S3: Perform time-domain and frequency-domain feature extraction on the axial stress wave signal to obtain mechanical characteristics; S4: Input visual and mechanical features into the feature-level multi-source heterogeneous data fusion mapping model, and output corn kernel feeding amount detection parameters and moisture content detection parameters.

8. The dual-parameter detection method for maize kernels based on physical prior and multi-source fusion according to claim 7, characterized in that, The physical prior-based low-light enhancement, dehazing, and target segmentation of corn kernel images includes the following steps: S21: Estimating motion vectors between adjacent frames of corn kernel images based on ORB feature point matching; S22: Input the motion vector as a priori parameter of motion blur into the low-light enhancement and dehazing coupled network; S23: Low-light enhancement and defogging coupled network reconstructs corn kernel images based on dust absorption-scattering coupled degradation model, and outputs reconstructed images; S24: Edge cropping and alignment of the reconstructed image based on motion vectors; S25: Perform boundary-aware target segmentation on the reconstructed image after edge cropping and alignment to obtain the corn kernel region and extract visual features.

9. The dual-parameter detection method for maize kernels based on physical prior and multi-source fusion according to claim 8, characterized in that, The low-light enhancement and defogging coupling network adopts a single encoder-dual decoder structure. The single encoder is used to extract multi-scale shared features of corn kernel images, the first decoder is used to perform illuminance adaptive mapping, and the second decoder is used to introduce a dust absorption-scattering coupled degradation model. The transmittance map obtained according to the degradation model is multiplied element-wise with the feature map in the second decoder to achieve feature modulation under physical prior constraints.

10. The dual-parameter detection method for maize kernels based on physical prior and multi-source fusion according to claim 7, characterized in that, The visual features include the seed pixel area (Area) and the near-infrared average reflectance (R). avg And the contrast ratio Con of the gray-level co-occurrence matrix, the mechanical features include integral impulse, power spectral centroid frequency F c The energy ratio E in the 5kHz to 15kHz frequency band r The feature-level multi-source heterogeneous data fusion mapping model takes a six-dimensional feature vector V composed of visual and mechanical features as input. ; The parameter m for detecting corn kernel feed rate is calculated using a nonlinear compensation formula: Where a, k, and b are the compensation coefficients obtained through calibration; when If the effective pixel area ratio of a single frame is less than 10%, switch to the pure mechanics estimation branch. The pure mechanics estimation branch is calculated according to the following formula: Where c and d are the mechanical branch compensation coefficients obtained through calibration; the water content detection parameters are obtained by multilayer perceptron inversion with a six-dimensional feature vector V as input.

Images