A visual detection method and system for grain particle morphological characteristics

Abstract

This invention discloses a visual detection method and system for grain particle morphology features, relating to the field of grain detection technology. The detection method includes: setting up at least two polarization-adjustable cameras arranged in an array and aligned with the detection area; controlling each polarization-adjustable camera to acquire at least two polarization channel images with different polarization directions and at least one unpolarized image using a polarization classification method. This invention uses at least two polarization-adjustable cameras arranged in an array to perform stereo matching of polarization channel images with the same polarization direction within an overlapping area to obtain a preliminary depth map. Simultaneously, it calculates the surface normal gradient field pixel-by-pixel based on the polarization angle image. Then, it uses a Poisson reconstruction method to fuse multiple preliminary depth maps with the surface normal gradient field and solve the Poisson equation to reconstruct a high-resolution depth map, thereby extracting the three-dimensional contour and volume features of grain particles, significantly improving the ability to accurately identify imperfect grains and the accuracy of quality assessment.

Description

Technical Field

[0001] This invention relates to the field of grain detection technology, specifically to a visual detection method and system for the morphological characteristics of grain particles. Background Technology

[0002] Visual detection methods for grain particle morphology use cameras and algorithms to automatically measure the appearance and grade the quality of grains (such as rice, wheat, and corn).

[0003] For example, patent publication number "CN114037835A", entitled "Method, Apparatus, Equipment and Storage Medium for Estimating Grain Quality", describes a method that includes: acquiring a first multi-view image of the grain grain to be estimated; obtaining a second multi-view image of each grain grain based on the first multi-view image; and obtaining multiple contour images and category information for each grain grain based on the second multi-view image. This invention estimates the quality of a single grain grain by detecting its spatial location and category, and by estimating the quality of a single grain grain based on its morphological information. The estimation process can be achieved without sieving the grain grain, saving manpower and resources.

[0004] The above methods rely solely on two-dimensional contour maps to calculate the projected area, and then estimate the mass through area-mass conversion coefficients or polynomial equations. Their morphological feature representation is limited to two-dimensional planar information and cannot obtain real three-dimensional geometric parameters such as the fullness, groove depth, and volume of grain particles. They also lack effective means of perceiving morphological features such as surface depressions, cracks, or loose internal structures. As a result, when faced with grain particles with irregular shapes or complex appearance defects, it is difficult to accurately reflect the actual morphological features of the particles. Therefore, a visual detection method and system for the morphological features of grain particles has been invented. Summary of the Invention

[0005] The purpose of this invention is to provide a visual detection method and system for the morphological characteristics of grain particles to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a visual detection method for the morphological characteristics of grain particles, the detection method comprising:

[0007] Set up at least two polarization-adjustable cameras arranged in an array and aligned with the detection area. Control each polarization-adjustable camera to acquire at least two polarization channel images with different polarization directions and at least one unpolarized image through a polarization classification method. The polarization directions in the polarization channel images acquired by all polarization-adjustable cameras are consistent.

[0008] Within the detection area, establish an independent region covered by the field of view of a single polarization-tunable camera and an overlapping region covered by the fields of view of at least two adjacent polarization-tunable cameras.

[0009] For grain particles within the field of view of a single camera, the polarization degree image and polarization angle image are calculated pixel by pixel based on polarization channel images with different polarization directions. Basic particle morphological features are obtained through feature extraction methods, and the basic particle morphological features are divided into morphological features of independent regions and morphological features of overlapping regions through region cropping methods.

[0010] The particle morphology features of the overlapping area are fused: a first preliminary depth map and a second preliminary depth map are obtained by stereo matching, and the surface normal gradient field is calculated pixel by pixel based on the polarization angle image;

[0011] The first preliminary depth map, the second preliminary depth map, and the surface normal gradient field are fused using the Poisson reconstruction method, and the Poisson equation is solved to reconstruct a high-resolution depth map.

[0012] The morphological features of the germ region, wax layer region, loose internal tissue region, surface depression pattern, and drying crack morphology of several overlapping regions are used as two-dimensional feature maps and projected onto a high-resolution depth map. At the projected position, the feature values ​​of each two-dimensional feature map are vector-concatenated with the three-dimensional contour features and volume features. The weight of each feature channel is calculated through an attention weight matrix for the concatenated feature vectors. After weighted summation, the vectors are mapped through a fully connected layer to output the complete morphological features of the grain particles in the overlapping region.

[0013] Furthermore, the polarization classification method includes:

[0014] Provide supplemental lighting for grain grains;

[0015] By adjusting the voltage applied to the liquid crystal polarization element in the polarization-tunable camera, the polarization-tunable camera can be switched sequentially to multiple preset polarization directions and a non-polarization state.

[0016] Synchronously adjust the liquid crystal polarization elements of all polarization-adjustable cameras to the same preset polarization direction, and synchronously trigger all polarization-adjustable cameras to acquire images, thereby obtaining polarization channel images of all polarization-adjustable cameras under the same preset polarization direction;

[0017] The process of synchronously switching and acquiring all preset polarization directions is repeated in sequence, and a single polarization-adjustable camera can obtain polarization channel images with at least two different polarization directions.

[0018] Synchronously adjust the liquid crystal polarization elements of all polarization-adjustable cameras to a non-polarized state, and simultaneously trigger all polarization-adjustable cameras to acquire images, thereby obtaining non-polarized images.

[0019] Furthermore, the feature extraction method includes: calculating the polarization degree image and polarization angle image within the outer contour pixel by pixel based on the gray values ​​of the same pixel position in polarization channel images with at least two different polarization directions;

[0020] Adaptive threshold segmentation is performed on each polarization channel image and the unpolarized image using polarization degree images to obtain the external contour of grain particles;

[0021] Within the outer contour, polarization degree images are used to distinguish between regions dominated by specular reflection and regions dominated by diffuse reflection.

[0022] In the region dominated by specular reflection, the germ region and the waxy layer region are identified by the polarization degree value. In the region dominated by diffuse reflection, the region with loose internal tissue is identified by the local coefficient of variation of the polarization degree value.

[0023] The polarization angle gradient direction and gradient magnitude of each pixel within the outer contour are calculated using polarization angle images. The direction and depth of surface depression textures are extracted by the continuous change trajectory of the polarization angle gradient, and the morphology of drying cracks is extracted by connecting the abrupt change points of the polarization angle gradient.

[0024] The basic particle morphology characteristics are obtained by combining polarization degree image, polarization angle image, external contour, germ region, wax layer region, internal loose tissue region and drying crack morphology.

[0025] Furthermore, using polarization degree images to distinguish between regions dominated by specular reflection and regions dominated by diffuse reflection includes:

[0026] Histogram statistics were performed on the polarization degree image to obtain the distribution curve of polarization degree values ​​in the range of 0 to 1;

[0027] The bimodal characteristics in the distribution curve are detected, and the region corresponding to the first peak with a lower polarization degree value is determined as the diffuse reflection-dominant region, while the region corresponding to the second peak with a higher polarization degree value is determined as the specular reflection-dominant region.

[0028] Using the polarization degree value corresponding to the valley between two peaks as the segmentation threshold, the polarization degree image is binarized to obtain a specular reflection region mask and a diffuse reflection region mask. Based on the specular reflection region mask and the diffuse reflection region mask, the specular reflection dominant region and the diffuse reflection dominant region are distinguished.

[0029] Furthermore, in the specular reflection-dominant region, the germ region can be identified by the degree of polarization, including:

[0030] Within the specular reflection region mask, connected regions with polarization values ​​below a first threshold are marked as candidate germ regions; the first threshold is 0.3 to 0.5 times the average polarization value in the dominant specular reflection region.

[0031] Morphological operations are performed on the germ candidate regions to remove isolated noise points; the area, roundness, and centroid offset relative to the grain particle outline of each germ candidate region are calculated.

[0032] Candidate regions whose area falls within the preset embryo area range, whose roundness is greater than 0.6, and whose center of gravity offset is within the range of the embryo's physiological position are identified as embryo regions.

[0033] The presence, area, and relative position of the germ region are used as the morphological characteristics of the germ region.

[0034] Furthermore, the region cropping method includes:

[0035] Based on the field of view boundary of each polarization-tunable camera, the spatial boundary between independent and overlapping regions is determined, and independent region masks and overlapping region masks are generated.

[0036] Obtain the outer contour of grain particles from the basic particle morphology features, and compare the pixel coordinates within the outer contour with the independent region mask and the overlapping region mask pixel by pixel.

[0037] If all pixels within the outer contour are located within the independent region mask, then the basic particle morphological features are marked as the morphological features of the independent region as a whole.

[0038] If all pixels within the outer contour are located within the overlapping region mask, then the basic particle morphological features are marked as the morphological features of the overlapping region as a whole.

[0039] If pixels exist in both the independent region mask and the overlapping region mask, then the number of pixels in the two regions is compared: when the number of pixels in the independent region mask is greater than or equal to the number of pixels in the overlapping region mask, the basic particle morphology features are classified as the morphology features of the independent region as a whole; when the number of pixels in the independent region mask is less than the number of pixels in the overlapping region mask, the basic particle morphology features are classified as the morphology features of the overlapping region as a whole.

[0040] Furthermore, a first preliminary depth map is obtained by stereo matching at least two polarization channel images with the same polarization direction. This process is repeated along the polarization direction to obtain at least two first preliminary depth maps. The number of first preliminary depth maps is the same as the number of polarization channel images of a single polarization-adjustable camera. A second preliminary depth map is obtained by stereo matching at least two unpolarized images.

[0041] The stereo matching method includes:

[0042] The intrinsic parameter matrix, rotation matrix, and translation vector of the polarization-tunable camera array are pre-calibrated, and the epipolar constraint equation is established.

[0043] In two polarization channel images with the same polarization direction, the pixel matching search range is limited to the corresponding epipolar neighborhood based on the epipolar constraint equation, and polarization consistency constraints are introduced.

[0044] Retain matching point pairs in two images where the difference in polarization degree of corresponding matching points is less than a preset consistency threshold and the difference in polarization angle is less than a preset angle threshold;

[0045] A semi-global matching algorithm is used to calculate the disparity map for the filtered matching point pairs. The disparity map is then converted into a preliminary depth map by combining the camera baseline distance and focal length parameters.

[0046] The same semi-global matching algorithm and disparity transformation are used on at least two unpolarized images to obtain a second preliminary depth map.

[0047] Furthermore, the fusion of the preliminary depth map with the surface normal gradient field using the Poisson reconstruction method includes:

[0048] Based on the gradient consistency of the first and second preliminary depth maps in the grain particle edge region, the confidence weights of the depth maps are calculated respectively.

[0049] The initial depth map is weighted and averaged using the confidence weights to obtain the fused depth map.

[0050] Using the surface normal gradient field as a gradient constraint term and the fused depth map as a boundary condition, the Poisson equation is constructed and solved to obtain the high-resolution depth map.

[0051] A visual inspection system for grain particle morphology features, the inspection system comprising:

[0052] Polarization image acquisition module: Set up at least two polarization-adjustable cameras arranged in an array and aligned with the detection area. Control each polarization-adjustable camera to acquire at least two polarization channel images with different polarization directions and at least one unpolarized image through a polarization classification method. The polarization directions in the polarization channel images acquired by all polarization-adjustable cameras are consistent.

[0053] Region segmentation module: Within the detection area, establish an independent region covered by the field of view of a single polarization-tunable camera and an overlapping region covered by the field of view of at least two adjacent polarization-tunable cameras;

[0054] Feature extraction module: For grain particles within the field of view of a single camera, the polarization degree image and polarization angle image are calculated pixel by pixel based on polarization channel images with different polarization directions. Basic particle morphological features are obtained through feature extraction methods. The basic particle morphological features are divided into morphological features of independent regions and morphological features of overlapping regions through region cropping methods.

[0055] Calculation module: Obtains preliminary depth maps No. 1 and No. 2 through stereo matching method, and calculates the surface normal gradient field pixel by pixel based on polarization angle image;

[0056] Depth map reconstruction module: The module fuses the first preliminary depth map, the second preliminary depth map, and the surface normal gradient field using the Poisson reconstruction method, and solves the Poisson equation to reconstruct a high-resolution depth map.

[0057] Feature mapping output module: The morphological features of the germ region, wax layer region, loose internal tissue region, surface depression pattern direction, and drying crack morphology of several overlapping regions are taken as two-dimensional feature maps and projected onto a high-resolution depth map. At the projected position, the feature values ​​of each two-dimensional feature map are vector-concatenated with the three-dimensional contour features and volume features. The weight of each feature channel is calculated by the attention weight matrix for the concatenated feature vector. After weighted summation, it is mapped through a fully connected layer to output the complete morphological features of the grain particles in the overlapping region.

[0058] Compared with the prior art, the beneficial effects of the present invention are:

[0059] This visual detection method and system for grain particle morphology features uses an array of at least two polarization-adjustable cameras to perform stereo matching of polarization channel images with the same polarization direction in the overlapping area to obtain a preliminary depth map. Simultaneously, it calculates the surface normal gradient field pixel by pixel based on the polarization angle image. Then, it uses the Poisson reconstruction method to fuse multiple preliminary depth maps with the surface normal gradient field and solve the Poisson equation to reconstruct a high-resolution depth map, thereby extracting the three-dimensional contour and volume features of grain particles. This not only allows the acquisition of three-dimensional geometric parameters such as fullness, ventral groove depth, and volume, which are not obtainable by traditional two-dimensional contour methods, but also enables the simultaneous detection of surface and subsurface microstructural features, significantly improving the ability to finely identify imperfect particles and the accuracy of quality assessment.

[0060] Simultaneously, a polarization classification method is employed. By adjusting the voltage applied to the liquid crystal polarization element, all polarization-adjustable cameras are synchronously switched to multiple preset polarization directions and a non-polarized state, acquiring polarization channel images and non-polarized images in different polarization directions. Based on these images, polarization degree images and polarization angle images are calculated pixel by pixel. The polarization degree images are used to distinguish between specular reflection-dominated regions and diffuse reflection-dominated regions. In specular reflection regions, the germ region and wax layer region are identified by the polarization degree value. In diffuse reflection regions, the loose internal tissue region is identified by the local coefficient of variation of the polarization degree value. At the same time, the surface depression texture is extracted by the continuous change trajectory of the polarization angle gradient, and the drying crack morphology is extracted by the connection line of gradient abrupt change points. This method directly obtains information on the material distribution and structural integrity of the surface and interior of grain particles, providing a microscopic discrimination basis for grain quality grading that cannot be achieved by traditional visual inspection.

[0061] Based on the camera's field of view coverage, the detection area is divided into independent and overlapping regions. For grain particles in independent regions, morphological features are extracted directly from single-view polarized channel images and unpolarized images and output quickly. For grain particles in overlapping regions, stereo matching and polarization 3D reconstruction are performed using multi-view information from multiple adjacent cameras to obtain high-precision fusion results of 3D morphology and 2D microscopic features. The size of independent regions can be flexibly adjusted according to actual detection needs. When only basic appearance parameters are needed, the independent regions are enlarged to improve efficiency. When fine 3D parameters (such as fullness and germ integrity) are needed, the independent regions are shrunk or even set to zero, so that all particles enter the overlapping region for high-precision reconstruction. This partitioned adaptive processing strategy effectively balances computational resources and processing speed while ensuring detection accuracy. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of the detection method of the present invention;

[0063] Figure 2 This is a schematic diagram illustrating the fusion of overlapping regions in this invention;

[0064] Figure 3 This is a schematic diagram illustrating the region division of the present invention;

[0065] Figure 4 This is a schematic diagram illustrating the acquisition of basic particle morphological characteristics according to the present invention;

[0066] Figure 5 This is a schematic diagram of the overlapping and independent regions of the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] like Figure 1 - Figure 5 As shown, the present invention provides a technical solution: a visual detection method for the morphological characteristics of grain particles, the detection method comprising:

[0069] Set up at least two polarization-adjustable cameras arranged in an array and aligned with the detection area. Control each polarization-adjustable camera to acquire at least two polarization channel images with different polarization directions and at least one unpolarized image through a polarization classification method. The polarization directions in the polarization channel images acquired by all polarization-adjustable cameras are consistent.

[0070] Within the detection area, establish an independent region covered by the field of view of a single polarization-tunable camera and an overlapping region covered by the fields of view of at least two adjacent polarization-tunable cameras.

[0071] For grain particles within the field of view of a single camera, the polarization degree image and polarization angle image are calculated pixel by pixel based on polarization channel images with different polarization directions. Basic particle morphological features are obtained through feature extraction methods, and the basic particle morphological features are divided into morphological features of independent regions and morphological features of overlapping regions through region cropping methods.

[0072] The particle morphology features of the overlapping area are fused: a first preliminary depth map and a second preliminary depth map are obtained by stereo matching, and the surface normal gradient field is calculated pixel by pixel based on the polarization angle image;

[0073] The first preliminary depth map, the second preliminary depth map, and the surface normal gradient field are fused using the Poisson reconstruction method, and the Poisson equation is solved to reconstruct a high-resolution depth map.

[0074] The morphological features of the germ region, wax layer region, loose internal tissue region, surface depression pattern, and drying crack morphology of several overlapping regions are used as two-dimensional feature maps and projected onto a high-resolution depth map. At the projected position, the feature values ​​of each two-dimensional feature map are vector-concatenated with the three-dimensional contour features and volume features. The weight of each feature channel is calculated through an attention weight matrix for the concatenated feature vectors. After weighted summation, the vectors are mapped through a fully connected layer to output the complete morphological features of the grain particles in the overlapping region.

[0075] Polarization classification methods include:

[0076] Provide supplemental lighting for grain grains;

[0077] By adjusting the voltage applied to the liquid crystal polarization element in the polarization-tunable camera, the polarization-tunable camera can be switched sequentially to multiple preset polarization directions and a non-polarization state.

[0078] Synchronously adjust the liquid crystal polarization elements of all polarization-adjustable cameras to the same preset polarization direction, and synchronously trigger all polarization-adjustable cameras to acquire images, thereby obtaining polarization channel images of all polarization-adjustable cameras under the same preset polarization direction;

[0079] The process of synchronously switching and acquiring all preset polarization directions is repeated in sequence, and a single polarization-adjustable camera can obtain polarization channel images with at least two different polarization directions.

[0080] Synchronously adjust the liquid crystal polarization elements of all polarization-adjustable cameras to a non-polarized state, and simultaneously trigger all polarization-adjustable cameras to acquire images, thereby obtaining non-polarized images.

[0081] Feature extraction methods include: calculating the degree of polarization and the angle of polarization within the outer contour pixel by pixel based on the gray values ​​of the same pixel position in polarization channel images with at least two different polarization directions;

[0082] Adaptive threshold segmentation is performed on each polarization channel image and the unpolarized image using polarization degree images to obtain the external contour of grain particles;

[0083] Within the outer contour, polarization degree images are used to distinguish between regions dominated by specular reflection and regions dominated by diffuse reflection.

[0084] In the region dominated by specular reflection, the germ region and the waxy layer region are identified by the polarization degree value. In the region dominated by diffuse reflection, the region with loose internal tissue is identified by the local coefficient of variation of the polarization degree value.

[0085] The polarization angle gradient direction and gradient magnitude of each pixel within the outer contour are calculated using polarization angle images. The direction and depth of surface depression textures are extracted by the continuous change trajectory of the polarization angle gradient, and the morphology of drying cracks is extracted by connecting the abrupt change points of the polarization angle gradient.

[0086] The basic particle morphology characteristics are obtained by combining polarization degree image, polarization angle image, external contour, germ region, wax layer region, internal loose tissue region and drying crack morphology.

[0087] Distinguishing between specular reflection-dominant and diffuse reflection-dominant regions using polarization degree images includes:

[0088] Histogram statistics were performed on the polarization degree image to obtain the distribution curve of polarization degree values ​​in the range of 0 to 1;

[0089] By detecting the bimodal characteristics in the distribution curve, the region corresponding to the first peak with a lower degree of polarization is identified as the region dominated by diffuse reflection, and the region corresponding to the second peak with a higher degree of polarization is identified as the region dominated by specular reflection.

[0090] Using the polarization degree value corresponding to the valley between two peaks as the segmentation threshold, the polarization degree image is binarized to obtain a specular reflection region mask and a diffuse reflection region mask. Based on the specular reflection region mask and the diffuse reflection region mask, the specular reflection dominant region and the diffuse reflection dominant region are distinguished.

[0091] In regions dominated by specular reflection, the germ region can be identified by the degree of polarization, including:

[0092] Within the mask of the specular reflection region, connected regions with polarization values ​​lower than a first threshold are marked as candidate germ regions; the first threshold is 0.3 to 0.5 times the average polarization value in the dominant specular reflection region.

[0093] Morphological operations are performed on the germ candidate regions to remove isolated noise points; the area, roundness, and centroid offset relative to the grain outline of each germ candidate region are calculated.

[0094] Candidate regions whose area falls within the preset embryo area range, whose roundness is greater than 0.6, and whose center of gravity offset is within the range of the embryo's physiological position are identified as embryo regions.

[0095] The presence, area, and relative position of the germ region are used as morphological characteristics of the germ region.

[0096] Region cropping methods include:

[0097] Based on the field of view boundary of each polarization-tunable camera, the spatial boundary between independent and overlapping regions is determined, and independent region masks and overlapping region masks are generated.

[0098] Obtain the outer contour of grain particles from the basic particle morphology features, and compare the pixel coordinates within the outer contour with the independent region mask and the overlapping region mask pixel by pixel.

[0099] If all pixels within the outer contour are located within the independent region mask, then the basic particle morphological features are marked as the morphological features of the independent region as a whole.

[0100] If all pixels within the outer contour are located within the overlapping region mask, then the basic particle morphological features are marked as the morphological features of the overlapping region as a whole.

[0101] If pixels exist in both the independent region mask and the overlapping region mask, then the number of pixels in the two regions is compared: when the number of pixels in the independent region mask is greater than or equal to the number of pixels in the overlapping region mask, the basic particle morphology features are classified as the morphology features of the independent region as a whole; when the number of pixels in the independent region mask is less than the number of pixels in the overlapping region mask, the basic particle morphology features are classified as the morphology features of the overlapping region as a whole.

[0102] A first preliminary depth map is obtained by stereo matching at least two polarization channel images with the same polarization direction. This process is repeated with the polarization direction to obtain at least two first preliminary depth maps. The number of first preliminary depth maps is the same as the number of polarization channel images of a single polarization-adjustable camera. A second preliminary depth map is obtained by stereo matching at least two unpolarized images.

[0103] Stereo matching methods include:

[0104] The intrinsic parameter matrix, rotation matrix, and translation vector of the polarization-tunable camera array are pre-calibrated, and the epipolar constraint equation is established.

[0105] In two polarization channel images with the same polarization direction, the pixel matching search range is restricted to the neighborhood of the corresponding epipolar line based on the epipolar constraint equation, and polarization consistency constraint is introduced.

[0106] Retain matching point pairs in two images where the difference in polarization degree of corresponding matching points is less than a preset consistency threshold and the difference in polarization angle is less than a preset angle threshold;

[0107] A semi-global matching algorithm is used to calculate the disparity map for the selected matching point pairs. The disparity map is then converted into a preliminary depth map by combining the camera baseline distance and focal length parameters.

[0108] The same semi-global matching algorithm and disparity transformation are used on at least two unpolarized images to obtain a second preliminary depth map.

[0109] The initial depth map is fused with the surface normal gradient field using the Poisson reconstruction method, including:

[0110] Based on the gradient consistency of the first and second preliminary depth maps in the grain particle edge region, the confidence weights of the depth maps are calculated respectively.

[0111] The initial depth map is weighted and averaged using confidence weights to obtain the fused depth map.

[0112] By using the surface normal gradient field as the gradient constraint term and the fused depth map as the boundary condition, the Poisson equation is constructed and solved to obtain a high-resolution depth map.

[0113] A visual inspection system for grain particle morphology features, the system comprising:

[0114] Polarization image acquisition module: Set up at least two polarization-adjustable cameras arranged in an array and aligned with the detection area. Control each polarization-adjustable camera to acquire at least two polarization channel images with different polarization directions and at least one unpolarized image through a polarization classification method. The polarization directions in the polarization channel images acquired by all polarization-adjustable cameras are consistent.

[0115] Region segmentation module: Within the detection area, establish an independent region covered by the field of view of a single polarization-tunable camera and an overlapping region covered by the field of view of at least two adjacent polarization-tunable cameras;

[0116] Feature extraction module: For grain particles within the field of view of a single camera, the polarization degree image and polarization angle image are calculated pixel by pixel based on polarization channel images with different polarization directions. Basic particle morphological features are obtained through feature extraction methods. The basic particle morphological features are divided into morphological features of independent regions and morphological features of overlapping regions through region cropping methods.

[0117] Calculation module: Obtains preliminary depth maps No. 1 and No. 2 through stereo matching method, and calculates the surface normal gradient field pixel by pixel based on polarization angle image;

[0118] Depth map reconstruction module: The module fuses the first preliminary depth map, the second preliminary depth map, and the surface normal gradient field using the Poisson reconstruction method, and solves the Poisson equation to reconstruct a high-resolution depth map.

[0119] Feature mapping output module: The morphological features of the germ region, wax layer region, loose internal tissue region, surface depression pattern direction, and drying crack morphology of several overlapping regions are taken as two-dimensional feature maps and projected onto a high-resolution depth map. At the projected position, the feature values ​​of each two-dimensional feature map are vector-concatenated with the three-dimensional contour features and volume features. The weight of each feature channel is calculated by the attention weight matrix for the concatenated feature vector. After weighted summation, it is mapped through a fully connected layer to output the complete morphological features of the grain particles in the overlapping region.

[0120] like Figure 1 and Figure 2 As shown, at least two polarization-tunable cameras are arrayed in the detection area and supplemented with illumination. Each camera acquires multiple polarization channel images with different polarization directions and one unpolarized image using a polarization classification method. For any specific polarization direction, the polarization images acquired by all cameras in that direction are consistent with each other. The detection area is divided into independent regions and overlapping regions based on the camera's field of view coverage. When a grain particle is located in an independent region, feature sets are directly extracted from the corresponding polarization channel images and the unpolarized image and fused to obtain morphological features. When the particle is located in an overlapping region, multiple first feature sets are first obtained, and then the polarization is calculated based on the images with different polarization directions. Vibration map and polarization angle map are obtained. Two preliminary depth maps are obtained by stereo matching of polarized channel images and unpolarized images with the same polarization direction. The surface normal gradient field is calculated using the polarization angle map. Then, the two depth maps and the normal gradient field are fused and a high-resolution depth map is reconstructed using the Poisson equation to extract three-dimensional contour and volume features to form a second feature set. Finally, the first feature set and the second feature set are fused across the feature sets to output the morphological features of particles in the overlapping area. The three-dimensional contour features and volume features extracted based on the high-resolution depth map are regarded as the second feature set, and the part of the basic particle morphological features corresponding to the overlapping area is regarded as the first feature set.

[0121] When grain grains are located in overlapping areas, the advantage of multiple cameras simultaneously capturing the same grain is utilized to reconstruct three-dimensional information and fuse it with two-dimensional polarization features. Each camera covering the grain independently extracts a set of features (e.g., contour, surface texture, reflectivity, etc.) from its own polarization channel image and unpolarized image. The number of feature sets obtained depends on the number of cameras capturing the grain, and is called the "first feature set." Next, based on at least two images with different polarization directions acquired by the same camera, the degree of polarization image and the angle of polarization image are calculated pixel by pixel. The degree of polarization image can distinguish between specular reflection areas (such as the waxy layer) and diffuse reflection areas on the grain surface. For example, in chalky textures, polarization angle images can extract the direction of surface depressions and crack morphology by observing changes in the polarization angle. For two images with the same polarization direction (e.g., both cameras capturing images with a 0° polarization direction), stereo matching (i.e., calculating the distance based on the difference in the position of the same object in the two images) yields a preliminary depth map (map 1). The same stereo matching is performed on two unpolarized images to obtain a second preliminary depth map. Using the previously calculated polarization angle images, the orientation (i.e., the normal gradient field) of each small region on the particle surface can be calculated pixel-by-pixel. This is equivalent to a kind of fine three-dimensional information that does not rely on binocular parallax. The two preliminary depth maps are then compared with… The surface normal gradient fields are fused, and a high-resolution depth map is reconstructed by solving the Poisson equation. This eliminates perspective distortion at camera edges and obtains accurate depth values ​​for each pixel. From this high-resolution depth map, the three-dimensional contour and volume features of grain particles can be extracted to form a second feature set. The first feature set (two-dimensional polarization features, texture features, etc.) is fused with the second feature set (three-dimensional geometric features) to comprehensively determine the complete morphological features of grain particles in the overlapping area, including germ position, ventral groove depth, chalkiness, and size. At least two polarization-tunable phase arrays are used to achieve this. The machine performs stereo matching on polarization channel images with the same polarization direction within the overlapping region to obtain a preliminary depth map. Simultaneously, it calculates the surface normal gradient field pixel by pixel based on the polarization angle image. Then, it uses the Poisson reconstruction method to fuse multiple preliminary depth maps with the surface normal gradient field and solve the Poisson equation to reconstruct a high-resolution depth map. This allows for the extraction of three-dimensional contour and volume features of grain particles. This not only enables the acquisition of three-dimensional geometric parameters such as fullness, ventral groove depth, and volume, which are not obtainable by traditional two-dimensional contour methods, but also allows for the simultaneous detection of surface and subsurface microstructural features, significantly improving the ability to finely identify imperfect grains and the accuracy of quality assessment.

[0122] This application arranges at least two liquid crystal adjustable polarization cameras in an array, with each camera covering only a portion of the transport plane. The field of view of each camera can be controlled within a range with minimal distortion (e.g., field of view less than 30°), thereby significantly reducing geometric distortion at the edges of a single image. The array arrangement allows for overlapping areas between the fields of view of adjacent cameras. Within these overlapping areas, depth information can be obtained using dual-view stereo matching, thereby reconstructing the three-dimensional contours and volumetric features of the particles. This compensates for the inability of a single camera to acquire depth information. Furthermore, the array arrangement can linearly expand the detection width by increasing the number of cameras, adapting to transport planes of different specifications and exhibiting good scalability.

[0123] At least two polarization-tunable cameras are arrayed in the detection area and supplemental lighting is provided. The supplemental lighting uses a single unpolarized white light source to illuminate the grain particles in the detection area from one or more angles to avoid the influence of natural light fluctuations on the consistency of polarization information. Each camera acquires multiple polarization channel images in different polarization directions and one unpolarized image using a polarization classification method. For any given polarization direction, the polarization images acquired by all cameras in that direction are consistent with each other. Based on the camera's field of view coverage, the detection area is divided into independent regions and overlapping regions: independent regions (which may even be zero, i.e., all regions are overlapping regions) are located within the field of view of a single camera. Near the center, imaging distortion is minimal. Therefore, when grain particles are in this region, feature extraction and fusion of single-view polarization channel images and unpolarized images acquired by the camera can quickly output accurate morphological features, simplifying the analysis process. However, the overlapping area is located at the edge of the camera's field of view, where significant barrel distortion or perspective distortion exists. A single image can cause particle contour distortion. Therefore, when grain particles are in this region, complex processing using multi-view images simultaneously captured by multiple adjacent cameras is necessary. First, multiple initial feature sets are obtained. Then, polarization degree maps and polarization angle maps are calculated based on images with different polarization directions, and the feature sets are merged. Simultaneously, images with the same polarization... Two preliminary depth maps are obtained by stereo matching of polarized channel images and unpolarized images. The surface normal gradient field is calculated using the polarization angle map. Then, the two depth maps and the normal gradient field are fused using the Poisson equation to reconstruct a high-resolution depth map, extracting 3D contour and volume features to form a second feature set. Finally, the first and second feature sets are fused across feature sets to output the morphological features of particles in the overlapping region. A polarization classification method is used, and by adjusting the voltage applied to the liquid crystal polarization element, all polarization-adjustable cameras are synchronously switched to multiple preset polarization directions and unpolarized states to acquire polarization channel images and unpolarized images in different polarization directions. The method uses polarization images to calculate polarization degree images and polarization angle images pixel by pixel. The polarization degree images are used to distinguish between specular reflection-dominated regions and diffuse reflection-dominated regions. In specular reflection regions, the germ region and waxy layer region are identified by the polarization degree value. In diffuse reflection regions, the loose internal tissue region is identified by the local coefficient of variation of the polarization degree value. At the same time, the surface depression texture is extracted by the continuous change trajectory of the polarization angle gradient, and the drying crack morphology is extracted by the connecting line of gradient abrupt change points. This method directly obtains the material distribution and structural integrity information of the surface and interior of grain particles, providing a microscopic discrimination basis for grain quality grading that cannot be achieved by traditional visual inspection.

[0124] like Figure 5As shown, the fields of view of adjacent cameras overlap in the detection area, and this overlapping position is called the overlapping region. Meanwhile, independent regions are also limited by the size of the detection area. Regions whose fields of view exceed the detection area size are not classified as overlapping or independent regions. The size of the independent region is not fixed but can be flexibly adjusted according to the actual detection needs of the visual detection method for grain particle morphology. Different grain detection scenarios have different requirements for morphological features. When only basic appearance indicators such as particle size, color, integrity, and surface defects need to be determined, high-precision three-dimensional information is not required. In this case, the independent region can be set larger, and the results can be quickly output using single-view imaging to meet the needs. However, when it is necessary to obtain the fullness of the grain, the depth of the ventral groove, etc., the independent region can be larger. When determining fine morphological parameters such as germ integrity and surface cracks, it is necessary to rely on multi-view stereo matching and polarization 3D reconstruction of overlapping areas to obtain high-precision 3D morphological information. Therefore, it is necessary to reduce the size of independent regions or even set them to zero so that the entire detection area is covered by overlapping regions. This ensures that each grain can obtain the required fine features through complex processing. The size of the independent region depends on the extraction requirements of morphological feature parameters in actual applications. The more the requirements are biased towards fine 3D parameters (such as fullness, ventral groove depth, and germ integrity), the smaller or even non-existent the independent region will be; the more the requirements are biased towards basic appearance parameters, the larger the independent region can be. This configurability allows this method to flexibly balance processing speed and feature accuracy according to different detection levels and accuracy requirements.

[0125] The optical characteristics of a camera lens determine that distortion is small near the center of the field of view and large at the edges. While complex multi-view 3D reconstruction is accurate for all particles, it is computationally intensive and inefficient. Conversely, simple single-view analysis for all particles leads to distortion and inaccurate measurements in edge regions due to distortion. This method differentiates based on the actual location of the particles: for independent regions with negligible distortion, simple and fast single-view analysis meets basic appearance inspection needs; for overlapping edge regions with significant distortion, complex and precise multi-view polarization 3D reconstruction is used to obtain fine morphological parameters. The size of independent regions can be dynamically adjusted (even set to zero) according to actual inspection needs, flexibly balancing accuracy and efficiency. This partitioned processing strategy ensures both overall inspection speed and accurate measurement of particles in edge regions. For detection accuracy, the detection area is divided into independent and overlapping regions based on the camera's field of view coverage. For grain particles in independent regions, morphological features are extracted directly from single-view polarized channel images and unpolarized images and output quickly. For grain particles in overlapping regions, stereo matching and polarization 3D reconstruction are performed using multi-view information from multiple adjacent cameras to obtain high-precision fusion results of 3D morphology and 2D microscopic features. The size of independent regions can be flexibly adjusted according to actual detection needs. When only basic appearance parameters are needed, the independent regions are enlarged to improve efficiency. When fine 3D parameters (such as fullness and germ integrity) are needed, the independent regions are shrunk or even set to zero, so that all particles enter the overlapping region for high-precision reconstruction. This partitioned adaptive processing strategy effectively balances computing resources and processing speed while ensuring detection accuracy.

[0126] Several polarization directions (e.g., two or more from 0°, 45°, and 90°) are pre-set according to the detection requirements. For each preset polarization direction, when all arrayed polarization-adjustable cameras acquire images, the liquid crystal adjustable polarization elements mounted in front of the camera lenses are uniformly adjusted to the same polarization transmission direction. If the preset polarization direction is 0°, the liquid crystal adjustable polarization element of each polarization-adjustable camera is set to a strict 0° transmission axis; if the preset polarization direction is 45°, the liquid crystal adjustable polarization elements of all cameras are uniformly set to a 45° transmission axis. There is no angular deviation between different cameras, and the global shutter is synchronously triggered to ensure that all cameras acquire images of the same preset polarization direction at the same time, thereby avoiding inconsistencies in polarization information caused by changes in illumination or particle movement. The central controller sends the same voltage control signal to the liquid crystal adjustable polarization elements of all cameras, so that the polarization channel images of the same preset polarization direction captured by different cameras are identical.

[0127] For a single grain within the field of view of a camera, the images acquired in four polarization directions are used. , The image representing a deflection direction of 0° is calculated pixel-by-pixel for the degree of polarization (DoP) and the angle of polarization (AoP) based on the Stokes vector method:

[0128] ;

[0129] ;

[0130] Where (x, y) are pixel coordinates. DoP represents the brightness of the current pixel coordinate. The value range of DoP is [0,1], and the value range of AoP is [0°,180°].

[0131] Using the degree of polarization (DoP) image for unpolarized images Adaptive threshold segmentation is performed because grain particles and the background of the detection area typically have significant differences in DoP (particle surface exhibits a mixture of specular and diffuse reflection, while the background is either low-polarization or pure noise). The Otsu algorithm is used to automatically calculate the segmentation threshold. A binary mask is obtained. Its outline is the outer outline of the grain. The Otsu algorithm determines the image binarization segmentation threshold without being affected by image brightness and contrast. It is used in the image segmentation process to automatically calculate an optimal global threshold.

[0132] The polarization degree of grain images (DoP) were statistically analyzed using a 256-level grayscale histogram. Since the grain surface simultaneously contains a smooth, almost mirror-like region (high DoP) and a rough, diffuse reflection region (low DoP), the histogram usually exhibits a distinct bimodal distribution.

[0133] The histogram is fitted using a Gaussian mixture model (GMM) to automatically detect the locations of the two peaks. (Low DoP, corresponding to diffuse peak) and (High DoP, corresponding to specular reflection peak), calculate the valley position between the two peaks. As a segmentation threshold, a binary mask is generated:

[0134] ;

[0135] Obtain a mask for the specular reflection area With diffuse reflection region mask ;

[0136] exist Within this range, the DoP value will be lower than the threshold. =0.4× ( Connected regions (where DoP is the mean of the specular reflection area) are marked as germ candidate regions. After performing a 3×3 pixel morphological opening operation on the candidate regions to remove isolated noise points, the area A, roundness C=4πA / P² (P is the perimeter), and centroid offset Δr relative to the centroid of the particle outline are calculated for each candidate region. Morphological opening is a basic morphological operation in image processing, consisting of two steps: erosion and dilation. It is usually used to remove small noise points in images, break narrow connections, and smooth object boundaries while keeping the overall shape and area of ​​the object basically unchanged.

[0137] Taking rice as an example, the preset range of embryo area is [50, 500] pixels, the roundness threshold is 0.6, and the range of embryo physiological position is the bottom 1 / 3 of the grain outline (near the base of the caryopsis). Candidate regions that meet all the conditions are determined as embryo regions, and their existence, area and relative position are recorded as embryo region morphological features.

[0138] The waxy layer region is characterized by a DoP value higher than =0.8× Identify highly polarized connected regions and record their distribution masks;

[0139] exist Within each pixel neighborhood (e.g., a 7×7 window), the local coefficient of variation (CV) of the DoP value is calculated as CV = σ / μ. σ represents the standard deviation, reflecting the dispersion (fluctuation) of a set of data. The larger σ is, the more dispersed the data points are. μ represents the mean, reflecting the central tendency or average level of a set of data. σ / μ together are called the coefficient of variation, which is a dimensionless relative index used to measure the relative fluctuation of data. Regions with CV > 0.35 are marked as regions with loose internal structure. This is because loose structure causes light to be scattered multiple times inside the particles, resulting in a high degree of non-uniformity in local polarization.

[0140] Using the angle-of-polarization (AoP) image, calculate the gradient of each pixel within the contour:

[0141] ;

[0142] Gradient direction θ= gradient magnitude :

[0143] For surface depression textures, the AoP gradient exhibits a continuous and smooth change trajectory. The direction of the depression texture is extracted by tracing continuous pixel chains with gradient direction consistency > 0.85. The depression depth is estimated based on the gradient magnitude integral.

[0144] For dry cracks, the abrupt change in refractive index at the crack edge causes an abrupt change in AoP, forming a local maximum (abrupt point) of the gradient amplitude. Connecting adjacent abrupt points (distance < 5 pixels and direction difference < 30°) forms a crack skeleton line, and its length, width and bifurcation shape are recorded as dry crack morphological features.

[0145] A checkerboard calibration plate was used to jointly calibrate all polarization-tunable cameras in the array, and the intrinsic parameter matrix of each camera was obtained. Rotation matrix and translation vector For adjacent cameras i and j, calculate the fundamental matrix. and essential matrix Establish polar constraint equations;

[0146] ,in , These are the homogeneous coordinates of the corresponding pixel.

[0147] Taking the 0° polarization direction as an example, take the polarization channel images acquired by camera i and camera j in this direction. ,for Points to be matched in Based on the polar constraint, it is placed in The matching search range is limited to the neighborhood of the corresponding epipolar line (epidial line distance < 3 pixels).

[0148] Introducing polarization consistency constraints: for candidate matching point pairs ( , ), calculate its DoP difference Difference from AoP Only retain those that meet the requirements. <0.08 and Matching point pairs <12°;

[0149] For the filtered matching point pairs, a semi-global matching algorithm (SGM) is used to calculate the disparity map. The process of calculating the disparity map using the semi-global matching algorithm (SGM) is as follows: First, for the matching point pairs filtered by epipolar constraints and polarization consistency constraints, the matching cost under different disparities is calculated pixel by pixel in the epipolar neighborhood. One-dimensional path cost aggregation is performed along multiple directions (such as left, right, up, down and the four diagonals). The disparity continuity and depth edge jump are balanced by a penalty term. For each pixel, the disparity value with the minimum total aggregation cost is selected as its disparity. Finally, after bidirectional consistency verification and sub-pixel interpolation optimization, the disparity map d of the entire image is output. The SGM parameter settings are: penalty term P1=10, P2=150, number of paths 8. Combined with the camera baseline distance B=150mm and focal length f=16mm, the disparity map d is converted into a depth map.

[0150] ;

[0151] Where ε is a minimum value to prevent division by zero, this depth map is the first preliminary depth map. The above process is repeated for the polarization directions of 0°, 45°, 90° and 135° to obtain the first preliminary depth map.

[0152] The same SGM process was used on the unpolarized image (but without introducing polarization consistency constraints) to obtain the second preliminary depth map.

[0153] For the first preliminary depth map (taking four maps as an example) and the second preliminary depth map, firstly calculate the gradient magnitude consistency in the grain particle edge region of each depth map (detect the contour by the Canny operator), and define the edge gradient consistency score;

[0154] ;

[0155] in This is the k-th preliminary depth map. The gradient mean at the edge is the value of multiple depth maps, and λ = 2.0 is the attenuation coefficient. Normalization yields the confidence weights ,satisfy =1;

[0156] The fused depth map is obtained by weighting the initial depth maps using confidence scores. ;

[0157] ;

[0158] Using the surface normal gradient field (p, q) as the gradient constraint term, As boundary conditions, construct the Poisson equation;

[0159] ;

[0160] The above equations are solved within the particle mask region using the sparse conjugate gradient method to obtain a smooth and detail-preserving high-resolution depth map. This depth map simultaneously integrates the geometric consistency of stereo matching with the surface micro-normal information of polarized photometric stereo.

[0161] Based on the field of view boundary of each polarization-tunable camera and the width of the overlapping area between adjacent cameras, two binary mask images are pre-generated in the detection plane coordinate system. The first image is an independent region mask, where the value of each pixel indicates whether the location is covered by the field of view of only a single camera (i.e., it does not overlap with the field of view of other cameras). The second image is an overlapping region mask, where the value of each pixel indicates whether the location is covered by the field of view of at least two adjacent cameras at the same time. The two masks are mutually exclusive, meaning that any pixel on the detection plane belongs to at most one of the masks.

[0162] like Figure 3 As shown, after obtaining the binary image of the outer contour of the grain, the coordinates of all pixels within the contour are compared pixel by pixel with the independent region mask and the overlapping region mask respectively. The number of pixels within the contour that fall in the independent region mask is counted and recorded as the first quantity; the number of pixels within the contour that fall in the overlapping region mask is counted and recorded as the second quantity.

[0163] Based on the statistical results, the following attribution determinations are made: If the second quantity is zero, meaning all pixels are located within the independent region mask, then the basic particle morphological features of the current particle are marked as the morphological features of the independent region as a whole; if the first quantity is zero, meaning all pixels are located within the overlapping region mask, then the basic particle morphological features are marked as the morphological features of the overlapping region as a whole; if neither the first nor the second quantity is zero, then their magnitudes are compared: when the first quantity is greater than or equal to the second quantity, the basic particle morphological features are classified as the morphological features of the independent region as a whole; conversely, when the first quantity is less than the second quantity, they are classified as the morphological features of the overlapping region as a whole.

[0164] like Figure 4 As shown, the independent region mask corresponds to the set of pixels covered by only a single camera's field of view, while the overlapping region mask corresponds to the set of pixels covered by at least two adjacent camera's field of view. After obtaining the outer contour of the grain particle through feature extraction, the coordinates of all pixels within the contour are compared pixel by pixel with the two masks. The number of pixels falling within the independent region and the number of pixels falling within the overlapping region are counted. Based on the statistical results, the "majority pixel principle" is used for classification: if all pixels within the contour are located in the independent region, the basic particle morphology features of the particle are marked as the morphology features of the independent region; if all pixels are located in the overlapping region, the particle is marked as the morphology features of the overlapping region; if pixels exist in both types of regions (i.e., the particle crosses the region boundary), the number of pixels is compared. When the number of pixels in the independent region is greater than or equal to the number of pixels in the overlapping region, the particle is classified as an independent region; otherwise, it is classified as an overlapping region. This region clipping method avoids splitting the same particle into multiple parts for separate processing, ensuring the integrity of the particle morphology features and the simplicity of the processing flow, while providing a clear decision basis for subsequent adaptive detection of partitions.

[0165] The essence of projection alignment is to use a high-resolution depth map as a "spatial anchor point" to backproject two-dimensional semantic features from the image plane to a three-dimensional surface, achieving strict registration of the two types of features on the particle surface geometry. For each effective pixel in the two-dimensional feature map (i.e., the pixel located within the grain particle mask), its image coordinates and corresponding high-resolution depth value are known. Combined with the camera intrinsic parameter matrix, its three-dimensional spatial coordinates can be calculated through perspective backprojection. Two-dimensional semantic feature values ​​(such as germ markers, wax layer probabilities, etc.) are assigned three-dimensional coordinates along with the pixel, thus completing the transformation from "image plane label" to "spatial surface attribute". This process ensures that if a pixel in the two-dimensional feature is marked as "crack", then the mark is precisely attached to the corresponding spatial position on the three-dimensional surface, rather than floating in the abstract feature space.

[0166] Building upon projection alignment, two types of feature values ​​exist simultaneously at the same spatial location: material / defect attributes from the 2D semantic feature map and shape / scale attributes from the 3D geometric analysis. These two are concatenated along the channel dimension to form a cross-modal feature vector at that location. However, the concatenated feature tensor suffers from large differences in numerical scale between channels and uneven discriminative importance. An attention mechanism, through a learnable weight matrix, dynamically calculates the contribution of each feature channel to the final representation. This attention mechanism simulates selective attention in human vision, automatically suppressing irrelevant or redundant information and enhancing information relevant to the current task when faced with multi-source information. This is achieved through adaptive weighting of feature channels, using global average pooling (GAP) and global max pooling (GMP) on the feature tensor F to capture... The overall response level and extreme response level of each channel are evaluated. GAP reflects the "average activity" of the channel, and GMP reflects the "maximum discriminative power" of the channel. The outputs of GAP and GMP are concatenated and input into a two-layer fully connected network. The first layer realizes cross-channel information interaction (dimensionality reduction), and the second layer generates normalized channel weight vectors. This process ensures that the weight of each channel depends not only on its own response but also on the modulation of the responses of other channels. If a channel is highly correlated with another channel (such as "crack semantics" and "crack depth"), the attention network can reduce the weight of one of them to avoid redundancy. The generated weight vector is multiplied with the original feature tensor channel by channel to obtain the attention-enhanced feature tensor. The feature values ​​of high-weight channels are amplified, while low-weight channels are suppressed, realizing adaptive fusion of "on-demand enhancement".

[0167] The attention-weighted feature tensor has a high dimension (number of channels × number of spatial pixels), and the spatial structure information has been transformed into channel-level statistics through attention pooling. The role of the fully connected layer is to compress this high-dimensional sparse tensor into a low-dimensional, dense, and semantically rich morphological feature vector. Cross-modal feature fusion is achieved through four progressive layers: "projection alignment to solve spatial misalignment, vector concatenation to achieve information aggregation, attention weighting to eliminate dimensional differences, and fully connected mapping to compress semantic space".

[0168] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended embodiments and their equivalents.

Claims

1. A visual detection method for the morphological characteristics of grain particles, characterized in that, The detection method includes: Set up at least two polarization-adjustable cameras arranged in an array and aligned with the detection area. Control each polarization-adjustable camera to acquire at least two polarization channel images with different polarization directions and at least one unpolarized image through a polarization classification method. The polarization directions in the polarization channel images acquired by all polarization-adjustable cameras are consistent. Within the detection area, establish an independent region covered by the field of view of a single polarization-tunable camera and an overlapping region covered by the fields of view of at least two adjacent polarization-tunable cameras. For grain particles within the field of view of a single camera, the polarization degree image and polarization angle image are calculated pixel by pixel based on polarization channel images with different polarization directions. Basic particle morphological features are obtained through feature extraction methods, and the basic particle morphological features are divided into morphological features of independent regions and morphological features of overlapping regions through region cropping methods. The particle morphology features of the overlapping area are fused: a first preliminary depth map and a second preliminary depth map are obtained by stereo matching, and the surface normal gradient field is calculated pixel by pixel based on the polarization angle image; The first preliminary depth map, the second preliminary depth map, and the surface normal gradient field are fused using the Poisson reconstruction method, and the Poisson equation is solved to reconstruct a high-resolution depth map. The morphological features of the germ region, wax layer region, loose internal tissue region, surface depression pattern, and drying crack morphology of several overlapping regions are used as two-dimensional feature maps and projected onto a high-resolution depth map. At the projected position, the feature values ​​of each two-dimensional feature map are vector-concatenated with the three-dimensional contour features and volume features. The weight of each feature channel is calculated through an attention weight matrix for the concatenated feature vectors. After weighted summation, the vectors are mapped through a fully connected layer to output the complete morphological features of the grain particles in the overlapping region.

2. The visual detection method for grain particle morphology features according to claim 1, characterized in that: The polarization classification method includes: Provide supplemental lighting for grain grains; By adjusting the voltage applied to the liquid crystal polarization element in the polarization-tunable camera, the polarization-tunable camera can be switched sequentially to multiple preset polarization directions and a non-polarization state. Synchronously adjust the liquid crystal polarization elements of all polarization-adjustable cameras to the same preset polarization direction, and synchronously trigger all polarization-adjustable cameras to acquire images, thereby obtaining polarization channel images of all polarization-adjustable cameras under the same preset polarization direction; The process of synchronously switching and acquiring all preset polarization directions is repeated in sequence, and a single polarization-adjustable camera can obtain polarization channel images with at least two different polarization directions. Synchronously adjust the liquid crystal polarization elements of all polarization-adjustable cameras to a non-polarized state, and simultaneously trigger all polarization-adjustable cameras to acquire images, thereby obtaining non-polarized images.

3. The visual detection method for grain particle morphology features according to claim 1, characterized in that: The feature extraction method includes: calculating the polarization degree image and polarization angle image of the outer contour pixel by pixel based on the gray value of the same pixel position in polarization channel images with at least two different polarization directions; Adaptive threshold segmentation is performed on each polarization channel image and the unpolarized image using polarization degree images to obtain the external contour of grain particles; Within the outer contour, polarization degree images are used to distinguish between regions dominated by specular reflection and regions dominated by diffuse reflection. In the region dominated by specular reflection, the germ region and the waxy layer region are identified by the polarization degree value. In the region dominated by diffuse reflection, the region with loose internal tissue is identified by the local coefficient of variation of the polarization degree value. The polarization angle gradient direction and gradient magnitude of each pixel within the outer contour are calculated using polarization angle images. The direction and depth of surface depression textures are extracted by the continuous change trajectory of the polarization angle gradient, and the morphology of drying cracks is extracted by connecting the abrupt change points of the polarization angle gradient. The basic particle morphology characteristics are obtained by combining polarization degree image, polarization angle image, external contour, germ region, wax layer region, internal loose tissue region and drying crack morphology.

4. The visual detection method for grain particle morphology features according to claim 3, characterized in that: Distinguishing between specular reflection-dominant and diffuse reflection-dominant regions using polarization degree images includes: Histogram statistics were performed on the polarization degree image to obtain the distribution curve of polarization degree values ​​in the range of 0 to 1; The bimodal characteristics in the distribution curve are detected, and the region corresponding to the first peak with a lower polarization degree value is determined as the diffuse reflection-dominant region, while the region corresponding to the second peak with a higher polarization degree value is determined as the specular reflection-dominant region. Using the polarization degree value corresponding to the valley between two peaks as the segmentation threshold, the polarization degree image is binarized to obtain a specular reflection region mask and a diffuse reflection region mask. Based on the specular reflection region mask and the diffuse reflection region mask, the specular reflection dominant region and the diffuse reflection dominant region are distinguished.

5. The visual detection method for grain particle morphology features according to claim 3, characterized in that: In regions dominated by specular reflection, the germ region can be identified by the degree of polarization, including: Within the specular reflection region mask, connected regions with polarization values ​​below a first threshold are marked as candidate germ regions; the first threshold is 0.3 to 0.5 times the average polarization value in the dominant specular reflection region. Morphological operations are performed on the germ candidate regions to remove isolated noise points; the area, roundness, and centroid offset relative to the grain particle outline of each germ candidate region are calculated. Candidate regions whose area falls within the preset embryo area range, whose roundness is greater than 0.6, and whose center of gravity offset is within the range of the embryo's physiological position are identified as embryo regions. The presence, area, and relative position of the germ region are used as the morphological characteristics of the germ region.

6. The visual detection method for grain particle morphology features according to claim 1, characterized in that: The region cropping method includes: Based on the field of view boundary of each polarization-tunable camera, the spatial boundary between independent and overlapping regions is determined, and independent region masks and overlapping region masks are generated. Obtain the outer contour of grain particles from the basic particle morphology features, and compare the pixel coordinates within the outer contour with the independent region mask and the overlapping region mask pixel by pixel. If all pixels within the outer contour are located within the independent region mask, then the basic particle morphological features are marked as the morphological features of the independent region as a whole. If all pixels within the outer contour are located within the overlapping region mask, then the basic particle morphological features are marked as the morphological features of the overlapping region as a whole. If pixels exist in both the independent region mask and the overlapping region mask, then the number of pixels in the two regions is compared: when the number of pixels in the independent region mask is greater than or equal to the number of pixels in the overlapping region mask, the basic particle morphology features are classified as the morphology features of the independent region as a whole; when the number of pixels in the independent region mask is less than the number of pixels in the overlapping region mask, the basic particle morphology features are classified as the morphology features of the overlapping region as a whole.

7. The visual detection method for grain particle morphology features according to claim 1, characterized in that: A first preliminary depth map is obtained by stereo matching at least two polarization channel images with the same polarization direction. This process is repeated with the polarization direction to obtain at least two first preliminary depth maps. The number of first preliminary depth maps is the same as the number of polarization channel images of a single polarization-adjustable camera. A second preliminary depth map is obtained by stereo matching at least two unpolarized images. The stereo matching method includes: The intrinsic parameter matrix, rotation matrix, and translation vector of the polarization-tunable camera array are pre-calibrated, and the epipolar constraint equation is established. In two polarization channel images with the same polarization direction, the pixel matching search range is limited to the corresponding epipolar neighborhood based on the epipolar constraint equation, and polarization consistency constraints are introduced. Retain matching point pairs in two images where the difference in polarization degree of corresponding matching points is less than a preset consistency threshold and the difference in polarization angle is less than a preset angle threshold; A semi-global matching algorithm is used to calculate the disparity map for the filtered matching point pairs. The disparity map is then converted into a preliminary depth map by combining the camera baseline distance and focal length parameters. The same semi-global matching algorithm and disparity transformation are used on at least two unpolarized images to obtain a second preliminary depth map.

8. The visual detection method for grain particle morphology features according to claim 1, characterized in that: The process of fusing the preliminary depth map with the surface normal gradient field using the Poisson reconstruction method includes: Based on the gradient consistency of the first and second preliminary depth maps in the grain particle edge region, the confidence weights of the depth maps are calculated respectively. The initial depth map is weighted and averaged using the confidence weights to obtain the fused depth map. Using the surface normal gradient field as a gradient constraint term and the fused depth map as a boundary condition, the Poisson equation is constructed and solved to obtain the high-resolution depth map.

9. A visual inspection system for the morphological characteristics of grain particles, characterized in that: The detection system includes: Polarization image acquisition module: Set up at least two polarization-adjustable cameras arranged in an array and aligned with the detection area. Control each polarization-adjustable camera to acquire at least two polarization channel images with different polarization directions and at least one unpolarized image through a polarization classification method. The polarization directions in the polarization channel images acquired by all polarization-adjustable cameras are consistent. Region segmentation module: Within the detection area, establish an independent region covered by the field of view of a single polarization-tunable camera and an overlapping region covered by the field of view of at least two adjacent polarization-tunable cameras; Feature extraction module: For grain particles within the field of view of a single camera, the polarization degree image and polarization angle image are calculated pixel by pixel based on polarization channel images with different polarization directions. Basic particle morphological features are obtained through feature extraction methods. The basic particle morphological features are divided into morphological features of independent regions and morphological features of overlapping regions through region cropping methods. Calculation module: Obtains preliminary depth maps No. 1 and No. 2 through stereo matching method, and calculates the surface normal gradient field pixel by pixel based on polarization angle image; Depth map reconstruction module: The module fuses the first preliminary depth map, the second preliminary depth map, and the surface normal gradient field using the Poisson reconstruction method, and solves the Poisson equation to reconstruct a high-resolution depth map. Feature mapping output module: The morphological features of the germ region, wax layer region, loose internal tissue region, surface depression pattern direction, and drying crack morphology of several overlapping regions are taken as two-dimensional feature maps and projected onto a high-resolution depth map. At the projected position, the feature values ​​of each two-dimensional feature map are vector-concatenated with the three-dimensional contour features and volume features. The weight of each feature channel is calculated by the attention weight matrix for the concatenated feature vector. After weighted summation, it is mapped through a fully connected layer to output the complete morphological features of the grain particles in the overlapping region.

Images