Method and device for extracting rice region of interest, and electronic equipment

Abstract

The application provides a rice region of interest extraction method and device and electronic equipment, the method is applied to computer vision technical field, the method comprises the following steps: preprocessing the acquired hyperspectral image of rice to obtain a reference hyperspectral image; obtain a structure constraint mask, a sparse adjustment mask and a random mask; fuse the structure constraint mask, the sparse adjustment mask and the random mask to obtain a target mask, and use the target mask to mask process the reference hyperspectral image to obtain a target hyperspectral image; extract the target feature of the target hyperspectral image; process the target feature, reconstruct the hyperspectral image, and output the boundary probability graph of the region of interest; determine the region of interest mask of the rice according to the boundary probability graph, and extract the region of interest from the hyperspectral image. The method can combine the structure constraint mask with the random mask, avoid manual intervention, suppress background noise, and improve the accuracy of region of interest extraction.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more specifically, to a method, apparatus, and electronic device for extracting regions of interest in rice grains. Background Technology

[0002] Rice is an important food crop, and to verify its food safety, it is necessary to classify rice by year. Traditional year classification relies on human sensory evaluation (color, odor) or chemical testing (fatty acid value, malondialdehyde content), which has drawbacks such as being time-consuming (single sample testing requires more than 48 hours), costly, and destructive (chemical methods require crushing samples), making it difficult to meet the modern grain storage requirements of "fast, non-destructive, and large-scale".

[0003] In related technologies, hyperspectral images are used to capture the spatial morphology and spectral features of rice. However, hyperspectral image data is high-dimensional (typically 200-1000 bands) and highly redundant (correlation between adjacent bands > 0.9), which can easily lead to the "curse of dimensionality" when directly inputting it into a classification model. In the process of analyzing rice grains using hyperspectral images, it is crucial to accurately locate the core regions of the rice (such as the endosperm and aleurone layer), eliminate background interference (rice husks, broken rice, impurities), and extract the region of interest (ROI) from the hyperspectral image. The quality of this step directly affects the subsequent classification performance. In practical applications, the extraction of ROI suffers from the following problems: reliance on manual interaction: traditional threshold segmentation requires manual parameter adjustment and has poor generalization; insufficient feature fusion: existing spectral-spatial joint screening methods often use step-by-step processing (first spectral band selection, then spatial clustering), ignoring the collaborative discriminative information between spectral and spatial methods; and ambiguous key region localization: no specific constraints are designed for the anatomical structure of rice (endosperm proportion > 80%, aleurone layer located outside the endosperm), which can easily lead to misclassification of non-core regions (such as rice husk fragments) as ROIs. Summary of the Invention

[0004] This application provides a method, apparatus, and electronic device for extracting regions of interest (ROIs) in rice grains. This method can improve the accuracy of ROI extraction by combining structural constraint masks with random masks, avoiding manual intervention, suppressing background noise, and improving the accuracy of ROI extraction.

[0005] Firstly, a method for extracting regions of interest in rice grains is provided, the method comprising:

[0006] The initial hyperspectral image of the acquired rice was subjected to spectral preprocessing and spatial preprocessing to obtain a reference hyperspectral image. The spectral preprocessing was used to suppress noise and correct spectral scattering effects, while the spatial preprocessing was used to enhance the morphology and structure of the rice grains and suppress background interference.

[0007] Based on the prior anatomical structure of the rice, a structural constraint mask is generated, and based on the structural constraint mask, a sparse adjustment mask is generated to control the sparsity of the structural constraint mask region. The structural constraint mask region is the region obtained by masking the reference hyperspectral image based on the structural constraint mask.

[0008] In the reference hyperspectral image, the pixel regions other than the structure-constrained mask region are defined as unstructured regions; a random mask is generated based on these unstructured regions.

[0009] The structural constraint mask, the sparse adjustment mask, and the random mask are fused together to obtain the target mask. The target mask is then used to mask the reference hyperspectral image to obtain the target hyperspectral image.

[0010] The spectral and spatial features of the target hyperspectral image are extracted by an encoder, and feature fusion is performed to obtain the target features.

[0011] The target features are processed using a decoder with a dual-branch parallel structure for feature reconstruction and boundary segmentation. The hyperspectral image is reconstructed through the feature reconstruction branch, while the boundary segmentation branch outputs the boundary probability map of the region of interest. The two branches are jointly optimized using a weighted loss function.

[0012] The boundary probability map is used to determine the mask of interest for the rice grain, and the region of interest is extracted from the initial hyperspectral image.

[0013] In conjunction with the first aspect, in some possible implementations, the reference hyperspectral image includes denoised spectral features and denoised spatial features. The initial hyperspectral image of the acquired rice is subjected to spectral preprocessing and spatial preprocessing to obtain the reference hyperspectral image, including:

[0014] Obtain the initial hyperspectral image of the rice grain;

[0015] Wavelet thresholding is applied to the initial hyperspectral image to obtain the denoised hyperspectral image.

[0016] Multivariate scattering correction is performed on the denoised hyperspectral image to obtain the denoised spectral features in the reference hyperspectral image;

[0017] Morphological filtering and background suppression are performed on the initial hyperspectral image to obtain the denoised spatial features in the reference hyperspectral image.

[0018] In conjunction with the first aspect, in some possible implementations, the prior knowledge of the anatomical structure of rice is used to generate a structural constraint mask, including:

[0019] The endosperm center of the rice grain is obtained by weighting the pixel coordinates of the rice grain in the reference hyperspectral image with a first weight value; the first weight value is determined by the spectral purity in the reference hyperspectral image.

[0020] A circular mask template is generated with the endosperm center as the center and a preset radius and pixel coordinates of the rice grain as the center.

[0021] Calculate the edge gradient magnitude of the pixel for the rice grain;

[0022] A pixel region representing a predetermined proportion of the edge gradient magnitude is used as a candidate region mask for the aleurone layer of the rice.

[0023] Perform an XOR operation between the circular mask template and the candidate region mask to obtain the structural constraint mask.

[0024] In conjunction with the first aspect, in some possible implementations, the generation of a sparse adjustment mask based on the structural constraint mask for controlling the sparsity of the structural constraint mask region includes:

[0025] The sparsity of the structure constraint mask region is controlled by the first preset parameter to obtain the sparsity adjustment mask;

[0026] The random mask generated based on this unstructured region includes:

[0027] The unstructured region is masked with a first preset probability to obtain a random mask.

[0028] In conjunction with the first aspect, in some possible implementations, the structural constraint mask, the sparse adjustment mask, and the random mask are fused to obtain the target mask, including:

[0029] The structural constraint mask is fused with the sparse adjustment mask to obtain the reference mask;

[0030] The target mask is obtained by XORing the reference mask with the random mask.

[0031] In conjunction with the first aspect, in some possible implementations, the extraction of spectral and spatial features of the target hyperspectral image by the encoder, and the subsequent feature fusion, includes:

[0032] The spectral dimension of the hyperspectral image of the target is reduced by using a first preset convolution kernel, and the spectral features of the hyperspectral image of the target are extracted.

[0033] The spatial features of the target hyperspectral image are extracted using a second preset convolution kernel, which is larger than the first preset convolution kernel;

[0034] The spectral feature is then reconstructed to a size that matches the spatial feature using bilinear interpolation.

[0035] The spectral feature, whose size matches that of the spatial feature, is concatenated with the spatial feature and then max-pooled using a third preset convolution kernel to obtain the target feature. The third preset convolution kernel is smaller than the second preset convolution kernel.

[0036] In conjunction with the first aspect, in some possible implementations, the target feature is processed using a decoder with a dual-branch parallel structure for feature reconstruction and boundary segmentation. The hyperspectral image is reconstructed through the feature reconstruction branch, while the boundary segmentation branch outputs a boundary probability map of the region of interest. This includes:

[0037] By upsampling layer by layer through transposed convolution, the features of the target are restored to obtain a reconstructed hyperspectral image;

[0038] Calculate the mean square error loss between the reconstructed hyperspectral image and the target hyperspectral image;

[0039] The channel weights of the decoder are calculated based on an attention mechanism;

[0040] The target hyperspectral map is weighted and convolved based on the channel weights to obtain the boundary probability map.

[0041] In conjunction with the first aspect, in some possible implementations, determining the interest mask of the rice grain based on the boundary probability map and extracting the region of interest from the initial hyperspectral image includes:

[0042] The boundary probability map is then thresholded and binarized to obtain the mask of interest.

[0043] The region of interest is obtained by multiplying the initial hyperspectral image with the mask of interest pixel by pixel.

[0044] Secondly, a device for extracting regions of interest in rice grains is provided, the device comprising:

[0045] The preprocessing module is used to perform spectral preprocessing and spatial preprocessing on the acquired initial hyperspectral image of rice to obtain a reference hyperspectral image; wherein, the spectral preprocessing is used to suppress noise and correct spectral scattering effects; the spatial preprocessing is used to enhance the morphological structure of rice grains and suppress background interference.

[0046] The generation module is used to generate a structural constraint mask based on the prior anatomical structure of the rice, and to generate a sparse adjustment mask based on the structural constraint mask to control the sparsity of the structural constraint mask region. The structural constraint mask region is the region obtained by masking the reference hyperspectral image based on the structural constraint mask.

[0047] The definition and generation module is also used to define the pixel regions in the reference hyperspectral image other than the structure-constrained mask region as unstructured regions; and to generate random masks based on the unstructured regions.

[0048] The fusion and processing module is used to fuse the structural constraint mask, the sparse adjustment mask and the random mask to obtain the target mask, and use the target mask to perform masking processing on the reference hyperspectral image to obtain the target hyperspectral image;

[0049] The fusion module is used to extract the spectral and spatial features of the target hyperspectral image through the encoder, and perform feature fusion to obtain the target features;

[0050] The processing and optimization module is used to process the target features using a decoder with a dual-branch parallel structure of feature reconstruction and boundary segmentation. It reconstructs the hyperspectral image through the feature reconstruction branch and outputs the boundary probability map of the region of interest through the boundary segmentation branch. The two branches are jointly optimized using a weighted loss function.

[0051] The determination and extraction module is used to determine the mask of interest for the rice grain based on the boundary probability map and extract the region of interest from the initial hyperspectral image.

[0052] Thirdly, an electronic device is provided, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, causing the electronic device to perform the method described above for extracting the region of interest of rice.

[0053] Fourthly, a computer program product is provided, comprising: computer program code, which, when run on a computer, causes the computer to execute the method described above for extracting the region of interest of rice.

[0054] Fifthly, a computer-readable storage medium is provided that stores computer program code, which, when run on a computer, causes the computer to perform the method described above for extracting the region of interest of rice.

[0055] This application provides a method for extracting regions of interest (ROIs) from rice grains. This method performs spectral and spatial preprocessing on an initial hyperspectral image of the acquired rice grain to obtain a reference hyperspectral image. A structural constraint mask is generated based on the prior anatomical structure of the rice grain. Based on this structural constraint mask, a sparse adjustment mask is generated to control the sparsity of the structural constraint mask region. Pixel regions in the reference hyperspectral image other than the structural constraint mask region are defined as unstructured regions. A random mask is generated based on these unstructured regions. The structural constraint mask, the sparse adjustment mask, and the random mask are fused to obtain a target mask. This target mask is then used to mask the reference hyperspectral image to obtain the target hyperspectral image. In other words, the target mask not only incorporates random perturbation but also the structural constraint mask of the rice grain, avoiding manual intervention while suppressing background noise. Therefore, the background noise of the target hyperspectral image processed based on this target mask is suppressed. The encoder extracts spectral and spatial features from the hyperspectral image of the target and fuses these features to obtain the target features. A decoder with a dual-branch parallel structure for feature reconstruction and boundary segmentation processes these features. The feature reconstruction branch reconstructs the hyperspectral image, while the boundary segmentation branch outputs a boundary probability map of the region of interest (ROI). A weighted loss function is used to jointly optimize the two branches. Based on this boundary probability map, the ROI mask of the rice grain is determined, and the ROI is extracted from the initial hyperspectral image. In other words, by extracting a higher purity ROI mask from the target hyperspectral image and using this mask to extract the ROI from the initial hyperspectral image, the accuracy of the ROI extraction is improved. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the implementation environment for a method for extracting regions of interest in rice provided in an embodiment of this application;

[0057] Figure 2 This is a schematic flowchart illustrating a method for extracting regions of interest in rice provided in an embodiment of this application;

[0058] Figure 3 This is a schematic diagram of the structure of a method and apparatus for extracting regions of interest in rice provided in an embodiment of this application;

[0059] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0060] The technical solutions in this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0061] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0062] Before introducing the methods of the embodiments of this application, the technical terms that may be involved in the embodiments of this application will be explained first.

[0063] Hyperspectral images (HSI) are a type of image containing rich spectral information. Hyperspectral images of rice can record the reflectance or radiance intensity of each pixel across multiple consecutive spectral bands. For example, a single pixel in a hyperspectral image contains tens to hundreds of consecutive spectral bands. In practical applications, hyperspectral imaging technology can simultaneously capture the spatial morphology (such as grain shape and surface cracks) and spectral characteristics (such as moisture absorption peaks in the near-infrared region and protein / starch characteristics in the short-wave infrared region) of rice, providing richer discriminative information for determining the age and classification of rice. However, in practical applications, rice from different years exhibits significant differences in spectral curves in the 1000-2500 nm band due to variations in starch aging and lipid oxidation during storage (e.g., the absorbance of 3-year-old rice at 1720 nm is 15%-20% higher than that of new rice).

[0064] Spectral preprocessing is the process of removing random noise from a hyperspectral image before analysis.

[0065] Spectral scattering is the phenomenon where the direction of light propagation changes after light shines on the surface of an object or enters the interior of an object.

[0066] Morphological structure is used to characterize the physical structure of rice grains. For example, morphological structure includes information such as the shape, size, and surface smoothness of rice grains.

[0067] Anatomical priors are a priori knowledge of the internal structure of rice. These anatomical priors include a priori knowledge of the location of the endosperm and the location of the aleurone layer in rice.

[0068] Spectral features are used to characterize the chemical composition of rice corresponding to pixels in hyperspectral images, because different chemical components have unique absorption and reflection characteristics for different wavelengths of light.

[0069] Spatial features are used to characterize the shape, texture, outline size, and relationship with the surrounding environment of pixels in hyperspectral images of rice.

[0070] In the following description of the embodiments of this application, it is used as... Figure 1 Taking an example, the implementation environment of the embodiments of this application will be introduced.

[0071] For example, such as Figure 1 As shown, the implementation environment includes server 110 and client 120.

[0072] Server 110 is a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms. Server 110 includes a network communication unit, processor, and memory, etc. Specifically, server 110 is used to analyze the hyperspectral images acquired by client 120 to extract regions of interest from the hyperspectral images.

[0073] Client 120 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc. Client 120 is a physical device with image acquisition and basic image processing capabilities. Specifically, client 120 can acquire hyperspectral images using hyperspectral imaging technology. Client 120 and server 110 are connected directly or indirectly via wired or wireless communication.

[0074] Figure 2 This is a schematic flowchart illustrating a method for extracting regions of interest in rice provided in an embodiment of this application.

[0075] For example, such as Figure 2 As shown, taking the server as the execution subject as an example, this application describes a method for extracting the region of interest of rice grains. The method 200 includes the following steps.

[0076] Step 201: Perform spectral preprocessing and spatial preprocessing on the acquired initial hyperspectral image of rice to obtain a reference hyperspectral image; wherein, the spectral preprocessing is used to suppress noise and correct spectral scattering effects; the spatial preprocessing is used to enhance the morphological structure of rice grains and suppress background interference.

[0077] It should be understood that in practical applications, electronic noise and ambient light interference in sensors can mask spectral features in hyperspectral images, thereby reducing the accuracy of extracting regions of interest from rice grains. Differences in the size, shape, and surface roughness (morphological structure) of rice grains cause varying degrees of light scattering. This results in translation and rotation of the spectral baseline (multiplicative / additive effects), causing rice grains with the same chemical composition to have different spectra due to morphological differences (different spectra for the same substance), or rice grains with different compositions to have similar spectra due to scattering effects (similar spectra for different substances). Therefore, spectral preprocessing is used to suppress noise and correct for spectral scattering effects.

[0078] In practical applications, the initial hyperspectral image inevitably contains background elements (such as conveyor belts, sample trays, black velvet cloth, etc.) in addition to rice grains. The spectrum of the background is completely different from that of the rice grains. If these background pixels are not removed, they will act as "outliers" or "misclassified points," severely interfering with the training and prediction of the model. Therefore, spatial preprocessing is used to enhance the morphological structure of rice grains and suppress background interference.

[0079] The spectral preprocessing is used to suppress noise and correct spectral scattering effects; the spatial preprocessing is used to enhance the morphology and structure of rice grains and suppress background interference.

[0080] In one possible implementation, an initial hyperspectral image of the rice is acquired; wavelet thresholding is applied to the initial hyperspectral image to obtain a denoised hyperspectral image; multivariate scattering correction is performed on the denoised hyperspectral image to obtain the denoised spectral features in the reference hyperspectral image; morphological filtering and background suppression are performed on the initial hyperspectral image to obtain the denoised spatial features in the reference hyperspectral image.

[0081] It should be understood that in practical applications, the initial hyperspectral images of rice may contain uneven lighting, noise, and a lot of background interference. Therefore, it is necessary to perform spectral preprocessing and spatial processing on the initial hyperspectral images of rice to eliminate the interference in the initial hyperspectral images.

[0082] Wavelet thresholding denoising is used to eliminate random noise in the initial hyperspectral image. Multivariate scattering correction is used to eliminate physical scattering effects. Morphological filtering and background suppression are used to remove irrelevant regions such as background shadows and extract the rice grain regions. The reference hyperspectral image includes denoised spectral features and denoised spatial features.

[0083] In this implementation, since the initial hyperspectral image may contain noise and background interference, noise in the spectral features of the initial hyperspectral image is removed based on wavelet thresholding, and noise in the spatial features of the initial hyperspectral image is removed based on morphological filtering and background suppression, thereby eliminating interference in the initial hyperspectral image.

[0084] To provide a more detailed explanation of the above embodiments, the following description is divided into several parts.

[0085] The first part describes the content of obtaining the initial hyperspectral image of the rice.

[0086] In some embodiments, an initial hyperspectral image of rice is acquired using a hyperspectral imaging camera.

[0087] The second part describes the content of the denoised hyperspectral image obtained by performing wavelet threshold denoising on the initial hyperspectral image.

[0088] It should be understood that the initial hyperspectral image contains a large amount of spatial and spectral information. However, instrument thermal noise and scattering noise can interfere with this information, leading to inaccurate subsequent spectral analysis. Therefore, it is necessary to process the initial hyperspectral image by using wavelet thresholding.

[0089] In some embodiments, wavelet thresholding is used to perform three-level discrete wavelet decomposition on each band of the spectral information in the initial hyperspectral image to obtain high-frequency coefficients. A soft thresholding function is then applied to these high-frequency coefficients to obtain the denoised hyperspectral image.

[0090] The expression for the soft threshold function is as follows.

[0091]

[0092] in, Let N be the noise standard deviation, and N be the number of pixels in the initial hyperspectral image.

[0093] The third part describes the denoised hyperspectral image obtained by performing multivariate scattering correction on the denoised hyperspectral image and then explaining the denoised spectral features in the reference hyperspectral image.

[0094] It should be understood that when light shines on granular, non-uniform samples like rice, photons are absorbed by the chemical components within the sample (such as water, protein, and starch). These photons interact with the surface and internal structure of the sample particles, altering their propagation direction. In other words, the initial hyperspectral image exhibits scattering effects, causing the spectral curve to shift vertically and its slope to change, resulting in significant interference in the uncorrected spectral curve. In this case, multivariate scattering correction is performed on the denoised hyperspectral image to obtain the final spectral curve.

[0095] In one possible implementation, the scattering differences caused by rice grain size and surface roughness are eliminated based on a multi-far scattering correction formula.

[0096] in, For the spectral curve, For fitting parameters, The spectral mean and standard deviation of standard rice samples were used. calculate.

[0097] In some embodiments, the spectral mean of all bands in the initial hyperspectral image is calculated, and the standard deviation of all bands in the initial hyperspectral image is calculated.

[0098] Part Four describes the morphological filtering and background suppression performed on the initial hyperspectral image to obtain the denoising spatial features of the reference hyperspectral image.

[0099] In some embodiments, a 3×3 structuring element is used to perform a closing operation on the initial hyperspectral image. That is, a dilation-erosion morphological filter is first performed on the initial hyperspectral image to fill the microcracks inside the endosperm of the rice grains and connect the broken edges of the rice grains, thereby reducing noise in the initial hyperspectral image.

[0100] In some embodiments, based on the morphological watershed algorithm, combined with the convex hull of rice grains as the segmentation criterion, adhering rice grains are effectively separated to obtain the denoised spatial features in the reference hyperspectral image of a single rice grain.

[0101] Step 202: Based on the prior anatomical structure of the rice, a structural constraint mask is generated, and based on the structural constraint mask, a sparse adjustment mask is generated to control the sparsity of the structural constraint mask region. The structural constraint mask region is the region obtained by masking the reference hyperspectral image based on the structural constraint mask.

[0102] The anatomical priors include endosperm location priors and aleurone layer location priors. The structural constraint mask region is the overlapping area between the endosperm center and the aleurone layer edge.

[0103] It should be understood that in related technologies, traditional masked autoencoder (MAE) randomized pixel masking does not take into account the structural priors of rice (such as the endosperm being concentrated in the center of the grain and the aleurone layer surrounding the endosperm), resulting in inaccurate extraction of the region of interest. In this case, the focus on the structurally constrained masked region in the rice grain is enhanced by the anatomical priors of the rice grain, that is, the focus is on the key regions of the endosperm center and the aleurone layer.

[0104] In this framework, anatomical priors are used to determine the internal structure of rice grains. For example, the interior of a rice grain includes the endosperm center and the aleurone layer. A structural constraint mask is used to more accurately locate the aleurone layer and endosperm center of the rice grain; the structural constraint mask region is the area obtained by masking the reference hyperspectral image based on this structural constraint mask. A sparsity adjustment mask is used to adjust the sparsity of the structural constraint mask region. In some embodiments, the sparsity adjustment mask enables the structural constraint mask region to more accurately fit an image of real rice grains.

[0105] It should be understood that rice grains contain a wealth of chemical information. If the average spectrum of the entire rice grain is taken, the local and key chemical information within the rice grain will be "diluted" and "mixed", making it impossible to accurately reflect changes in specific qualities (such as lipid rancidity and starch gelatinization). Therefore, structural constraint masks can accurately define regions such as the embryo and endosperm on a reference hyperspectral image.

[0106] Understandably, based on prior knowledge of anatomical structure, it can be determined that the endosperm accounts for 80%-90% of the rice grain area and is distributed in an approximately circular shape at the center of the grain. The center of the endosperm can be determined based on the morphological centroid.

[0107] In one possible implementation, the pixel coordinates of rice grains in the reference hyperspectral image are weighted and averaged with a first weight value to obtain the endosperm center of the rice grain; a circular mask template is generated with the endosperm center as the center and a preset radius and the pixel coordinates of the rice grain; the edge gradient magnitude of the pixels of the rice grain is calculated; a preset proportion of the pixel region in the edge gradient magnitude is used as the candidate region mask of the aleurone layer of the rice grain; the circular mask template and the candidate region mask are XORed to obtain the structural constraint mask.

[0108] The first weight value is determined by the spectral purity in the reference hyperspectral image. The endosperm center is a high-density aggregation area of ​​starch granules and proteins in rice, and is also the main site for storing nutrients in rice. Spectral purity is the similarity between the endosperm spectrum in the reference hyperspectral image and the standard spectrum; that is, the higher the spectral purity, the closer the spectrum of that pixel is to the standard spectrum. The aleurone layer is a special cell layer on the outer layer of the grain seed, located between the seed coat and the endosperm core. The aleurone layer is approximately 50-100 μm thick and is located on the outer layer of the endosperm. There is a clear boundary between the aleurone layer and the endosperm center, meaning that the spectral gradient changes drastically in the reference hyperspectral image.

[0109] In this implementation, since the endosperm and aleurone layer contain abundant chemical components, they are important regions for analyzing the year classification of rice. Therefore, the structural constraint mask of rice is calculated in the target mask based on the prior anatomical structure of rice to improve the learning efficiency and recognition accuracy of identifying the region of interest of rice.

[0110] To provide a more detailed explanation of the above embodiments, the following description is divided into several parts.

[0111] The first part explains how the pixel coordinates of rice grains in the reference hyperspectral image are weighted and averaged with a first weight value to obtain the contents of the endosperm center of the rice grain.

[0112] It should be understood that the different chemical compositions and physical structures of different parts of rice result in different spectral responses (i.e., "first weighting values") in different bands of hyperspectral images. The endosperm, as the main component, exhibits the most significant spectral characteristics. By utilizing this weighting, it is possible to determine the "centroid" representing the endosperm, rather than simply its geometric center.

[0113] In some embodiments, the spectral reflectance intensity of each pixel of a rice grain in a reference hyperspectral image is used as a first weight.

[0114] In some embodiments, the x-coordinate and y-coordinate of each pixel are weighted and averaged with the corresponding first weight to obtain the endosperm center of the rice.

[0115] The second part explains how to generate a circular mask template with the endosperm center as the center, a preset radius, and the pixel coordinates of the rice grain.

[0116] The preset radius is a radius automatically determined by the server. This application embodiment does not limit this, for example, the preset radius is 0.4.

[0117] In some embodiments, the diagonal length of the rice grain is determined based on the pixel coordinates of the rice grain; a reference radius is obtained by multiplying a preset radius by the diagonal length; a circular mask template is generated with the endosperm center as the center and the reference radius as the radius, wherein the central region of the circular mask template is 1 and the outer region is 0.

[0118] The third part explains how to calculate the edge gradient magnitude of the pixels of the rice grain.

[0119] It should be understood that the spectral gradient of the aleurone layer changes drastically in the reference hyperspectral image, so it is necessary to calculate the edge gradient magnitude of the pixels of the rice grain.

[0120] In some embodiments, the gradient magnitude at the particle edge is calculated based on the Sobel operator.

[0121] Part Four explains the content of using a preset proportion of pixel regions in the edge gradient magnitude as candidate region masks for the aleurone layer of the rice.

[0122] It should be understood that, since the spectral gradient of the aleurone layer changes drastically, determining the pixel region with a preset proportion of the edge gradient amplitude as the candidate region mask of the aleurone layer has high accuracy.

[0123] The preset ratio is 10%, which means that the top 10% of pixels in the edge gradient magnitude are used as candidate region masks for the atomizing layer.

[0124] Part Five explains how to XOR the circular mask template with the candidate region mask to obtain the content of the structural constraint mask.

[0125] Understandably, in order to preserve the overlapping area between the endosperm center and the aleurone layer edge, the circular mask template and the candidate region mask are XORed to obtain the structural constraint mask.

[0126] In some embodiments, the formula for calculating the synthetic structural constraint mask is as follows.

[0127]

[0128] in, For structural constraint masks; For the candidate region mask of the aleurone layer; It is a circular mask template.

[0129] It should be understood that in practical applications, in order for the structural constraint mask to fit the real rice more accurately, it is also necessary to control the sparsity of the structural constraint mask. That is, a small portion of the most representative pixels are selectively and sparsely extracted within the structural constraint mask region. In this case, a sparse adjustment mask is generated based on the structural constraint mask to control the sparsity of the structural constraint mask region.

[0130] In one possible implementation, the sparsity of the structural constraint mask region is controlled by a first preset parameter to obtain the sparsity adjustment mask.

[0131] In some embodiments, a reference hyperspectral image is masked based on a structural constraint mask to obtain a structural constraint mask region. The sparsity of the structural constraint mask region is controlled by a first preset parameter, and the structural constraint mask is matrix binarized to obtain a sparsity adjustment mask.

[0132] The formula for calculating the sparse adjustment mask is as follows.

[0133]

[0134] in, This is a sparse adjustment mask with row i and column j. The first preset parameter is... Used to control how the threshold is generated, according to Determine the proportion of pixels to retain. It can be 0.2, which means that only the top 20% of the pixels with the largest pixel in the structural constraint mask region are retained.

[0135] In this implementation, controlling the sparsity of the structural constraint mask region with a first preset parameter can make the fitted structural constraint mask closer to the chemical composition of real rice grains, thus reducing errors.

[0136] Step 203: Define the pixel regions in the reference hyperspectral image other than the structure-constrained mask region as unstructured regions; generate a random mask based on the unstructured regions.

[0137] It should be understood that the unstructured region is the pixel region in the reference hyperspectral image excluding the structure constraint mask region. The random mask is used to prevent the reference hyperspectral image from overfitting, that is, to avoid the reference hyperspectral image from failing to fit irregularly shaped rice grains.

[0138] In some embodiments, the unstructured region is masked with a first preset probability to obtain a random mask, and the formula for calculating the random mask is as follows.

[0139]

[0140] in, Let i be a random mask with i as the row and j as the column. This is the first preset probability. The first preset probability can be 0.3.

[0141] In this implementation, the unstructured region is masked with a first preset probability to obtain a random mask, thereby introducing random perturbation. This prevents the encoder from overfitting the prior anatomical structure of rice, that is, it prevents the encoder from being unable to simulate the spectral image of irregularly shaped rice.

[0142] Step 204: The structural constraint mask, the sparse adjustment mask, and the random mask are fused to obtain the target mask, and the target mask is used to perform masking processing on the reference hyperspectral image to obtain the target hyperspectral image.

[0143] It should be understood that the related technology of masking the reference hyperspectral image by random masking does not take into account the physical structure of rice. In this case, adding structural constraint mask, sparse adjustment mask and random mask to the target mask makes the target hyperspectral image processed by the target mask closer to the real hyperspectral image of rice.

[0144] In some embodiments, a structural constraint mask and a sparse random mask are fused to obtain a reference mask, and the reference mask is XORed with the random mask to obtain the target mask. The calculation formula for the target mask is as follows.

[0145]

[0146] One method for fusing structural constraint masks and sparse random masks is to perform a bitwise XOR operation on the structural constraint mask and the sparse random mask. For the target mask, For structural constraint mask, For sparse adjustment mask, Use a random mask.

[0147] In this implementation, the target mask is generated based on the structural constraint mask, the sparse adjustment mask, and the random mask. That is, the structural constraint mask and the sparse condition mask are added to the target mask, which enables the encoder to more accurately identify the endosperm and aleurone layer of rice.

[0148] In some embodiments, the target mask is used to mask the reference hyperspectral image to obtain a target hyperspectral image representing the pure endosperm spectral data extracted from the reference hyperspectral image.

[0149] Step 205: Extract the spectral and spatial features of the target hyperspectral image using an encoder, and perform feature fusion to obtain the target features.

[0150] The encoder is used to extract features from the incomplete hyperspectral image of the target after masking; specifically, it extracts the spectral and spatial features of the chemical components of the rice grains. The target features are obtained by fusing the spectral and spatial features.

[0151] It should be understood that the target hyperspectral image carries a lot of spectral and spatial information, so the encoder needs to analyze the target hyperspectral image, that is, extract the spectral and spatial features of the target hyperspectral image.

[0152] In one possible implementation, a first preset convolutional kernel is used to reduce the spectral dimension of the target hyperspectral image and extract the spectral features of the target hyperspectral image; a second preset convolutional kernel is used to extract the spatial features of the target hyperspectral image, the second preset convolutional kernel being larger than the first preset convolutional kernel; the spectral features are restored to a size matching the spatial features through bilinear interpolation; the spectral features matching the size of the spatial features are concatenated with the spatial features and max-pooled by a third preset convolutional kernel to obtain the target features, the third preset convolutional kernel being smaller than the second preset convolutional kernel.

[0153] The first preset convolution kernel, the second preset convolution kernel, and the third preset convolution kernel are convolution kernels automatically set by the server. This application embodiment does not limit them. For example, the first preset convolution kernel can be a 1×1 convolution kernel, the second preset convolution kernel can be a 3×3 convolution kernel, and the third preset convolution kernel can be a 2×2 convolution kernel.

[0154] In this implementation, since the target hyperspectral image contains a large number of bands, direct processing would involve a huge amount of computation. Therefore, dimensionality reduction of the target hyperspectral image based on a first preset convolutional kernel can significantly reduce the pressure of subsequent computations. Fusing spectral features with spatial features facilitates learning from the fused target features, resulting in better learning outcomes.

[0155] To provide a more detailed explanation of the above embodiments, the following description is divided into several parts.

[0156] The first part explains how to reduce the spectral dimension of the hyperspectral image of the target using a first preset convolution kernel and extract the spectral features of the hyperspectral image of the target.

[0157] In some embodiments, a target hyperspectral image is input into an encoder. The first size of the target hyperspectral image is a first height, a first width, and a first number of bands. The first size of the target hyperspectral image is reduced to a second size using a 1×1 convolution kernel to extract the spectral features of the target hyperspectral image.

[0158] For example, the first dimension is H×W×B, and the second dimension is H×W×C1 (C1=B / 4).

[0159] The second part explains the extraction of spatial features from the target hyperspectral image using the second preset convolution kernel.

[0160] In some embodiments, a target hyperspectral image is input into an encoder. The first dimension of the target hyperspectral image is a first height, a first width, and a first number of bands. The first dimension of the target hyperspectral image is reduced to a third dimension by a 3×3 convolution kernel in order to extract the spatial features of the target hyperspectral image.

[0161] For example, the first dimension is H×W×B, and the third dimension is H / 2×W / 2×C2 (C1=C2).

[0162] The third part explains how to recover the spectral feature to a size that matches the spatial feature using bilinear interpolation.

[0163] It should be understood that since spectral features and spatial features must be identical in spatial dimensions to be fused, it is necessary to scale the second dimension of the spectral features to the same size as the third dimension of the spatial features based on bilinear interpolation upsampling.

[0164] In some embodiments, the second dimension of the spectral feature is recovered to the third dimension of the spatial feature based on the bilinear difference.

[0165] For example, the second dimension is restored to H / 2×W / 2×C1.

[0166] The fourth part explains how the spectral feature, whose size matches that of the spatial feature, is concatenated with the spatial feature, and then max-pooled using a third preset convolution kernel to obtain the target feature.

[0167] In some embodiments, the spectral features are concatenated with the spatial features to obtain reference features, the size of which is H / 2×W / 2×2C1. The reference features are then max-pooled using a 2×2 convolution kernel to obtain the target features H / 4×W / 4×2C1.

[0168] Step 206: The target features are processed using a decoder with a dual-branch parallel structure for feature reconstruction and boundary segmentation. The hyperspectral image is reconstructed through the feature reconstruction branch, while the boundary segmentation branch outputs the boundary probability map of the region of interest. The two branches are then jointly optimized using a weighted loss function.

[0169] It should be understood that in practical applications, since the encoder compresses and extracts the spectral and spatial features in the target hyperspectral image, the decoder is also required to reconstruct these spectral and spatial features in order to obtain detailed information on the spectral or spatial features of the region of interest.

[0170] Feature reconstruction involves reconstructing the spectral features from the target hyperspectral image. The decoder is used to reconstruct the spectral spatial details of the region of interest (ROI) in the hyperspectral image. Boundary segmentation is used to determine the boundaries of the ROI in the reconstructed hyperspectral image. The reconstructed hyperspectral image is the spectral image after reprocessing by the encoder. The ROI is the spectral region and image region that reflects the chemical changes in rice. The boundary probability map is used to accurately reconstruct the boundaries of the ROI in the hyperspectral image.

[0171] It should be understood that in order for the decoder to learn richer knowledge, the target features need to be processed after receiving them from the encoder.

[0172] In one possible implementation, the target features are restored by upsampling layer by layer through transposed convolution to obtain a reconstructed hyperspectral image; the mean square error loss between the reconstructed hyperspectral image and the target hyperspectral image is calculated; the channel weights of the decoder are calculated based on the attention mechanism; and the target hyperspectral image is weighted by convolution based on the channel weights to obtain a boundary probability map.

[0173] The transposed convolution uses a 3×3 kernel with a stride of 2. A stride of 2 means that the height and width of the feature map are doubled in each operation, while the number of channels is reduced. The mean squared error loss is the error loss between the reconstructed hyperspectral image and the target hyperspectral image. The boundary probability map is used to accurately reconstruct the boundaries of the region of interest in the hyperspectral image.

[0174] In this implementation, reconstructing a hyperspectral image enables the model to learn accurate spectral and spatial features, and allows the extraction of boundary probability maps of the region of interest from the reconstructed hyperspectral image.

[0175] To provide a more detailed explanation of the above embodiments, the following description is divided into several parts.

[0176] The first part explains how the reconstructed hyperspectral image is obtained by upsampling the target features layer by layer through transposed convolution and then restoring them.

[0177] In some embodiments, target features are input into a transposed convolution to output a reconstructed hyperspectral image.

[0178]

[0179] Where k is the number of decoding layers (4 layers in total). This represents the number of channels in the k-th layer. That is, the number of channels in the 1st layer is 2. The number of channels in the second layer is The number of channels in the third layer is The fourth layer has 3 channels. This indicates that the convolution kernel is 3×3. This indicates a step size of 2. The final layer outputs the reconstructed hyperspectral image.

[0180] The second part explains the calculation of the mean square error loss between the reconstructed hyperspectral image and the target hyperspectral image.

[0181] In some embodiments, the mean square error loss between the reconstructed hyperspectral image and the target hyperspectral image is calculated based on the mean square error algorithm, wherein the calculation formula is as follows.

[0182]

[0183] in, For the target hyperspectral image, To reconstruct the hyperspectral image, The dimensions of the target hyperspectral image and the reconstructed hyperspectral image, This represents the mean square error loss.

[0184] The third part explains how the channel weights of the decoder are calculated based on the attention mechanism.

[0185] In some embodiments, channel weights are calculated based on the SE Block at the third layer of the decoder.

[0186]

[0187] in, Channel weights; For activation functions; It is a linear rectified function; These are the parameters for the fully connected layer; These are the parameters for the fully connected layer; This is the third layer of transposed convolution.

[0188] Part Four explains the boundary probability map obtained by performing weighted convolution on the hyperspectral image of the target based on the channel weight.

[0189] In some embodiments, the target hyperspectral image is weighted based on the channel weight and convolved with a 1×1 convolution kernel to obtain a boundary probability map, wherein the boundary probability map has a size of H×W×1.

[0190] Optionally, after performing the above steps, the following steps may also be performed.

[0191] In one possible implementation, a similarity loss function is calculated based on the boundary probability map and the true boundary label; a weighted loss function is calculated based on the mean squared error loss function and the similarity loss function; and the weighted loss function is used to jointly optimize the feature reconstruction and boundary segmentation branches.

[0192] It should be understood that in practical applications, since losses may occur in the decoder during the calculation process, it is necessary to jointly optimize the feature reconstruction and boundary segmentation dual-branch structure based on the weighted loss function. That is, the weighted loss function is input into the feature reconstruction and boundary segmentation dual-branch structure.

[0193] In some embodiments, the formula for calculating the similarity (Dice) loss function is as follows.

[0194]

[0195] in, The similarity loss function; For boundary probability plots; For true boundary labels.

[0196] In some embodiments, the weighted loss function is calculated using the following formula.

[0197]

[0198] in, For weighted loss functions; Weights for feature reconstruction branches, for example, It is 0.7; This is the mean square error loss; The weights of the branches that split at the boundaries, for example, It is 0.3; This is the similarity loss function.

[0199] Step 207: Determine the mask of interest for the rice grain based on the boundary probability map, and extract the region of interest from the initial hyperspectral image.

[0200] The region of interest (ROI) for rice is defined as the spectral and image regions that reflect the chemical changes in rice. An ROI mask is used to extract the ROI from the initial hyperspectral image.

[0201] It should be understood that since the interest mask is a mask obtained through multiple processing steps and is extracted from the boundary probability map of the region of interest, the region of interest of rice can be obtained by processing the initial hyperspectral image through the interest mask.

[0202] In one possible implementation, the boundary probability map is thresholded and binarized to obtain the mask of interest; the initial hyperspectral image is then multiplied pixel by pixel with the mask of interest to obtain the region of interest.

[0203] In this context, the threshold in threshold binarization is a threshold automatically determined by the server. This application embodiment does not limit this threshold. For example, the threshold can be 0.5. That is, the boundary probability map output by the decoder is binarized with a threshold > 0.5 to obtain the mask of interest.

[0204] In some embodiments, the initial hyperspectral image is multiplied pixel-by-pixel with the mask of interest to obtain the region of interest.

[0205] In this implementation, since the initial hyperspectral image contains a large amount of complex and useless information, and the interest mask can eliminate irrelevant background information interference to the greatest extent, the truly needed region of interest can be preserved based on the interest mask.

[0206] Optionally, after performing the above steps, the following steps may also be performed.

[0207] It should be understood that in practical applications, after obtaining a relatively accurate region of interest, it is necessary to analyze that region of interest. Therefore, the region of interest can be input into a lightweight classifier for analysis.

[0208] In one possible implementation, the region of interest is fed into a lightweight classifier for year classification.

[0209] The lightweight classifier can be MobileNetV3.

[0210] This application provides a method for extracting regions of interest (ROIs) from rice grains. The method performs spectral and spatial preprocessing on an initial hyperspectral image of the acquired rice grains to obtain a reference hyperspectral image. A structural constraint mask is generated based on the prior anatomical structure of the rice grains. Based on this structural constraint mask, a sparse adjustment mask is generated to control the sparsity of the structural constraint mask region. Pixel regions in the reference hyperspectral image other than the structural constraint mask region are defined as unstructured regions. A random mask is generated based on these unstructured regions. The structural constraint mask, the sparse adjustment mask, and the random mask are fused to obtain a target mask. This target mask is then used to mask the reference hyperspectral image to obtain the target hyperspectral image. In other words, the target mask not only incorporates random perturbation but also the structural constraint mask of the rice grains, avoiding manual intervention while suppressing background noise. Therefore, the background noise of the target hyperspectral image processed based on this target mask is suppressed. The encoder extracts spectral and spatial features from the hyperspectral image of the target and fuses these features to obtain the target features. A decoder with a dual-branch parallel structure for feature reconstruction and boundary segmentation processes these features. The feature reconstruction branch reconstructs the hyperspectral image, while the boundary segmentation branch outputs a boundary probability map of the region of interest (ROI). A weighted loss function is used to jointly optimize the two branches. Based on this boundary probability map, the ROI mask of the rice grain is determined, and the ROI is extracted from the initial hyperspectral image. In other words, by extracting a higher purity ROI mask from the target hyperspectral image and using this mask to extract the ROI from the initial hyperspectral image, the accuracy of the ROI extraction is improved.

[0211] Figure 3 This is a schematic diagram of a device for extracting regions of interest in rice provided in an embodiment of this application.

[0212] For example, the device 300 includes:

[0213] The preprocessing module 301 is used to perform spectral preprocessing and spatial preprocessing on the acquired initial hyperspectral image of rice to obtain a reference hyperspectral image; wherein, the spectral preprocessing is used to suppress noise and correct spectral scattering effects; the spatial preprocessing is used to enhance the morphological structure of rice grains and suppress background interference.

[0214] The generation module 302 is used to generate a structural constraint mask based on the prior anatomical structure of the rice, and to generate a sparse adjustment mask based on the structural constraint mask to control the sparsity of the structural constraint mask region. The structural constraint mask region is the region obtained by masking the reference hyperspectral image based on the structural constraint mask.

[0215] The definition and generation module 303 is further configured to define the pixel regions in the reference hyperspectral image other than the structure-constrained mask region as unstructured regions; and generate random masks based on the unstructured regions.

[0216] The fusion and processing module 304 is used to fuse the structural constraint mask, the sparse adjustment mask and the random mask to obtain a target mask, and use the target mask to perform masking processing on the reference hyperspectral image to obtain a target hyperspectral image;

[0217] The fusion module 305 is used to extract the spectral and spatial features of the target hyperspectral image through the encoder, and perform feature fusion to obtain the target features;

[0218] The processing and optimization module 306 is used to process the target feature using a decoder with a dual-branch parallel structure of feature reconstruction and boundary segmentation. It reconstructs the hyperspectral image through the feature reconstruction branch and outputs the boundary probability map of the region of interest through the boundary segmentation branch. The two branches are jointly optimized using a weighted loss function.

[0219] The determination and extraction module 307 is used to determine the mask of interest of the rice grain based on the boundary probability map and extract the region of interest from the initial hyperspectral image.

[0220] In one possible implementation, the preprocessing module 301 is specifically used for:

[0221] Obtain the initial hyperspectral image of the rice grain;

[0222] Wavelet thresholding is applied to the initial hyperspectral image to obtain the denoised hyperspectral image.

[0223] Multivariate scattering correction is performed on the denoised hyperspectral image to obtain the denoised spectral features in the reference hyperspectral image;

[0224] Morphological filtering and background suppression are performed on the initial hyperspectral image to obtain the denoised spatial features in the reference hyperspectral image.

[0225] In one possible implementation, the generation module 302 is specifically used for:

[0226] The endosperm center of the rice grain is obtained by taking the weighted average of the pixel coordinates of the rice grain in the reference hyperspectral image and the first weight value.

[0227] The first weight value is determined by the spectral purity in the reference hyperspectral image;

[0228] A circular mask template is generated with the endosperm center as the center and a preset radius and pixel coordinates of the rice grain as the center.

[0229] Calculate the edge gradient magnitude of the pixel for the rice grain;

[0230] A pixel region representing a predetermined proportion of the edge gradient magnitude is used as a candidate region mask for the aleurone layer of the rice.

[0231] Perform an XOR operation between the circular mask template and the candidate region mask to obtain the structural constraint mask.

[0232] In one possible implementation, the generation module 302 is specifically used for:

[0233] The sparsity of the structure constraint mask region is controlled by the first preset parameter to obtain the sparsity adjustment mask;

[0234] In one possible implementation, the definition and generation module 303 are also used for:

[0235] The unstructured region is masked with a first preset probability to obtain a random mask.

[0236] In one possible implementation, the fusion and processing module 304 is specifically used for:

[0237] The structural constraint mask is fused with the sparse adjustment mask to obtain the reference mask;

[0238] The target mask is obtained by XORing the reference mask with the random mask.

[0239] In one possible implementation, the fusion module 305 is specifically used for:

[0240] The spectral dimension of the hyperspectral image of the target is reduced by using a first preset convolution kernel, and the spectral features of the hyperspectral image of the target are extracted.

[0241] The spatial features of the target hyperspectral image are extracted using a second preset convolution kernel, which is larger than the first preset convolution kernel;

[0242] The spectral feature is then reconstructed to a size that matches the spatial feature using bilinear interpolation.

[0243] The spectral feature, whose size matches that of the spatial feature, is concatenated with the spatial feature and then max-pooled using a third preset convolution kernel to obtain the target feature. The third preset convolution kernel is smaller than the second preset convolution kernel.

[0244] In one possible implementation, the processing and optimization module 306 is specifically used for:

[0245] By upsampling layer by layer through transposed convolution, the features of the target are restored to obtain a reconstructed hyperspectral image;

[0246] Calculate the mean square error loss between the reconstructed hyperspectral image and the target hyperspectral image;

[0247] The channel weights of the decoder are calculated based on an attention mechanism;

[0248] The target hyperspectral map is weighted and convolved based on the channel weights to obtain the boundary probability map.

[0249] In one possible implementation, the determination and extraction module 307 is specifically used for:

[0250] The boundary probability map is then thresholded and binarized to obtain the mask of interest.

[0251] The region of interest is obtained by multiplying the initial hyperspectral image with the mask of interest pixel by pixel.

[0252] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0253] For example, such as Figure 4 As shown, the electronic device 400 includes a memory 401 and a processor 402. The memory 401 stores executable program code 403, and the processor 402 is used to call and execute the executable program code 403 to perform a method for extracting regions of interest in rice.

[0254] Furthermore, this application also protects an apparatus that may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to perform a method for extracting regions of interest in rice provided in this application.

[0255] This embodiment can divide the device into functional modules based on the above method example. For example, each module can correspond to a separate function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0256] It should be understood that the apparatus provided in this embodiment is used to perform the above-described method for extracting the region of interest of rice grains, and therefore can achieve the same effect as the above-described implementation method.

[0257] When using integrated units, the device may include a processing module and a storage module. When applied to an electronic device, the processing module can be used to control and manage the operation of the electronic device. The storage module can be used to support the execution of relevant program code by the electronic device.

[0258] The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits shown in conjunction with the disclosure of this application. The processor may also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and microprocessors, etc., and the storage module may be a memory.

[0259] In addition, the device provided in the embodiments of this application may specifically be a chip, component or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute a method for extracting the region of interest of rice provided in the above embodiments.

[0260] This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the above-described method steps to implement the method for extracting the region of interest of rice provided in the above embodiment.

[0261] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the method for extracting the region of interest of rice provided in the above embodiment.

[0262] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0263] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0264] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0265] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of extracting a region of interest of paddy rice, characterized by, The method includes: The initial hyperspectral image of the acquired rice grains is subjected to spectral preprocessing and spatial preprocessing to obtain a reference hyperspectral image; wherein, the spectral preprocessing is used to suppress noise and correct spectral scattering effects; the spatial preprocessing is used to enhance the morphological structure of rice grains and suppress background interference; Based on the prior knowledge of the anatomical structure of the rice, a structural constraint mask is generated, and based on the structural constraint mask, a sparse adjustment mask is generated to control the sparsity of the structural constraint mask region. The structural constraint mask region is the region obtained by masking the reference hyperspectral image based on the structural constraint mask. In the reference hyperspectral image, the pixel regions other than the structure-constrained mask region are defined as unstructured regions; a random mask is generated based on the unstructured regions. The structural constraint mask, the sparse adjustment mask, and the random mask are fused to obtain the target mask, and the target mask is used to mask the reference hyperspectral image to obtain the target hyperspectral image. The spectral and spatial features of the target hyperspectral image are extracted by an encoder, and feature fusion is performed to obtain the target features; The target features are processed using a decoder with a dual-branch parallel structure for feature reconstruction and boundary segmentation. The hyperspectral image is reconstructed through the feature reconstruction branch, while the boundary segmentation branch outputs the boundary probability map of the region of interest. The two branches are jointly optimized using a weighted loss function. The interest mask of the rice is determined based on the boundary probability map, and the region of interest is extracted from the initial hyperspectral image; The generation of a structural constraint mask based on the prior knowledge of the anatomical structure of the rice includes: The endosperm center of the rice grain is obtained by weighting the pixel coordinates of the rice grains in the reference hyperspectral image with a first weight value; the first weight value is determined by the spectral purity in the reference hyperspectral image. A circular mask template is generated with the endosperm center as the center and a preset radius and pixel coordinates of the rice grains. Calculate the edge gradient magnitude of the pixels of the rice grains; A pixel region with a preset proportion of the edge gradient magnitude is used as a candidate region mask for the aleurone layer of the rice. The structural constraint mask is obtained by performing an XOR operation between the circular mask template and the candidate region mask. The process of fusing the structural constraint mask, the sparse adjustment mask, and the random mask to obtain the target mask includes: The structural constraint mask and the sparse adjustment mask are fused to obtain a reference mask; The target mask is obtained by XORing the reference mask with the random mask.

2. The method of claim 1, wherein, The reference hyperspectral image includes denoised spectral features and denoised spatial features. The process of performing spectral preprocessing and spatial preprocessing on the acquired initial hyperspectral image of rice to obtain the reference hyperspectral image includes: Obtain an initial hyperspectral image of the rice grains; The initial hyperspectral image is subjected to wavelet threshold denoising to obtain a denoised hyperspectral image; Multivariate scattering correction is performed on the denoised hyperspectral image to obtain the denoised spectral features in the reference hyperspectral image; Morphological filtering and background suppression are performed on the initial hyperspectral image to obtain the denoised spatial features in the reference hyperspectral image.

3. The method of claim 1, wherein, The step of generating a sparse adjustment mask based on the structural constraint mask to control the sparsity of the structural constraint mask region includes: The sparsity of the structural constraint mask region is controlled by a first preset parameter to obtain the sparsity adjustment mask; The generation of a random mask based on the unstructured region includes: The unstructured region is masked with a first preset probability to obtain a random mask.

4. The method of claim 1, wherein, The step of extracting the spectral and spatial features of the target hyperspectral image through an encoder and performing feature fusion includes: The spectral dimension of the target hyperspectral image is reduced by using a first preset convolution kernel, and the spectral features of the target hyperspectral image are extracted. The spatial features of the target hyperspectral image are extracted using a second preset convolution kernel, wherein the second preset convolution kernel is larger than the first preset convolution kernel; The spectral features are restored to a size that matches the spatial features using bilinear interpolation. The spectral features that match the size of the spatial features are concatenated with the spatial features, and then max pooled using a third preset convolution kernel to obtain the target features. The third preset convolution kernel is smaller than the second preset convolution kernel.

5. The method of claim 1, wherein, The process utilizes a decoder with a dual-branch parallel structure for feature reconstruction and boundary segmentation to process the target features. The feature reconstruction branch reconstructs the hyperspectral image, while the boundary segmentation branch outputs a boundary probability map of the region of interest. This includes: By upsampling layer by layer through transposed convolution, the target features are restored to obtain a reconstructed hyperspectral image; Calculate the mean square error loss between the reconstructed hyperspectral image and the target hyperspectral image; The channel weights of the decoder are calculated based on an attention mechanism; The target hyperspectral map is weighted and convolved based on the channel weights to obtain the boundary probability map.

6. The method of claim 1, wherein, The step of determining the interest mask of the rice grain based on the boundary probability map and extracting the region of interest from the initial hyperspectral image includes: The boundary probability map is then thresholded and binarized to obtain the mask of interest. The region of interest is obtained by multiplying the initial hyperspectral image with the mask of interest pixel by pixel.

7. A device for extracting a region of interest of paddy, characterized by, The device includes: The preprocessing module is used to perform spectral preprocessing and spatial preprocessing on the acquired initial hyperspectral image of rice to obtain a reference hyperspectral image; wherein, the spectral preprocessing is used to suppress noise and correct spectral scattering effects; the spatial preprocessing is used to enhance the morphological structure of rice grains and suppress background interference. A generation module is used to generate a structural constraint mask based on the prior anatomical structure of the rice grain, and to generate a sparse adjustment mask based on the structural constraint mask to control the sparsity of the structural constraint mask region. The structural constraint mask region is the region obtained by masking the reference hyperspectral image based on the structural constraint mask. The module performs a weighted average of the pixel coordinates of the rice grains in the reference hyperspectral image with a first weight value to obtain the endosperm center of the rice grain. The first weight value is determined by the spectral purity in the reference hyperspectral image. A circular mask template is generated with the endosperm center as the center and a preset radius and the pixel coordinates of the rice grains. The module calculates the edge gradient magnitude of the pixels of the rice grains. A preset proportion of the pixel regions in the edge gradient magnitude is used as candidate region masks for the aleurone layer of the rice grain. The circular mask template and the candidate region mask are XORed to obtain the structural constraint mask. The structural constraint mask is fused with the sparse adjustment mask to obtain a reference mask. The reference mask is XORed with a random mask to obtain a target mask. The definition and generation module is further configured to define pixel regions in the reference hyperspectral image other than the structure-constrained mask region as unstructured regions; and generate random masks based on the unstructured regions. The fusion and processing module is used to fuse the structural constraint mask, the sparse adjustment mask and the random mask to obtain a target mask, and use the target mask to perform masking processing on the reference hyperspectral image to obtain a target hyperspectral image; The fusion module is used to extract the spectral and spatial features of the target hyperspectral image through the encoder, and perform feature fusion to obtain the target features; The processing and optimization module is used to process the target features using a decoder with a dual-branch parallel structure of feature reconstruction and boundary segmentation. It reconstructs the hyperspectral image through the feature reconstruction branch and outputs the boundary probability map of the region of interest through the boundary segmentation branch. The two branches are jointly optimized using a weighted loss function. The determination and extraction module is used to determine the interest mask of the rice grain based on the boundary probability map and extract the region of interest from the initial hyperspectral image.

8. An electronic device, comprising: The electronic device includes: Memory, used to store executable program code; A processor for calling and running the executable program code from the memory, causing the electronic device to perform the method as described in any one of claims 1 to 6.

Images