A feed mildew identification method based on hyperspectral vision feature fusion

Abstract

The present application provides a kind of feed mildew identification method based on hyperspectral visual feature fusion, belong to the mildew detection technical field based on computer vision;First, the hyperspectral-rgb bimodal image of feed sample is synchronously collected and physically priori pretreated;Then, feature extraction is carried out through one-dimensional spectral encoder and two-dimensional space encoder to obtain spectral feature map and spectral feature vector, space feature map and space texture feature vector;Subsequently, based on band-space collaborative gating fusion module, after weight calculation, outer product operation, weighted fusion obtains a space joint feature vector;After the space joint feature vector is projected in turn, the constructed particle adjacency graph is input into the particle level graph attention network, and the graph attention update feature is obtained;Finally, based on mildew classification head, output mildew grade label.The present application fuses hyperspectral spectral feature and visual spatial feature, realizes the rapid nondestructive accurate identification method of feed mildew.

Description

Technical Field

[0001] This invention belongs to the field of computer vision-based mold detection technology, and particularly relates to a method for identifying mold in feed based on hyperspectral visual feature fusion. Background Technology

[0002] Feed mold is a significant factor leading to economic losses and animal health risks in the livestock industry. Mycotoxins in moldy feed can be transmitted through the food chain, affecting livestock growth performance and the safety of livestock products. With the expansion of feed raw material trade and the extension of storage cycles, compound feed is susceptible to temperature, humidity, and microbial contamination during storage and transportation, resulting in localized or latent mold growth. Traditional detection methods rely heavily on manual sensory evaluation and laboratory physicochemical analysis, which suffer from high subjectivity, sample destruction, and time-consuming and labor-intensive issues. Machine vision methods based on RGB images have limited information dimensions and struggle to identify early mold growth and latent mold spots. While methods based on single hyperspectral imaging can acquire spectral information, they have high data dimensionality and redundancy, and do not fully utilize prior knowledge of the physicochemical processes caused by mold growth, making it difficult to achieve pixel-level localization and classification of moldy areas in scenarios with stacked, irregularly shaped, and complex backgrounds in compound feed. Therefore, developing a feed mold identification method that integrates hyperspectral spectral features and RGB visual spatial features, incorporates prior knowledge of mold growth and particle-level structural relationships, and possesses online domain adaptive capabilities is of significant engineering importance.

[0003] Currently, the main methods for identifying moldy feed include the following: (1) Traditional detection methods based on sensory evaluation and physicochemical analysis: This method relies on the inspector to observe the color, odor and mold spots of the feed, and is supplemented by laboratory analysis such as high performance liquid chromatography (HPLC), gas chromatography (GC) or enzyme-linked immunosorbent assay (ELISA). The operation process is mature and the results are accurate, but sensory evaluation is highly subjective and moldy samples pose a threat to the health of the inspector; physicochemical analysis requires the sample to be crushed, extracted and pretreated, which is complicated, time-consuming and costly, and difficult to meet the needs of online non-destructive testing. (2) RGB image detection method based on machine vision: RGB images of the feed surface are obtained through industrial cameras, color, texture and morphological features are extracted, and classification is achieved by combining support vector machine (SVM) or shallow neural network. Such methods are fast and low-cost, but RGB images only contain three channels of information in the visible light band, which cannot reflect the changes in internal chemical composition caused by mold. They are not good at identifying early mold and latent mold spots. At the same time, they are greatly affected by lighting conditions and shooting angle, and the generalization performance of the model is limited. (3) Detection methods based on single hyperspectral imaging or near-infrared spectroscopy: Hyperspectral imaging (HSI) is used to obtain continuous narrow band spectral information, or near-infrared spectroscopy (NIRS) is used to analyze the internal chemical composition of the sample. Such methods can obtain spectral features and are sensitive to chemical changes. However, due to the high dimensionality and redundancy of hyperspectral data, the single spectral model has high computational complexity and poor real-time performance. Moreover, relying solely on spectral information cannot fully utilize the spatial distribution and morphological texture information of the moldy area, making it difficult to achieve pixel-level positioning and grading of the moldy area. In addition, existing methods do not consider the occlusion, shadow and mold propagation between particles caused by the stacking of compound feed particles. The equipment cost is high and the model is highly dependent on the variety and environment, which limits the field deployment.

[0004] In summary, existing methods for identifying moldy feed have certain limitations in terms of non-destructiveness, real-time performance, spatial resolution, utilization of physical mechanisms, and generalization stability: traditional physicochemical analysis destroys samples and is inefficient; RGB machine vision information has a single dimension; single hyperspectral methods lack spatial semantic fusion, do not introduce physical priors for mold-sensitive bands, and do not specifically model the characteristics of compound feed pellet stacking, irregular shape, and complex background. Summary of the Invention

[0005] To address the above problems, this invention provides a method for identifying moldy feed based on hyperspectral visual feature fusion, comprising the following steps: S1, perform simultaneous acquisition of hyperspectral-RGB dual-modal images and physical prior preprocessing on feed samples to obtain sparse physical prior weight vectors. A normalized hyperspectral block, One normalized RGB block; S2, will Each normalized hyperspectral block is sequentially input into a one-dimensional spectral encoder for feature extraction. Each spectral feature map and spectral feature vector will Each normalized RGB block is sequentially input into a two-dimensional spatial encoder for feature extraction. Each spatial feature map and spatial texture feature vector; S3, will Each spectral feature vector and spatial texture feature vector are sequentially input into the band-space collaborative gating fusion module, and after weight calculation, outer product operation, and weighted fusion, the result is obtained. Joint eigenvectors of spatial spectra; The joint feature vectors of the empty spectrum are projected sequentially and input into the granular graph attention network along with the constructed granular adjacency graph. The graph attention update features are obtained through adjacency relationship propagation. S4 inputs the graph attention update features into the mold classification head after global average pooling, and outputs a mold level label calibrated by temperature scaling.

[0006] Preferably, the S1 process specifically includes: Using a pushbroom hyperspectral imaging system and an RGB industrial camera as the acquisition center, feed samples laid on a black low-reflectivity stage were simultaneously imaged, yielding hyperspectral cube data and RGB visual images. Based on the physical mechanisms of increased moisture content, enhanced OH bond absorption, and early color change during the mold growth process of compound feed, a physical prior spectral weight vector was constructed. Sparse spectral selection was then performed on the hyperspectral cube data based on this physical prior spectral weight vector, resulting in a normalized sparse hyperspectral cube and a sparse physical prior spectral weight vector. An early mold index map was constructed based on the reflectance ratio of specific bands in the hyperspectral cube data. This early mold index map was then used as a single-channel prior feature map and stitched with the RGB visual image along the channel dimension to form a four-channel input image. Feed particle mask generation, single-particle instance segmentation, and block cropping were performed on the normalized sparse hyperspectral cube and the four-channel input image to obtain… A normalized hyperspectral block and a normalized RGB block.

[0007] Preferably, the specific process for obtaining the sparse physical prior weight vector is as follows: First, sensitive bands for mold growth were selected based on the physical mechanisms of increased moisture content, enhanced OH bond absorption, and early color change during the mold growth process in compound feed. Second, Gaussian weighted windows were set around each sensitive band to construct a physical prior spectral weight vector. The dimension of the physical prior spectral weight vector was 224, with weights higher than 0.5 at sensitive bands and close to 0 at non-sensitive bands. Third, the physical prior spectral weight vector was multiplied band-by-band by the hyperspectral cube data to obtain a weighted hyperspectral cube. Then, the cubes were sorted from highest to lowest weight and the top weights were selected. A sparse hyperspectral cube is obtained by retaining certain bands; finally, a normalized sparse hyperspectral cube is obtained by performing max-min normalization on the sparse hyperspectral cube; simultaneously, based on the retained bands... Each band index is used to extract the corresponding weight from the physical prior spectral weight vector, resulting in a dimension of [missing information]. The sparse physical prior weight vector.

[0008] Preferably, the construction of the early mold index map involves first extracting the 760 nm and 800 nm reflectance images from the hyperspectral cube data and calculating the early mold index at each pixel location; secondly, performing min-max normalization on the early mold index to map its value range to the interval between 0 and 1, thus obtaining the early mold index map; finally, using the early mold index map as a single-channel prior feature map and stitching it with the RGB visual image in the channel dimension to form a four-channel input image.

[0009] Preferably, the A normalized hyperspectral block, The process of obtaining a normalized RGB block includes: First, a single-band grayscale image located at 550 nm in the normalized sparse hyperspectral cube is selected as the reference image. The Otsu's inter-class variance method is used to perform adaptive threshold segmentation on the reference image to obtain an initial binary mask. Second, morphological closing operations and connected component analysis are performed on the initial binary mask to obtain feed particle masks. Third, a distance transform is performed on the feed particle masks, and the watershed algorithm is used to segment the contacting feed particles, resulting in a set of single-particle instance masks, where each mask corresponds to an independent feed particle. Finally, based on the single-particle instance mask set, samples are cropped from the normalized sparse hyperspectral cube and the four-channel input image, respectively. A normalized hyperspectral block and a normalized RGB block.

[0010] Preferably, the specific data processing procedure of S3 includes: firstly, ... Each spectral feature vector and spatial texture feature vector is sequentially input into the band-space co-gated fusion module. The band-gated generation network outputs a physically constrained band-gated vector, and the spatial gated generation network outputs a spatial gated matrix. These are then combined through an outer product operation to generate a band-space joint saliency map. Finally, an adaptive weighted fusion is performed on the spectral and spatial feature maps based on this joint saliency map to obtain the final fusion result. First, a joint feature vector of the empty spectrum is generated; second, a particle adjacency graph is constructed based on the Euclidean distance between the centroids of each particle in the single-particle instance mask set; third, a joint feature vector of the empty spectrum is generated; fourth, a joint feature vector of the empty spectrum is generated; fifth, a joint feature vector of the empty spectrum is generated; sixth, a joint feature vector of the empty spectrum is generated; seventh, a joint feature vector of the empty spectrum is generated; and finally, a joint feature vector of the empty spectrum is The joint feature vectors of the empty spectrum are sequentially projected to 128 dimensions through a fully connected layer to obtain K particle graph node features. Then, the K particle graph node features and the particle adjacency graph are input into the particle-level graph attention network. After graph attention calculation and residual fusion, the output graph attention update features are obtained.

[0011] Preferably, the specific process for obtaining the graph attention update features is as follows: First, construct a particle adjacency graph based on the Euclidean distance between the centroids of each particle in the single-particle instance mask set; second, ... The joint feature vectors of the spatial spectrum are obtained by sequentially projecting them through a fully connected layer. Each particle map node feature, then... Each particle graph node feature and its adjacency graph are input into a granular graph attention network, which consists of two graph attention layers. In the graph attention layer, for adjacent particle pairs with connecting edges in the particle adjacency graph, the attention coefficient is calculated as the LeakyReLU nonlinear mapping result of the inner product of the node features after linear transformation, and then normalized by the Softmax function. The adjacent particle features are then weighted and aggregated based on the normalized attention coefficients. Finally, the output of the second-layer graph attention is added to the residual of the original particle graph node features to obtain the graph attention update features.

[0012] Preferably, the specific process of temperature scaling calibration is as follows: Let the Logit values ​​of the four categories output by the second fully connected layer of the mold classification head be respectively... , , and The temperature scaling parameter is denoted as First, divide each Logit value by... , obtain the scaled Logit value Next, the scaled Logit value is input into the Softmax function to obtain the calibrated mold probability distribution. When T is greater than 1, the probability distribution tends to be smoother, the peak confidence decreases, and the model’s overconfidence in the training samples is suppressed. When T equals 1, the temperature scaling degenerates into the standard Softmax function. The temperature scaling parameter T is searched on the validation set by minimizing the cross-entropy loss. The candidate sets are 1.0, 1.2, 1.5 and 2.0. The T value that minimizes the cross-entropy loss on the validation set is selected as the final inference parameter.

[0013] Compared with the prior art, the present invention has the following beneficial effects: (1) Improve the accuracy of feed moldy area location and early mold identification ability. By introducing physical prior spectral weight vector and early mold index map, the model can prioritize mold-related bands and weak response areas; by band-space collaborative gating fusion, the focusing ability of key bands and key spatial locations is enhanced; by using particle-level graph attention network, the mold propagation is modeled by particle adjacency relationship, which improves the identification ability of particles in occluded and shaded areas; (2) Improve the real-time performance, lightweight nature, and cross-scenario generalization ability of online detection. By using physical prior sparse spectral selection, the input band is reduced from 224 dimensions to 48 dimensions, reducing the computational load of the one-dimensional spectral encoder; by replacing high-dimensional feature splicing with band-space collaborative gating fusion, the spatial-spectral joint feature dimension is reduced; by using real-time illumination compensation parameters and domain adaptive alignment features to suppress environmental disturbances, the model failure caused by variety changes and the need for frequent retraining are reduced. (3) Enhance the traceability and batch early warning capabilities of mold identification results. By uploading mold grade labels, mold probability distribution, granular graph attention weights and domain adaptive alignment features to the quality traceability system, the entire chain of feed raw materials from sample collection to mold determination is realized, and batch-level mold early warning based on granular abnormal aggregation patterns is supported. Attached Figure Description

[0014] Figure 1 This is a complete flowchart of the overall technical route of the present invention.

[0015] Figure 2 This is a flowchart of the hyperspectral-RGB dual-modal acquisition and physical prior preprocessing of the present invention.

[0016] Figure 3 This is a schematic diagram illustrating the feature extraction of the normalized hyperspectral block and the normalized RGB block in this invention.

[0017] Figure 4 This is a flowchart of the graph attention update feature calculation process of the present invention.

[0018] Figure 5 This is a flowchart of the mold grade label prediction process of the present invention.

[0019] Figure 6This is a diagram showing the physical prior spectral weight vector and band gating distribution in an embodiment of the present invention.

[0020] Figure 7 This is an RGB visual image diagram from an embodiment of the present invention.

[0021] Figure 8 This is a diagram of the early mold index in an embodiment of the present invention.

[0022] Figure 9 This is a band-space joint saliency map in an embodiment of the present invention.

[0023] Figure 10 This is a visualization of the attention weights at the granular level in an embodiment of the present invention.

[0024] Figure 11 This is a diagram showing the recognition accuracy under different lighting conditions in an embodiment of the present invention.

[0025] Figure 12 This is a graph showing the cross-variety generalization accuracy of wheat-soybean meal in an embodiment of the present invention. Detailed Implementation

[0026] This invention provides a method for identifying feed mold based on hyperspectral visual feature fusion, the overall technical process of which is as follows: Figure 1 As shown: S1, using a pushbroom hyperspectral imaging system and an RGB industrial camera as the acquisition center, synchronously acquires images of feed samples laid on a black low-reflectivity stage, obtaining hyperspectral cube data and RGB visual images; based on the physical mechanism of increased moisture, enhanced OH bond absorption, and early color change during the mold growth of compound feed, a physical prior spectral weight vector is constructed, and sparse spectral selection is performed on the hyperspectral cube data according to the physical prior spectral weight vector to obtain normalized sparse hyperspectral cubes and sparse physical prior spectral weight vectors; based on the reflectance ratio of specific bands of the hyperspectral cube data, an early mold index map is constructed, and the early mold index map is used as a single-channel prior feature map and stitched with the RGB visual image in the channel dimension to form a four-channel input image; feed particle mask generation, single particle instance segmentation, and block clipping are performed on the normalized sparse hyperspectral cubes and the four-channel input image to obtain a set of single particle instance masks. A normalized hyperspectral block and a normalized RGB block; S2, will Normalized hyperspectral blocks are sequentially input into a one-dimensional spectral encoder, and spectral feature maps and spectral feature vectors are extracted along the band dimension; Each normalized RGB block is sequentially input into a two-dimensional spatial encoder, which extracts spatial feature maps and spatial texture feature vectors along the spatial dimension; ultimately obtaining... Each spectral feature map and spectral feature vector, and Each spatial feature map and spatial texture feature vector; S3, Band-Spatial Cooperative Gated Fusion and Granular Graph Attention Inference; First, ... Each spectral feature vector and spatial texture feature vector is sequentially input into the band-space co-gated fusion module. The band-gated generation network outputs a physically constrained band-gated vector, and the spatial gated generation network outputs a spatial gated matrix. These are then combined through an outer product operation to generate a band-space joint saliency map. Finally, an adaptive weighted fusion is performed on the spectral and spatial feature maps based on this joint saliency map to obtain the final fusion result. First, a joint feature vector of the empty spectrum is generated; second, a particle adjacency graph is constructed based on the Euclidean distance between the centroids of each particle in the single-particle instance mask set; third, a joint feature vector of the empty spectrum is generated; fourth, a joint feature vector of the empty spectrum is generated; fifth, a joint feature vector of the empty spectrum is generated; sixth, a joint feature vector of the empty spectrum is generated; seventh, a joint feature vector of the empty spectrum is generated; and finally, a joint feature vector of the empty spectrum is The joint feature vectors of the spatial spectrum are projected sequentially through a fully connected layer to 128 dimensions to obtain K particle graph node features. Then, the K particle graph node features and the particle adjacency graph are input into the particle-level graph attention network. After graph attention calculation and residual fusion, the graph attention update features are output. S4, mold level identification: After global average pooling of the graph attention update features, the input is fed into the mold classification head. Fully connected dimensionality reduction, Dropout regularization and Softmax probability mapping are performed in sequence to output the mold probability distribution. After temperature scaling calibration, the category corresponding to the maximum probability is selected as the mold level label.

[0027] The present invention will be further described below with reference to embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0028] S1. Hyperspectral-RGB Dual-Modal Data Acquisition and Physical Prior Preprocessing The overall process is as follows Figure 2 As shown: S1-1 Dual-modal image synchronous acquisition. First, feed samples are evenly spread on a black, low-reflectivity stage with a surface reflectivity of less than 5% to suppress background reflection interference. A pushbroom hyperspectral imaging system is used to acquire hyperspectral cube data at a spectral resolution of 2.8 nm within the 400 nm to 1000 nm band range. The spatial resolution of the hyperspectral cube data is 1024 pixels × 1024 pixels, and the number of spectral bands is 224. Second, RGB visual images are acquired at a resolution of 2048 × 1536 pixels using an RGB industrial camera under the same field of view. The hyperspectral imaging system and the RGB industrial camera achieve synchronous exposure via an external hardware trigger signal, emitted by a motion controller at a frequency of 10 Hz. Third, the hyperspectral cube data and RGB visual images are timestamped. A timestamp is added to each frame of data based on the trigger time, and hyperspectral frames with a time difference of less than 50 milliseconds are paired with RGB frames using nearest neighbor matching. Finally, the hyperspectral cube data and RGB visual images are saved as 32-bit floating-point arrays with dimensions of spatial height × spatial width × number of spectral bands and spatial height × spatial width × 3, respectively. S1-2 Construction of Physical Prior Spectral Weights and Selection of Sparse Spectra. First, based on the physical mechanism of increased moisture content, enhanced OH bond absorption, and early color change during the mold growth process of compound feed, the 550 nm early color change band, the 760 nm moisture absorption peak band, and the 980 nm OH bond broad absorption band were selected as the mold-sensitive bands. Secondly, Gaussian weighted windows with bandwidths of 40 nm, 80 nm, and 100 nm are set as the center for each mold-sensitive band to construct a physical prior spectral weight vector. The dimension of the physical prior spectral weight vector is 224. The weight values ​​are higher than 0.5 in the mold-sensitive bands and close to 0 in the non-sensitive bands. The Gaussian weights are calculated according to the formula... Calculation, where The center wavelength, Half the bandwidth; Next, the physical prior spectral weight vector is multiplied band-by-band with the hyperspectral cube data to obtain a weighted hyperspectral cube; then, it is sorted from highest to lowest weight and the top weights are selected. By retaining certain bands, a sparse hyperspectral cube is obtained, in which the spectral dimension of the sparse hyperspectral cube is reduced from 224 to [missing value]. Finally, a min-max normalization process is performed on the sparse hyperspectral cube to map the reflectance of each band to the interval between 0 and 1, resulting in a normalized sparse hyperspectral cube; simultaneously, based on the preserved... Each band index is used to extract the corresponding weight from the physical prior spectral weight vector, resulting in a dimension of [missing information]. sparse physical prior weight vector; S1-3 Early Mold Index Generation: First, the reflectance images of the 760 nm and 800 nm bands were extracted from the hyperspectral cube data. The reason for choosing these two bands is that the 760 nm band is located in the transition region from the red edge to the near-infrared, and is more sensitive to changes in the internal moisture and cell structure of feed particles. After mold growth, the reflectance in this region decreases with increasing moisture content. The 800 nm band is located at the beginning of the near-infrared plateau, and is relatively less affected by moisture absorption, with relatively stable reflectance, so it can be used as a normalized reference band. An early mold index is constructed using 760 nm and 800 nm reflectance. This can, to some extent, offset the multiplicative interference caused by differences in light intensity and sample surface orientation, highlighting the weak moisture response caused by early mold growth, thereby enhancing the two-dimensional spatial encoder's ability to perceive latent mold spots. Secondly, the early mold index at each pixel location is calculated. The early mold index is defined as the difference between the 760 nm and 800 nm reflectance divided by their sum, i.e., the early mold index equals 760 nm reflectance minus 800 nm reflectance divided by 760 nm reflectance plus 800 nm reflectance. Thirdly, the early mold index is subjected to min-max normalization, mapping its value range to the 0-1 interval, resulting in an early mold index map. Finally, the early mold index map is used as a single-channel prior feature map and stitched with the RGB visual image in the channel dimension to form a four-channel input image. This four-channel input image is used to enhance the two-dimensional spatial encoder's ability to perceive the weak response of early mold growth. S1-4 Feed Particle Mask Generation and Single Particle Instance Segmentation. First, a single-band grayscale image located at 550 nm in the normalized sparse hyperspectral cube is selected as the reference image. The Otsu's method is used to perform adaptive threshold segmentation on the reference image to obtain an initial binary mask. Second, morphological closing operations are performed on the initial binary mask, with a 5×5 square kernel as the structuring element, to close small gaps between particles and eliminate isolated noise points. Connected component analysis is then used to filter out noise regions with an area smaller than a preset threshold of 50 pixels, resulting in the feed particle mask. Third, a distance transform is performed on the feed particle mask, and the peak value of the distance transform is used as the seed point. The watershed algorithm is used to segment contacting feed particles, obtaining a set of single-particle instance masks, where each mask corresponds to an independent feed particle. Finally, based on the single-particle instance mask set, a single particle is cropped from the normalized sparse hyperspectral cube. Each size is 32×32× Normalized hyperspectral blocks; cropped from the four-channel input image based on a set of single-particle instance masks. Each normalized RGB block has a size of 224×224×4. Each normalized hyperspectral block and normalized RGB block cover the same complete feed particle and its neighborhood. That is, in the single particle instance mask set, each mask corresponds to a feed particle with a normalized hyperspectral block and a normalized RGB block. The 550 nm band was chosen as the reference image for mask generation for the following reasons: First, this band is in the visible green light region, where the reflectance difference between the feed particle surface and the black low-reflectance stage is significant, which is beneficial for separating the particles from the background through threshold segmentation. Second, 550 nm is a sensitive band for early color change in mold growth, where normal particles and moldy particles already have a certain contrast, but its main function is to provide a single-band grayscale image with a high signal-to-noise ratio for subsequent particle localization and mask generation. Third, the spatial resolution of this band is consistent with the original hyperspectral cube data, which can accurately reflect the particle edges and stacking structure, providing reliable input for subsequent watershed instance segmentation. The specific steps for normalized hyperspectral block cropping are as follows: First, perform connected component analysis on each mask in the single-particle instance mask set to obtain the pixel set and centroid coordinates of each independent feed particle; second, expand 16 pixels outwards from the centroid to determine a 32-pixel × 32-pixel spatial window. If the window exceeds the image boundary, it is padded to the limit of the image boundary; third, map the above spatial window to the corresponding spatial position of the normalized sparse hyperspectral cube, and extract all pixels within the window. The reflectance values ​​are obtained in sparse bands; finally, the cut hyperspectral block is subjected to max-min normalization to obtain a size of 32×32× Normalized hyperspectral blocks. Each normalized hyperspectral block covers an entire feed pellet and its local neighborhood to ensure that the spectral features simultaneously include information about the pellet body and boundary transitions; The specific steps for normalized RGB block cropping are as follows: First, perform connected component analysis on each mask in the single-particle instance mask set to obtain the pixel coordinate set and centroid coordinates of each independent feed particle; the four-channel input image is composed of an RGB visual image and an early mold index map stitched together in the channel dimension, with a spatial resolution consistent with the RGB visual image and a channel number of 4; second, using the particle centroid as the center of the cropping window, extend 112 pixels outwards in all directions to determine a 224 pixel × 224 pixel spatial window; if the window exceeds the boundary of the four-channel input image, it is truncated to the image boundary, and areas less than 224 pixels × 224 pixels after truncation are filled using mirror filling or zero filling. The target size is then determined. Next, the aforementioned spatial window is mapped to the corresponding spatial position of the four-channel input image. The values ​​of all pixels within the window in the R, G, B, and early mold index channels are extracted, resulting in an RGB-early mold index block of size 224×224×4. Then, the mean and standard deviation of the RGB channels from the ImageNet dataset are used to perform channel-by-channel standardization on the first three RGB channels. The early mold index channels are independently standardized using the mean and standard deviation of the early mold index from the training set, ensuring that the values ​​of the four channels are at a compatible scale. Finally, the standardized 224×224×4 block is output as a normalized RGB block. Each normalized RGB block is associated with its corresponding 32×32×... The normalized hyperspectral block covers the same complete feed pellet and its local neighborhood to ensure that the subsequent spectral branches and spatial branches have a one-to-one correspondence in spatial location. S1-5 Construction of the Training Sample Set. First, data augmentation is performed synchronously on all normalized hyperspectral blocks and normalized RGB blocks corresponding to the feed samples. Specifically, this includes random horizontal flipping, random vertical flipping, brightness jittering, spectral noise augmentation, and early mold index image jittering. The probabilities of random horizontal and vertical flipping are both set to 0.5. The adjustment coefficient for brightness jittering is uniformly and randomly selected between 0.8 and 1.2. Spectral noise augmentation uses Gaussian noise with a mean of 0 and a standard deviation of 0.01, and early mold index image jittering uses additive noise with a mean of 0 and a standard deviation of 0.02. Second, each feed sample is labeled as one of four categories: normal, slightly moldy, moderately moldy, or severely moldy. All particle-sized blocks within the feed sample share the same category label, resulting in labeled samples. Finally, all labeled samples are divided into a training sample set, a validation sample set, and a test sample set in a ratio of 8:1:1.

[0029] S2, Particle-level spatial spectral feature extraction Feature extraction of normalized hyperspectral blocks and normalized RGB blocks, as follows Figure 3 As shown: S2-1 One-Dimensional Spectral Feature Extraction: [The following text appears to be incomplete and requires further context:] 32×32× Normalized hyperspectral blocks of 1×10⁻⁶ dimensionality are sequentially input into a one-dimensional spectral encoder. For a single normalized hyperspectral block, it is first compressed into a shape of 1×10⁻⁶ dimensionality along the spatial dimension using a global average pooling layer. The initial spectral vector, i.e., with 1 channel and a sequence length of . The pooling operation averages the values ​​for each band within a 32×32 spatial region. Next, the initial spectral vector is passed sequentially through three one-dimensional convolutional layers. The first layer has a kernel size of 11, 64 output channels, a stride of 1, and uses the same padding method, resulting in an output feature shape of 64×32. The second convolutional layer has a kernel size of 7, 128 output channels, a stride of 1, and the same padding method, outputting a 128×48 spectral feature map. The third convolutional layer has a kernel size of 3, 128 output channels, a stride of 1, and the same padding method, outputting a 128×48 spectral feature map. Next, each one-dimensional convolutional layer is followed by a batch normalization layer and a ReLU activation function. Finally, a global average pooling layer compresses the 128×48 spectral feature map into a 128-dimensional spectral feature vector along the sequence length dimension. This spectral feature vector is used to characterize the deep discriminative information of feed mold in the spectral dimension. The final result is... Each spectral feature map and spectral feature vector; S2-2 Two-Dimensional Spatial Texture Feature Extraction: K normalized RGB blocks of 224×224×4 dimensions are sequentially input into a two-dimensional spatial encoder; the two-dimensional spatial encoder consists of 3 two-dimensional convolutional layers; the first layer has a kernel size of 7×7, 32 output channels, a stride of 2, and the same padding method, with an output feature map shape of 112×112×32; the second layer has a kernel size of 5×5, 64 output channels, a stride of 2, and the same padding method, with an output feature map shape of 56×56×64; the third layer has a kernel size of... The output is a 28×28×128 spatial feature map, with a 3×3 matrix, 128 output channels, a stride of 2, and the same padding method. Next, after each 2D convolutional layer, a batch normalization layer and a ReLU activation function are sequentially connected. Then, after the third 2D convolutional layer, an adaptive average pooling layer is connected, compressing the 28×28×128 spatial feature map to 128 dimensions. Finally, a 128-dimensional spatial texture feature vector is output, which is used to characterize the color, texture, morphology, and early mold index response information of the moldy area in the feed. The final result is... A spatial feature map and a spatial texture feature vector.

[0030] S3, Band-Spatial Cooperative Gated Fusion and Granular Graph Attention Reasoning The band-space collaborative gating fusion module includes a band-gated generation network, a spatial gated generation network, and a granular graph attention network. The graph attention update feature calculation process is as follows: Figure 4 As shown: S3-1 Band-gated vector generation. A 128-dimensional spectral feature vector is input into the band-gated generation network. This network consists of two fully connected layers. The first layer has 64 output nodes and uses the ReLU activation function. The second layer has [number of output nodes missing]. First, the Sigmoid activation function is used; second, the output of the second layer is normalized using the Softmax function, resulting in... A dimensional band-gated vector is used, where each element corresponds to an importance weight of a sparse spectral band, with values ​​ranging from 0 to 1. Finally, the band-gated vector is multiplied element-wise with the sparse physical prior spectral weight vector to obtain a physically constrained band-gated vector, enabling the network to retain the physical prior of mold-sensitive bands while focusing on data-driven importance.

[0031] S3-2 Spatial Gating Matrix Generation. A 128-dimensional spatial texture feature vector is input into a spatial gating generation network. This network consists of two fully connected layers: the first layer has 64 output nodes and uses the ReLU activation function, while the second layer has 196 output nodes and uses the Sigmoid activation function. Next, the 196-dimensional output is reshaped into a 14×14 spatial gating matrix, where each element corresponds to a spatial location importance weight. Then, min-max normalization is performed on the spatial gating matrix to make the sum of all its elements equal to 1, resulting in a normalized spatial gating matrix. Finally, the normalized spatial gating matrix is ​​upsampled to the same 28×28 resolution as the original spatial feature map, yielding the spatial gating matrix.

[0032] S3-3 Band-Space Cooperative Gated Fusion. First, the physical constraint band gating vector and the spatial gating matrix are multiplied by an outer product to generate... A 28×28 band-spatial joint saliency map is generated, where each element reflects the importance of both the corresponding band and spatial location for mold detection. Secondly, the 128×28 band-spatial joint saliency map output from the third layer convolution of the one-dimensional spectral encoder is used... The spectral feature map is extended to 128× using linear interpolation. ×1, then the 28×28×128 spatial feature map output from the third convolution layer of the 2D spatial encoder is transposed to 128×28×28, aligning the two feature paths in the channel dimension; next, the band-space joint saliency map is multiplied element-wise with the aligned spectral feature map and spatial feature map respectively to obtain a weighted spectral feature map and a weighted spatial feature map; then, global average pooling is performed on the weighted spectral feature map along the band and spatial dimensions to obtain a 128-dimensional weighted spectral feature vector, and global average pooling is performed on the weighted spatial feature map along the spatial dimension to obtain a 128-dimensional weighted spatial feature vector; finally, the weighted spectral feature vector and the weighted spatial feature vector are concatenated to obtain a 256-dimensional spatial-spectral joint feature vector; For all feed pellets, the spectral feature maps and spectral feature vectors, spatial feature maps and spatial texture feature vectors are sequentially processed by performing band-gated vector generation, spatial gating matrix generation, and band-space co-gated fusion. Joint eigenvectors of the empty spectrum.

[0033] S3-4 Granular Graph Attention Propagation. First, a particle adjacency graph is constructed based on the Euclidean distance between the centroids of particles in the single-particle instance mask set. If the distance between the centroids of two particles is less than a preset adjacency threshold of 80 pixels and their masks are in contact, an edge is connected in the particle adjacency graph. Second, ... The joint eigenvectors of the spatial spectrum are sequentially projected onto a 128-dimensional array through a fully connected layer to obtain... Each particle map node feature, then... Each particle graph node feature and particle adjacency graph are input into a granular graph attention network. The granular graph attention network consists of two graph attention layers, each containing eight attention heads, each outputting 16-dimensional features. After concatenation, a 128-dimensional graph attention output is obtained. Next, in the graph attention layer, for adjacent particle pairs with connecting edges in the particle adjacency graph, the attention coefficient is calculated as the LeakyReLU nonlinear mapping result of the inner product of the node features after linear transformation, and normalized using the Softmax function. The adjacent particle features are then weighted and aggregated based on the normalized attention coefficients. Finally, the second-layer graph attention output is added to the residual of the original particle graph node features to obtain the graph attention update feature. This graph attention update feature simultaneously includes the spatial spectrum discrimination information of the current particle and the mold propagation information of neighboring particles.

[0034] S4. Mold Level Identification and Online Domain Adaptive Output S4-1 Mold Probability Distribution Prediction. First, the graph attention update features are globally averaged to obtain sample-level feature vectors. Second, the sample-level feature vectors are input into the mold classification head, which consists of a first fully connected layer, a Dropout layer, and a second fully connected layer. The first fully connected layer has 128 input nodes and 128 output nodes, uses the ReLU activation function, and has a Dropout ratio of 0.5. The second fully connected layer has 4 output nodes, corresponding to four categories: normal, mild mold, moderate mold, and severe mold. Third, the output of the second layer is input into the Softmax function to obtain the posterior probability of each category, i.e., the mold probability distribution. Finally, the mold probability distribution is calibrated by temperature scaling. The temperature scaling parameters are selected from candidate sets 1.0, 1.2, 1.5, and 2.0 through grid search on the validation set to minimize the cross-entropy loss of the validation set. The calibrated mold probability distribution is used for subsequent mold level label determination.

[0035] The specific process of temperature scaling calibration is as follows: Let the Logit values ​​of the four categories output by the second fully connected layer of the mold classification head be respectively... , , and The temperature scaling parameter is denoted as First, divide each Logit value by... , obtain the scaled Logit value Next, the scaled Logit value is input into the Softmax function to obtain the calibrated mold probability distribution. When T is greater than 1, the probability distribution tends to be smoother, the peak confidence decreases, and the model's overconfidence in the training samples can be suppressed; when T equals 1, the temperature scaling degenerates into the standard Softmax function. The temperature scaling parameter T is determined by a grid search on the validation set to minimize the cross-entropy loss. The candidate sets are 1.0, 1.2, 1.5, and 2.0. The T value that minimizes the cross-entropy loss on the validation set is selected as the final inference parameter. S4-2 Mold Degradation Grade Label Determination and Output. First, compare the probabilities of the four categories in the mold probability distribution and select the category index corresponding to the highest probability. Second, map the category index to the corresponding mold degradation grade label, which includes normal, mild mold, moderate mold, and severe mold. Third, output the mold degradation grade label and the mold probability distribution together to the host computer's quality traceability system. Finally, when the determination result is moderate or severe mold, the system generates a batch-level mold degradation warning and pushes it to the warehouse management personnel's terminal. The above process is as follows: Figure 5 As shown; S1-1 and S1-3 are executed on the training sample set and the new batch of unlabeled samples, respectively, to obtain the corresponding four-channel input images. Then, granular block cropping in S1-4 is performed on the four-channel input images of the training sample set and the new batch of unlabeled samples to obtain granular normalized RGB blocks. The granular normalized RGB blocks of the training sample set and the new batch of unlabeled samples are input together into the two-dimensional spatial encoder to extract intermediate layer features. The maximum mean difference between the two feature distributions is calculated to obtain domain-adaptive alignment features. Next, the class-weighted cross-entropy loss and the maximum mean difference auxiliary loss are added. The class-weighted cross-entropy loss comes only from the labeled training samples. Backpropagation is performed using the AdamW optimizer to update the two-dimensional spatial encoder parameters, aligning the feature distribution of the new batch of samples with the distribution of the training sample set. Finally, during the domain-adaptive update process, the parameters of the one-dimensional spectral encoder and the granular graph attention network are fixed, and only the shallow convolution parameters of the two-dimensional spatial encoder are updated to prevent overfitting of the new batch of noise. The specific calculation process for the maximum mean difference between the two characteristic distributions is as follows: First, n source domain sample features are randomly selected from both the training sample set and the new batch of unlabeled samples. Features of target domain samples The features are 128-dimensional feature vectors obtained by global average pooling after the third layer convolution of the two-dimensional spatial encoder. Secondly, a Gaussian radial basis kernel function is used to perform a nonlinear mapping on the source and target domain sample features. Thirdly, the mean embeddings of the source and target domain sample features in the mapping space are calculated respectively. and Then, the Euclidean distance between the two mean embeddings is calculated to obtain the maximum mean difference. ,in The reproducing kernel Hilbert space norm is represented; in practical implementation, a sample approximation formula is used. Finally, As an auxiliary loss term, it is added to the category-weighted cross-entropy loss and drives the shallow convolution parameters of the two-dimensional spatial encoder to update during backpropagation, so that the feature distribution of the new batch of samples is aligned with the distribution of the training sample set. Model Training and Parameter Update (S4-4) First, the original hyperspectral cube data and RGB visual images of the training sample set are batch-input into the network, and forward propagation (S1 to S4) is performed sequentially to obtain the sample-level mold probability distribution. Second, the class-weighted cross-entropy loss is calculated, and backpropagation is performed using the AdamW optimizer to update the learnable parameters of the one-dimensional spectral encoder, two-dimensional spatial encoder, band-spatial collaborative gating fusion module, granular graph attention network, and mold classification head. Third, a cosine annealing learning rate scheduling strategy is adopted, with an initial learning rate set to 0.001 and a minimum learning rate set to 0.00001. The learning rate decays according to a cosine curve after each training epoch. Finally, the model loss is evaluated on the validation set. If the validation set loss does not decrease for 30 consecutive epochs, an early stopping mechanism is triggered, and the model parameters with the lowest validation set loss are saved as the final inference model.

[0036] Example 2: To quantitatively evaluate the recognition accuracy, early mold detection capability, and cross-scenario generalization capability of the feed mold profiling identification method based on hyperspectral visual feature fusion proposed in this invention in the task of identifying mold profiling levels in compound feed, this embodiment constructs a simulation experimental system based on a self-built corn-soybean meal compound feed mold sample library. The experimental dataset covers four categories: normal, mild mold, moderate mold, and severe mold, obtained by artificial inoculation with Aspergillus flavus and Aspergillus ochraceus and cultured under controlled temperature and humidity conditions, with 680, 650, 670, and 680 samples for each category, respectively. Each sample group simultaneously acquired hyperspectral cubic data in the 400 nm to 1000 nm band and 2048×1536 pixel RGB visual images. The experimental environment was based on the Ubuntu 22.04 operating system, using Python 3.10 and the PyTorch 2.0 deep learning framework, and was trained and tested on an NVIDIA RTX 3090 GPU. The optimizer used was AdamW, with an initial learning rate set to 0.001, and a cosine annealing learning rate scheduling strategy was adopted. Evaluation metrics include recognition accuracy, macro-precision, macro-recall, macro-F1 score, and Kappa coefficient.

[0037] I. Analysis of the effectiveness of physical prior sparse spectrum selection Figure 6 The distributions of the physical prior spectral weight vector, the data learning band gating vector, and the physical constraint band gating vector across 224 original bands are shown. Analysis Figure 6 It is observed that the physical prior spectral weight vector exhibits three distinct peaks near 550 nm, 760 nm, and 980 nm, consistent with the physical mechanisms of early color change, increased moisture content, and enhanced OH bond absorption caused by mold growth. The data-learned band-gated vector also shows high responses in some non-prior bands, indicating that data-driven approaches can capture discriminative information not covered by physical priors. The physical constraint band-gated vector integrates both types of information, improving feature discriminativity while maintaining physical interpretability. Using only 48 sparse bands, the method achieved a test set accuracy of 96.12%, only 0.73 percentage points lower than the baseline model's 96.85% using all 224 bands, but with a 62.4% reduction in single-sample inference time. This verifies the effectiveness of selecting physical prior sparse spectra in improving real-time performance while maintaining accuracy.

[0038] II. Effectiveness Analysis of Band-Space Cooperative Gating Fusion Figure 7 , Figure 8 and Figure 9 From RGB visual images, early mold index maps, and band-spatial joint significance, respectively Figure 3 This section showcases the characteristic responses of a typical moldy sample from various angles.

[0039] Figure 7 This is an RGB visual image of a typical moldy sample. The image shows color changes and localized mold spots on the surface of the feed pellets. However, some early moldy areas show little color difference from the surface of normal pellets, making accurate identification difficult based solely on RGB visual information.

[0040] Figure 8 This is the early mold index map for the corresponding sample. The early mold index is constructed based on the reflectance of the 760 nm and 800 nm bands, which can enhance the weak moisture response caused by early mold growth. The high response areas in the map basically correspond to the suspected mold areas in the RGB visual image, indicating that the early mold index map has a certain enhancing effect on latent mold spots.

[0041] Figure 9This is a joint saliency map of band and space. The map shows a high response near the moldy area and a lower response in the background and normal particle areas. Compared to methods using spatial gating matrices or band gating vectors alone, the joint saliency map can simultaneously locate the spatial position of mold occurrence and the sensitive bands, reducing background and shadow interference. Ablation experiments show that after removing the band-space co-gating fusion module and directly splicing the bi-branch features, the macro-average F1 score decreased from 96.06% to 94.21%, indicating that the co-gating mechanism can effectively enhance the expressive power of key features.

[0042] III. Granular Graph Attention Network Contribution Analysis Figure 10 This section presents a visualization of attention weights in a granular graph. Nodes in the graph represent individual feed particles, and the color intensity of the edges indicates the magnitude of the attention coefficient between adjacent particles. Analysis Figure 10 It can be seen that in the mold-affected area, the attention coefficient between adjacent particles is significantly higher than that between isolated particles or in normal areas, indicating that the particle-level graph attention network can capture the structural associations of mold spores spreading across particles. For particles occluded by shaded areas, the attention weights are mainly directed towards neighboring moldy particles, thus improving the recognition accuracy of occluded particles by leveraging adjacency information. Confusion matrix analysis shows that after introducing the particle-level graph attention network, the misclassification rate between mild and moderate mold decreased from 7.2% to 4.8%, verifying the role of the graph attention mechanism in improving the discrimination ability of boundary samples.

[0043] IV. Analysis of Online Domain Adaptation and Cross-Variety Generalization Ability Figure 10 and Figure 11 The generalization ability of the model under different lighting conditions and across variety test sets is demonstrated.

[0044] Figure 11 The figures show the recognition accuracy curves under different light intensity variations. Without the domain adaptation mechanism, the model accuracy drops to 88.34% when the light intensity change exceeds 30%. After introducing real-time light compensation parameters, the accuracy in light-changing scenarios recovers to 94.51%. Further introducing domain adaptive alignment features allows the model to maintain stable recognition performance under varying light conditions.

[0045] Figure 12The figure shows the accuracy curves of online adaptive testing on the wheat-soybean meal cross-variety test set. Without the domain adaptation mechanism, the model accuracy was 84.62%. After introducing a full-network fine-tuning strategy, the accuracy gradually improved to 92.45%. Using a domain adaptation strategy that only updates the shallow parameters of the two-dimensional spatial encoder, the accuracy further improved to 92.87%, while reducing the risk of overfitting to new batch noise. These results indicate that the online domain adaptation mechanism can effectively suppress distribution drift caused by changes in ambient light and feed variety, improving the stability of the model in actual deployment on feed production lines.

[0046] V. Comparison of Identification Performance and Ablation Experiment Table 1 details the recognition accuracy, macro-average precision, macro-average recall, macro-average F1 score, and Kappa coefficient of this method and three baseline models on the test set.

[0047] Table 1 Comparison of mold detection performance of four methods

[0048] Analysis of Table 1 shows that our proposed method outperforms the three baseline methods in terms of recognition accuracy, macro-average F1 score, and Kappa coefficient. Compared to the single RGB branch model, our method effectively captures the chemical composition changes caused by mold growth by introducing hyperspectral spectral features. Compared to the single hyperspectral branch model, our method fully utilizes the spatial distribution of moldy areas and the propagation relationship between particles by fusing RGB visual spatial features with a particle-level image attention mechanism. Compared to the simple stitching fusion model, our method further enhances feature discrimination ability by adaptively weighting key bands and key spatial locations through a band-space collaborative gating fusion module.

[0049] Table 2 presents the results of the ablation experiment to verify the contribution of each module to the recognition performance.

[0050] Table 2 Comparison of ablation test results

[0051] Analysis of Table 2 shows that the model performance decreased to varying degrees after removing any module. Specifically, removing the physical prior sparse spectral selection resulted in an accuracy decrease of approximately 2.3 percentage points, indicating that the prior constraints of the mold-sensitive bands play an important role in feature discrimination. Removing the early mold index map significantly reduced the recall rate of mild mold samples, verifying the ability of the early mold index map to enhance the weak response of latent mold spots. Removing the band-spatial collaborative gating fusion module reduced the macro-average F1 score by approximately 1.8 percentage points, indicating that the collaborative gating mechanism can effectively focus on key bands and key spatial locations. Removing the particle-level graph attention network increased the misclassification rate of adjacent mold levels, indicating that the propagation of particle adjacency relationships has a positive contribution to the identification of particles in occluded and shadowed areas. Removing the online domain adaptive mechanism reduced the cross-variety test accuracy by approximately 3.3 percentage points, indicating that the domain adaptive alignment feature has an inhibitory effect on environmental and varietal changes.

[0052] VI. Category Recognition Performance Analysis To further quantify and analyze the recognition performance of this method for each category, Table 3 provides detailed numerical values ​​for the confusion matrix of the test set.

[0053] Table 3. Test Set Confusion Matrix

[0054] Analysis of Table 3 shows that this method maintains high recognition accuracy across all four categories. The recognition accuracy for normal samples and severely moldy samples reaches 97.8% and 97.8%, respectively, while the accuracy for mildly moldy samples and moderately moldy samples is 93.8% and 94.8%, respectively. False positives are mainly concentrated between adjacent mold levels. Normal samples are occasionally misclassified as mildly or moderately moldy, and there are a few bidirectional false positives between mildly and moderately moldy samples. The false positive rate is lowest for severely moldy samples. The confusion matrix data above indicates that this method maintains high recognition accuracy across all four categories, and false positives are mainly concentrated between adjacent mold levels, consistent with the physical law of gradual change in mold severity.

[0055] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0056] While the specific embodiments of the present invention have been described above, they are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for identifying moldy feed based on hyperspectral visual feature fusion, characterized in that, The process includes the following: S1, perform simultaneous acquisition of hyperspectral-RGB dual-modal images and physical prior preprocessing on feed samples to obtain sparse physical prior weight vectors. A normalized hyperspectral block, One normalized RGB block; S2, will Each normalized hyperspectral block is sequentially input into a one-dimensional spectral encoder for feature extraction. Each spectral feature map and spectral feature vector will Each normalized RGB block is sequentially input into a two-dimensional spatial encoder for feature extraction. Each spatial feature map and spatial texture feature vector; S3, will Each spectral feature vector and spatial texture feature vector are sequentially input into the band-space collaborative gating fusion module, and after weight calculation, outer product operation, and weighted fusion, the result is obtained. Joint eigenvectors of spatial spectra; The joint feature vectors of the empty spectrum are projected sequentially and input into the granular graph attention network along with the constructed granular adjacency graph. The graph attention update features are obtained through adjacency relationship propagation. S4 inputs the graph attention update features into the mold classification head after global average pooling, and outputs a mold level label calibrated by temperature scaling.

2. The feed mold identification method based on hyperspectral visual feature fusion as described in claim 1, characterized in that: The specific process of S1 includes: Using a pushbroom hyperspectral imaging system and an RGB industrial camera as the acquisition center, feed samples laid on a black low-reflectivity stage were simultaneously imaged, yielding hyperspectral cube data and RGB visual images. Based on the physical mechanisms of increased moisture content, enhanced OH bond absorption, and early color change during the mold growth process of compound feed, a physical prior spectral weight vector was constructed. Sparse spectral selection was then performed on the hyperspectral cube data based on this physical prior spectral weight vector, resulting in a normalized sparse hyperspectral cube and a sparse physical prior spectral weight vector. An early mold index map was constructed based on the reflectance ratio of specific bands in the hyperspectral cube data. This early mold index map was then used as a single-channel prior feature map and stitched with the RGB visual image along the channel dimension to form a four-channel input image. Feed particle mask generation, single-particle instance segmentation, and block cropping were performed on the normalized sparse hyperspectral cube and the four-channel input image to obtain… A normalized hyperspectral block and a normalized RGB block.

3. A method for identifying feed mold based on hyperspectral visual feature fusion as described in claim 1 or 2, characterized in that: The specific process for obtaining the sparse physical prior weight vector is as follows: First, sensitive bands for mold growth were selected based on the physical mechanisms of increased moisture content, enhanced OH bond absorption, and early color change during the mold growth process in compound feed. Second, Gaussian weighted windows were set around each sensitive band to construct a physical prior spectral weight vector. The dimension of the physical prior spectral weight vector was 224, with weights higher than 0.5 at sensitive bands and close to 0 at non-sensitive bands. Third, the physical prior spectral weight vector was multiplied band-by-band by the hyperspectral cube data to obtain a weighted hyperspectral cube. Then, the cubes were sorted from highest to lowest weight and the top weights were selected. Each band is retained as a band, resulting in a sparse hyperspectral cube; Finally, a minimum-max normalization is performed on the sparse hyperspectral cube to obtain a normalized sparse hyperspectral cube; simultaneously, based on the preserved... Each band index is used to extract the corresponding weight from the physical prior spectral weight vector, resulting in a dimension of [missing information]. The sparse physical prior weight vector.

4. The feed mold identification method based on hyperspectral visual feature fusion as described in claim 2, characterized in that: The construction of the early mold index map involves first extracting the 760 nm and 800 nm reflectance images from the hyperspectral cube data and calculating the early mold index at each pixel location; secondly, performing min-max normalization on the early mold index to map its value range to the interval between 0 and 1, thus obtaining the early mold index map; finally, using the early mold index map as a single-channel prior feature map and stitching it with the RGB visual image in the channel dimension to form a four-channel input image.

5. A method for identifying feed mold based on hyperspectral visual feature fusion as described in claim 1 or 2, characterized in that: The A normalized hyperspectral block, The process of obtaining a normalized RGB block includes: First, a single-band grayscale image located at 550 nm in the normalized sparse hyperspectral cube is selected as the reference image. The Otsu's inter-class variance method is used to perform adaptive threshold segmentation on the reference image to obtain an initial binary mask. Second, morphological closing operations and connected component analysis are performed on the initial binary mask to obtain feed particle masks. Third, a distance transform is performed on the feed particle masks, and the watershed algorithm is used to segment the contacting feed particles, resulting in a set of single-particle instance masks, where each mask corresponds to an independent feed particle. Finally, based on the single-particle instance mask set, samples are cropped from the normalized sparse hyperspectral cube and the four-channel input image, respectively. A normalized hyperspectral block and a normalized RGB block.

6. The feed mold identification method based on hyperspectral visual feature fusion as described in claim 1, characterized in that, The specific data processing procedure of S3 includes: firstly, ... Each spectral feature vector and spatial texture feature vector is sequentially input into the band-space co-gated fusion module. The band-gated generation network outputs a physically constrained band-gated vector, and the spatial gated generation network outputs a spatial gated matrix. These are then combined through an outer product operation to generate a band-space joint saliency map. Finally, an adaptive weighted fusion is performed on the spectral and spatial feature maps based on this joint saliency map to obtain the final fusion result. First, a joint feature vector of the empty spectrum is generated; second, a particle adjacency graph is constructed based on the Euclidean distance between the centroids of each particle in the single-particle instance mask set; third, a joint feature vector of the empty spectrum is generated; fourth, a joint feature vector of the empty spectrum is generated; fifth, a joint feature vector of the empty spectrum is generated; sixth, a joint feature vector of the empty spectrum is generated; seventh, a joint feature vector of the empty spectrum is generated; and finally, a joint feature vector of the empty spectrum is The joint feature vectors of the empty spectrum are sequentially projected to 128 dimensions through a fully connected layer to obtain K particle graph node features. Then, the K particle graph node features and the particle adjacency graph are input into the particle-level graph attention network. After graph attention calculation and residual fusion, the output graph attention update features are obtained.

7. The feed mold identification method based on hyperspectral visual feature fusion as described in claim 6, characterized in that: The specific process for obtaining the graph attention update features is as follows: First, construct a particle adjacency graph based on the Euclidean distance between the centroids of each particle in the single-particle instance mask set; second, ... The joint feature vectors of the spatial spectrum are obtained by sequentially projecting them through a fully connected layer. Each particle map node feature, then... Each particle graph node feature and its adjacency graph are input into a granular graph attention network, which consists of two graph attention layers. In the graph attention layer, for adjacent particle pairs with connecting edges in the particle adjacency graph, the attention coefficient is calculated as the LeakyReLU nonlinear mapping result of the inner product of the node features after linear transformation, and then normalized by the Softmax function. The adjacent particle features are then weighted and aggregated based on the normalized attention coefficients. Finally, the output of the second-layer graph attention is added to the residual of the original particle graph node features to obtain the graph attention update features.

8. The feed mold identification method based on hyperspectral visual feature fusion as described in claim 1, characterized in that: The specific process of temperature scaling calibration is as follows: Let the Logit values ​​of the four categories output by the second fully connected layer of the mold classification head be respectively... , , and The temperature scaling parameter is denoted as First, divide each Logit value by... , obtain the scaled Logit value Next, the scaled Logit value is input into the Softmax function to obtain the calibrated mold probability distribution. ; When T is greater than 1, the probability distribution tends to be smoother, the peak confidence decreases, and the model’s overconfidence in the training samples is suppressed. When T equals 1, the temperature scaling degenerates into the standard Softmax function. The temperature scaling parameter T is searched on the validation set by minimizing the cross-entropy loss. The candidate sets are 1.0, 1.2, 1.5 and 2.

0. The T value that minimizes the cross-entropy loss on the validation set is selected as the final inference parameter.

Images