A product ingredient content analysis method and system based on multispectral imaging

Abstract

This application provides a product component content analysis method and system based on multispectral imaging, belonging to the technical field of product component content analysis. The method includes: determining a set of characteristic bands for multispectral imaging based on the type of product to be tested and the component detection requirements, acquiring multispectral images under this set for preprocessing and image quality assessment, generating preprocessed image data and image quality scores; extracting spectral and spatial feature vectors of the region of interest from the preprocessed image data; calculating the first fusion weight of the spectral feature vector and the second fusion weight of the spatial feature vector based on the image quality scores, performing weighted fusion to generate a multimodal fusion feature vector, inputting it into a component content quantitative analysis model to obtain the predicted value of the target component content, generating a component content distribution matrix and performing image segmentation to identify and locate abnormal component regions; fusing the component content distribution matrix with the preprocessed image data to generate a spatial distribution cloud map of the target component.

Description

Technical Field

[0001] This application relates to the technical field of product component content analysis, and in particular to a product component content analysis method and system based on multispectral imaging. Background Technology

[0002] Existing methods for analyzing product composition include chemical analysis and spectroscopic analysis. Chemical analysis requires product sampling and determines composition through complex chemical reactions and experimental procedures. This process is cumbersome, requires specialized equipment and technicians, and has a long analysis cycle, failing to meet the needs of rapid detection. Spectroscopic analysis utilizes the absorption and emission properties of substances at different wavelengths of light to analyze composition. However, existing spectroscopic methods only focus on spectral characteristics, neglecting the spatial characteristics of the product, resulting in incomplete and inaccurate analysis results. The cumbersome operation and long cycle of chemical analysis cannot meet the demands of rapid testing in modern production. Spectroscopic analysis, relying solely on spectral characteristics, cannot utilize the spatial characteristics of the product, cannot accurately identify and locate abnormal regions of composition, and cannot visually display the spatial distribution of components.

[0003] Therefore, there is an urgent need for a product component content analysis method and system based on multispectral imaging. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a method and system for analyzing product component content based on multispectral imaging.

[0005] A first aspect of this application provides a method for analyzing the content of product components based on multispectral imaging, comprising: Based on the type of product to be tested and the requirements for component detection, the set of characteristic bands for multispectral imaging is determined. Acquire multispectral images of the product under test under the set of characteristic bands; The multispectral image is preprocessed and image quality is evaluated to generate preprocessed image data and corresponding image quality scores; The spectral feature vector and spatial feature vector of the region of interest are extracted from the preprocessed image data, respectively. Based on the image quality score, calculate the first fusion weight of the spectral feature vector and the second fusion weight of the spatial feature vector; Based on the first fusion weight and the second fusion weight, the spectral feature vector and the spatial feature vector are weighted and fused to generate a multimodal fusion feature vector; The multimodal fusion feature vector is input into a preset quantitative analysis model for component content to obtain the predicted value of the target component content of the product to be tested. Based on the predicted values ​​of the target component content, a component content distribution matrix is ​​generated, and the component content distribution matrix is ​​segmented to identify and locate abnormal component regions. The component content distribution matrix is ​​fused with the preprocessed image data to generate a spatial distribution cloud map of the target components.

[0006] A second aspect of this application provides a product component content analysis system based on multispectral imaging, comprising: The band determination module is used to determine the set of characteristic bands for multispectral imaging based on the type of the product to be tested and the requirements for component detection. An image acquisition module is used to acquire multispectral images of the product under test under the set of characteristic bands. The processing and evaluation module is used to preprocess the multispectral image and evaluate its image quality, generating preprocessed image data and corresponding image quality scores. The feature extraction module is used to extract the spectral feature vector and spatial feature vector of the region of interest from the preprocessed image data, respectively. The weight calculation module is used to calculate the first fusion weight of the spectral feature vector and the second fusion weight of the spatial feature vector based on the image quality score; The feature fusion module is used to perform weighted fusion of the spectral feature vector and the spatial feature vector according to the first fusion weight and the second fusion weight to generate a multimodal fusion feature vector; The content prediction module is used to input the multimodal fusion feature vector into a preset component content quantitative analysis model to obtain the predicted value of the target component content of the product to be tested. An anomaly localization module is used to generate a component content distribution matrix based on the predicted value of the target component content, and to perform image segmentation on the component content distribution matrix to identify and locate abnormal component regions. The cloud map generation module is used to fuse the component content distribution matrix with the preprocessed image data to generate a target component spatial distribution cloud map.

[0007] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described product component content analysis method based on multispectral imaging.

[0008] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the product component content analysis method based on multispectral imaging.

[0009] The beneficial effects of the product component content analysis method and system based on multispectral imaging provided in this application are as follows: This application determines a suitable set of characteristic bands to acquire multispectral images based on the type of product to be tested and the component detection requirements. After preprocessing and quality assessment, spectral and spatial feature vectors are extracted. The fusion weight of the two is determined according to the image quality score, and a multimodal fusion feature vector is generated. The predicted value of the target component content is obtained using a quantitative analysis model, and then a component content distribution matrix is ​​generated to identify and locate abnormal component areas. Finally, the matrix is ​​fused with the preprocessed image to generate a target component spatial distribution cloud map, thereby achieving accurate analysis, anomaly location, and visualization of the component content of the product to be tested. Attached Figure Description

[0010] Figure 1 A schematic flowchart of a product component content analysis method based on multispectral imaging provided in an embodiment of this application; Figure 2 This is a structural block diagram of a product component content analysis system based on multispectral imaging provided in an embodiment of this application; Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0011] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0012] To make the purpose, technical solution, and advantages of this application clearer, the following will be described in conjunction with the appendix. Figure 1-3 The following is an explanation using specific examples.

[0013] Please refer to Figure 1 , Figure 1 A flowchart illustrating a product component content analysis method based on multispectral imaging provided in an embodiment of this application is shown. The method includes: S101: Based on the type of product to be tested and the requirements for component detection, determine the set of characteristic bands for multispectral imaging.

[0014] In this embodiment, the product to be tested is a target sample that requires component content detection, quality analysis, or quality grading. It can be food, traditional Chinese medicine, industrial products, agricultural products, pharmaceuticals, etc. The product type refers to the category to which the product to be tested belongs, such as grains, meat, tablets, granules, medicinal materials, chemical raw materials, etc. Different types of products have different spectral response characteristics. Component detection requirements are the detection targets specified by the user or the detection task, including the name of the component to be detected, its content range, detection accuracy, distribution requirements, etc.

[0015] In this embodiment, multispectral imaging is an imaging method that simultaneously acquires images of the product under test under multiple discrete feature bands to obtain a three-dimensional data cube that includes both spatial and spectral information. The feature band set is an optimal combination of bands that is sensitive to the target components, can effectively distinguish changes in component content, and is less affected by environmental interference, and is used for subsequent spectral feature extraction and quantitative component analysis.

[0016] S102: Acquire multispectral images of the product under test in the characteristic band set.

[0017] In this embodiment, the characteristic band set is a set of optimal imaging bands that are sensitive to changes in the content of the target component and are less susceptible to interference; these are the dedicated bands for this multispectral imaging. A multispectral image is a set of images acquired separately under multiple characteristic bands, with each image corresponding to one band, forming a three-dimensional data structure (image cube) that includes both spatial and spectral information. Acquisition is performed using a multispectral imaging device under uniform illumination, position, and parameters to image the product under test, obtaining digital image data.

[0018] S103: Perform preprocessing and image quality assessment on multispectral images to generate preprocessed image data and corresponding image quality scores.

[0019] In this embodiment, preprocessing involves a series of corrections, denoising, and standardization processes applied to the original multispectral image. The aim is to eliminate interference from image noise, systematic errors, and uneven illumination, thereby improving image quality and providing reliable data for feature extraction. Image quality assessment uses quantitative indicators to objectively evaluate the multispectral images before and after preprocessing, quantifying indicators such as image sharpness, signal-to-noise ratio, and uniformity to generate quality quantification values ​​that can be used for subsequent weight allocation. Preprocessed image data is multispectral image data that has undergone preprocessing (bad pixel correction, denoising, reflectance calibration, etc.) to eliminate interference, improve consistency, retain valid product information, and remove invalid interference. The image quality score, obtained through a weighted comprehensive evaluation method, is a quantitative score representing the overall quality of the preprocessed image, ranging from 0 to 100 points. It can be further decomposed into sub-scores in the spectral dimension (signal-to-noise ratio) and the spatial dimension (sharpness), providing a basis for weight calculation.

[0020] S104: Extract the spectral feature vector and spatial feature vector of the region of interest from the preprocessed image data.

[0021] In this embodiment, the preprocessed image data is multispectral image data that has undergone preprocessing such as bad pixel correction, reflectance calibration, denoising, and filtering. It has a high signal-to-noise ratio and good consistency, and is used for subsequent feature extraction. The region of interest (ROI) is the portion of the image that truly belongs to the product under test after removing invalid areas such as background, shadows, and highlights, ensuring that features originate solely from the product itself. The spectral feature vector is a set of numerical features extracted from the multispectral data that represent differences in the content of product components, such as absorption peaks, reflectance, spectral derivatives, and principal components. The spatial feature vector is a set of features such as shape, texture, and statistical distribution extracted from the spatial information of the image, used to describe the product's surface structure, uniformity, and texture variations. Within the same ROI, spectral and spatial features are extracted simultaneously to form two independent feature vectors for subsequent multimodal fusion.

[0022] S105: Based on the image quality score, calculate the first fusion weight of the spectral feature vector and the second fusion weight of the spatial feature vector.

[0023] In this embodiment, the image quality score is used to represent a comprehensive rating of the multispectral image quality, further broken down into a spectral signal-to-noise ratio score and a spatial sharpness score, used to determine the reliability of spectral and spatial information. The first fusion weight is calculated based on the spectral signal-to-noise ratio score, representing the importance of the spectral feature vector in subsequent multimodal fusion. The second fusion weight is calculated based on the spatial sharpness score, representing the importance of the spatial feature vector in subsequent multimodal fusion. The fusion weight is a coefficient used to weight the spectral and spatial feature vectors, satisfying the condition: First fusion weight + Second fusion weight = 1. The spectral feature vector is a feature vector extracted from the region of interest, representing component content information. The spatial feature vector is a feature vector extracted from the region of interest, representing product morphology, texture, and structural information.

[0024] S106: Based on the first fusion weight and the second fusion weight, the spectral feature vector and the spatial feature vector are weighted and fused to generate a multimodal fusion feature vector.

[0025] In this embodiment, the first fusion weight is a coefficient calculated from the image quality score, representing the proportion of spectral feature vectors in the fusion, and is positively correlated with the spectral signal-to-noise ratio. The second fusion weight is a coefficient calculated from the image quality score, representing the proportion of spatial feature vectors in the fusion, and is positively correlated with spatial sharpness. The spectral feature vector is a high-dimensional feature vector extracted from multispectral data, representing the product's component content and absorption characteristics. The spatial feature vector is a high-dimensional feature vector extracted from the image, representing the product's shape, texture, and distribution uniformity. The multimodal fusion feature vector is a comprehensive feature vector obtained by weighted fusion of spectral and spatial information, including both component and structural information, and is used as input for subsequent quantitative analysis models.

[0026] S107: Input the multimodal fusion feature vector into the preset component content quantitative analysis model to obtain the predicted value of the target component content of the product to be tested.

[0027] In this embodiment, the preset quantitative analysis model of component content is a regression / prediction model that is pre-trained with a large number of samples and is used to map multimodal features to component content. In this embodiment, the quantitative analysis model of component content adopts the support vector regression model.

[0028] Specifically, the Support Vector Regression (SVR) model used for quantitative analysis of component content consists of a feature mapping layer, a kernel function calculation layer, and a regression output layer. The settings of each layer and its parameters are tailored to the multispectral component detection scenario, as follows: The feature mapping layer maps the input multimodal fusion feature vector to a high-dimensional feature space, with no additional adjustable parameters, only performing the nonlinear transformation of the feature dimension; the kernel function calculation layer is the core of the model, using a radial basis function (RBF) as the kernel function, with the kernel function parameter γ set to 0.01 (optimized through grid search to adapt to the nonlinear correlation characteristics of multimodal features); the regularization parameter C is set to 10.0 to balance model fit and generalization ability, avoiding overfitting; the insensitive loss function parameter ε is set to 0.001 to control the model's tolerance to prediction errors, adapting to the high-precision requirements of component content detection; in addition, the sample weight coefficient is set to 1.0 to ensure a balanced contribution of each batch of samples to model training, and the libsvm solver is used, with an iteration precision set to 1. This makes the solution converge.

[0029] In this embodiment, the target component is the specific component in the product to be tested that needs to be measured in terms of content, such as moisture, sugar, protein, active ingredients, and harmful components. The predicted content value is a numerical value representing the percentage or concentration of the target component in the product, output by the quantitative analysis model of component content, providing basic data for subsequent distribution analysis.

[0030] S108: Based on the predicted values ​​of the target component content, generate a component content distribution matrix, and perform image segmentation on the component content distribution matrix to identify and locate abnormal component regions.

[0031] In this embodiment, the predicted target component content is the component content value corresponding to each pixel / region in the product to be tested. The component content distribution matrix is ​​a two-dimensional matrix formed by arranging the component content at each position on the product surface according to the image coordinates. Each element represents the content value at the corresponding position, indicating the spatial distribution of the component.

[0032] In this embodiment, image segmentation divides the component content distribution matrix into multiple sub-regions with similar attributes based on the numerical value or spatial structure of the content. Abnormal component regions are those with excessively high / low content, uneven distribution, local clustering, or sparse distribution that do not meet normal standards. Identification and localization automatically determines whether a region is abnormal and provides information such as location, area, mean, standard deviation, and centroid coordinates, achieving automatic anomaly detection.

[0033] S109: Fuse the component content distribution matrix with the preprocessed image data to generate a spatial distribution cloud map of the target components.

[0034] In this embodiment, the component content distribution matrix is ​​presented in a two-dimensional matrix form. Each matrix element corresponds to the predicted value of the target component content at a certain pixel within the region of interest of the product under test, intuitively representing the differences in component content distribution across the product space. The preprocessed image data is multispectral image data processed by bad pixel correction, noise reduction, and reflectance calibration, preserving the clear spatial outline and surface details of the product, serving as the underlying image carrier for the distribution cloud map. Fusion involves spatially registering and overlaying the abstract component content values ​​(distribution matrix) with the concrete product image (preprocessed image), ensuring a precise correspondence between the component content distribution and the actual spatial location of the product, achieving a visual combination of image and data. The target component spatial distribution cloud map is the visualized image generated after fusion. Using the preprocessed image as a background, different colors are used to map different component content levels, intuitively presenting the spatial distribution, uniformity, and location of abnormal areas of the target component on the product under test, facilitating quick and intuitive viewing.

[0035] As can be seen from the above, this application determines a suitable set of characteristic bands to acquire multispectral images based on the type of product to be tested and the requirements for component detection. After preprocessing and quality assessment, spectral and spatial feature vectors are extracted. The fusion weight of the two is determined according to the image quality score, and a multimodal fusion feature vector is generated. The predicted value of the target component content is obtained using a quantitative analysis model, and then a component content distribution matrix is ​​generated to identify and locate abnormal component areas. Finally, the matrix is ​​fused with the preprocessed image to generate a spatial distribution cloud map of the target component, thereby achieving accurate analysis, anomaly location, and visualization of the component content of the product to be tested.

[0036] In one embodiment of this application, based on the type of the product to be tested and the requirements for component detection, a set of characteristic bands for multispectral imaging is determined, including: Construct a prior knowledge base, which includes information on characteristic bands, optional characteristic bands, and interference bands corresponding to different product types and target components. Obtain the type identifier of the product to be tested and the target component detection requirements, and query the prior knowledge base based on the type identifier to obtain the initial feature band candidate set; Acquire rapid pre-scanning multispectral images of the product under test, perform spectral analysis on the rapid pre-scanning multispectral images, and identify spectral interference characteristics under the current environment; Based on spectral interference characteristics, severely interfered bands are removed from the initial candidate set of feature bands to obtain the remaining bands, which are then ranked by importance. Based on the preset upper limit of the number of bands and their importance ranking, the optimal subset of feature bands is selected from the remaining bands to form the final set of feature bands.

[0037] In this embodiment, the prior knowledge base is a pre-built and stored database, including characteristic bands, optional bands, and interference bands corresponding to different products and components. Characteristic bands are those that have a clear spectral response to the target component and can effectively distinguish changes in its content. Optional characteristic bands can serve as supplementary bands, possessing some distinguishing ability but not essential. Interference bands are those that are affected by illumination, environment, and background noise, causing signal distortion.

[0038] In this embodiment, the type identifier is a number, name, or label used to uniquely identify the product category and is used for quick lookup of the prior knowledge base. The initial feature band candidate set is a combination of feature bands initially screened from the knowledge base based on the product type. The rapid pre-scanning multispectral image is a preview image quickly acquired across the entire spectrum, used for environmental interference analysis, but not for final component calculation. Spectral analysis involves comparing and calculating spectral curves to identify interference information such as background interference, intensity attenuation, and baseline drift. Spectral interference features are characteristics that affect the accuracy of spectral signals, such as noise, attenuation, and drift introduced by the environment, lighting, and equipment. Bands severely affected by interference are those with low signal-to-noise ratios, susceptible to environmental influences, and unsuitable for quantitative analysis. The importance ranking is based on the bands' ability to distinguish components, signal-to-noise ratio, and stability, sorted from highest to lowest.

[0039] In this embodiment, the upper limit on the number of bands is the maximum number of selectable bands set based on imaging efficiency, computational load, and model input dimensions. The optimal feature band subset is the final set of bands with the strongest discriminative power, minimal interference, and best suited for quantitative component analysis within the quantity limit.

[0040] As can be seen from the above, this embodiment can summarize the band information corresponding to different product types and target components by constructing a prior knowledge base, providing a foundation for subsequent queries; querying the prior knowledge base to obtain an initial candidate set of feature bands can preliminarily determine possible feature bands based on the type of product to be tested; spectral analysis of the rapid pre-scanned multispectral image to identify spectral interference features can help understand the impact of the current environment on the spectrum; removing and sorting severely interfered bands can eliminate interference factors, making the remaining bands more representative; selecting the optimal subset of feature bands as the final feature band set can make the feature band set accurate and efficient, improving the accuracy and reliability of subsequent product component content analysis.

[0041] In one embodiment of this application, a rapid pre-scan multispectral image of the product under test is acquired, and spectral analysis is performed on the rapid pre-scan multispectral image to identify spectral interference characteristics in the current environment, including: Acquire rapid pre-scanning multispectral images of the product under test, and extract the spectral curves of the background region in the rapid pre-scanning multispectral images as the environmental background spectrum; The environmental background spectrum is compared with a pre-stored standard environmental spectrum library to calculate spectral angle similarity and spectral correlation coefficient, thereby identifying the type of environmental background. Extract the spectral curve of the region of interest from the rapid pre-scan multispectral image as the preliminary spectrum of the product; The intensity difference between the ambient background spectrum and the standard ambient spectrum is calculated to generate the attenuation coefficients for each band. Calculate the baseline shift between the preliminary spectrum of the product and the spectrum of the standard product; Based on the type of environmental background, attenuation coefficient, and baseline drift, the spectral interference characteristics under the current environment are comprehensively determined, and the signal-to-noise ratio influence coefficients corresponding to each band are calculated.

[0042] In this embodiment, the background area is the region in the image that does not belong to the product under test but belongs to the environment / carrier / substrate. The spectral curve is the reflectance / intensity variation curve of a pixel or region at different wavelengths, representing the spectral response law. The environmental background spectrum is the spectral curve extracted from the background area, representing the spectral characteristics of the current detection environment. The standard environmental spectral library is a pre-established standard spectral database including various typical backgrounds (e.g., whiteboard, black box, tray, paper, etc.). Spectral angle similarity is calculated by treating two spectra as vectors and measuring the angle between them, used to measure the similarity of spectral shapes. The spectral correlation coefficient represents the correlation between the changing trends of two spectral curves, with values ​​between [-1, 1]. The type of environmental background is the background category identified through spectral matching, such as metal tray, white ceramic, black light-absorbing material, conveyor belt, etc.

[0043] In this embodiment, the region of interest is the valid region in the image that belongs to the product under test. The preliminary spectrum of the product is the uncorrected raw spectrum extracted from the product region. The intensity difference is the difference in intensity between the ambient background spectrum and the corresponding standard background spectrum in each band, representing illumination attenuation, optical path loss, etc. The attenuation coefficient is a coefficient calculated based on the intensity difference and used to characterize the degree of signal attenuation in each band.

[0044] In this embodiment, the standard product spectrum is the spectrum of a standard sample measured under known composition and interference-free conditions. Baseline drift is the overall offset between the initial product spectrum and the standard product spectrum, representing systematic errors, uneven illumination, etc. The signal-to-noise ratio (SNR) impact coefficient is a quantified coefficient that assesses the degree of interference in each band and its impact on the SNR after considering environmental background, attenuation, and drift.

[0045] As can be seen from the above, this embodiment can acquire basic data for analysis by acquiring rapid pre-scanning multispectral images; extracting the spectral curve of the background region as the environmental background spectrum and comparing it with the standard environmental spectral library can identify the type of environmental background; extracting the spectral curve of the region of interest as the preliminary spectrum of the product can be used for subsequent analysis; calculating the intensity difference between the environmental background spectrum and the standard environmental spectrum to obtain the attenuation coefficient, and calculating the baseline drift between the preliminary spectrum of the product and the standard product spectrum, thereby comprehensively determining the spectral interference characteristics and calculating the signal-to-noise ratio influence coefficient, which helps to remove severely interfered bands from the initial feature band candidate set, improve the accuracy of the final determined feature band set, and provide a more reliable basis for product component content analysis.

[0046] In one embodiment of this application, extracting spectral feature vectors of regions of interest from preprocessed image data includes: Semantic segmentation is performed on the preprocessed image data to generate a semantic segmentation mask. The semantic segmentation mask is used to identify the region category to which each pixel in the preprocessed image data belongs. The region categories include the product effective region, background region, shadow region, and highlight region. Based on semantic segmentation masking, the effective region of the product is extracted as the region of interest; Obtain the spectral reflectance values ​​of all pixels within the region of interest in each feature band, and construct a three-dimensional spectral data cube. Outlier removal and smoothing filtering are performed on the three-dimensional spectral data cube to obtain the corrected spectral data. Calculate the mean spectrum of the corrected spectral data in each band to generate the average spectral curve; The average spectral curve is processed to extract spectral absorption characteristic parameters, including absorption peak position, absorption depth, absorption width, absorption symmetry, and absorption area. The average spectral curve is transformed by first and second derivatives to extract spectral change rate and spectral curvature features; Independent component analysis algorithm is used to reduce the dimensionality of the corrected spectral data and extract the spectral principal component score vector; The spectral absorption characteristic parameters, spectral rate of change characteristics, spectral curvature characteristics, and spectral principal component score vectors are correlated to generate spectral feature vectors.

[0047] In this embodiment, the preprocessed image data is multispectral image data that has undergone bad pixel correction, reflectance calibration, and denoising. It has a high signal-to-noise ratio and good consistency, serving as the basic input for feature extraction. Semantic segmentation is an image segmentation technique that automatically identifies the category (e.g., product, background) of each pixel in an image through algorithms, achieving precise division of different regions. The semantic segmentation mask is a two-dimensional binary / label matrix generated after semantic segmentation, where each pixel corresponds to a region category label, used to quickly identify and distinguish different regions. Region categories are the image region types divided by semantic segmentation, including valid product regions, background regions, shadow regions, and highlight regions, used to eliminate invalid regions and retain valid product information.

[0048] In this embodiment, the effective product region is the area in the image that belongs to the product under test and can be used for component detection, after removing interference areas such as background, shadows, and highlights. The region of interest (ROI) is the core region for feature extraction; here, it is the effective product region extracted from the semantic segmentation mask, ensuring that features originate solely from the product itself. Spectral reflectance value is the ratio of reflected light intensity to incident light intensity at a pixel in a multispectral image at a specific wavelength, representing data indicating the product's component characteristics. The three-dimensional spectral data cube is a data set formed by organizing the spectral reflectance values ​​of all pixels within the ROI at various feature wavelengths according to a three-dimensional structure of spatial coordinates (x, y) + wavelength bands, including both spatial and spectral information.

[0049] In this embodiment, outlier removal involves identifying and deleting abnormal pixels (e.g., noise points, abnormal reflection points) that deviate from the normal range in the three-dimensional spectral data cube to avoid interfering with feature extraction. Smoothing filtering is applied to the spectral data after outlier removal to suppress random noise, resulting in a smoother spectral curve and preserving effective spectral features. The corrected spectral data, after outlier removal and smoothing filtering, is the more reliable three-dimensional spectral data, free from interference, and is used for spectral feature extraction.

[0050] In this embodiment, the mean spectrum is a single spectral curve obtained by calculating the average reflectance of all pixels in each characteristic band of the corrected spectral data, representing the overall spectral characteristics of the product. The average spectral curve is a curve plotted with the band as the horizontal axis and the average reflectance as the vertical axis, visually presenting the spectral response of the product in different bands. Spectral absorption characteristic parameters are quantitative parameters extracted from the average spectral curve that represent the absorption characteristics of the components, including absorption peak position, absorption depth, absorption width, absorption symmetry, and absorption area, which are directly related to the content of product components.

[0051] In this embodiment, the first and second derivative transformations are mathematical transformations of the average spectral curve. The first derivative represents the rate of change (slope) of the spectral curve, and the second derivative represents the curvature of the spectral curve, used to enhance absorption peak characteristics and distinguish overlapping absorption peaks. The spectral rate of change feature is extracted from the first derivative of the average spectral curve and represents how quickly the spectral reflectance changes with the wavelength, used to enhance the boundary characteristics of component absorption peaks. The spectral curvature feature is extracted from the second derivative of the average spectral curve and represents the degree of curvature of the spectral curve, used to identify weak absorption peaks and distinguish spectral differences between similar components.

[0052] In this embodiment, Independent Component Analysis (ICA) is a data dimensionality reduction algorithm used to separate independent feature components from high-dimensional spectral data, eliminate redundant information, and simplify feature dimensions. Dimensionality reduction reduces the dimensionality of spectral data, thereby reducing subsequent computational load and avoiding overfitting while preserving effective features. The spectral principal component score vector, obtained after dimensionality reduction via ICA, is a low-dimensional vector representing the core information of the spectrum and is an important component of spectral features.

[0053] Specifically, the Independent Component Analysis (ICA) algorithm is divided into a data preprocessing layer, an independent component decomposition layer, and a principal component screening layer. Each layer and its parameters are strictly adapted to multispectral component detection scenarios. The specific settings are as follows: Data Preprocessing Layer: No algorithm layer parameters are used. Only the input corrected spectral data is centered (the spectral value of each band is subtracted from the mean of that band) to eliminate the influence of baseline offset on the decomposition results; Independent Component Decomposition Layer: The FastICA algorithm (fast independent component analysis, the mainstream implementation method in industrial scenarios) is adopted. The core parameter settings are: Independence Criterion: Negentropy is selected as the non-Gaussianity measure to adapt to the non-Gaussian distribution characteristics of spectral data; Iterative Optimization Algorithm: The fixed-point iteration method is adopted, with the iteration step size set to 1.0 and the iteration accuracy threshold set to 1. (Iteration stops when the change in the weight matrix between two consecutive iterations is less than the iteration accuracy threshold); Maximum number of iterations: set to 1000 times to avoid non-convergence of the algorithm; Principal component screening layer: set the principal component contribution rate threshold to 95%, that is, select independent components with a cumulative contribution rate greater than or equal to 95% as principal components to generate spectral principal component score vectors; if the cumulative contribution rate of the first k independent components reaches 95%, the dimension of the principal component score vector is k (for example, k=5, the vector dimension is 5).

[0054] In this embodiment, feature association involves splicing and integrating different types of spectral features (absorption parameters, rate of change, curvature, principal components) to form a complete feature set. The spectral feature vector is the final generated high-dimensional vector that includes all spectrally related features, capable of comprehensively representing the spectral characteristics of the product components, and is used for subsequent multimodal fusion and quantitative component analysis.

[0055] As can be seen from the above, this embodiment accurately extracts the effective region of the product as the region of interest by using semantic segmentation to generate a mask, constructs a three-dimensional spectral data cube, and performs outlier removal and smoothing filtering to obtain corrected spectral data. Then, the mean spectrum is calculated to generate the average spectral curve, and spectral absorption feature parameters, spectral rate of change features, and spectral curvature features are extracted. The independent component analysis algorithm is used to reduce the dimensionality and extract the spectral principal component score vector. Finally, these features are associated to generate a spectral feature vector. This can effectively extract spectral features that represent the component information of the product to be tested, providing a reliable data foundation for accurately analyzing the component content of the product.

[0056] In one embodiment of this application, spatial feature vectors of regions of interest are extracted from preprocessed image data, including: Based on semantic segmentation mask, obtain the boundary contour information of the effective area of ​​the product; Perform Fourier descriptor transform on the boundary contour of the effective area of ​​the product to extract contour shape features; Based on the three-dimensional spectral data cube, the image channels corresponding to the characteristic absorption bands of the target components are selected as the reference images for spatial analysis. The gray-level co-occurrence matrix is ​​calculated on the spatial analysis benchmark image to extract texture features within the effective area of ​​the product; texture features include energy, contrast, correlation, entropy, inverse moment, and variance; Local binary mode transform is performed on the spatial analysis benchmark image to extract local texture micro-pattern features; Based on the spectral reflectance values ​​of each pixel within the effective area of ​​the product in the characteristic absorption band, the statistical characteristics of the reflectance values ​​are calculated; the statistical characteristics include the maximum value, minimum value, mean, standard deviation, skewness, and kurtosis. Feature association is performed on contour shape features, texture features, local texture micro-pattern features and statistical features to generate spatial feature vectors.

[0057] In this embodiment, the semantic segmentation mask is a generated two-dimensional label matrix used to identify the region category (product effective region, background region, etc.) to which each pixel in the image belongs, providing a basis for extracting the product effective region. The product effective region is the core region of the product itself, marked in the semantic segmentation mask, which eliminates invalid regions such as background, shadows, and highlights, and is the only object for spatial feature extraction. Boundary contour information is the edge contour data of the product effective region, representing the spatial morphological features of the product such as shape, size, and boundary orientation, and is the basis for extracting shape features.

[0058] In this embodiment, the Fourier descriptor transform is an image shape feature extraction algorithm. By performing a Fourier transform on the product boundary contour, the spatial shape of the contour is transformed into a series of quantized parameters (Fourier descriptors) to characterize the overall shape of the contour. The contour shape features are features extracted by the Fourier descriptor transform that characterize the shape of the effective area of ​​the product, representing the overall contour, symmetry, complexity, etc. of the product.

[0059] In this embodiment, the three-dimensional spectral data cube is a three-dimensional data set organized according to spatial coordinates (x, y) + characteristic bands, including both spatial and spectral information of the product. The characteristic absorption bands of the target component are bands exhibiting significant spectral absorption characteristics (selected from the set of characteristic bands determined in S101), and the images of these bands can represent the spatial distribution differences of the target component. Image channels are independent image layers corresponding to individual characteristic bands in a multispectral image, with each channel corresponding to the image information of one characteristic band. The spatial analysis reference image is selected from the three-dimensional spectral data cube, using the image channel corresponding to the characteristic absorption band of the target component as a reference, for subsequent extraction of spatial texture and statistical features, ensuring that the extracted spatial features are correlated with the target component.

[0060] In this embodiment, the gray-level co-occurrence matrix (GLCM) is a matrix used to extract image texture features. By statistically analyzing the co-occurrence probability of pixels with different gray values ​​in an image at a specific direction and distance, it represents the texture coarseness, uniformity, and other characteristics of the image. Texture features are quantitative features extracted from a spatial analysis benchmark image that represent the surface texture structure of a product. These features include energy, contrast, correlation, entropy, inverse moment, and variance, and are used to describe the roughness, texture uniformity, etc., of the product surface.

[0061] In this embodiment, Local Binary Pattern Transform (LBP) is an algorithm for extracting local textures in an image. By comparing the grayscale values ​​of each pixel with those of its neighboring pixels, it generates local binary codes to capture subtle texture patterns on the product surface. Local texture micro-pattern features, extracted through LBP, characterize the subtle texture structure of the product surface and can capture local details that cannot be covered by macroscopic textures, such as contour shapes. Spectral reflectance values ​​are the reflectance data of each pixel within the effective area of ​​the product in the feature absorption band, representing the spectral response characteristics at that location and are related to the content of the target component. Statistical features are quantitative features obtained by statistically calculating the spectral reflectance values ​​of each pixel within the effective area of ​​the product, including maximum, minimum, mean, standard deviation, skewness, and kurtosis, representing the distribution pattern, dispersion, and distribution shape of reflectance.

[0062] In this embodiment, feature association involves splicing and integrating different types of spatial features (contour shape, texture, local micro-patterns, statistics) to form a complete set of spatial features, thus comprehensively covering the spatial information of the product. The spatial feature vector is the final generated high-dimensional vector that includes all spatially related features, which can comprehensively represent the spatial characteristics of the product, such as its shape, surface texture, and reflectivity distribution, and is used for subsequent fusion weight calculation and multimodal fusion.

[0063] As can be seen from the above, this embodiment can accurately obtain the boundary contour information of the effective area of ​​the product by using a semantic segmentation mask. The contour shape features extracted by Fourier descriptor transform can represent the product's shape characteristics. By selecting the image channel corresponding to the absorption band of the target component feature as the reference image, texture features and local texture micro-mode features are extracted by using gray-level co-occurrence matrix calculation and local binary mode transformation, which helps to grasp the micro-texture features of the product. Statistical features are calculated based on the spectral reflectance values ​​of pixels in the effective area of ​​the product, which can quantify the reflectance distribution characteristics. Finally, multiple features are associated to generate a spatial feature vector, which can comprehensively present the spatial structure and attributes of the product, and help to more accurately analyze the product's component content and distribution.

[0064] In one embodiment of this application, calculating a first fusion weight of the spectral feature vector and a second fusion weight of the spatial feature vector based on the image quality score includes: The image quality score is decomposed into spectral signal-to-noise ratio score and spatial sharpness score; Based on the spectral signal-to-noise ratio score, the first fusion weight of the spectral feature vector is calculated using a preset continuous weighting function; Based on the spatial clarity score, the second fusion weight of the spatial feature vector is calculated through a preset continuous weight function; The sum of the first fusion weight and the second fusion weight is 1. If the spectral signal-to-noise ratio score is less than the first failure threshold, or the spatial sharpness score is less than the second failure threshold, the current image is determined to be a failed image, and a re-acquisition command is triggered.

[0065] In this embodiment, the spectral signal-to-noise ratio (SNR) score is a sub-score derived from the image quality score, representing only the spectral dimension quality. It indicates the ratio of spectral signal to noise and is directly related to the reliability of the spectral feature vector. The spatial sharpness score is a sub-score derived from the image quality score, representing only the spatial dimension quality. It indicates the sharpness of spatial details (contours, textures) in the image and is directly related to the reliability of the spatial feature vector. The preset continuous weighting function is a predefined continuous function that takes the quality score as input and outputs weight values ​​between 0 and 1, such as a sigmoid function or a linear function. It is used to convert the quality score into corresponding fusion weights, enabling dynamic adjustment of the weights according to the quality. The first fusion weight is the weight coefficient corresponding to the spectral feature vector, calculated from the spectral SNR score using the continuous weighting function. A larger value indicates a higher proportion of the spectral feature vector in the fusion. The second fusion weight is the weight coefficient corresponding to the spatial feature vector, calculated from the spatial sharpness score using the continuous weighting function. A larger value indicates a higher proportion of the spatial feature vector in the fusion.

[0066] In this embodiment, the first failure threshold is a pre-set lower limit for the spectral signal-to-noise ratio (SNR) score. A value below this threshold indicates excessive noise in the spectral signal, making the spectral feature vector unreliable. The second failure threshold is a pre-set lower limit for the spatial sharpness score. A value below this threshold indicates blurred spatial details in the image, making the spatial feature vector unreliable. A failed image is one whose spectral SNR score is less than the first failure threshold or whose spatial sharpness score is less than the second failure threshold. Its feature extraction results are unreliable and cannot be used for subsequent fusion and analysis. The re-acquisition command is an automatically triggered command by the system when an image is determined to be a failed image. It is used to re-acquire the multispectral image of the product under test, ensuring that subsequent analysis is based on high-quality data.

[0067] Specifically, the calculation process for the first and second failure thresholds is as follows: First, construct a calibration sample set: select 100 standard samples with known true component content, covering different quality levels with different spectral signal-to-noise ratios and spatial sharpness; second, acquire multispectral images for each standard sample, preprocess them, calculate the spectral signal-to-noise ratio score and spatial sharpness score, and input them into the component content quantitative analysis model to obtain the predicted value; then, calculate the absolute error of component content prediction for each sample, and set the maximum allowable error. =0.5%; subsequently, those with prediction errors less than or equal to The minimum spectral signal-to-noise ratio score among the samples is taken as the first failure threshold, and the minimum spatial clarity score is taken as the second failure threshold. Finally, the 95% confidence lower limit of the threshold is taken as the final effective first failure threshold and second failure threshold. For example, the first failure threshold is set to 28 points and the second failure threshold is set to 25 points.

[0068] As can be seen from the above, this embodiment decomposes the image quality score into a spectral signal-to-noise ratio score and a spatial sharpness score. It can calculate the fusion weights of the spectral feature vector and the spatial feature vector based on these two scores using a preset continuous weighting function. This makes the fusion weight allocation more reasonable and ensures that the sum of the first fusion weight and the second fusion weight is 1, thus ensuring the effectiveness of feature fusion. At the same time, by setting a failure threshold, it is possible to promptly identify failed images and trigger a re-acquisition command, ensuring image quality and the accuracy of subsequent analysis.

[0069] In one embodiment of this application, a multimodal fusion feature vector is generated by weighted fusion of the spectral feature vector and the spatial feature vector according to a first fusion weight and a second fusion weight, including: The spectral feature vector is normalized to obtain the standard spectral feature vector. The spatial feature vectors are standardized to obtain standard spatial feature vectors. Multiply the first fusion weight by the standard spectral feature vector to obtain the weighted spectral feature vector; Multiply the second fusion weight by the standard space feature vector to obtain the weighted space feature vector; Feature correlation is performed on the weighted spectral feature vector and the weighted spatial feature vector to generate a primary fusion feature vector; The primary fusion feature vector is input into the cross-modal attention mechanism network. The weighted spectral feature vector is used as the query matrix, and the weighted spatial feature vector is used as the key matrix and value matrix. Attention weights are calculated and attention-enhanced feature vectors are generated. The primary fusion feature vector and the attention-enhanced feature vector are residually concatenated to obtain the multimodal fusion feature vector.

[0070] In this embodiment, feature standardization is a preprocessing operation that normalizes / standardizes the feature vectors, such as Z-score standardization or Min-Max standardization, to eliminate dimensional differences between different feature dimensions, thus ensuring the fairness of weighted fusion. The standard spectral feature vector is the vector obtained after standardizing the spectral feature vectors; its feature values ​​in each dimension are of the same order of magnitude, allowing it to directly participate in the weighted calculation. The standard spatial feature vector is the vector obtained after standardizing the spatial feature vectors; its dimensions are consistent with the standard spectral feature vectors, facilitating fusion calculation.

[0071] In this embodiment, the weighted spectral feature vector is obtained by element-wise multiplication of the standard spectral feature vector with the first fusion weight, highlighting the contribution of spectral features under high weights. The weighted spatial feature vector is obtained by element-wise multiplication of the standard spatial feature vector with the second fusion weight, highlighting the contribution of spatial features under high weights. Feature association is an operation that concatenates / fused two feature vectors, such as vector concatenation or element-wise addition, to generate a new vector containing both types of feature information. The initial fusion feature vector is the initial fusion vector obtained after feature association between the weighted spectral feature vector and the weighted spatial feature vector, initially integrating spectral and spatial information.

[0072] In this embodiment, the cross-modal attention mechanism network is a deep learning network module that can automatically identify information more important for component analysis in different modal features (spectral / spatial), strengthening key features and suppressing redundant information. The query matrix is ​​used in the attention mechanism to retrieve key information; here, it is a matrix transformed from weighted spectral feature vectors. The key matrix is ​​used in the attention mechanism to match query information; here, it is a matrix transformed from weighted spatial feature vectors. The value matrix is ​​used in the attention mechanism to output matched information; here, it is a matrix transformed from weighted spatial feature vectors.

[0073] In this embodiment, attention weights are coefficients calculated by a cross-modal attention mechanism, representing the importance of different feature dimensions, and are used to enhance key features. The attention-enhanced feature vector is a vector obtained by weighting the primary fused feature vector through the attention mechanism, highlighting feature information valuable for component analysis. Residual connection: This operation adds the original feature vector to the enhanced feature vector, preserving the original feature information while superimposing the attention enhancement effect, avoiding feature loss. The multimodal fused feature vector is the final generated high-dimensional vector that fuses spectral and spatial features and is attention-enhanced, possessing the advantages of both types of features, and is used for subsequent quantitative analysis of component content.

[0074] As can be seen from the above, this embodiment can eliminate the influence of different dimensions between spectral feature vectors and spatial feature vectors by performing feature standardization processing, making the features comparable; multiplying the first fusion weight and the second fusion weight with the standard spectral feature vector and the standard spatial feature vector respectively can reasonably allocate the proportion of spectral and spatial features according to image quality; performing feature association on the weighted feature vectors to generate primary fusion feature vectors can initially integrate spectral and spatial feature information; using a cross-modal attention mechanism network, with the weighted spectral feature vector as the query matrix and the weighted spatial feature vector as the key matrix and value matrix, attention weights are calculated and attention-enhanced feature vectors are generated, which can focus on key information and improve feature expression ability; performing residual connection between the primary fusion feature vector and the attention-enhanced feature vector to obtain a multimodal fusion feature vector helps to alleviate the gradient vanishing problem, improve model training efficiency and performance, and finally generate a more accurate and effective multimodal fusion feature vector for product component content analysis.

[0075] In one embodiment of this application, image segmentation is performed on the component content distribution matrix to identify and locate abnormal component regions, including: An adaptive threshold segmentation algorithm is used to segment the component content distribution matrix, classifying pixels into different component level regions according to their content values ​​to obtain preliminary segmentation regions; or, The product edge gradient map is calculated based on preprocessed image data. The watershed algorithm is used to segment the product edge gradient map to obtain a preliminary segmentation region based on morphology. This region is then mapped to the component content distribution matrix. Based on the mean content and standard deviation of each initial sub-region, a region merging algorithm is used to merge the initial sub-regions whose mean content difference is less than the preset merging threshold and are spatially adjacent, to obtain the final segmented region. Calculate the mean content, standard deviation of content, area of ​​each final segmented region, and coordinates of the centroid of the region. The regions with a mean content greater than the preset normal content range and a standard deviation of content less than the preset uniformity threshold are marked as regions with abnormal component uniformity. Regions with a mean content within a preset normal content range and a standard deviation of content greater than a preset uniformity threshold are marked as regions with uneven component distribution. Regions with a mean content greater than the preset normal content range and a standard deviation greater than the preset uniformity threshold are marked as severely abnormal regions. Calculate the total number of various abnormal areas, their total area percentage, and their relative position to the product boundary, and generate an abnormal area analysis report.

[0076] In this embodiment, the component content distribution matrix represents the target component content at each location of the product under test in two-dimensional matrix form, with matrix element values ​​corresponding to the content magnitude. Image segmentation divides the component content distribution matrix into multiple sub-regions with similar attributes based on component content values ​​or product morphology. Abnormal component regions are areas with excessively high / low content, uneven distribution, local clustering, or sparse distribution, failing to meet normal quality standards. The adaptive threshold segmentation algorithm automatically calculates a segmentation threshold based on the content value distribution within the matrix, separating regions of different content levels; it is suitable for segmentation based on content values. Component level regions are different levels of regions divided according to content magnitude. The initial segmented regions are the initial sub-regions obtained through adaptive thresholding or watershed algorithms, without region merging.

[0077] Specifically, the adaptive threshold segmentation algorithm for component content distribution matrix segmentation consists of a data preprocessing layer, a threshold adaptive calculation layer, and a pixel classification segmentation layer. Each layer and its parameters are strictly adapted to the numerical characteristics of the component content distribution matrix. The specific settings are as follows: The data preprocessing layer has no additional adjustable parameters and only performs normalization processing on the input component content distribution matrix (mapping the content values ​​to the [0,255] interval) to adapt to the numerical range of the threshold calculation; the threshold adaptive calculation layer adopts the Otsu algorithm (maximum inter-class variance method, adapted to the bimodal distribution characteristics of component content). The core parameter settings are: the inter-class variance calculation window size is set to 3×3 (to match the local continuity of the component content distribution), and the iteration precision threshold is set to 1. (Iteration stops when the difference between the inter-class variances calculated in two consecutive iterations is less than the iteration accuracy threshold). The maximum number of iterations is set to 100 to avoid the algorithm from failing to converge. Pixel classification segmentation layer parameters: Set three component level thresholds (low content threshold T1, medium content threshold T2, and high content threshold T3). T1 = 1 / 2 of the threshold when the inter-class variance is minimum, T2 = the threshold when the inter-class variance is minimum, and T3 = 1.5 times the threshold when the inter-class variance is minimum. Divide the pixels into three component level regions of low, medium, and high content according to their content values ​​to generate the initial segmentation region.

[0078] In this embodiment, the product edge gradient map is a gradient image calculated from the preprocessed image, representing the changes in the product's edges and contours. The watershed algorithm is an algorithm for segmentation based on image edges and gradient information, suitable for region segmentation according to product shape and contour. Mapping maps the segmented regions obtained based on shape to the same spatial location in the component content distribution matrix.

[0079] Specifically, the watershed algorithm for product edge gradient map segmentation consists of a gradient map generation layer, a marker extraction layer, and a watershed segmentation layer. Each layer and its parameters are strictly adapted to the morphological features of the preprocessed image data. The specific settings are as follows: The gradient map generation layer uses the Sobel operator to calculate the product edge gradient. The parameters are: the operator kernel size is 3×3, the gradient calculation direction is bidirectional (x and y), the gradient magnitude fusion weight is 1:1 (the gradients in the x and y directions are equally weighted), and the gradient values ​​are normalized to the [0, 255] interval. The core parameters of the marker extraction layer are: the foreground marker threshold is set to the 20th percentile of the gradient map grayscale value (to select low-gradient product interior areas as foreground), the background marker threshold is set to the 80th percentile of the gradient map grayscale value (to select high-gradient background / edge areas as background), and the minimum marker area is set to 50 pixels (to remove isolated small markers and avoid over-segmentation). The parameters of the watershed segmentation layer are: the iteration stop threshold is set to 1. (Stop when the water level change between adjacent iterations is less than this value), the maximum number of iterations is set to 500, and the water area merging threshold is set to 5 (merge adjacent water areas when the gradient difference is less than 5 to reduce over-segmentation).

[0080] In this embodiment, the initial sub-region is a collection of small, scattered regions obtained after preliminary segmentation. The mean content is the average content of all pixels within a sub-region, representing the overall content level of that region. The standard deviation of content represents the dispersion of content within a sub-region; a larger value indicates a more uneven distribution. The region merging algorithm merges spatially adjacent and similar sub-regions with similar content into a unified region, reducing over-segmentation. The preset merging threshold is the upper limit of the content difference used to determine whether two regions are similar; regions with content differences less than this threshold are merged. The final segmented region is the stable and reasonable region division result obtained after segmentation and merging.

[0081] In this embodiment, the area segmentation refers to the number of pixels occupied by the region, representing the size of the abnormal region. The centroid coordinates of the region are the coordinates of the center point of the region, used to locate the abnormal position. The preset normal content range is the acceptable content range of the target component, determined by prior knowledge or standard samples. The preset uniformity threshold is the upper limit of the standard deviation for judging whether the content within a region is uniform; if it is greater than this, it is considered non-uniform. A component-uniform abnormal region is a region where the overall content is too high or too low, but the internal distribution is uniform. A component-distribution-uneven region is a region where the content is within the normal range, but the internal fluctuations are large and uneven. A severely abnormal region is a region where the content is outside the normal range and the internal distribution is extremely uneven. The relative positional relationship with the product boundary describes the location of the abnormal region within the product, such as the edge or corner. The abnormal region analysis report includes a quantitative statistical report of the abnormal type, quantity, area, proportion, and location.

[0082] Specifically, the calculation process for the preset normal content range is as follows: Select 100 qualified standard samples, use the gold standard method to determine the true content of the target component, calculate the content mean μ and standard deviation σ, determine the normal range as [μ-3σ,μ+3σ] based on the 3σ principle, and correct it based on industry standards.

[0083] The calculation process for the preset uniformity threshold is as follows: Select 50 qualified samples with uniform component distribution, calculate the standard deviation of their content distribution sub-regions, take the 95th percentile and multiply by an engineering margin of 1.1 to obtain the preset uniformity threshold.

[0084] The calculation process for the preset merging threshold is as follows: Calculate the difference in the mean content of adjacent initial sub-regions of qualified samples, take the 90th percentile as the preset merging threshold, and for gradient-type products, the threshold can be finely adjusted according to the direction. After verifying the merging effect, determine the final value.

[0085] As can be seen from the above, this embodiment can identify and locate abnormal regions with uniform component distribution, uneven component distribution, and severe abnormal regions by performing image segmentation on the component content distribution matrix. It can also generate an abnormal region analysis report that includes the total number of various abnormal regions, the total area ratio, and the relative positional relationship with the product boundary, which can more accurately analyze the abnormality of product component content.

[0086] In one embodiment of this application, the component content distribution matrix is ​​fused with preprocessed image data to generate a target component spatial distribution cloud map, including: The component content distribution matrix is ​​registered with the preprocessed image data to obtain the registered component content distribution matrix. Based on a preset color mapping table, the registered component content distribution matrix is ​​converted into RGB color values ​​to generate an initial spatial distribution cloud map; Content scale bars and statistical information are overlaid on the initial spatial distribution cloud map to generate a spatial distribution cloud map of the target component.

[0087] In this embodiment, image registration aligns the component content distribution matrix with the preprocessed image data in terms of spatial coordinates, size, and position, ensuring that each content value corresponds to the correct product location. The registered component content distribution matrix, after coordinate alignment and size standardization, is a content matrix that corresponds perfectly one-to-one with the preprocessed image. A preset color mapping table is a pre-defined table of color-to-content correspondences, such as blue for low content, green for medium content, and red for high content, used to convert numerical values ​​into colors. RGB color values ​​are color values ​​composed of red, green, and blue channels, used to render the content matrix into a color image.

[0088] In this embodiment, the initial spatial distribution cloud map is a component distribution image generated solely from the colorization of the content matrix, without any overlay of text, scales, or other information. The content scale bar is a color bar that displays the correspondence between colors and content, used to intuitively read the content ranges represented by different colors. The statistical information is the content statistical result of this test, including: maximum content, minimum content, mean content, and standard deviation of content. The target component spatial distribution cloud map is the final generated color visualization image, simultaneously displaying the product shape, component spatial distribution, content levels, location of abnormal areas, and statistical indicators.

[0089] As can be seen from the above, this embodiment can make the component content distribution matrix and the preprocessed image data correspond accurately by performing image registration. The initial spatial distribution cloud map generated by converting the color mapping table to RGB color values ​​can intuitively present the component content distribution. The content scale bar and statistical information are superimposed on the initial spatial distribution cloud map, which can facilitate users to read the component content information and perform analysis. Finally, a cloud map that can clearly show the spatial distribution of the target component is generated.

[0090] In one embodiment of this application, preprocessing and image quality assessment are performed on a multispectral image to generate preprocessed image data and corresponding image quality scores, including: The original multispectral image was subjected to defect detection and correction. Median filtering combined with spectral domain correlation analysis was used to identify and repair abnormal pixels, resulting in a corrected image. The reflectance of the corrected image is calibrated, and the spectral reflectance values ​​of each band are calculated based on the reference spectrum of the standard white board. Calculate the gray-level gradient matrix of each band image and extract the image sharpness evaluation factor; Calculate the non-uniformity parameters of images in each band to evaluate the uniformity of the spatial distribution of illumination; Calculate the signal-to-noise ratio of images in each band and evaluate the noise level of the spectral signal; Based on the sharpness evaluation factor, non-uniformity parameter and signal-to-noise ratio, an image quality score is generated through a weighted comprehensive evaluation method, and the image quality score is decomposed into spectral signal-to-noise ratio score and spatial sharpness score.

[0091] In this embodiment, the original multispectral image is the initial image directly output by the camera without any correction processing. Defect pixel detection and correction identifies pixels in the image that have abnormal responses, are malfunctioning, or have excessive noise, and repairs or replaces them. Median filtering is a denoising method used to remove salt-and-pepper noise and isolated noise points. Spectral domain correlation analysis uses the magnitude of spectral correlation between bands to identify abnormal pixels that deviate from normal spectral patterns. Abnormal pixels are pixels with abnormal responses, excessive noise, or damage, which can interfere with subsequent analysis. The corrected image is the image after defect pixel repair and denoising.

[0092] In this embodiment, reflectance calibration converts the original image grayscale values ​​into physically meaningful spectral reflectance, eliminating the influence of illumination and optical path. The standard white board is a standard calibration board with known reflectance close to 100%, used for reflectance calculation. The reference spectrum is the standard reflectance spectrum of the standard white board at various wavelengths. The spectral reflectance value is the ratio of the intensity of reflected light to the intensity of incident light, and is a core physical quantity for component detection.

[0093] In this embodiment, the gray-level gradient matrix is ​​a matrix composed of the gray-level change rates of image pixels, used to evaluate sharpness. The image sharpness evaluation factor is an indicator characterizing the clarity of image contours, textures, and details. The non-uniformity parameter is an evaluation indicator characterizing whether the illumination and darkness distribution of an image is uniform. The uniformity of illumination spatial distribution refers to whether the illumination is stable and consistent at different locations on the product surface.

[0094] In this embodiment, the signal-to-noise ratio (SNR) is the ratio of effective signal to noise, representing the purity of the spectral signal. The noise level of the spectral signal is the degree of interference from random noise and system noise. The weighted comprehensive evaluation method weights multiple evaluation indicators according to preset weights to obtain a comprehensive quality score. The spectral SNR score is a sub-score decomposed from the image quality score, representing the quality of the spectral signal. The spatial sharpness score is a sub-score decomposed from the image quality score, representing the spatial sharpness of the image.

[0095] As can be seen from the above, this embodiment can remove abnormal pixels and improve image quality by performing bad pixel detection and correction on the original multispectral image; calibrating the reflectance of the corrected image can accurately calculate the spectral reflectance values ​​of each band; calculating the gray-level gradient matrix to extract image sharpness evaluation factors can assess image sharpness; calculating non-uniformity parameters to assess the uniformity of illumination spatial distribution can provide an understanding of illumination conditions; calculating the signal-to-noise ratio to assess the noise level of the spectral signal can provide an understanding of signal quality; and generating an image quality score through a weighted comprehensive evaluation method and decomposing it into a spectral signal-to-noise ratio score and a spatial sharpness score can provide quantitative image quality indicators for subsequent analysis.

[0096] In one embodiment of this application, outlier removal and smoothing filtering are performed on a three-dimensional spectral data cube to obtain corrected spectral data, including: A three-dimensional tensor decomposition algorithm is used to perform low-rank sparse decomposition on a three-dimensional spectral data cube, separating low-rank principal component tensors and sparse noise tensors. Based on the statistical distribution of sparse noise tensors, an adaptive thresholding method is used to identify and remove outlier pixels. For the three-dimensional spectral data cube after removing outliers, a joint bilateral filtering algorithm is used for smoothing. The depth dimension weight of the filter kernel is calculated based on the spectral angular distance, and the spatial dimension weight is calculated based on the spatial Euclidean distance. The filtered low-rank principal component tensor and the portion of the removed sparse noise tensor that is less than a preset noise threshold are reconstructed to generate the corrected spectral data.

[0097] In this embodiment, the 3D tensor decomposition algorithm performs matrix / tensor decomposition on a 3D spectral data cube, splitting the data into effective principal components and noise components. Low-rank sparse decomposition is a method of decomposing 3D data into two parts: a low-rank principal component tensor and a sparse noise tensor. The low-rank principal component tensor is the effective signal component representing the true spectral information and spatial distribution patterns of the product; the data is smooth and highly correlated. The sparse noise tensor is a sparse interference component composed of random noise, bad pixels, and abnormal impulses, concentrated on a few pixels.

[0098] In this embodiment, the adaptive thresholding method automatically determines the threshold based on the statistical distribution (mean, standard deviation) of the noise itself to identify abnormal pixels. Abnormal pixels are those whose spectral reflectance deviates from the normal range, affecting feature extraction and quantitative accuracy. The joint bilateral filtering algorithm performs filtering simultaneously in the spatial and spectral domains, effectively removing noise while preserving edge and spectral features. The filter kernel is the neighborhood calculation window used during filtering, including both spatial and spectral band neighborhoods. The depth dimension weight is the filtering weight along the spectral band direction, determined by the spectral angular distance. Spectral angular distance is an indicator of the difference in shape between two spectral curves; a larger distance results in a lower weight, preserving true spectral features.

[0099] In this embodiment, the spatial dimension weight is the filtering weight of the image spatial neighborhood, determined by the spatial Euclidean distance. The spatial Euclidean distance is the spatial coordinate distance between pixels; the closer the distance, the greater the weight. The preset noise threshold is a threshold used to distinguish between weak noise that can be retained and strong anomalies that must be removed. Reconstruction is the process of recombinating the filtered effective principal components with reliable weak noise to generate the final corrected spectral data.

[0100] Specifically, the calculation process for the preset noise threshold is as follows: First, perform three-dimensional tensor low-rank sparse decomposition on the three-dimensional spectral data cube, extract the sparse noise tensor and expand it into a one-dimensional array, and calculate the mean of the array. and standard deviation Secondly, a threshold coefficient k is set according to the imaging environment (k=3 for industrial sites), and then the threshold coefficient is set according to the formula. = +k× Calculate the initial preset noise threshold; then, verify the outlier removal rate: if the removal rate is not within the reasonable range of 0.5%-5%, fine-tune the coefficient k and recalculate the threshold; finally, verify the signal-to-noise ratio and spectral feature integrity of the corrected spectral data to confirm the final preset noise threshold.

[0101] As can be seen from the above, this embodiment uses a three-dimensional tensor decomposition algorithm to perform low-rank sparse decomposition on the three-dimensional spectral data cube, which can separate the low-rank principal component tensor and the sparse noise tensor, making it easier to process. Based on the statistical distribution of the sparse noise tensor, an adaptive threshold method is used to identify and remove outlier pixels, which can remove abnormal interference in the data. A joint bilateral filtering algorithm is used for smoothing, and the depth dimension weight and spatial dimension weight of the filter kernel are calculated according to the spectral angular distance and spatial Euclidean distance, respectively, which can achieve a smoothing effect while preserving spectral features. Finally, the filtered low-rank principal component tensor and the portion of the removed sparse noise tensor that is less than the preset noise threshold are reconstructed to generate corrected spectral data, improve data quality, and provide a more accurate data foundation for product component content analysis.

[0102] Corresponding to the product component content analysis method based on multispectral imaging in the above embodiments, Figure 2 This is a structural block diagram of a product component content analysis system based on multispectral imaging, provided as an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 2 The product component content analysis system 20 based on multispectral imaging includes: a band determination module 21, an image acquisition module 22, a processing and evaluation module 23, a feature extraction module 24, a weight calculation module 25, a feature fusion module 26, a content prediction module 27, an anomaly localization module 28, and a cloud map generation module 29.

[0103] Among them, the band determination module 21 is used to determine the set of characteristic bands for multispectral imaging based on the type of the product to be tested and the requirements for component detection; Image acquisition module 22 is used to acquire multispectral images of the product under test in a set of characteristic bands; The processing and evaluation module 23 is used to preprocess and evaluate the image quality of the multispectral image, and generate preprocessed image data and corresponding image quality scores. Feature extraction module 24 is used to extract the spectral feature vector and spatial feature vector of the region of interest from the preprocessed image data, respectively; The weight calculation module 25 is used to calculate the first fusion weight of the spectral feature vector and the second fusion weight of the spatial feature vector based on the image quality score; The feature fusion module 26 is used to perform weighted fusion of the spectral feature vector and the spatial feature vector according to the first fusion weight and the second fusion weight to generate a multimodal fusion feature vector; The content prediction module 27 is used to input the multimodal fusion feature vector into the preset component content quantitative analysis model to obtain the predicted value of the target component content of the product to be tested. The anomaly localization module 28 is used to generate a component content distribution matrix based on the predicted value of the target component content, and to perform image segmentation on the component content distribution matrix to identify and locate abnormal component regions. The cloud map generation module 29 is used to fuse the component content distribution matrix with the preprocessed image data to generate a spatial distribution cloud map of the target component.

[0104] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the modules in the aforementioned device embodiments, for example... Figure 2 The functions of the following modules are shown: band determination module 21, image acquisition module 22, processing and evaluation module 23, feature extraction module 24, weight calculation module 25, feature fusion module 26, content prediction module 27, anomaly localization module 28, and cloud map generation module 29.

[0105] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0106] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0107] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store device type information.

[0108] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation methods described in any embodiment of the product component content analysis method based on multispectral imaging provided in the embodiments of this application, or they can execute the implementation methods of the electronic devices described in the embodiments of this application, which will not be repeated here.

[0109] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0110] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0111] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0112] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.

[0114] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0115] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0116] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for analyzing the content of product components based on multispectral imaging, characterized in that, include: Based on the type of product to be tested and the requirements for component detection, the set of characteristic bands for multispectral imaging is determined. Acquire multispectral images of the product under test under the set of characteristic bands; The multispectral image is preprocessed and image quality is evaluated to generate preprocessed image data and corresponding image quality scores; The spectral feature vector and spatial feature vector of the region of interest are extracted from the preprocessed image data, respectively. Based on the image quality score, calculate the first fusion weight of the spectral feature vector and the second fusion weight of the spatial feature vector; Based on the first fusion weight and the second fusion weight, the spectral feature vector and the spatial feature vector are weighted and fused to generate a multimodal fusion feature vector; The multimodal fusion feature vector is input into a preset quantitative analysis model for component content to obtain the predicted value of the target component content of the product to be tested. Based on the predicted values ​​of the target component content, a component content distribution matrix is ​​generated, and the component content distribution matrix is ​​segmented to identify and locate abnormal component regions. The component content distribution matrix is ​​fused with the preprocessed image data to generate a spatial distribution cloud map of the target components.

2. The product component content analysis method based on multispectral imaging according to claim 1, characterized in that, The determination of the characteristic band set for multispectral imaging based on the type of product to be tested and the requirements for component detection includes: Construct a prior knowledge base, which includes information on characteristic bands, optional characteristic bands, and interference bands corresponding to different product types and different target components; Obtain the type identifier and target component detection requirements of the product to be tested, and query the prior knowledge base based on the type identifier to obtain an initial feature band candidate set; Acquire rapid pre-scanning multispectral images of the product under test, perform spectral analysis on the rapid pre-scanning multispectral images, and identify spectral interference characteristics under the current environment; Based on the spectral interference characteristics, severely interfered bands are removed from the initial feature band candidate set to obtain the remaining bands and sort them by importance; Based on the preset upper limit of the number of bands and the importance ranking, the optimal subset of feature bands is selected from the remaining bands as the final determined set of feature bands.

3. The product component content analysis method based on multispectral imaging according to claim 2, characterized in that, The process of acquiring a rapid pre-scanning multispectral image of the product under test, performing spectral analysis on the rapid pre-scanning multispectral image, and identifying spectral interference characteristics under the current environment includes: Acquire a rapid pre-scanning multispectral image of the product under test, and extract the spectral curve of the background region in the rapid pre-scanning multispectral image as the environmental background spectrum; The environmental background spectrum is compared with a pre-stored standard environmental spectrum library to calculate spectral angle similarity and spectral correlation coefficient, thereby identifying the type of environmental background. Extract the spectral curve of the region of interest from the rapid pre-scan multispectral image as the preliminary spectrum of the product; Calculate the intensity difference between the background environmental spectrum and the standard environmental spectrum to generate the attenuation coefficient for each band; Calculate the baseline shift between the preliminary spectrum of the product and the spectrum of the standard product; Based on the type of the environmental background, the attenuation coefficient, and the baseline drift, the spectral interference characteristics under the current environment are comprehensively determined, and the signal-to-noise ratio influence coefficient corresponding to each band is calculated.

4. The product component content analysis method based on multispectral imaging according to claim 1, characterized in that, The step of extracting spectral feature vectors of regions of interest from the preprocessed image data includes: The preprocessed image data is semantically segmented to generate a semantic segmentation mask. The semantic segmentation mask is used to identify the region category to which each pixel in the preprocessed image data belongs. The region category includes product effective region, background region, shadow region, and highlight region. Based on the semantic segmentation mask, the effective region of the product is extracted as the region of interest; Obtain the spectral reflectance values ​​of all pixels in the region of interest under each feature band, and construct a three-dimensional spectral data cube; The three-dimensional spectral data cube is subjected to outlier removal and smoothing filtering to obtain corrected spectral data; Calculate the mean spectrum of the corrected spectral data in each band to generate an average spectral curve; The average spectral curve is processed to extract spectral absorption characteristic parameters, including absorption peak position, absorption depth, absorption width, absorption symmetry, and absorption area. The average spectral curve is transformed by first and second derivatives to extract spectral change rate and spectral curvature features; The dimensionality reduction of the corrected spectral data is performed using an independent component analysis algorithm to extract the spectral principal component score vector. The spectral feature vector is generated by performing feature correlation on the spectral absorption characteristic parameters, spectral rate of change characteristics, spectral curvature characteristics, and spectral principal component score vector.

5. The product component content analysis method based on multispectral imaging according to claim 4, characterized in that, The step of extracting spatial feature vectors of regions of interest from the preprocessed image data includes: Based on the semantic segmentation mask, obtain the boundary contour information of the effective area of ​​the product; Perform a Fourier descriptor transform on the boundary contour of the effective area of ​​the product to extract the contour shape features; Based on the three-dimensional spectral data cube, the image channel corresponding to the characteristic absorption band of the target component is selected as the reference image for spatial analysis. The gray-level co-occurrence matrix is ​​calculated on the spatial analysis reference image to extract texture features within the effective area of ​​the product; the texture features include energy, contrast, correlation, entropy, inverse moment, and variance. The spatial analysis reference image is subjected to local binary mode transformation to extract local texture micro-pattern features; Based on the spectral reflectance values ​​of each pixel within the effective area of ​​the product in the characteristic absorption band, the statistical characteristics of the reflectance values ​​are calculated; the statistical characteristics include the maximum value, minimum value, mean, standard deviation, skewness, and kurtosis. The spatial feature vector is generated by associating the contour shape features, texture features, local texture micro-pattern features, and statistical features.

6. The product component content analysis method based on multispectral imaging according to claim 1, characterized in that, The step of calculating the first fusion weight of the spectral feature vector and the second fusion weight of the spatial feature vector based on the image quality score includes: The image quality score is decomposed into a spectral signal-to-noise ratio score and a spatial sharpness score; Based on the spectral signal-to-noise ratio score, the first fusion weight of the spectral feature vector is calculated using a preset continuous weighting function; Based on the spatial clarity score, the second fusion weight of the spatial feature vector is calculated using a preset continuous weighting function; Wherein, the sum of the first fusion weight and the second fusion weight is 1; If the spectral signal-to-noise ratio score is less than the first failure threshold, or the spatial sharpness score is less than the second failure threshold, the current image is determined to be a failed image, and a re-acquisition command is triggered.

7. The product component content analysis method based on multispectral imaging according to claim 1, characterized in that, The step of weighted fusing the spectral feature vector and the spatial feature vector according to the first fusion weight and the second fusion weight to generate a multimodal fusion feature vector includes: The spectral feature vector is subjected to feature normalization processing to obtain a standard spectral feature vector; The spatial feature vectors are subjected to feature standardization processing to obtain standard spatial feature vectors; Multiply the first fusion weight by the standard spectral feature vector to obtain the weighted spectral feature vector; Multiply the second fusion weight by the standard spatial feature vector to obtain the weighted spatial feature vector; The weighted spectral feature vector and the weighted spatial feature vector are correlated to generate a primary fusion feature vector; The primary fusion feature vector is input into the cross-modal attention mechanism network. The weighted spectral feature vector is used as the query matrix, and the weighted spatial feature vector is used as the key matrix and value matrix. Attention weights are calculated and attention-enhanced feature vectors are generated. The primary fusion feature vector and the attention-enhanced feature vector are residually concatenated to obtain the multimodal fusion feature vector.

8. The product component content analysis method based on multispectral imaging according to claim 1, characterized in that, The step of performing image segmentation on the component content distribution matrix to identify and locate abnormal component regions includes: An adaptive threshold segmentation algorithm is used to segment the component content distribution matrix, classifying pixels into different component level regions according to their content values ​​to obtain preliminary segmentation regions; or, The product edge gradient map is calculated based on the preprocessed image data. The watershed algorithm is used to segment the product edge gradient map to obtain a preliminary segmentation region based on morphology. This region is then mapped to the component content distribution matrix. Based on the mean content and standard deviation of each initial sub-region, a region merging algorithm is used to merge the initial sub-regions whose mean content difference is less than the preset merging threshold and are spatially adjacent, to obtain the final segmented region. Calculate the mean content, standard deviation of content, area of ​​each final segmented region, and coordinates of the centroid of the region. The regions where the average content is greater than the preset normal content range and the standard deviation of the content is less than the preset uniformity threshold are marked as regions with abnormal component uniformity. The regions where the average content is within the preset normal content range and the standard deviation of the content is greater than the preset uniformity threshold are marked as regions with uneven component distribution. The regions where the average content is greater than the preset normal content range and the standard deviation of the content is greater than the preset uniformity threshold are marked as severely abnormal regions. Calculate the total number of various abnormal areas, their total area percentage, and their relative position to the product boundary, and generate an abnormal area analysis report.

9. The product component content analysis method based on multispectral imaging according to claim 1, characterized in that, The step of fusing the component content distribution matrix with the preprocessed image data to generate a target component spatial distribution cloud map includes: The component content distribution matrix is ​​image registered with the preprocessed image data to obtain the registered component content distribution matrix. Based on a preset color mapping table, the registered component content distribution matrix is ​​converted into RGB color values ​​to generate an initial spatial distribution cloud map; Content scale bars and statistical information are superimposed on the initial spatial distribution cloud map to generate a spatial distribution cloud map of the target component.

10. A product component content analysis system based on multispectral imaging, characterized in that, include: The band determination module is used to determine the set of characteristic bands for multispectral imaging based on the type of the product to be tested and the requirements for component detection. An image acquisition module is used to acquire multispectral images of the product under test under the set of characteristic bands. The processing and evaluation module is used to preprocess the multispectral image and evaluate its image quality, generating preprocessed image data and corresponding image quality scores. The feature extraction module is used to extract the spectral feature vector and spatial feature vector of the region of interest from the preprocessed image data, respectively. The weight calculation module is used to calculate the first fusion weight of the spectral feature vector and the second fusion weight of the spatial feature vector based on the image quality score; The feature fusion module is used to perform weighted fusion of the spectral feature vector and the spatial feature vector according to the first fusion weight and the second fusion weight to generate a multimodal fusion feature vector; The content prediction module is used to input the multimodal fusion feature vector into a preset component content quantitative analysis model to obtain the predicted value of the target component content of the product to be tested. An anomaly localization module is used to generate a component content distribution matrix based on the predicted value of the target component content, and to perform image segmentation on the component content distribution matrix to identify and locate abnormal component regions. The cloud map generation module is used to fuse the component content distribution matrix with the preprocessed image data to generate a target component spatial distribution cloud map.

Images