A hyperspectral imaging detection method for a new material of a fatty acyl amino acid salt

Abstract

The present application relates to the field of image detection, and more particularly to a hyperspectral imaging detection method for fatty acyl amino acid salt new material. The method comprises: obtaining batch metadata of a sample to be detected and collecting configuration to generate a set of collection parameters, collecting dark field data cubes, white field data cubes and original spectral data cubes of the sample to be detected to generate a collection data package; dark and white correction of the collection data package to generate a standardized spectral matrix and a pixel index; constructing a reference spectral library based on the standardized spectral matrix and the pixel index, and completing label alignment and feature band screening, the reference spectral library containing structure type labels, water content state labels and impurity state labels, generating model parameters and discrimination rules; performing pixel-level reasoning based on the model parameters and the discrimination rules, combining batch metadata to perform drift compensation, and fusing confidence evaluation and spatial consistency constraints to output component distribution maps, index estimation results and detection reports. The present application can effectively enhance the consistency and traceability across batches.

Description

Technical Field

[0001] This invention relates to the field of image detection, specifically to the field of hyperspectral imaging detection, and particularly to a hyperspectral imaging detection method for a novel fatty acyl amino acid salt material. Background Technology

[0002] In the field of image detection, especially hyperspectral imaging detection, existing methods for testing samples of novel fatty acyl amino acid salts typically revolve around acquiring the original spectral data cube of the sample through imaging calibration. This is followed by combining dark-field and white-field data cubes for dark / white correction, and then performing denoising, bad pixel repair, band clipping, and region of interest extraction on the acquired data. This results in a spectral matrix and spatial mapping information that can be used for model training or discrimination, ultimately outputting a component distribution map, index estimation results, and a test report. However, these methods suffer from limitations such as inconsistent acquisition parameter sets, difficulty in incorporating batch metadata into the discrimination process, and difficulty in quantifying pixel-level inference reliability. Existing methods largely rely on fixed acquisition configurations and the repeatability of single imaging calibration acquisitions. Furthermore, under varying acquisition timestamps, ambient temperature, humidity, light source status, and exposure parameters, baseline differences between the reflectance characterization data cube and the standardized spectral matrix can accumulate, and pixel index mapping of the region of interest can become unstable. This makes it difficult to meet the requirement of continuous consistency in the output component distribution map, index estimation results, and test report. For the joint processing of batch metadata, acquisition parameter sets, and feature sets within the same link, existing technologies generally lack unified data caliber constraints between reference spectral library construction, label alignment, feature band selection, and cross-validation. This makes it difficult for the discrimination rules formed during the training dataset and model training phase to maintain consistent input conditions with the acquisition parameter sets of different batches when transferred to the pixel-level inference stage. Consequently, this leads to insufficient traceability of detection report fields and difficulties in comparing batches during production and processing. It also makes it difficult to form a stable output link for abnormal pixel masks at the level of confidence assessment and spatial consistency constraint fusion. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a hyperspectral imaging detection method for novel fatty acyl amino acid salt materials, comprising:

[0004] S100: Obtain batch metadata and acquisition configuration of the sample to be tested, perform fixed imaging geometry processing, and generate acquisition parameter set;

[0005] S200. Based on the acquisition parameter set, perform imaging calibration acquisition processing to generate an original acquisition data packet; the original acquisition data packet includes a dark field data cube, a white field data cube, and the original spectral data cube of the sample to be tested;

[0006] S300. Based on the original acquired data packet, perform dark white correction, bad pixel repair and band clipping processing to generate a reflectance characterization data cube.

[0007] S400. Based on the reflectance characterization data cube, perform noise reduction, scattering correction and baseline correction in a linked process, extract the region of interest, and generate a standardized spectral matrix and pixel index.

[0008] S500: Based on the standardized spectral matrix and the pixel index, perform reference spectral library construction, label alignment and feature band filtering to generate a training dataset and a feature set; the reference spectral library includes structure type labels, water-containing state labels and impurity state labels.

[0009] S600. Based on the training dataset and the feature set, perform model training, cross-validation and discrimination rule setting processes to generate model parameters and discrimination rules;

[0010] S700. Based on the model parameters and the discrimination rules, perform pixel-level inference; perform drift compensation based on the batch metadata, and perform confidence assessment and spatial consistency constraint fusion to output component distribution map, index estimation results and detection report.

[0011] Furthermore, the set of acquisition parameters includes sample identifier, batch number, raw material batch number, process section identifier, acquisition timestamp, ambient temperature, ambient humidity, light source working status, exposure parameters, working distance, field of view, white field reference object identifier, and dark field acquisition condition identifier.

[0012] Furthermore, the fixed imaging geometry includes mounting the hyperspectral camera on the positioning bracket and locking the pitch and yaw angles, mounting the light source on the fixed bracket and locking the incident angle, positioning the sample carrier stage at the working distance, and writing the pitch angle, yaw angle, incident angle, working distance, and field of view into the acquisition parameter set.

[0013] Furthermore, the dark-white correction includes performing dark field subtraction processing on the cubic original spectral data of the sample under test, and performing radiometric normalization processing based on the cubic white field data; the bad pixel repair includes generating a bad pixel mask and performing neighborhood interpolation reconstruction on the bad pixel pixels.

[0014] Furthermore, the denoising, scattering correction, and baseline correction linkage processing includes: performing sliding window smoothing on the cubic reflectance characterization data; selecting a scattering correction branch based on the structure type identifier in the acquired parameter set and generating a scattering correction result; performing morphological baseline fitting on the scattering correction result and generating a baseline correction result; expanding the baseline correction result based on the region of interest to generate the normalized spectral matrix and simultaneously generating the pixel index.

[0015] Furthermore, the construction of the reference spectral library includes establishing an index of spectral library entries based on sample identifiers; each spectral library entry includes a spectral vector, a structure type label, a water content label, an impurity status label, a physicochemical control field, an acquisition parameter set reference identifier, and a preprocessing link identifier.

[0016] Furthermore, the feature band screening includes generating a band scoring table by grouping the reference spectral library according to the structure type label, the water-bearing state label and the impurity state label, and generating the feature set by filtering the band index set based on the band scoring table and the scoring threshold. The feature set carries a feature set version number.

[0017] Furthermore, the drift compensation includes extracting the cumulative working time of the light source, ambient temperature, ambient humidity and exposure parameters from the batch metadata to generate a drift factor vector, and performing drift correction on the feature set of the pixel-level inference input to generate a compensation feature set.

[0018] Furthermore, the confidence assessment includes outputting a category score sequence for each pixel and generating a pixel confidence based on the difference between the maximum score and the second largest score; the spatial consistency constraint fusion includes establishing a neighborhood connected region based on the pixel index, performing neighborhood voting replacement on low-confidence pixels, and generating an abnormal pixel mask.

[0019] Furthermore, the detection report includes sample identifier, batch metadata, acquisition parameter set, preprocessing link identifier, feature set version number, model parameter version number, discrimination rule version number, component distribution map index, index estimation result field, and abnormal pixel mask field.

[0020] The key innovations of this invention include:

[0021] (1) Extract the cumulative working time of the light source, ambient temperature, ambient humidity and exposure parameters from the batch metadata to generate a drift factor vector, and perform drift correction on the feature set of the pixel-level inference input to generate a compensation feature set. Then, use the compensation feature set together with the model parameters and the discrimination rules for the input organization of the pixel-level inference.

[0022] (2) Output the category score sequence of each pixel during the pixel-level inference process, and generate pixel confidence based on the difference between the maximum score and the second largest score. Further establish a neighborhood connected region based on the pixel index, perform neighborhood voting replacement on low confidence pixels and generate an abnormal pixel mask, so that the spatial consistency constraint fusion and the detection report field writing form a connection link.

[0023] (3) Construct the reference spectral library based on the standardized spectral matrix and the pixel index and perform label alignment, wherein each spectral library entry includes a spectral vector, a structure type label, a water-bearing state label, an impurity state label, a physicochemical control field, a collection parameter set reference identifier, and a preprocessing link identifier; and generate a band scoring table by grouping the reference spectral library according to the structure type label, the water-bearing state label, and the impurity state label, and generate the feature set carrying the feature set version number based on the band scoring table and the scoring threshold, thereby generating the training dataset for the model training and the discrimination rule setting.

[0024] The following are its main beneficial effects:

[0025] (1) By applying the drift factor vector to the feature set and generating the compensation feature set, the input caliber of the pixel-level inference is made to correspond with the batch metadata. When the model parameters and the discrimination rules call the compensation feature set, they can work together with the acquisition timestamp, ambient temperature, ambient humidity, light source working status and exposure parameters reflected by the acquisition parameter set, thereby forming a traceable drift compensation basis in the generation chain of the component distribution map, the index estimation result and the detection report.

[0026] (2) By linking the category score sequence, the pixel confidence and the neighborhood connected region, the spatial consistency constraint fusion is triggered by the pixel confidence and performs neighborhood voting replacement on low confidence pixels. At the same time, the abnormal pixel mask is generated and entered into the detection report as a field. The component distribution map index and the abnormal pixel mask field can be verified under the mapping relationship of the pixel index, thereby alleviating the link breakpoint problem when relying only on the pixel-level inference output and lacking uncertainty representation.

[0027] (3) By embedding the acquisition parameter set reference identifier and the preprocessing link identifier in the reference spectral library entries, and generating the band scoring table and the feature set version number after performing the label alignment on the structure type label, the water content label and the impurity status label, the training dataset, the feature set, the model parameters and the discrimination rule maintain a consistent calling relationship under the caliber of the feature set version number, the model parameter version number and the discrimination rule version number, thereby forming an auditable data link with the same caliber between the model training, the cross-validation and the pixel-level inference, reducing the difficulty of discrimination rule adaptation caused by the inconsistency between the spectral vector source and the preprocessing link. Attached Figure Description

[0028] Figure 1This is a schematic flowchart of a hyperspectral imaging detection method for a novel fatty acyl amino acid salt material provided in an embodiment of this application. Detailed Implementation

[0029] In the field of image detection, especially in the field of hyperspectral imaging detection, reference Figure 1 This is a flowchart illustrating a hyperspectral imaging detection method for a novel fatty acyl amino acid salt material provided in an embodiment of the present invention. The process may include at least steps S100-S700:

[0030] S100: Obtain batch metadata and acquisition configuration of the sample to be tested, perform fixed imaging geometry processing, and generate acquisition parameter set;

[0031] S200. Based on the acquisition parameter set, perform imaging calibration acquisition processing to generate an original acquisition data packet; the original acquisition data packet includes a dark field data cube, a white field data cube, and the original spectral data cube of the sample to be tested;

[0032] S300. Based on the original acquired data packet, perform dark white correction, bad pixel repair and band clipping processing to generate a reflectance characterization data cube.

[0033] S400. Based on the reflectance characterization data cube, perform noise reduction, scattering correction and baseline correction in a linked process, extract the region of interest, and generate a standardized spectral matrix and pixel index.

[0034] S500: Based on the standardized spectral matrix and the pixel index, perform reference spectral library construction, label alignment and feature band filtering to generate a training dataset and a feature set; the reference spectral library includes structure type labels, water-containing state labels and impurity state labels.

[0035] S600. Based on the training dataset and the feature set, perform model training, cross-validation and discrimination rule setting processes to generate model parameters and discrimination rules;

[0036] S700. Based on the model parameters and the discrimination rules, perform pixel-level inference; perform drift compensation based on the batch metadata, and perform confidence assessment and spatial consistency constraint fusion to output component distribution map, index estimation results and detection report.

[0037] S100: Obtain batch metadata and acquisition configuration of the sample to be tested, perform fixed imaging geometry processing, and generate acquisition parameter set;

[0038] In this step, the batch metadata of the sample to be tested is received and aggregated by the acquisition control unit of the detection terminal. Its input sources include batch records issued by the production execution system, reading results of the sample's outer packaging label, and on-site environmental sampling results. The batch metadata of the sample to be tested is used to characterize the traceability attributes of the sample within the same batch. This batch metadata includes sample identifier, batch number, raw material batch number, process segment identifier, and acquisition timestamp. The sample identifier is written by a barcode or RFID tag reading module; the batch number and raw material batch number are written by the production record interface; the process segment identifier is written by the process node code; and the acquisition timestamp is written by the acquisition control unit calling the system clock. The acquisition configuration is generated by the configuration management unit of the detection terminal and sent to the acquisition control unit. The acquisition configuration is used to constrain the available range of spectral range, spatial sampling scale, scanning method, and exposure parameters for hyperspectral imaging acquisition. The scanning method includes pushbroom acquisition and area array acquisition. The spectral range and spatial sampling scale are jointly determined by the model parameters and lens parameters of the hyperspectral camera and written into the acquisition configuration. Understandably, the acquisition control unit triggers this step when it receives a new sample identifier or batch number change, and triggers this step to regenerate the acquisition parameter set when there are changes in the ambient temperature, ambient humidity, or light source working status. The change determination is completed by the configuration management unit by comparing the corresponding fields in the most recent acquisition parameter set. When an input field is missing or the format verification fails, the acquisition control unit generates an abnormal record and writes the abnormal identifier into the batch metadata of the sample to be tested, while keeping the acquisition configuration in a locked state until the abnormal record is reviewed and closed, after which the subsequent fixed imaging geometry process begins.

[0039] The fixed imaging geometry is achieved collaboratively by a mechanical positioning component and a geometric verification component. The mechanical positioning component includes a positioning bracket, a fixed bracket, and a sample carrier stage. The geometric verification component includes an angle scale reading module and a distance measurement module. Specifically, the acquisition control unit drives the positioning bracket into a preset installation position. The maintenance personnel install the hyperspectral camera on the positioning bracket and lock it in place. The pitch and yaw angles corresponding to the locking action are acquired by the angle scale reading module and written into the acquisition parameter set. Subsequently, the light source is installed on the fixed bracket and locked in place. The incident angle corresponding to the locking action is acquired by the angle scale reading module and written into the acquisition parameter set. The sample carrier stage is used to carry the sample to be tested and define the coverage position of the sample surface within the field of view. The acquisition control unit moves the sample carrier stage to the positioning point corresponding to the working distance. The working distance is obtained by the distance measurement module by measuring the optical center of the hyperspectral camera and the reference surface of the sample carrier stage and written into the acquisition parameter set. The field of view is determined by the matching result of the imaging preview frame of the hyperspectral camera and the calibration lines of the sample carrier stage and written into the acquisition parameter set. To ensure that the dark field data cube, white field data cube, and the original spectral data cube of the sample under test are acquired under the same geometric constraints, the acquisition control unit performs a geometric consistency check after writing the fields for pitch angle, yaw angle, incident angle, working distance, and field of view. This geometric consistency check includes locking status reading, angle reading range verification, and working distance deviation verification. When the locking status is not locked or the angle reading exceeds the acquisition configuration limit range, the acquisition control unit generates a geometric anomaly record and freezes the sample stage movement command. The geometric consistency check is re-executed after the geometric anomaly record is corrected and closed. The white field reference object identifier is written based on the white field reference object identifier reading result. The white field reference object is used for white field data cube acquisition in the subsequent imaging calibration acquisition stage. The dark field acquisition condition identifier is generated and written jointly by the shading component status and the light source operating status. The shading component status is obtained from the shading switch input, and the light source operating status is reported by the light source driver module. When the white field reference object identifier is inconsistent with the white field reference object list registered by the configuration management unit, the acquisition control unit writes a white field reference anomaly identifier and generates an anomaly record.

[0040] After completing the access of the batch metadata of the sample to be tested, the distribution of the acquisition configuration, and the fixing of the imaging geometry, the acquisition control unit generates the acquisition parameter set and performs versioned storage. The acquisition parameter set, as a structured data object, includes sample identifier, batch number, raw material batch number, process section identifier, acquisition timestamp, ambient temperature, ambient humidity, light source working status, exposure parameters, working distance, field of view, white field reference object identifier, and dark field acquisition condition identifier. Among them, the ambient temperature and ambient humidity are collected and written by the environmental sensor, and the exposure parameters are selected and written by the acquisition control unit within the acquisition configuration limit range in combination with the preview frame saturation detection results. The saturation detection includes threshold determination of the brightness histogram distribution of the preview frame and generation of exposure parameter change records. When the exposure parameters change, the acquisition control unit writes the change source, change timestamp, and exposure parameters before and after the change into the audit field of the acquisition parameter set. The audit field and the abnormal record are stored together in the acquisition parameter set repository. Understandably, the acquisition parameter set repository assigns a unique version number to each generated acquisition parameter set and retains a historical version index. The version number is written into the version field of the acquisition parameter set, and the historical version index is stored in association with the sample identifier and batch number for subsequent cross-batch drift compensation calls to the reference information of the same batch. After being output in this step, the acquisition parameter set is written into the original acquisition task queue. The original acquisition task queue provides the acquisition parameter set as input to the imaging calibration acquisition of S200, enabling S200 to call the hyperspectral camera, light source, and sample stage according to the acquisition parameter set to complete the acquisition of dark field data cube, white field data cube, and the original spectral data cube of the sample under test.

[0041] In summary, the technical effects of this step are as follows: Through structured access to the batch metadata of the sample under test and the acquisition configuration, the acquisition control unit solidifies the pre-acquisition conditions into an auditable set of acquisition parameters. By fixing the imaging geometry and writing in the pitch angle, yaw angle, incident angle, working distance, and field of view, the dark field data cube, white field data cube, and the original spectral data cube of the sample under test obtain a consistent geometric constraint basis. Through versioned storage of the acquisition parameter set and closed-loop anomaly recording, the S200's imaging calibration acquisition obtains a stable input benchmark and forms a traceable link.

[0042] S200. Based on the acquisition parameter set, perform imaging calibration acquisition processing to generate an original acquisition data packet; the original acquisition data packet includes a dark field data cube, a white field data cube, and the original spectral data cube of the sample to be tested;

[0043] This step uses the acquisition parameter set output from the previous steps as the sole input source. The acquisition parameter set is read from the acquisition parameter set repository by the acquisition control unit and written into the original acquisition task queue, triggering execution. Triggering conditions include changes in sample identification, batch number, exposure parameter changes, white field reference object identification changes, or dark field acquisition condition identification changes. When there are incomplete tasks with the same batch number in the original acquisition task queue, the acquisition control unit sorts them by acquisition timestamp and locks the execution order. In this step, the imaging calibration acquisition is completed by the acquisition control unit in conjunction with the hyperspectral camera, light source driving module, light-shielding component, and sample stage. The imaging calibration acquisition is used to perform time-series acquisition of the dark field data cube, white field data cube, and the original spectral data cube of the sample under the fixed imaging geometric constraints, forming a traceable original acquisition data package. The hyperspectral camera is a spectral imaging device with pushbroom or array acquisition capabilities, and internally includes a spectral dispersive component, an imaging sensor, and an acquisition buffer unit; the light source driving module is a light source power supply and status acquisition device, and internally includes a steady-state driving unit and a working status reporting unit; the light-shielding component is a light-shielding mechanism for dark field acquisition, and internally includes a light-shielding actuator and a light-shielding status detection unit; the sample carrier stage is a mechanism for carrying the sample to be tested and completing the scanning path execution, and internally includes a displacement actuator and a position readback unit.

[0044] Specifically, after reading the acquisition parameter set, the acquisition control unit first performs a consistency check on the light source operating status and the exposure parameters. This consistency check includes comparing the light source operating status readback value with the fields in the acquisition parameter set, checking the upper and lower limits of the exposure parameters, and verifying saturation detection. When an inconsistency is found during the consistency check, the acquisition control unit generates a calibration acquisition anomaly record and writes the anomaly identifier into the audit field of the corresponding task in the original acquisition task queue, while simultaneously pausing the trigger output to the hyperspectral camera. Further, the acquisition control unit sends a preheating command to the light source driving module and records the preheating start timestamp. After the light source driving module reports the light source stability identifier, it enters the acquisition stage. The light source stability identifier is calculated by the operating status reporting unit based on the light source current fluctuation and output intensity sampling sequence and written into the process log of this step. Understandably, the light source stability identifier is a necessary condition for imaging calibration acquisition. If the light source stability identifier is not met, the acquisition control unit maintains the acquisition trigger off and periodically reads the light source operating status until it is met before entering the next sub-process.

[0045] In the dark field data cube acquisition sub-process, the acquisition control unit reads the dark field acquisition condition identifier and drives the shading component to close. At the same time, it sends a shutdown command to the light source driving module and reads the light source working status readback value. When the shading status detection unit returns the shading closure identifier and the light source working status readback value corresponds to the closed state, the acquisition control unit sends a dark field acquisition trigger command to the hyperspectral camera. The dark field acquisition trigger command carries the sample identifier, acquisition timestamp and exposure parameters, and writes the working distance and the field of view into the acquisition metadata buffer of the hyperspectral camera. The dark field data cube is a spectral data cube acquired under conditions of shading and light source shutdown. It contains the original response values ​​of spatial and spectral dimensions. After dark field acquisition is completed, the acquisition control unit reads the acquisition buffer unit of the hyperspectral camera and writes the data into the dark field data cube storage area. At the same time, it writes the dark field acquisition condition identifier and the shading state readback value to form a dark field acquisition record. When the dark field data cube experiences frame loss, line synchronization abnormality, or saturation detection abnormality, the acquisition control unit writes the abnormal information into the calibration acquisition abnormality record and triggers dark field reacquisition. The number of reacquisitions is counted by the acquisition control unit in the process log and written into the audit field.

[0046] In the white-field data cube acquisition sub-process, the acquisition control unit drives the shading component to open and sends an activation command to the light source drive module. After reading the light source's operating status and verifying the light source's stability, it instructs the sample stage to position the white-field reference object within the field of view. The white-field reference object positioning is completed by comparing the position readback unit with the corresponding stage scribe line reference within the field of view. The positioning completion flag is written to the process log. Subsequently, the acquisition control unit sends a white-field acquisition trigger command to the hyperspectral camera. This command carries the white-field reference object identifier, acquisition timestamp, and exposure parameters, and includes the pitch angle, yaw angle, and incident angle as white-field acquisition geometric fields. The white field data cube is a cube of spectral data acquired from a white field reference object under the fixed imaging geometry and the working state of the light source. After the white field acquisition is completed, the acquisition control unit writes the white field data cube into the white field data cube storage area and writes the white field reference object identifier, the white field acquisition geometry field, and the light source working state readback value into the white field acquisition record. When the white field data cube shows abnormal brightness distribution or insufficient field of view coverage, the acquisition control unit generates a white field abnormality record and triggers the white field reference object to be repositioned before acquisition. The white field abnormality record and the number of repositioning times are written into the audit field.

[0047] In the cubic acquisition sub-process of the raw spectral data of the sample to be tested, the acquisition control unit controls the sample carrier stage to move the sample to be tested to the coverage area of ​​the field of view and lock the position. After the locking status is reported by the position readback unit, the acquisition triggering stage begins. Specifically, the acquisition control unit selects either a pushbroom acquisition path or an area array acquisition path according to the scanning method in the acquisition configuration. The pushbroom acquisition path is formed by the sample carrier stage moving at a constant speed along a preset scanning direction and being triggered by the hyperspectral camera. The area array acquisition path is formed by the hyperspectral camera being triggered by acquisition at a fixed position. The scanning path parameters and exposure parameters are jointly written into the sample acquisition metadata field. The acquisition control unit sends a sample acquisition trigger command to the hyperspectral camera. The sample acquisition trigger command carries the sample identifier, batch number, acquisition timestamp, exposure parameters, working distance, and field of view. During the acquisition process, the horizontal synchronization status and buffer occupancy status of the hyperspectral camera are read in real time. When the horizontal synchronization status is abnormal or the buffer occupancy exceeds the threshold, the acquisition control unit triggers an acquisition interruption and generates an acquisition anomaly record. At the same time, the identifier of the acquired segment is written into the audit field for subsequent review. The original spectral data cube of the sample under test is the original spectral data cube of the sample acquired under the fixed imaging geometry and the working state of the light source. It contains the original response values ​​of the spatial dimension and the band dimension. After the acquisition is completed, the acquisition control unit reads the complete data from the acquisition buffer unit and writes it into the storage area of ​​the original spectral data cube of the sample under test. It also writes the acquisition timestamp, exposure parameters, light source working state and scanning path parameters into the sample acquisition record.

[0048] After acquiring the dark-field data cube, white-field data cube, and the raw spectral data cube of the sample under test, the acquisition control unit associates and encapsulates the three types of data cubes with the acquisition parameter set to generate a raw acquisition data package. This raw acquisition data package is a structured data object containing a sample identifier, batch number, acquisition timestamp, dark-field data cube reference identifier, white-field data cube reference identifier, raw spectral data cube reference identifier of the sample under test, and acquisition parameter set reference identifier. The dark-field data cube reference identifier, white-field data cube reference identifier, and raw spectral data cube reference identifier of the sample under test are index keys for their respective storage areas, and the acquisition parameter set reference identifier is the version index key of the acquisition parameter set repository. Further, the acquisition control unit writes the raw acquisition data package into the raw acquisition data package queue and generates a data integrity verification record. This record includes the results of data cube size consistency verification, field-of-view consistency verification, and acquisition geometry field consistency verification. When inconsistencies exist in the data integrity verification record, the acquisition control unit writes the inconsistencies into the calibration acquisition anomaly record and marks the raw acquisition data package as pending review. The original acquisition data packet is used as the output of this step and is called by S300 for dark and white correction, bad pixel repair and band clipping. S300 reads the dark field data cube, white field data cube and the original spectral data cube of the sample to be tested from the original acquisition data packet and performs subsequent processing.

[0049] In summary, the technical effects of this step are as follows: By using the acquired parameter set as the trigger and constraint input for imaging calibration acquisition, the dark field data cube, white field data cube, and the original spectral data cube of the sample under test achieve a consistent acquisition metadata binding relationship. Through joint verification of the light source's working status, shading status, and sample stage position readback, the imaging calibration acquisition generates an auditable process log and anomaly record. Through the structured encapsulation and reference identification association of the original acquisition data packet, S300 obtains traceable data cube input and forms cross-step connections.

[0050] S300. Based on the original acquired data packet, perform dark white correction, bad pixel repair and band clipping processing to generate a reflectance characterization data cube.

[0051] This step uses the raw acquisition data packet output from the previous step as the input source. The raw acquisition data packet is written into the raw acquisition data packet queue by the acquisition control unit, triggering the processing flow. The triggering conditions include the raw acquisition data packet being in a closed state for review, the data integrity verification record meeting consistency requirements, and the acquisition parameter set version pointed to by the acquisition parameter set reference identifier being in a valid state. Understandably, the dark / white field correction, bad pixel repair, and band clipping are completed by the preprocessing execution unit. The preprocessing execution unit is a computing module deployed on the detection terminal or processing server, internally containing a data reading component, a correction calculation component, an anomaly detection component, and a data writing component. The data reading component loads the dark field data cube, white field data cube, and original spectral data cube of the sample to be tested according to the dark field data cube reference identifier, the white field data cube reference identifier, and the original spectral data cube reference identifier of the sample to be tested, respectively. The correction calculation component performs radiometric correction calculations. The anomaly detection component generates a bad pixel mask and records the anomaly location. The data writing component generates and stores the reflectance characterization data cube and writes it into the processing audit field. The dark-white correction is used to convert the cubic original spectral data of the sample under test from the sensor's original response domain to the reflectance characterization domain. The bad pixel repair is used to perform structured replacement of abnormal pixel responses. The band clipping is used to effectively filter band dimensions and solidify the clipping rules.

[0052] Specifically, the preprocessing execution unit parses the sample identifier, batch number, acquisition timestamp, and acquisition parameter set reference identifier from the original acquisition data packet. It then reads the acquisition parameter set corresponding to the acquisition parameter set reference identifier from the acquisition parameter set repository. The exposure parameters, light source operating status, field of view, and dark field acquisition condition identifier in the acquisition parameter set serve as input constraint fields for the calibration calculation component and are also written to the process log of this step as processing audit fields. Further, the data reading component performs a size consistency check on the dark field data cube, white field data cube, and the original spectral data cube of the sample under test. This size consistency check includes spatial dimension alignment check and band dimension alignment check. When inconsistencies occur, the preprocessing execution unit generates a preprocessing anomaly record and writes the anomaly identifier into the audit field of the original acquisition data packet, while simultaneously returning the original acquisition data packet to the original acquisition data packet queue for further review. For samples that pass the size consistency verification, the correction calculation component performs dark field subtraction processing on the original spectral data cube of the sample under test according to pixel position and band position. This dark field subtraction processing involves subtracting the response values ​​of the dark field data cube at the same position and band to generate a dark field subtracted data cube. Subsequently, radiometric normalization processing is performed based on the white field data cube. This radiometric normalization processing normalizes the dark field subtracted data cube according to the response values ​​of the white field data cube at the same position and band to generate a reflectance characterization candidate data cube. The normalization process is constrained by the exposure parameters and the operating state of the light source. When the exposure parameters change, the correction calculation component is triggered to write a correction version identifier and generate a correction recalculation record. Understandably, the dark-white correction is an essential technical feature of this invention. Its minimum set includes two operations: dark field subtraction processing and radiometric normalization processing, and their input reference identifier binding relationships. The reflectance characterization candidate data cube serves as the common input basis for subsequent bad pixel repair and band clipping.

[0053] In the defective pixel repair sub-process, the anomaly detection component generates a defective pixel mask based on the reflectance characterization candidate data cube. This mask is a mask structure consistent with the spatial dimension, marking the location of defective pixels and associating it with their band-dimensional anomaly range. Specifically, defective pixel determination is performed by the anomaly detection component using multi-condition joint discrimination. This multi-condition joint discrimination includes response saturation discrimination, response underflow discrimination, neighborhood mutation discrimination, and row-column stripe discrimination. Response saturation and response underflow discrimination are obtained by threshold detection of the pixel response range of the reflectance characterization candidate data cube. Neighborhood mutation discrimination is obtained by consistency detection of the spatial neighborhood difference within the same band. Row-column stripe discrimination is obtained by fluctuation detection of the row and column mean sequences in the spatial dimension. When the proportion of defective pixels in the defective pixel mask exceeds a preset upper limit, the preprocessing execution unit generates a defective pixel proportion anomaly record and writes the anomaly identifier into the process log. Furthermore, the data writing component performs neighborhood interpolation reconstruction on the bad pixel pixels marked by the bad pixel mask. This neighborhood interpolation reconstruction involves selecting a set of valid neighboring pixels within the same band and generating replacement values ​​based on distance weights. These replacement values ​​are written back to the reflectance characterization candidate data cube to form the bad pixel repair data cube. When the set of valid neighboring pixels is insufficient, the neighborhood interpolation reconstruction switches to cross-band adjacency interpolation, and the interpolation mode identifier is written to the bad pixel repair log. The bad pixel repair log and the bad pixel mask are written together into the audit field for review. Understandably, bad pixel repair is an essential technical feature of this invention. Its minimum set includes the bad pixel mask generation rules, the bad pixel interpolation reconstruction path, and the interpolation mode identifier writing mechanism. The bad pixel repair data cube serves as the input for subsequent band clipping.

[0054] In the band clipping sub-process, the preprocessing execution unit performs effective band interval filtering along the band dimension on the defective pixel repair data cube and generates clipping rules. The clipping rules are structured fields that record the retained and rejected band intervals and are written to the process log. Specifically, the clipping determination is calculated by the anomaly detection component in combination with sensor edge effects and signal-to-noise anomalies. The signal-to-noise anomalies are obtained by jointly judging the spatial response variance, defective pixel ratio, and response underflow ratio of each band. When a band meets the condition that the underflow ratio or defective pixel ratio exceeds the threshold within the field of view, the anomaly detection component marks the band as a rejected band and writes it into the clipping rules. Further, the clipping determination also reads the light source working status and exposure parameters in the acquisition parameter set. When there are fluctuation records in the light source working status or recalculation records in the exposure parameters, the preprocessing execution unit adds a clipping version identifier to the clipping rules and retains the previous version clipping rule index. The previous version clipping rule index is written into the audit field to form a version link. The data writing component performs band clipping on the defective pixel repair data cube according to the clipping rules to obtain a reflectance characterization data cube. The reflectance characterization data cube is a data cube that has undergone dark and white correction, defective pixel repair, and band clipping, maintaining the structured organization of spatial and band dimensions. It also establishes an association index with the sample identifier, batch number, acquisition timestamp, dark field data cube reference identifier, white field data cube reference identifier, original spectral data cube reference identifier of the sample under test, acquisition parameter set reference identifier, defective pixel mask reference identifier, and clipping rule reference identifier.

[0055] At the end of this step, the preprocessing execution unit writes the reflectance characterization data cube into the reflectance characterization data cube storage area and generates a reflectance characterization data cube reference identifier; simultaneously, it writes the bad pixel mask reference identifier and the clipping rule reference identifier into the audit field of the reflectance characterization data cube and generates a preprocessing link identifier, which is used in the preprocessing link identifier field of the subsequent inspection report. The reflectance characterization data cube, as the output of this step, is processed by the S400's denoising, scattering correction, and baseline correction in a coordinated manner, and the region of interest is extracted as input. The S400 reads the band dimension clipping result from the reflectance characterization data cube and constrains the subsequent processing link according to the clipping rules.

[0056] In summary, the technical effects of this step are as follows: Through dark and white field correction, pixel-level and band-level radiometric normalization correlations are established between the original spectral data cube of the sample under test and the dark field and white field data cubes, forming candidate data cubes for reflectance characterization. Through bad pixel mask generation and neighborhood interpolation reconstruction, the candidate data cubes for reflectance characterization obtain consistent data integrity markers and repair records. Through band clipping and clipping rule versioning, the reflectance characterization data cubes carry traceable descriptions of the effective band ranges and are integrated with subsequent processing by the S400.

[0057] S400. Based on the reflectance characterization data cube, perform noise reduction, scattering correction and baseline correction in a linked process, extract the region of interest, and generate a standardized spectral matrix and pixel index.

[0058] This step follows the output of S300. The reflectance characterization data cube is loaded into the preprocessing execution unit from the reflectance characterization data cube storage area and then triggered to run. The triggering conditions include that the audit field of the reflectance characterization data cube is in a processable state, the associated sample identifier and batch number are verified, and the version of the acquisition parameter set pointed to by the acquisition parameter set reference identifier is in a valid state. Understandably, the preprocessing execution unit is a computing module deployed on the detection terminal or processing server, internally containing a denoising component, a scattering correction component, a baseline correction component, a region extraction component, and a matrixing component. The denoising component receives the reflectance characterization data cube and outputs the denoising result; the scattering correction component receives the denoising result and outputs the scattering correction result; the baseline correction component receives the scattering correction result and outputs the baseline correction result; the region extraction component performs region of interest extraction on the baseline correction result and outputs the region of interest mask structure; and the matrixing component expands the baseline correction result according to the region of interest mask structure to generate a normalized spectral matrix and simultaneously generates pixel indices. This step involves spectral analysis and image feature extraction. The region of interest extraction and pixel index generation are processing steps that link the spectral and spatial dimensions. The preprocessing link identifier is written and the version link is continued in this step.

[0059] Specifically, the preprocessing execution unit first parses the audit field of the reflectance characterization data cube to obtain the clipping rule reference identifier, bad pixel mask reference identifier, and acquisition parameter set reference identifier, and then the parameter reading component loads the acquisition parameter set. The acquisition parameter set includes sample identifier, batch number, acquisition timestamp, light source working status, exposure parameters, working distance, field of view, and structure type identifier. The structure type identifier represents the structural morphology category of the sample under test and is written by the acquisition configuration. The structural morphology category includes particle morphology identifier and thin film morphology identifier. The denoising component performs sliding window smoothing on the reflectance characterization data cube. The sliding window smoothing constructs a window in the band dimension and generates a smoothed value according to the local mean of the band response within the window. At the same time, a boundary completion strategy is used for the bands at the window boundary to generate smoothed values. The sliding window length and the boundary completion strategy are written as denoising configuration fields into the process log. When the process log records a fluctuation mark in the light source working status, the denoising component performs a linkage adjustment on the sliding window length and writes it into the denoising recalculation record. After receiving the denoising result from the denoising component, the scattering correction component selects a scattering correction branch based on the structure type identifier in the acquired parameter set and generates a scattering correction result. The scattering correction branch selection logic is driven by a branch mapping table, which serves as the configuration object for the scattering correction component and is included in version management. Technical solution one involves a branch for particle morphology identifiers, where the scattering correction component performs a standard normal transformation on the denoising result and outputs the scattering correction result. Technical solution two involves a branch for thin film morphology identifiers, where the scattering correction component performs multiplicative scattering correction on the denoising result and outputs the scattering correction result. Technical solution three involves a branch with missing or conflicting structure type identifiers, where the scattering correction component performs quantization normalization on the denoising result and outputs the scattering correction result. Simultaneously, abnormal structure type identifier records are written to the process log, and the scattering correction branch identifier is written to the audit field. The scattering correction result retains the same spatial and band dimension organization as the cubic reflectance characterization data and carries the scattering correction branch identifier, scattering correction configuration version number, and input acquired parameter set reference identifier for the baseline correction component to read.

[0060] The baseline correction component performs morphological baseline fitting on the scattering correction result and generates a baseline correction result. The morphological baseline fitting obtains a baseline estimation sequence by constructing a structuring element in the band dimension and performing morphological opening and closing operations on the spectral response sequence. Subsequently, the baseline correction result is obtained by subtracting the baseline estimation sequence from the scattering correction result. The structuring element length, the order of opening and closing operations, and the baseline estimation smoothing strategy are written as baseline correction configuration fields into the process log and form a baseline correction configuration version number. When the process log records a change in the pruning rule reference identifier, the baseline correction component triggers baseline recalculation and writes a baseline recalculation record. The baseline recalculation record and the pruning rule reference identifier together constitute the version link information of the preprocessing link identifier. After receiving the baseline correction result, the region extraction component performs region of interest extraction. The region of interest is the effective imaging area of ​​the sample under test within the field of view. The region extraction component performs threshold segmentation on the baseline correction result in the spatial dimension to generate candidate regions, and performs rejection processing on the candidate regions by combining the bad pixel mask reference identifier with the bad pixel mask. At the same time, it performs connectivity screening on the boundary of the candidate regions to generate the region of interest mask structure. When the area of ​​the candidate region is less than the preset lower limit or there is a multi-connected component conflict, the region extraction component generates a region anomaly record and writes it into the audit field. The matrix component writes the anomaly identifier into the subsequent pixel index accordingly. The matrix component expands the baseline correction result based on the region of interest mask structure to generate a normalized spectral matrix. This normalized spectral matrix is ​​organized by pixel rows and by cropped bands columns, with each row associated with the sample identifier, batch number, and acquisition timestamp. The synchronously generated pixel index has the same row order as the normalized spectral matrix and includes pixel row number, pixel column number, region number, anomaly identifier, and field of view reference fields. The pixel row number and pixel column number are obtained by mapping the spatial coordinates of the baseline correction result, and the region number is obtained by mapping the connected component numbers of the region of interest mask structure. The normalized spectral matrix and the pixel index are written as outputs of this step into the matrix storage area and index storage area, respectively. The preprocessing link identifier, acquisition parameter set reference identifier, and baseline correction configuration version number are written to the audit field. In cross-step transitions, the normalized spectral matrix and pixel index serve as inputs for the S500's reference spectral library construction, label alignment, and feature band filtering. The S500 reads the pixel index to build a spectral library entry index and references the normalized spectral matrix to extract spectral vectors.

[0061] In summary, the technical effects of this step are as follows: By smoothing the sliding window and mapping the scattering correction branches, the preprocessing unit incorporates spectral response fluctuations caused by structural morphology differences into a unified link configuration and forms an auditable version record. Through morphological baseline fitting and region-of-interest mask structure generation, the baseline correction results and the effective region boundaries establish a consistent organizational relationship in both spatial and band dimensions. Through the synchronized output of the standardized spectral matrix and pixel index, subsequent steps obtain an input carrier traceable to spatial coordinates row by row and establish the version reference relationship between the spectral library and the model link.

[0062] S500: Based on the standardized spectral matrix and the pixel index, perform reference spectral library construction, label alignment and feature band filtering to generate a training dataset and a feature set; the reference spectral library includes structure type labels, water-containing state labels and impurity state labels.

[0063] This step follows the output of S400. The standardized spectral matrix and the pixel index are loaded into the spectral library and modeling data processing unit from the matrix storage area and the index storage area, respectively, and then the unit is triggered to run. The triggering conditions include that the sample identifier and batch number have a valid mapping in the batch metadata, the preprocessing link identifier is in a traceable state, and the acquisition parameter set version pointed to by the acquisition parameter set reference identifier is in a valid state. The spectral library and modeling data processing unit is a computing module deployed on the detection terminal or processing server. It contains a reference spectral library construction component, a label alignment component, and a feature band screening component, along with a process log component and a version management component. The reference spectral library construction component receives the standardized spectral matrix, the pixel index, the batch metadata, and the acquisition parameter set reference identifier, generates a reference spectral library entry index, and writes it into the reference spectral library. The label alignment component performs alignment and writing of the structure type label, water content label, and impurity status label for the reference spectral library entries and generates a label consistency record. The feature band screening component reads the reference spectral library with completed label alignment, generates a band scoring table by label grouping, and filters the band index set according to the scoring threshold, thereby generating a feature set and carrying a feature set version number. Understandably, the reference spectral library is a spectral library data structure oriented towards sample identification, and the reference spectral library entry is the smallest recording unit of the spectral library; each reference spectral library entry includes a spectral vector, a structure type label, a water content label, an impurity status label, a physicochemical control field, an acquisition parameter set reference identifier, and a preprocessing link identifier, wherein the spectral vector is a single-pixel or region-aggregated spectral sequence extracted from the normalized spectral matrix, and the physicochemical control field is a laboratory test record field associated with the entry, which includes a control batch number, a control timestamp, and a control conclusion enumeration value, and the source is recorded by the process log component.

[0064] Specifically, the reference spectral library construction component establishes a spectral library entry index based on the sample identifier. This index is a mapping structure from sample identifiers to the entry set, and the batch number and acquisition timestamp are written into the audit field of the index key for easy traceability. The reference spectral library construction component locates the row record of the normalized spectral matrix based on the pixel row number and pixel column number of the pixel index, extracts the spectral sequence of that row to form a spectral vector, and writes the region number, anomaly identifier, and field-of-view reference fields from the pixel index into the spatial positioning field of the reference spectral library entry. When the anomaly identifier indicates that the pixel is in an anomalous pixel candidate state, the reference spectral library construction component marks the entry as isolated and writes the isolation reason. Isolated entries are not included in the subsequent statistical grouping for feature band filtering. The reference spectrum library construction component synchronously writes the acquisition parameter set reference identifier and the preprocessing link identifier. The acquisition parameter set reference identifier is used to trace back the exposure parameters, working distance, and field of view during acquisition. The preprocessing link identifier is used to trace back the link versions of dark white correction, bad pixel repair, band clipping, noise reduction, scattering correction, and baseline correction. The version management component registers the set of newly added entries in this batch as the incremental version of the reference spectrum library and writes the reference spectrum library version number and incremental range into the process log component. In technical solution one, the spectral vector adopts a single-pixel spectral vector method, with a one-to-one correspondence between the reference spectral library entries and the pixel indices, suitable for modeling scenarios that require preservation of spatial details. In technical solution two, the spectral vector adopts a region-aggregated spectral vector method, where the reference spectral library construction component aggregates multiple spectral records from the same region according to region number and performs robust statistical aggregation to generate a region spectral vector. At the same time, the set of pixel row numbers within the region is written into the spatial positioning field, suitable for scenarios with high noise or significant material surface texture. In technical solution three, the spectral vector adopts a hierarchical aggregation method, first aggregating by region number and then removing abnormal pixels by anomaly identifier to generate dual-version spectral vectors. Both versions of the spectral vector are written into the reference spectral library entries, and the versions are distinguished in the entry label field, facilitating subsequent comparison and auditing.

[0065] Furthermore, the label alignment component performs label alignment processing on the reference spectral library entries. This label alignment is a processing link that establishes a consistent mapping between the label fields in external control records or process records and the reference spectral library entry index. The label alignment component first reads the sample identifier, batch number, raw material batch number, and process segment identifier from the batch metadata, and then reads the control batch number, control timestamp, and control conclusion enumeration value from the physicochemical control field. Subsequently, it performs primary key matching based on the sample identifier and batch number to form an association mapping from the entry to the control record. When a one-to-many or many-to-one conflict occurs, the label alignment component selects the primary associated record according to the proximity rule between the acquisition timestamp and the control timestamp, and writes the conflict information into the label consistency record. The structure type label is a category marker for the material's structural morphology. It is mapped from the structure type identifier and bound to the acquisition parameter set reference identifier. The water content label is a category marker for the material's water content state. It is mapped from the control conclusion enumeration value in the physicochemical control field and bound to the control timestamp. The impurity state label is a category marker for the impurity state. It is jointly mapped from the process section identifier and the control conclusion enumeration value and bound to the raw material batch number. The label alignment component writes the structure type label, water content label, and impurity state label into the label field of the reference spectral library entry, and writes the label source identifier, mapping rule version number, and alignment timestamp into the label audit field. When a label field is missing or the mapping rule version number does not match, the label alignment component marks the entry as misaligned and writes the reason. Misaligned entries are not included in the statistical scope of the band scoring table. The version management component generates a label alignment version number after label alignment is completed and writes it into the reference spectral library version link for subsequent training dataset traceability.

[0066] In the feature band selection stage, the feature band selection component reads the reference spectral library with aligned labels and generates a band scoring table by grouping according to structure type label, water content label, and impurity status label. This band scoring table is a quantitative record structure of the distinguishing ability and stability of each band, and includes at least a band index, group difference score, intra-group stability score, redundancy marker, and comprehensive score fields. Specifically, the feature band selection component performs intra-group statistics on the spectral vectors within each label group to generate an intra-group stability score, and performs difference comparisons on the spectral vectors between different label groups to generate a group difference score. When a band exhibits contradictory scoring trends across different label dimensions, the feature band selection component writes a conflict flag in the redundancy marker field and reduces the weight of that band; the reduction record is written to the process log component. The scoring threshold is maintained by the version management component and bound to the scoring threshold version number. The feature band selection component selects a set of band indexes from the band scoring table based on the scoring threshold. The set of band indexes is one of the smallest sets of core parameters of the feature set. Simultaneously, the feature band selection component encapsulates the band indexes and the scoring threshold version number to generate a feature set and assigns a feature set version number, which is written into the audit field of the feature set. Subsequently, the spectral library and modeling data processing unit extracts the spectral vectors corresponding to the aligned entries according to the band indexes in the feature set to form feature vectors. These feature vectors are then encapsulated together with the structure type label, water content label, and impurity status label to generate a training dataset. The training dataset is a data structure oriented towards model training. The training dataset includes at least the sample identifier, batch number, feature vector, structure type label, water content label, impurity status label, acquisition parameter set reference identifier, preprocessing link identifier, reference spectral library version number, and label alignment version number. The training dataset version number is written into the audit field. The training dataset is written to the training dataset storage area and is available for the S600's "model training, cross-validation, and discrimination rule setting" functions to read. The feature set is written to the feature set storage area and is available for the S600's "model training, cross-validation, and discrimination rule setting" functions to call. At the same time, the reference spectrum library version number, the feature set version number, and the training dataset version number form a traceable link in the process log component. Subsequently, when the scoring threshold version number is updated, the mapping rule version number is updated, or a new incremental version of the reference spectrum library appears, the version management component triggers this step to be recalculated and writes the recalculation record, so that the training dataset and feature set continue to maintain an auditable version status.In an engineering embodiment, under the scenario of production batch sampling inspection, the detection terminal completes imaging under the same working distance and field of view, and outputs the standardized spectral matrix and the pixel index via S400. The spectral library and modeling data processing unit reads the physicochemical control fields returned by the laboratory and completes label alignment. Subsequently, a band scoring table is generated and the scoring threshold version number is fixed. The training dataset and feature set are output for S600 to train model parameters and generate discrimination rules. The process log component synchronously records the sample identifier, batch number, reference spectral library version number, training dataset version number, and feature set version number, which facilitates the traceability of subsequent test reports.

[0067] In summary, the technical effects of this step are as follows: Through incremental version registration of the reference spectral library construction and entry index, spectral vectors, acquisition parameter set reference identifiers, and preprocessing link identifiers form a unified and traceable carrier. By using a label alignment link, structure type labels, water content labels, and impurity status labels are written into the reference spectral library entries, generating label consistency records, thus providing an auditable representation of the label source and mapping rule version of the training samples. Feature band selection driven by a band scoring table generates feature sets carrying feature set version numbers, ensuring a stable version link between the training dataset and the feature set, and directly connecting to the S600 model training input.

[0068] S600. Based on the training dataset and the feature set, perform model training, cross-validation and discrimination rule setting processes to generate model parameters and discrimination rules;

[0069] This step follows the output of S500, loading the training dataset and feature set by the training management unit and triggering execution. The training management unit is a computing module deployed on the detection terminal or processing server, internally containing a data consistency verification component, a model training component, a cross-validation component, and a discrimination rule setting component, and connected to a version management component and a process log component. Specifically, the training dataset is an encapsulated data structure containing sample identifier, batch number, feature vector, structure type label, water content label, impurity status label, acquisition parameter set reference identifier, preprocessing link identifier, reference spectral library version number, and label alignment version number; the feature set is an encapsulated structure containing a band index set and a feature set version number, and carries a scoring threshold version number. When accessing the training dataset, the training management unit first reads the sample identifier, batch number, and preprocessing link identifier to establish a sample index table, and performs a consistency comparison between the dimension of the feature vector and the band index set of the feature set. When samples with inconsistent dimensions, missing labels, or unresolved acquisition parameter set reference identifiers are found, the data consistency verification component writes the sample into the isolated sample record and records the reason for isolation in the process log component. Simultaneously, the isolated sample identifier is removed from the sample index table, ensuring that subsequent model training and cross-validation operate under a unified standard. Furthermore, the training management unit writes the training dataset version number, feature set version number, and scoring threshold version number into the training task audit field as the upstream basis for the output version link in this step.

[0070] In the model training phase, the model training component takes the sample index table as input and generates training sample indices and validation sample indices based on the batch number and structure type label joint constraints. The index generation rules are then written into the index audit field. These rules include at least constraints describing layering by batch number, balancing by structure type label, and coverage by impurity state label. Understandably, the model training is a supervised learning process based on the feature vectors, structure type labels, water content labels, and impurity state labels. During runtime, the model training component first loads the model structure definition and training hyperparameter configuration. The training hyperparameter configuration is the minimum set of core parameters required for the training process, including at least a sample partitioning rule identifier, a class weight configuration identifier, a termination condition identifier, and a random seed identifier. The class weight configuration identifier is used to adjust the contribution of training samples when different label distributions are uneven. The termination condition identifier is used to end training when the loss converges or the iteration stabilizes. The random seed identifier is used to make the training process reproducible and auditable by the process log component. The model training component generates a training task number and a model candidate version number for each training task. It writes the training task number, training dataset version number, feature set version number, collection parameter set reference identifier coverage, and preprocessing link identifier coverage into the training task audit field. During training, it writes key intermediate states into the process log component. These key intermediate states include at least the current iteration state, sample coverage statistics, label consistency statistics, and abnormal sample count. When training is interrupted, data reading fails, or label conflicts cannot be resolved, the model training component writes the failure reason and recovery point identifier. If the recovery point identifier is available, execution continues, thus enabling the training task to have traceable breakpoint recovery capabilities in engineering scenarios.

[0071] In the cross-validation stage, the cross-validation component performs cross-validation for the first time in this step. Cross-validation is a process chain that repeatedly trains and validates the training dataset on multiple subsets and forms a stability evaluation record. Specifically, the cross-validation component reads the generation rules of the training sample index and the validation sample index, and groups the sample index table based on the batch number to generate a fold number and a fold sample list. The fold sample list is then written into the cross-validation audit field. Subsequently, the cross-validation component drives the model training component to repeatedly train on each fold to form a candidate model set, and outputs validation statistics records for the candidate model set. The validation statistics records at least include a confusion statistics field grouped by structure type label, water content label, and impurity status label, a confidence distribution statistics field, and an anomaly sample sensitivity field. The cross-validation component associates and stores validation statistics records with training task audit fields, and selects the target model candidate version number based on preset model selection criteria. When multiple candidate models have selection conflicts on different label dimensions, the cross-validation component writes the conflict information into the model selection conflict record and calls the version management component to generate a retraining trigger identifier. The retraining trigger identifier carries the trigger reason, trigger timestamp, and suggested category weight configuration identifier update item, enabling the training management unit to automatically initiate retraining in subsequent tasks without manual process modification.

[0072] In the discrimination rule setting stage, the discrimination rule setting component takes the verification statistics record and the target model candidate version number as input to generate discrimination rules that match the pixel-level inference. The discrimination rule is an encapsulation structure of the parsing method for the output category score sequence, the threshold determination method for pixel confidence, and the calling parameters for spatial consistency constraint fusion. Specifically, the discrimination rule setting component reads the confidence distribution statistics field from the verification statistics record, generates a pixel confidence threshold parameter and a threshold source audit field, and registers the pixel confidence threshold parameter as one of the minimum sets of core parameters of the discrimination rule. The discrimination rule setting component further reads the abnormal sample sensitivity field, generates the writing conditions for the abnormal pixel mask field and the abnormal identifier mapping rule, and writes the version number of the abnormal identifier mapping rule into the discrimination rule audit field. To establish a connection with the drift compensation processing of the S700, the discrimination rule setting component writes the batch metadata field mapping table required for drift compensation into the discrimination rules. This batch metadata field mapping table at least points to the field names corresponding to the cumulative working time of the light source, ambient temperature, ambient humidity, and exposure parameters. This allows the S700 to directly locate the drift compensation input fields and execute the drift compensation chain after reading the discrimination rules. The discrimination rule setting component also writes the neighborhood connectivity parameters, neighborhood voting replacement switch, and low-confidence pixel processing strategy for spatial consistency constraint fusion into the discrimination rules. This enables the spatial consistency constraint fusion of the S700 to be rule-driven in engineering implementation, and records the training dataset version number, feature set version number, and reference spectral library version number at the time of rule generation in the process log component.

[0073] During the output of this step, the version management component solidifies the candidate version number of the target model into a model parameter version number, and encapsulates the target model's model structure definition, training hyperparameter configuration summary, feature set version number reference, training dataset version number reference, and preprocessing link identifier coverage to generate model parameters. Simultaneously, it encapsulates the rule content, discrimination rule version number, threshold source audit field, drift compensation field mapping table, and spatial consistency constraint fusion parameters of the discrimination rule to generate the discrimination rule. The model parameters and the discrimination rule are written to the model and rule storage area, and the model parameter version number and discrimination rule version number are respectively marked in the storage record. The process log component synchronously records the training task number, cross-validation audit field, and model selection criterion identifier. The model parameters and the discrimination rule serve as direct input to the next main step S700, and are invoked when S700 executes "input the model parameters and the discrimination rule into pixel-level inference." Simultaneously, the model parameter version number and the discrimination rule version number are written to the version field of the detection report to form an end-to-end auditable link. In the engineering implementation, in the scenario of incoming material inspection on the production line, the training management unit reads the latest training dataset version number and feature set version number from the model and rule storage area at a fixed pace. When the reference spectrum library version number is incrementally updated and there is no conflict in the label consistency record, the training task is automatically triggered. When the cross-validation component outputs a model selection conflict record or a retraining trigger flag, the training management unit writes the trigger flag into the pending queue and automatically executes retraining in the next training window, thereby completing the iterative update of model parameters and discrimination rules without changing the S700 inference interface.

[0074] In summary, the technical effects of this step are as follows: Through consistency verification between the training dataset and the feature set, cross-validation auditing, and version solidification, a traceable version chain is formed between model parameters and discrimination rules. By encapsulating pixel confidence threshold parameters, drift compensation field mapping tables, and spatial consistency constraint fusion parameters in the discrimination rules, the input for pixel-level inference has a consistent rule-driven entry point in engineering implementation. Through the collaboration of retraining trigger flags and breakpoint recovery records, the training process maintains a continuous and auditable running path under batch updates and abnormal scenarios.

[0075] S700. Based on the model parameters and the discrimination rules, perform pixel-level inference; perform drift compensation based on the batch metadata, and perform confidence assessment and spatial consistency constraint fusion to output component distribution map, index estimation results and detection report.

[0076] This step is executed by a pixel-level inference module on the detection terminal or processing server. This pixel-level inference module is connected to the model and rule storage area, the result storage area, and the log auditing unit, and is used to perform inference and output generation based on the pixel-level spectral features of the sample under test. Specifically, the input sources for this step include: the model parameters and discrimination rules output and fixed by S600, the batch metadata and acquisition parameter set output by S100, and the standardized spectral matrix and pixel index output by S400. Furthermore, the feature set output by S500 can be called for feature extraction caliber alignment. When a task is triggered, the pixel-level inference module first reads the sample identifier, batch number, and collection timestamp from the collected parameter set. It then combines the model parameter version number, the discrimination rule version number, and the feature set version number to generate an inference task number. The inference task number, sample identifier, batch number, preprocessing link identifier, and version link information are written to the log audit unit. When the detection terminal receives a batch switching signal, a model parameter version number update signal, or a discrimination rule version number update signal, the pixel-level inference module automatically enters the version reload process. It reloads the model parameters and discrimination rules that match the inference task number. After loading is completed, the loading verification record is written to the log audit unit, thus ensuring a traceable correspondence between the inference process and version evolution.

[0077] In the drift compensation stage, the pixel-level inference module calls the drift compensation unit to perform the drift compensation. The drift compensation unit is a processing component that performs batch-level correction on the input features. The input is the batch metadata and the feature set to be inferred, and the output is the compensation feature set. Specifically, the drift compensation unit extracts the cumulative working time of the light source, ambient temperature, ambient humidity, and exposure parameters from the batch metadata. It combines the light source working state and dark field acquisition condition identifier in the acquisition parameter set to generate a drift factor vector, and maps the field source of the drift factor vector as drift compensation audit fields. Among them, the cumulative working time of the light source is used to characterize the offset of the light source state with the change of running time, the ambient temperature and ambient humidity are used to characterize the spectral response offset caused by the change of the environment, and the exposure parameters are used to characterize the response offset caused by the change of the acquisition link gain. The drift factor vector is a structured combination of the above fields in the current batch. Furthermore, the drift compensation unit performs drift correction on the feature values ​​corresponding to the feature set based on the drift factor vector. The drift correction processing path includes performing batch-by-batch offset correction and scale correction on the feature values, and writing the corrected features into the compensation feature set. When the batch metadata is missing any drift factor vector source field, the drift compensation unit writes the missing field record and calls the backoff branch. The backoff branch uses the ambient temperature, ambient humidity and exposure parameters in the collection parameter set as substitute inputs to generate the drift factor vector, and writes the backoff branch identifier into the drift compensation audit field. The compensation feature set is still used as the direct input for subsequent pixel-level inference.

[0078] In the pixel-level inference stage, the pixel-level inference module calls the feature assembly unit and the inference execution unit to work together. The feature assembly unit is used to extract and assemble the normalized spectral matrix into a pixel input feature stream according to the band index set of the feature set. The inference execution unit is used to perform forward inference on the pixel input feature stream and output a category score sequence. Specifically, the feature assembly unit uses the pixel index as the pixel positioning input, expands the normalized spectral matrix into a pixel spectral vector sequence according to the pixel index, and performs band extraction and feature arrangement according to the band index set of the feature set to form a pixel feature vector sequence consistent with the requirements of the model parameters. When the feature set version number is inconsistent with the feature set version number recorded in the model parameters, the feature assembly unit writes a version inconsistency record and triggers rule fallback. The rule fallback reloads the feature set according to the feature set version number reference in the discrimination rule and then generates the pixel feature vector sequence. Subsequently, the inference execution unit receives the pixel feature vector sequence corresponding to the compensation feature set, loads the model structure definition and model weight parameters in the model parameters, outputs the category score sequence for each pixel, and synchronously writes it into the pixel-level inference intermediate result package; the pixel-level inference intermediate result package contains at least the pixel index, category score sequence and model parameter version number reference identifier, and serves as the common input for confidence evaluation and spatial consistency constraint fusion.

[0079] In the confidence assessment stage, the pixel-level inference module calls the confidence assessment unit to perform the confidence assessment on the pixel-level inference intermediate result package. The input of the confidence assessment unit is the category score sequence and the discrimination rule, and the output is the pixel confidence and the initial pixel classification. Specifically, the confidence assessment unit extracts the maximum and second-largest scores from the category score sequence of each pixel, and generates the pixel confidence based on the difference between the maximum and second-largest scores. At the same time, the category corresponding to the maximum score is written into the initial pixel classification field. The pixel confidence threshold parameter is provided by the discrimination rule. The confidence assessment unit performs threshold comparison on the pixel confidence based on this, generates a low-confidence pixel identifier field, and writes it into the abnormal candidate record. Further, when the discrimination rule contains a threshold strategy segmented by structure type label, the confidence assessment unit uses the structure type identifier in the collected parameter set as the threshold strategy selection input, generates the pixel confidence judgment result according to the corresponding segment threshold, and writes the threshold strategy branch identifier into the confidence audit field, so that the confidence judgment caliber of the same inference task under different structure type labels remains auditable and consistent.

[0080] In the spatial consistency constraint fusion stage, the pixel-level inference module calls the spatial consistency constraint fusion unit to perform the spatial consistency constraint fusion. The spatial consistency constraint fusion unit takes as input a pixel index, a pixel initial classification field, a low-confidence pixel identifier field, and a discrimination rule, and outputs a fused pixel classification field and an abnormal pixel mask. Specifically, the spatial consistency constraint fusion unit establishes a neighboring connected region based on the pixel index. The neighboring connected region is a connected set constructed according to the spatial adjacency relationship of the pixel index. The adjacency relationship parameter is provided by the discrimination rule and written into the spatial constraint audit field. Within the connected region, the spatial consistency constraint fusion unit performs neighborhood voting replacement on low-confidence pixels. Neighboring voting replacement is a processing path that statistically analyzes the pixel initial classification distribution of high-confidence pixels within the neighboring connected region and selects the dominant category to replace the low-confidence pixels. During the replacement process, the identifier of the replacement source connected region and the pixel index being replaced are written simultaneously. Furthermore, when the number of high-confidence pixels in the connected region is insufficient or the category distribution shows multi-peak conflict, the spatial consistency constraint fusion unit calls the conflict handling branch. The conflict handling branch retains the low-confidence pixel as the original pixel's initial category and marks it as an abnormal pixel. The abnormal pixel is written into the abnormal pixel mask field with the pixel index as the key. The abnormal pixel mask field and the low-confidence pixel identifier field together form an abnormal pixel mask, which serves as a traceable basis for anomaly prompts in subsequent detection reports.

[0081] In the output generation stage, the pixel-level inference module calls the result aggregation unit and the report generation unit to complete the output of the component distribution map, the index estimation result, and the detection report. The result aggregation unit generates a spatial mapping result using the fused pixel category field and the pixel index as input, while the report generation unit generates the detection report using the spatial mapping result, the batch metadata, the acquisition parameter set, and the version link information as input. Specifically, the result aggregation unit backfills the fused pixel category field to a pixel grid with the same spatial size as the sample to be tested according to the pixel index, forming a component distribution map and generating a component distribution map index. The component distribution map index is a location field pointing to the component distribution map file or data object in the result storage area. Simultaneously, the result aggregation unit performs batch-level aggregation based on the fused pixel category field and the abnormal pixel mask field to generate an index estimation result field. This index estimation result field is a structured encapsulation of pixel statistics and region statistics related to the structure type label, water content label, and impurity status label, and is stored in association with the inference task number. Furthermore, the report generation unit generates the test report content according to the field constraints of the test report, and writes the sample identifier, the batch metadata, the acquisition parameter set, the preprocessing link identifier, the feature set version number, the model parameter version number, the discrimination rule version number, the component distribution map index, the index estimation result field, and the abnormal pixel mask field into the test report. At the end of the test report, the generation timestamp and the inference task number are written as audit anchors. When the result storage area fails to write or the index generation fails, the report generation unit writes the failure reason and marks the test report as pending completion status. At the same time, the generated index estimation result field and abnormal pixel mask field are retained, so that the engineering site can retry or supplement based on the pending completion status without changing the main inference link.

[0082] In summary, the technical effects of this step are as follows: By incorporating the batch metadata into drift compensation and performing pixel-level inference under the version chain, batch differences in the inference input are included in the processing chain under the same discrimination caliber. Through the linkage of confidence assessment and spatial consistency constraints, the pixel-level inference output forms a consistent mapping result in the spatial dimension, and simultaneously forms a traceable abnormal pixel mask field. By structurally solidifying the component distribution map index, index estimation result field, and detection report version field, an auditable correspondence is established between the output product and the upstream acquisition and preprocessing chain.

Claims

1. A hyperspectral imaging detection method for novel fatty acyl amino acid salt materials, characterized in that, include: S100: Obtain batch metadata and acquisition configuration of the sample to be tested, perform fixed imaging geometry processing, and generate acquisition parameter set; S200. Based on the acquisition parameter set, perform imaging calibration acquisition processing to generate an original acquisition data packet; the original acquisition data packet includes a dark field data cube, a white field data cube, and the original spectral data cube of the sample to be tested; S300. Based on the original acquired data packet, perform dark white correction, bad pixel repair and band clipping processing to generate a reflectance characterization data cube. S400. Based on the reflectance characterization data cube, perform noise reduction, scattering correction and baseline correction in a linked process, extract the region of interest, and generate a standardized spectral matrix and pixel index. S500: Based on the standardized spectral matrix and the pixel index, perform reference spectral library construction, label alignment and feature band filtering to generate a training dataset and a feature set; The reference spectral library includes structure type labels, water content state labels, and impurity state labels; S600. Based on the training dataset and the feature set, perform model training, cross-validation and discrimination rule setting processes to generate model parameters and discrimination rules; S700. Based on the model parameters and the discrimination rules, perform pixel-level inference; Based on the batch metadata, drift compensation is performed, and confidence assessment and spatial consistency constraint fusion are performed to output component distribution map, index estimation results and detection report.

2. The method according to claim 1, characterized in that, The set of acquisition parameters includes sample identifier, batch number, raw material batch number, process section identifier, acquisition timestamp, ambient temperature, ambient humidity, light source working status, exposure parameters, working distance, field of view, white field reference object identifier, and dark field acquisition condition identifier.

3. The method according to claim 1, characterized in that, The fixed imaging geometry includes mounting the hyperspectral camera on the positioning bracket and locking the pitch and yaw angles, mounting the light source on the fixed bracket and locking the incident angle, positioning the sample carrier stage at the working distance, and writing the pitch angle, yaw angle, incident angle, working distance, and field of view into the acquisition parameter set.

4. The method according to claim 1, characterized in that, The dark and white correction includes performing dark field subtraction processing on the cubic original spectral data of the sample under test, and performing radiometric normalization processing based on the cubic white field data; the bad pixel repair includes generating a bad pixel mask and performing neighborhood interpolation reconstruction on the bad pixel pixels.

5. The method according to claim 1, characterized in that, The noise reduction, scattering correction, and baseline correction linkage processing includes: performing sliding window smoothing on the cubic reflectance characterization data; selecting a scattering correction branch based on the structure type identifier in the acquired parameter set and generating a scattering correction result; performing morphological baseline fitting on the scattering correction result and generating a baseline correction result; expanding the baseline correction result based on the region of interest to generate the normalized spectral matrix and simultaneously generating the pixel index.

6. The method according to claim 1, characterized in that, The construction of the reference spectral library includes establishing an index of spectral library entries based on sample identifiers; each spectral library entry contains a spectral vector, a structure type label, a water content label, an impurity status label, a physicochemical control field, an acquisition parameter set reference identifier, and a preprocessing link identifier.

7. The method according to claim 1, characterized in that, The feature band screening includes grouping the reference spectral library according to the structure type label, the water-bearing state label and the impurity state label to generate a band scoring table, and filtering the band index set based on the band scoring table and the scoring threshold to generate the feature set, the feature set carrying a feature set version number.

8. The method according to claim 1, characterized in that, The drift compensation includes extracting the cumulative working time of the light source, ambient temperature, ambient humidity and exposure parameters from the batch metadata to generate a drift factor vector, and performing drift correction on the feature set of the pixel-level inference input to generate a compensation feature set.

9. The method according to claim 1, characterized in that, The confidence assessment includes outputting a category score sequence for each pixel and generating a pixel confidence based on the difference between the maximum and second-largest scores; the spatial consistency constraint fusion includes establishing a neighborhood connected region based on the pixel index, performing neighborhood voting replacement on low-confidence pixels, and generating an abnormal pixel mask.

10. The method according to claim 1, characterized in that, The detection report includes the sample identifier, the batch metadata, the collection parameter set, the preprocessing link identifier, the feature set version number, the model parameter version number, the discrimination rule version number, the component distribution map index, the index estimation result field, and the abnormal pixel mask field.

Images