Sensor data processing system, method and non-transitory storage medium

Abstract

The application discloses a kind of sensor data processing system, method and nonvolatile storage medium.Therein, the system includes: data acquisition module, for obtaining the encoding response data and equipment metadata of the object to be measured;Response file module is used to store and call response file;Multi-branch processing module is used to obtain one or more output branch units based on encoding response data, equipment metadata and the parameters obtained from response file, according to the initial data processing result of preset task configuration activation;Quality gate module is used for quality determination to initial data processing result, and target data processing result is generated based on quality determination result.The present application solves the technical problems of the current computing imaging technology, such as the hard split of sensor hardware, calibration parameters and task link, the lack of sharing mechanism and the dispersion of quality control, which causes the difficulty of adapting, the poor expansion and the inconsistency of the results of the related processing platform.

Description

Technical Field

[0001] This invention relates to the field of computational imaging and multimodal sensing technology, and more specifically, to a sensor data processing system, method, and non-volatile storage medium. Background Technology

[0002] Currently, the main approaches in computational imaging include multispectral image sensors and their filter array structures, joint design schemes for spectral coding front-ends and back-end reconstruction and recognition algorithms, and multispectral-assisted color processing, calibration compensation, and multimodal perception schemes. Although these technologies have made some progress in device design, algorithm optimization, color correction, and multimodal fusion, significant shortcomings remain in platform-based application.

[0003] First, the hardware, calibration, and task chains are fragmented. Sensor structures, calibration parameters, and task models are typically designed separately, leading to inconsistent data organization methods and making it easy to duplicate adaptations, which is detrimental to the subsequent expansion and maintenance of the system. Second, there is a lack of a shared access mechanism for response files. Spectral, color, and physical mode branches often rely on their own independent parameter sets, lacking a parameter architecture that is carried by a unified response file for multiple branches to share and access. In addition, output quality control is fragmented, with each mode function developed separately according to different chains, making it difficult to form a shared quality judgment chain and a stable result encapsulation method. Finally, other physical modes such as polarization and depth usually exist as independent solutions, making it difficult to integrate them into the same response file access relationship and quality judgment chain.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a sensor data processing system, method, and non-volatile storage medium to at least solve the technical problems of current computational imaging technology, such as difficulty in adapting related processing platforms, poor scalability, and inconsistent results caused by the disconnect between sensor hardware, calibration parameters, and task links, the lack of sharing mechanisms, and the decentralized quality control.

[0006] According to one aspect of the present invention, a sensor data processing system is provided, comprising: a data acquisition module for acquiring coded response data and device metadata of a test object, wherein the coded response data is acquired through an encoding front-end, the encoding front-end characterizes a sensor structure that can be described and invoked by a response file module, and the device metadata consists of parameter data of the sensor structure; a response file module for storing and invoking a response file, wherein the response file includes various parameters and rules required for the data processing process, and multiple output branch units share the parameters in the response file; a multi-branch processing module for enabling one or more output branch units according to a preset task configuration based on the coded response data, device metadata, and parameters invoked from the response file, to obtain an initial data processing result, wherein the output branch units include at least one of the following: a spectral output branch unit, a color output branch unit, a task output branch unit, and a physical mode output branch unit; and a quality gating module for performing quality judgment on the initial data processing result and generating a target data processing result based on the quality judgment result.

[0007] Optionally, the multi-branch processing module includes: a receiving unit for receiving encoded response data and device metadata; a calling unit for calling the response file to obtain parameters corresponding to the task configuration; and a starting unit for enabling the corresponding output branch unit according to the task configuration, processing the encoded response data based on the device metadata and parameters corresponding to the task configuration, and obtaining the initial data processing result. The processing result of the physical mode output branch unit is also used to correct the processing results of the spectral output branch unit, the color output branch unit, or the task output branch unit.

[0008] Optionally, the spectral output branch unit is used to output at least one of the following: reconstructed spectrum, spectrogram, low-dimensional spectral features, spectral angle, reconstruction quality score, and metamerism risk warning; the color output branch unit is used to output at least one of the following: true color map, display color map, color coordinate data, color confidence, comprehensive color difference or color correction map; the task output branch unit is used to output at least one of the following: material category, defect area, status level, control mark, result code, and confidence; the physical mode output branch unit is used to output at least one of the following: polarization map, depth map, angle map, event features, phase features, and wavefront features.

[0009] Optionally, it also includes a version maintenance module, which updates the response file module and the multi-branch processing module based on the quality judgment results, and redistributes the updated response file module and the updated multi-branch processing module through local writing, firmware update or network distribution.

[0010] Optionally, the response file includes at least one of the following types of fields: an identity field, including at least one of the following: device identifier of the sensor structure, module identifier of the sensor structure, batch identifier of the sensor structure, type of the sensor structure, and version of the sensor structure; a geometry field, including at least one of the following: pixel-to-channel mapping relationship in the sensor structure, filter array layout of the sensor structure, superpixel layout of the sensor structure, and region of interest relationship corresponding to the task configuration; a spectral field, including at least one of the following: spectral response matrix required for task configuration, quantum efficiency required for task configuration, dark current required for task configuration, crosstalk required for task configuration, and gain parameters required for task configuration; and a color field. Includes at least one of the following: color conversion parameters required for task configuration, illumination set required for task configuration, color correction parameters required for task configuration, and metamerism correction parameters required for task configuration; physical modality fields, including at least one of the following: polarization required for task configuration, depth required for task configuration, angle required for task configuration, initial calibration parameters required for task configuration, and joint response tensor required for task configuration; task fields, including at least one of the following: version of the task model corresponding to the task configuration, label system required for task configuration, and feature normalization rules required for task configuration; quality fields, including at least one of the following: quality threshold, rejection triggering rule, retest triggering rule, and confidence fusion parameters.

[0011] Optionally, the sensor structure includes at least one of the following: a multispectral filter array structure, a metasurface spectral structure, a three-channel and multispectral image dual-mode structure, a spectral and polarization combined structure, a spectral and depth or angle combined structure, a spectral and polarization and depth combined structure, and an event or time-series triggered structure.

[0012] Optionally, the parameter data includes at least one of the following: device identifier, module identifier, batch identifier, mode identifier, exposure time, gain, black level, temperature, sampling timing, region of interest information, superpixel layout information, pixel mapping index, main ray angle, and focus status.

[0013] According to another aspect of the present invention, a sensor data processing method is also provided, applied to any of the sensor data processing systems described above, comprising: acquiring coded response data and device metadata of a test object, wherein the coded response data is acquired through a coding front-end, the coding front-end characterizes a sensor structure that can be described and invoked by a response file module, and the device metadata is parameter data of the sensor structure; invoking a stored response file from the response file module, wherein the response file includes various parameters and rules required for the data processing process, and multiple output branch units share the parameters in the response file; based on the coded response data, device metadata, and parameters invoked from the response file, enabling one or more output branch units according to a preset task configuration to obtain an initial data processing result, wherein the output branch units include at least one of the following: a spectral output branch unit, a color output branch unit, a task output branch unit, and a physical mode output branch unit; performing a quality judgment on the initial data processing result, and generating a target data processing result based on the quality judgment result.

[0014] According to another aspect of the present invention, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above-described sensor data processing method.

[0015] According to another aspect of the present invention, a computer device is also provided, the computer device including a processor, the processor being configured to run a program, wherein the program executes the above-described sensor data processing method during runtime.

[0016] In this embodiment of the invention, a sensor data processing system is employed. A data acquisition module is used to acquire coded response data and device metadata of the object under test. The coded response data is acquired through a coding front-end, which characterizes the sensor structure that can be described and invoked by the response file module. The device metadata consists of parameter data of the sensor structure. A response file module is used to store and invoke response files, which include various parameters and rules required for the data processing. Multiple output branch units share the parameters in the response file. A multi-branch processing module is used to activate one or more output branch units based on the coded response data, device metadata, and parameters invoked from the response file, according to a preset task configuration, to obtain initial data processing results. The output branch units include… At least one of the following: spectral output branch unit, color output branch unit, task output branch unit, physical mode output branch unit; quality gating module, used to judge the quality of the initial data processing results and generate the target data processing results based on the quality judgment results, achieving the purpose of centralized management of response files and realizing shared calling of multiple output branches and consistent quality gating, thereby realizing the platform integration of hardware, sensor calibration parameters and task links, improving cross-task reuse capability, result consistency and system deliverability and scalability, and thus solving the technical problems of current computational imaging technology, such as the separation of sensor hardware, calibration parameters and task links, lack of sharing mechanism and decentralized quality control, resulting in difficulty in adapting related processing platforms, poor scalability and inconsistent results. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0018] Figure 1 This is a structural block diagram of a sensor data processing system provided according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the overall architecture of a sensor data processing system workflow provided by an optional embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of a process for generating one or more outputs from the same acquired data in a sensor data processing system, according to an optional embodiment of the present invention.

[0021] Figure 4 A hardware structure block diagram of a computer terminal for implementing a sensor data processing method is shown.

[0022] Figure 5This is a schematic flowchart of a sensor data processing method provided according to an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] According to embodiments of the present invention, a sensor data processing system is provided. Figure 1 This is a structural block diagram of a sensor data processing system provided according to an embodiment of the present invention, such as... Figure 1 As shown, the system includes:

[0026] The data acquisition module is used to acquire the coded response data and device metadata of the object under test. The coded response data is acquired through the coded front end, which represents the sensor structure that can be described and called by the response file module. The device metadata consists of the parameter data of the sensor structure.

[0027] In this module, the object under test (DUT) refers to the specific physical entity or scene observed, collected, and analyzed by the encoding front-end. It is the input source for the data processing flow; the physical signals emitted or reflected by the DUT (such as spectral radiation, polarization state, depth information, etc.) are captured by sensors and converted into coded response data. The specific form of the DUT depends on the application scenario: in industrial inspection, it may be a circuit board, wafer, or mechanical part used to detect defects, identify material types, or monitor status levels; in agriculture or environmental monitoring, it may be crops, vegetation, or water bodies used to analyze growth status or pollutant concentrations; in consumer electronics or security fields, it may be a face, object, or background scene used for color restoration, recognition, or 3D reconstruction. The DUT itself is not limited to a specific substance but refers broadly to any physical target that can be perceived by sensors and generate resolvable data through spectral, color, polarization, depth, or multimodal characteristics.

[0028] The data acquisition module can integrate an encoding front-end, an image sensor / CMOS (Complementary Metal-Oxide-Semiconductor) readout module, and a RAW and metadata acquisition module. The acquired encoded response data is not the raw physical signal, but rather the signal encoded by the encoding front-end, which is essentially the sensor structure. This sensor structure is defined as an object that can be described and invoked by the response file module. Hardware features such as multispectral filter arrays or metasurfaces are no longer fixed constraints but rather variables that can be parameterized. The device metadata records key parameters of the sensor structure during the acquisition process, such as device ID, module ID, batch ID, mode ID, exposure time, gain, black level, temperature, sampling sequence, ROI (Region of Interest) information, superpixel layout, pixel mapping index, principal ray angle, focus status, or other decoding-related acquisition information, used for subsequent data correction and decoding. Both types of data are ultimately integrated and output through the image sensor / CMOS readout module and the RAW and metadata acquisition module.

[0029] For example, taking PCB board defect detection in an industrial setting as an example, the object under test is a multilayer PCB board being transported on a production line. Its surface includes copper foil traces, solder mask layers, pads, and potential scratches, missing holes, or solder ball defects. The encoding front end used is a CMOS sensor equipped with a multi-spectral filter array (MSFA). Therefore, the encoded response data is the RAW data output by the sensor. Each pixel is not a single color value, but a mixed signal encoded by the filter array. For example, the original RAW value of a pixel may simultaneously contain a mixture of energy from the red light band and the near-infrared band. This undemodulated original pixel matrix, which mixes spatial and spectral information, is the encoded response data, reflecting the original physical response of the PCB surface under specific spectral and spatial encoding.

[0030] Device metadata is determined by sensor acquisition settings and is indirectly related to the object under test. It may include: Device ID (e.g., "PCB-Inspect-01"), Module ID, currently used mode ID (e.g., "High Contrast Defect Detection Mode"), Exposure Time (set to 5ms based on pipeline speed), Gain (set to 2.0 to compensate for insufficient lighting), Black Level (sensor dark current compensation value), Temperature (sensor current temperature 25℃, used for noise correction), Current Focus Status (sharp), Region of Interest (ROI) (only the PCB center inspection area is acquired), and Superpixel Layout (the current filter arrangement of the multispectral filter array). This data records "under what hardware conditions and environmental conditions the above encoded response data was acquired," which is used for subsequent decoding and quality assessment.

[0031] The data acquisition module collects the PCB's encoded response data (mixed RAW signal) and metadata (exposure, gain, temperature, etc.). The response file module calls the corresponding response file based on "PCB-Inspect-01" and "High-Contrast Defect Detection Mode". The response file contains the spectral response matrix (for demixing RAW signal), defect identification model version, quality threshold, etc., required for this specific PCB inspection task. Then, the multi-branch processing module works in parallel: the spectral branch uses the spectral response matrix in the response file to demodulate the encoded response data and reconstruct the true spectral curve of each pixel on the PCB surface to distinguish between copper foil and solder mask; the task branch uses the features extracted by the spectral branch and the defect model in the response file to identify "scratches" or "missing holes" and outputs defect coordinates and confidence levels; the color branch outputs a pseudo-color image of the PCB for easy manual review. The quality gating module checks the consistency of the results from each branch (e.g., whether the spectral reconstruction quality meets the standards, and whether the defect confidence level is higher than the threshold). If consistent, a result package containing defect location, type, confidence level, and original data fingerprint is generated.

[0032] In this way, the data acquisition module not only captures physical signals but also transforms hardware characteristics, i.e., device metadata, into a digital description that can be uniformly managed by the response file. Traditional computational imaging solutions often hard-code and bind sensor hardware with back-end decoding algorithms. Once the sensor structure is changed (e.g., from MSFA to metasurface), the back-end algorithm must be redeveloped. In this embodiment, however, the data acquisition module is only responsible for acquiring the most raw encoded response data (RAW) and basic device metadata (such as exposure, gain, and temperature). Regardless of whether the front end is MSFA, metasurface, or other composite structure, the data acquisition module outputs a standardized "raw pixel stream + metadata". This standardization eliminates the need to reconstruct the underlying hardware acquisition link due to changes in the front-end sensor, thus achieving universality. This allows the same acquisition base, i.e., the hardware acquisition link, to adapt to various different sensor structures and task requirements. This lays the foundation for subsequent multi-branch processing modules to share and call channel mappings, calibration parameters, and quality rules in the response file, thereby solving the problem of hardware and link separation in related technologies and improving the system's cross-task reuse capability and result consistency.

[0033] In addition, the acquired encoded response data can be preprocessed. For example, the sensor data processing system can perform pre-corrections on the encoded response data, such as black level, flat field, gain, temperature, and crosstalk. Subsequently, based on the superpixel layout, pixel mapping, or ROI rules in the response file, local windows, multi-channel vectors, or branch decoding tensors are constructed for subsequent branch decoding calls.

[0034] In computing imaging sensors, due to the use of multispectral filter arrays or metasurface encoding, a single physical pixel does not capture light of a single color, but rather a signal resulting from the mixing of multiple spectral bands. To reconstruct the complete spectral or color information, several or a group of adjacent physical pixels need to be combined for processing. This set of working physical pixels is called a "superpixel." Pixel mapping refers to the correspondence between the physical pixel positions on the sensor and their logical spectral channels or spatial positions. In complex encoding front-ends (such as irregularly arranged filters, metasurfaces, or sensors with dead pixels), physical pixel coordinates do not directly correspond to standard RGB or spectral channel indices. Pixel mapping establishes a lookup table or transformation rule that specifies which spectral bands the RAW data at a physical pixel primarily responds to, or which logical position in the output image it should be reconstructed to. ROI (Region of Interest) refers to the region of interest in an image. In the imaging process, not all pixels are equally important, or certain specific regions (such as areas with high defect rates or high-precision central areas) require special processing. ROI rules define which coordinate ranges in the image belong to the region of interest and what special processing strategies (such as higher resolution, different decoding parameters, or ignoring background noise) should be applied to these regions. A local window refers to a block of pixels within a certain range (e.g., 3x3, 5x5, 7x7) surrounding a central pixel or superpixel in an image processing algorithm. A multichannel vector is a one-dimensional array (vector) that arranges the data values ​​of all spectral bands belonging to the same superpixel or the same logical location. A branch decoding tensor is a multidimensional data structure specifically constructed for a particular output branch (such as a spectral output branch or a color output branch) that contains the necessary input data. Different output branches require different data formats. For example, a spectral output branch may require a 3D tensor (Height x Width x Bands) containing spatial information and spectral dimensions, while a task output branch may require a 2D feature map tensor after feature extraction. Constructing a branch decoding tensor requires extracting, reorganizing, and formatting a data structure suitable for the direct input of the algorithm for that branch, based on the requirements of the currently enabled branch, to avoid the problem of mismatch between general data formats and specific algorithm requirements.

[0035] The response file module is used to store and retrieve response files. These response files contain various parameters and rules required for the data processing, and multiple output branch units share the parameters retrieved from the response file.

[0036] In this module, the response file is not a single-dimensional calibration data, but a comprehensive configuration set including multiple dimensions such as identity, geometry, spectrum, color, physical modality, task and quality. Specifically, it may include at least one of the following: channel mapping, theoretical response, measured response, calibration parameters, model version, quality threshold, rejection rules, retesting rules, and version information.

[0037] The sensor is comprised of several fields: Identity Fields (configuration parameters for uniquely identifying the sensor hardware entity, production batch, and software version status); Geometric Fields (configuration parameters for describing the sensor's spatial distribution characteristics, pixel arrangement topology, and optical geometric properties); Physical Modality Fields (configuration parameters for describing the encoding mechanism and decoding parameters of physical quantities other than spectral intensity); Spectral Fields (configuration parameters for describing the sensor's response characteristics to light signals of various wavelengths and its noise model); Color Fields (configuration parameters for converting the sensor's raw signals into standard color space data and controlling color consistency); Task Fields (logical rules and model parameters for performing data analysis, identification, and decision-making in specific application scenarios); and Quality Fields (rules for evaluating the validity and confidence level of data processing results and triggering quality control actions).

[0038] Channel mapping defines the correspondence between the physical pixel array of the sensor and logical spectral channels or spatial locations, solving the spatial demodulation problem of raw pixel data to physically meaningful data; theoretical response refers to the spectral response matrix, quantum efficiency curve, or coding structure parameters pre-calculated based on the sensor design specifications or simulation model, serving as the initial benchmark value for the decoding algorithm; measured response refers to the correspondence between the actual output data of the sensor and the input optical signal obtained from the actual production line using a preset standard light source or calibration equipment, used to correct the deviation between theoretical values ​​and individual differences in actual hardware; calibration parameters cover one or more sets of values ​​such as black level, flat field correction coefficient, gain compensation, crosstalk correction, and temperature drift compensation to eliminate sensor noise and nonlinear distortion; model version identifier is used to distinguish data processing algorithms and deep learning networks at different iteration stages. The structure or feature extraction logic ensures that the backend module calls match the hardware characteristics of the processing model; the quality threshold sets numerical boundaries for judging the validity of the output results, such as the lower limit of signal-to-noise ratio, the upper limit of reconstruction error, or the confidence requirement. If the result exceeds the threshold, it is judged as a low-quality result; the rejection rule defines the logical judgment conditions for directly discarding the current processing result and marking it as invalid under specific conditions (such as data anomalies, low confidence, or consistency conflicts); the retest rule specifies the proactive intervention measures to be taken when rejection is triggered or potential uncertainties are discovered, such as adjusting the exposure gain, switching the illumination mode, changing the acquisition angle, or starting a secondary scan to obtain more accurate data; the version information records the release version number, build date, and change log of the response file, associated algorithm model, and firmware, which is used to realize the system's traceability, status management, and automated update mechanism.

[0039] This mechanism overcomes the shortcomings of traditional solutions where hardware, calibration, and task chains are fragmented, enabling different branches such as spectral output, color output, task recognition, and physical modality decoding to process in parallel based on the same acquired data, the same set of response parameters, and quality rules. This shared calling not only ensures the consistency of parameter sources in each branch during processing, effectively reducing conflicts caused by parameter dispersion and improving result consistency, but also enables descriptive constraints on the front-end encoding structure by the response file. Regardless of whether the front-end is a traditional filter array or a novel metasurface structure, it can be described by this file and called by the back-end on demand, thereby significantly improving cross-task reuse capability, system deliverability, and subsequent multimodal expansion capabilities.

[0040] The multi-branch processing module is used to enable one or more output branch units based on the encoded response data, device metadata, and parameters called from the response file, according to the preset task configuration, to obtain the initial data processing result. The output branch units include at least one of the following: spectral output branch unit, color output branch unit, task output branch unit, and physical mode output branch unit.

[0041] This module first receives the encoded response data and accompanying device metadata output from the encoding front-end. Then, based on the preset task configuration, it accurately extracts the corresponding channel mappings, calibration parameters, model parameters, and quality rules from the response file. Based on these shared parameters, the multi-branch processing module dynamically activates at least one of the following: spectral output branch unit, color output branch unit, task output branch unit, or physical mode output branch unit. Each branch unit executes decoding, reconstruction, or recognition algorithms in parallel or serially to generate initial data processing results. These results may include, for example, spectral curves output by the spectral output branch unit, color images output by the color output branch unit, material categories output by the task output branch unit, or polarization depth maps output by the physical mode output branch unit.

[0042] Specifically, the results from the physical modality output branch unit can be used to correct the outputs of other branches, thereby improving overall accuracy. In multimodal computational imaging systems, auxiliary physical dimension information output by the physical modality output branch unit, such as polarization, depth, angle, event, or fluorescence, is used as prior knowledge or compensation parameters to correct or optimize errors in other output branches. For example, when detecting defects on a smooth metal surface, the physical modality output branch unit outputs pixel-level polarization information, identifying high-polarization regions as specular reflection light. This identification result is then used as a mask or compensation coefficient to subtract the specular reflection component from the original signal of the spectral output branch unit, thus restoring the true diffuse reflection spectrum of the metal substrate and avoiding misjudging highlights as defects. Alternatively, when photographing objects with complex curved surfaces, the physical modality output branch unit outputs the surface normal vector for each pixel. This normal vector is used to calculate the incident angle and viewing angle, and then the geometric correction parameters in the response file are called to perform angle-dependent brightness attenuation compensation and color shift correction on the spectral reconstruction results of the spectral output branch unit, eliminating edge color differences caused by changes in viewing angle.

[0043] This multi-branch architecture based on shared calls to response files avoids the fragmentation of parameters across different links, enabling flexible combination and unified processing of different physical modalities and task requirements. It provides a standardized input foundation for subsequent quality gating and result encapsulation. By using the response file as the core hub, the originally independent hardware sensor encoding features, multimodal decoding logic, and business task objectives are parametrically decoupled and dynamically correlated. Regarding the flexible combination of physical modes, the geometric and physical mode fields in the response file define pixel mappings and response matrices for different encoding front-end structures, ranging from traditional multispectral filter arrays to metasurface polarization encoding. This allows the same acquisition link to adapt to various physical quantity acquisition needs without hardware changes. Regarding the flexible combination of task requirements, one or more of the spectral, color, task, or physical mode output branches are dynamically enabled based on the task configuration. For example, in a quality inspection scenario, the spectral output branch can be enabled simultaneously for material analysis, while the physical mode output branch can be enabled to eliminate reflective interference. In a conventional display scenario, only the color output branch can be enabled. This on-demand loading mechanism enables the elastic scaling of the processing link. Regarding unified processing, all enabled output branches share the same calibration parameters, quality thresholds, and model versions in the response file. The quality control module performs consistency checks and confidence assessments on the results of each output branch based on the same set of standards. Finally, a standardized result package containing multi-branch data, quality indicators, and version information is output through result encapsulation, thus breaking the limitations of isolated link parameters and scattered result formats in traditional solutions.

[0044] The quality gating module is used to determine the quality of the initial data processing results and generate the target data processing results based on the quality determination results.

[0045] This module receives initial data processing results generated by the multi-branch processing module. These results may come from one or more of the spectral, color, task, or physical modality output branches. Based on the quality fields stored in the response file, including quality thresholds, rejection rules, retest trigger rules, and confidence fusion parameters, the module performs quality judgment, checks consistency, scores quality, rejects results, reduces weight, or suggests retesting for the results of the enabled output branches. It compares the correlation between the enabled branches and calculates the quality score and confidence of each branch. Specifically, it first extracts configuration information from the response file, such as the quality threshold (the numerical boundary for determining whether a single output branch result is valid), rejection rules (the specific handling strategy when the result is invalid), retest trigger rules (under what conditions a secondary acquisition or correction action is initiated), and confidence fusion parameters (integrating the confidence scores of multiple parallel output branches, and using weighted averaging or other algorithmic logic to integrate the independent judgments of each branch into a unified final result quality assessment value). Then, it performs multi-dimensional evaluation and verification of the results of the enabled output branches. During the consistency check phase, the quality gating module calculates the correlation or logical consistency between the results of each enabled branch to determine whether there are data conflicts. For example, it checks whether the spectral reconstruction results of the spectral output branch unit and the color restoration results of the color output branch unit match in a physical sense, or whether the results of the physical mode output branch match in spatial structure. Based on this, the quality score and confidence of each branch are evaluated according to the quality threshold in the response file. If the score of a branch is lower than the quality threshold or the confidence is insufficient, a rejection rule is triggered, marking the result as invalid and refusing to output it, or performing a weight reduction process, that is, reducing the confidence weight of the branch result in the final result package instead of discarding it directly. At the same time, when uncertainty, inter-branch conflict or specific abnormal patterns are detected in the results, a retest suggestion is generated according to the retest trigger rule. This may include proactive intervention measures such as adjusting the acquisition parameters, switching physical modes or starting a second scan. Finally, the quality evaluation results of each branch are weighted and fused through the confidence fusion parameter to form a unified quality feedback, thereby ensuring the reliability, consistency and traceability of the multi-branch output results.

[0046] When the quality assessment result is outside the valid range or a conflict occurs between branches, the quality gating module determines it as unreliable, triggering rejection, weight reduction, or a suggestion for active retesting. This could involve adjusting the lighting or switching the acquisition angle. Adjusting the lighting refers to dynamically changing the physical parameters of the light source in the acquisition environment to optimize data acquisition conditions based on the cause of the current low image quality or branch conflict. For example, adjusting the light source intensity to resolve abnormal spectral saturation caused by underexposure or overexposure, changing the color temperature or spectral distribution of the light source to eliminate color distortion caused by metamerism, or switching the polarization state of the light source (e.g., switching from natural light to linearly polarized or circularly polarized light) to suppress specular reflection interference from smooth surfaces and thus extract true diffuse light. The system employs various methods to address 3D reconstruction. These methods include: spectral analysis, using multiple light sources simultaneously or alternately to illuminate objects at different angles to capture their features and resolve shadow occlusion issues; and switching the acquisition angle by controlling the sensor structure's mechanical movement to alter the geometric relative position of the image. This can be achieved by adjusting the principal ray angle or tilt angle of the sensor structure to avoid optical edge regions that cause severe distortion or crosstalk, changing the shooting distance to optimize depth of field and resolve spatial blur caused by defocus, thus affecting the consistency between the depth and spectral branches, rotating the object or sensor structure to acquire multi-view data from different sides of the object to eliminate parallax occlusion or structural blind spots from a single viewpoint, or using multi-focal plane scanning to acquire clear images at different depths to improve 3D reconstruction accuracy. Conversely, if the results meet quality requirements, no further processing is required. Subsequently, the quality gating module can uniformly encapsulate the initial data processing results, corresponding quality scores, confidence levels, reasons for rejection or retesting suggestions, and common fields such as response file version, device identifier, and batch identifier to obtain the target data processing results, forming a standardized result package. This mechanism ensures logical consistency of results from different modalities based on sharing the same response parameters, improving the deliverability, traceability, and stability of the system results in complex environments.

[0047] In this context, inter-branch conflicts refer to irreconcilable inconsistencies or contradictions in the results generated by different output branches when processing multiple outputs based on the same set of original acquired data and the same response file. For example, the image output by the color output branch unit may visually appear as a red object, but the spectral curve reconstructed by the spectral output branch unit shows that the region mainly has green band reflection characteristics. Or, the surface normal vector identified by the physical mode output branch unit may not match the geometric structure in spatial morphology, leading to results that violate optical principles or geometric common sense when interpreting the same object from different physical dimensions.

[0048] Through the above system, the goal of centralized management of response files and sharing and calling of multiple output branches and consistency quality gating is achieved. This realizes the platform integration of hardware, calibration and task links, improves cross-task reuse capability, result consistency and system deliverability and scalability. In turn, it solves the technical problems of current computational imaging technology, such as the separation of sensor hardware, calibration parameters and task links, lack of sharing mechanism and decentralized quality control, which leads to the difficulty in adapting related processing platforms, poor scalability and inconsistent results.

[0049] As an optional embodiment, the multi-branch processing module includes: a receiving unit for receiving encoded response data and device metadata; a calling unit for calling a response file to obtain parameters corresponding to the task configuration; and a starting unit for enabling the corresponding output branch unit according to the task configuration, processing the encoded response data based on the device metadata and parameters corresponding to the task configuration, and obtaining initial data processing results. The processing results of the physical mode output branch unit are also used to correct the processing results of the spectral output branch unit, the color output branch unit, or the task output branch unit.

[0050] Optionally, the multi-branch processing module serves as the core data processing unit of the platform. Its internal logic is implemented through the collaborative work of the receiving unit, the calling unit, and the starting unit. The receiving unit is responsible for acquiring the encoded response data collected by the encoding front-end and the device metadata related to the sensor structure, ensuring the complete input of raw data and physical state information. The calling unit extracts and loads a shared set of parameters from the response file module according to the current task configuration. These parameters cover channel mapping, calibration coefficients, model weights, and quality rules, providing a unified data benchmark for subsequent processing. The starting unit dynamically activates the corresponding output branch unit according to the preset task configuration, using the device metadata and calling parameters to decode and reconstruct the encoded response data.

[0051] Specifically, the calling unit first identifies the task the user wants to perform, such as "identifying material composition" or "generating a true-color photograph," which determines the subsequent processing path. Next, based on the current camera hardware model (e.g., device ID) and operating mode (e.g., mode ID), it retrieves a set of parameters tailored to the specific sensor structure from storage. Here, channel mapping indicates whether a pixel on the sensor corresponds to the red or blue light band, and what angle or polarization state a pixel corresponds to. Calibration coefficients are used to eliminate noise, dark current, or sensitivity differences between pixels, ensuring data accuracy. Model weights are core parameters of the intelligent algorithm; for example, deep learning network parameters for spectral reconstruction or classifier parameters for material classification determine how to process... Data; quality rules are the acceptance criteria, specifying what results are considered acceptable and what results should be rejected; subsequently, the startup unit does not run all functions simultaneously to save resources, but acts like a switch, only turning on the "spectral output branch unit" and "task output branch unit" if the task is "material identification", and only turning on the "color output branch unit" if the task is "taking pictures". This on-demand activation enables flexible combination; finally, combining the metadata recorded during acquisition (such as environmental information such as exposure time, temperature, gain, etc.) and the previously loaded calibration and mapping parameters, mathematical operations and demodulation are performed on the raw RAW data read from the sensor. For example, pixel signals that mix multiple spectral information are restored to pure spectral curves or RGB color values ​​through mathematical inversion or neural networks.

[0052] In this process, the physical mode output branch unit not only independently outputs additional physical quantities such as polarization, depth, or angle, but also feeds its processing results back to the spectral, color, or task output branches as correction quantities to compensate for errors caused by specular reflection, environmental interference, or sensor crosstalk, thereby improving the accuracy and robustness of the final result. This mechanism is reflected in the internal data flow and logical processing flow. That is, after the physical mode output branch (such as polarization or depth) completes the initial decoding, the specific numerical value or recognition feature output by it is not only stored as an independent result, but can also be extracted in real time and passed as input parameters to the processing algorithms of other branches. For example, when the physical mode output branch unit calculates the degree of polarization or polarization angle of each pixel, it identifies high polarization regions as specular reflection light and uses this identification result or estimated reflected light intensity as a "compensation amount" to subtract this interference from the original signal of the spectral output branch unit, thereby restoring the true diffuse reflection spectrum of the object. Alternatively, when the physical mode output branch unit outputs a surface normal vector, it uses this normal vector to calculate the incident angle of light and then retrieves the corresponding angle correction coefficient from the response file to compensate for geometric distortion or brightness attenuation of the color values ​​of the color output branch unit. Similarly, when the task output branch unit performs material identification or defect detection, it can call the non-reflective mask or 3D structural information provided by the physical mode output branch unit as prior knowledge to correct misjudgments caused by uneven lighting or changes in viewing angle. This operation of directly injecting the output of the physical mode branch into the calculation model or post-processing logic of other branches as a correction factor demonstrates its role as a correction amount. This architecture enables deep fusion and mutual correction of multimodal data, avoiding parameter conflicts or quality inconsistencies caused by isolated processing of each branch, and ensuring high consistency and traceability of multi-task outputs under the same response file constraint.

[0053] As an optional embodiment, the spectral output branch unit is used to output at least one of the following: reconstructed spectrum, spectrogram, low-dimensional spectral features, spectral angle, reconstruction quality score, and metamerism risk warning; the color output branch unit is used to output at least one of the following: true color map, display color map, color coordinate data, color confidence level, comprehensive color difference or color correction map; the task output branch unit is used to output at least one of the following: material category, defect area, status level, control mark, result code, and confidence level; the physical mode output branch unit is used to output at least one of the following: polarization map, depth map, angle map, event features, phase features, and wavefront features.

[0054] Optionally, the spectral output branch unit is responsible for reconstructing and outputting spectral curves, spectral distribution maps, low-dimensional feature vectors, spectral angular distances, reconstruction quality scores, and metamerism risk warnings from the raw coded data to meet the needs of spectral analysis. The color output branch unit outputs true color maps, display color maps, color coordinate data (e.g., XYZ data, Lab data, RGB data), color confidence scores, and comprehensive color difference or color correction maps based on color conversion parameters, ensuring the accuracy and consistency of visual presentation. The task output branch unit combines the task model to output material categories, defect areas, status levels, control markers, result codes, and confidence scores, directly serving industrial inspection and decision-making scenarios. The physical modality output branch unit extends the system's capabilities, outputting polarization maps, depth maps, angle maps, event features, phase features, and wavefront features, supporting the perception of more complex physical quantities. These branch units share parameters in the same response file, achieving unified processing from single acquired data to multi-dimensional structured results, demonstrating the technical advantages of the computational imaging platform in functional integration and modular output.

[0055] Among them, the spectral curve is a continuous function image describing the intensity of light reflected or emitted by an object at different wavelengths, reflecting the intrinsic optical properties of the material. Low-dimensional feature vectors are compact representations of high-dimensional spectral data, retaining key information for easier subsequent algorithm processing. Spectral angular distance measures the angular difference between two spectral vectors; a smaller value indicates higher spectral similarity and is often used for material identification. Metamerism risk refers to two objects appearing the same color under a specific light source but having completely different spectral distributions, resulting in inconsistent color performance under different light sources. Color coordinates XYZ and Lab are standardized color spaces that convert colors into numerical descriptions for accurate quantification and comparison. Comprehensive color difference is a numerical value measuring the degree of difference between two colors, used to evaluate the accuracy of color reproduction. Material category is the type of object determined based on spectral or image features, such as metal, plastic, or organic matter. Defect areas mark the specific locations of flaws, cracks, or contamination in the image. Status level is a classification of the object's condition, such as excellent, good, or substandard. Control marks and result codes are output instructions or status identifiers used for subsequent operational decisions of automated equipment. Polarization maps display the polarization state of reflected light from an object's surface, helping to differentiate materials and remove reflections. Depth maps provide distance information from each pixel in the scene to the sensor structure, constructing a 3D structure. Angle maps reflect the direction of light incidence or reflection, used to analyze surface geometry. Event features record the time points of drastic changes in light intensity, suitable for capturing high-speed motion scenes. Phase and wavefront features describe the wave properties of light waves; phase reflects the wave state, and wavefront describes the shape of the wavefront. These physical quantities can be used to analyze the stress distribution of transparent objects or the accuracy of optical components. All branches share calibration parameters in the response file, ensuring consistency in the output of different physical quantities, achieving unified processing from single RAW data to multi-dimensional structured information, and improving the system's functional integration.

[0056] As an optional embodiment, it also includes a version maintenance module, which is used to update the response file module and the multi-branch processing module based on the quality judgment result, and redistribute the updated response file module and the updated multi-branch processing module through local writing, firmware update or network distribution.

[0057] Optionally, the version maintenance module constructs a closed-loop feedback and iteration mechanism based on quality judgment results to continuously optimize system performance in response to changes in the field environment and model drift issues. This module receives quality judgment results from the quality gating module and automatically triggers an update process when insufficient confidence, high rejection rate, or specific error samples are detected. Specifically, the version maintenance module uses misjudged samples, drift data, and retest results collected on-site to generate an update queue, thereby correcting parameter configurations in the response file or adjusting processing logic and model weights in the multi-branch processing module.

[0058] The version maintenance module's process of generating an update queue begins with intelligent filtering and classification of data reported by various branches. First, from misclassified samples collected on-site, confidence thresholds and manual verification markers are used to identify difficult samples with low confidence, high error, or those that the model cannot classify correctly. Simultaneously, it monitors calibration parameter drift data caused by environmental changes during long-term operation and records of failed retests. Then, the version maintenance module encapsulates these high-value data in a unified format, adds source tags, timestamps, and error type identifiers, and stores them in a temporary cache. When the cached data accumulates to a predetermined quantity or triggers a specific update cycle, it automatically writes it in batches to a persistent update queue database. This queue not only stores the raw data but also includes preprocessed feature vectors and corresponding correct labels (if any), thus forming a structured feedback dataset for subsequent version iterations to use for model retraining or response file parameter calibration.

[0059] The correction process first involves cleaning and labeling the data in the update queue using offline analysis tools to identify feature patterns or calibration drift trends that lead to misjudgments. For response file parameters, calibration coefficients are recalculated based on new data. For example, drift data is used to correct the spectral response matrix to eliminate the influence of ambient light, or quality threshold rules are adjusted to adapt to new scene distributions. For processing logic and model weights, samples containing correct labels are used for incremental training or fine-tuning of the task model or reconstruction algorithm. This optimizes the weight parameters of the neural network to correct feature extraction biases, or the rule engine updates the logical branches of the decoding algorithm. This solidifies the correction knowledge accumulated on-site into the next version of the response file or model file, achieving closed-loop performance optimization.

[0060] The update process encompasses the evolution of response file and model versions, ensuring that the updated parameters and algorithms better align with the current physical modal characteristics and task requirements. Subsequently, the version maintenance module redistributes the updated response file module and multi-branch processing module to the target device or cloud server through various communication methods such as local writing, firmware upgrades, or network distribution, enabling online upgrades and self-improvement of system capabilities. The updated response file module and multi-branch processing module can include updated response files, model versions, quality thresholds, retesting rules, or processing parameters. This mechanism not only solves the maintenance difficulties caused by the disconnect between calibration and task links in related technologies but also significantly improves the platform's result consistency, deliverability, and adaptability to new physical modalities or lighting conditions during long-term operation, forming a complete lifecycle management closed loop from data acquisition and quality assessment to version iteration.

[0061] As an optional embodiment, the response file includes at least one of the following types of fields: Identity fields, including at least one of the following: device identifier of the sensor structure, module identifier of the sensor structure, batch identifier of the sensor structure, type of the sensor structure, and version of the sensor structure; Geometric fields, including at least one of the following: pixel-to-channel mapping relationship in the sensor structure, filter array layout of the sensor structure, superpixel layout of the sensor structure, and region of interest relationship corresponding to the task configuration; Spectral fields, including at least one of the following: spectral response matrix required for task configuration, quantum efficiency required for task configuration, dark current required for task configuration, crosstalk required for task configuration, and gain parameter required for task configuration; Color fields; The fields include at least one of the following: color conversion parameters required for task configuration, illumination set required for task configuration, color correction parameters required for task configuration, and metamerism correction parameters required for task configuration; physical mode fields include at least one of the following: polarization required for task configuration, depth required for task configuration, angle required for task configuration, initial calibration parameters required for task configuration, and joint response tensor required for task configuration; task fields include at least one of the following: version of the task model corresponding to the task configuration, label system required for task configuration, and feature normalization rules required for task configuration; quality fields include at least one of the following: quality threshold, rejection triggering rule, retest triggering rule, and confidence fusion parameters.

[0062] Optionally, the identity field ensures hardware traceability and configuration matching through device, module, batch identifier, and structural type version; the geometry field covers pixel channel mapping, filter array layout, and region of interest relationships, providing a spatial topology basis for decoding; the spectral field contains physical parameters such as response matrix, quantum efficiency, dark current, and crosstalk gain, supporting spectral reconstruction and preprocessing; the color field integrates conversion parameters, illumination sets, and metamerism correction parameters to ensure color reproduction accuracy; the physical mode field records initial calibration parameters such as polarization, depth, angle, and joint response tensor, supporting multimodal fusion and compensation; the task field associates model version, label system, and feature normalization rules to adapt to the algorithm requirements of different application scenarios; the quality field defines thresholds, rejection and retest trigger rules, and confidence fusion parameters to construct a unified quality gating mechanism. These fields together constitute a standardized parameter carrier, enabling different output branches to perform consistent processing based on the same response file, solving the problem of hardware, calibration, and task link fragmentation in related technologies, and improving system reusability, result consistency, and scalability.

[0063] Among them, the device, module, and batch identifiers serve as index keys to uniquely identify the sensor hardware and its production information, while the structure type version records the generation or modification history of the current sensor structure. The combination of these two ensures that the response file loaded by the system is completely matched with the physical hardware, preventing decoding errors caused by hardware mismatch. Pixel channel mapping describes the correspondence between a single pixel of the sensor and a specific spectral or physical mode channel. Filter array layout refers to the geometric arrangement rules of encoding structures such as MSFA or metasurfaces. Region of interest relationships define the pixel neighborhood range required for local decoding. These together constitute the spatial topological basis for restoring the original signal to physically meaningful data. The response matrix quantifies the sensitivity of each channel to different wavelengths of light, quantum efficiency reflects the ability of photons to convert into electrons, dark current is the noise current in the absence of light, and crosstalk gain describes the degree of signal interference between adjacent channels or wavelengths. These physical parameters are the core basis for black level correction, flat field correction, and spectral inversion reconstruction. Transformation parameters map the sensor's raw response to a standard color space such as XYZ or Lab. The illumination set defines the standard light source spectral distribution applicable to the algorithm. Metamerism correction parameters compensate for color deviations caused by variations in the light source. These three parameters work together to eliminate the influence of ambient light and achieve high-precision color reproduction. Initial calibration parameters such as polarization, depth, and angle record the baseline response characteristics of each physical mode. The joint response tensor describes the coupling relationship between different modes. These data are used for inter-modal compensation after decoding, such as using depth information to correct spectral distortion caused by viewing angle, achieving the fusion and enhancement of multimodal information. The model version identifies the currently used algorithm model number, the label system defines the specific category definition for the classification or regression task, and the feature normalization rules ensure that the input data conforms to the numerical distribution expected by the model. These fields enable the platform to flexibly adapt to the specific algorithm requirements of diverse industrial scenarios such as material identification and defect detection. The quality threshold defines the boundary value for determining whether the result is valid, the rejection rule clarifies the handling strategy when the result is unreliable, the retest trigger rule indicates when to start secondary data collection, and the confidence fusion parameter is used to integrate the confidence of multiple branches. These rules together construct a standardized quality gating mechanism to ensure the consistency and reliability of the final output result.

[0064] As an optional embodiment, the sensor structure includes at least one of the following: a multispectral filter array structure, a metasurface spectral structure, a three-channel and multispectral image dual-mode structure, a spectral and polarization combined structure, a spectral and depth or angle combined structure, a spectral and polarization and depth combined structure, and an event or time-series triggered structure.

[0065] Optionally, the core feature of the sensor structure lies in its ability to be described and invoked by the response file module, rather than being limited to a specific physical form. This structure can encompass: multispectral filter array structures, achieving spectral separation through specific filter layers; metasurface spectral structures, using nano-optical elements to encode the light field; three-channel and multispectral image dual-mode structures, fusing conventional RGB and multispectral data to enhance color reproduction; spectral and polarization joint structures, simultaneously acquiring spectral and polarization information to suppress reflection interference; spectral and depth or angle joint structures, integrating spatial geometric information to assist in 3D reconstruction; spectral, polarization, and depth joint structures, achieving synchronous perception of multidimensional physical quantities; and event or time-triggered structures, acquiring data based on dynamic event streams. These structures collectively constitute the platform's input foundation, and its output encoded response data includes multimodal information such as spectrum, color, polarization, and depth, with acquisition parameters recorded through device metadata. Regardless of the specific hardware form used, as long as its encoding mechanism can be described and parameterized by the geometric, spectral, or physical modal fields in the response file, the platform can flexibly enable different output branches according to task configuration, achieving cross-task reuse and result consistency control.

[0066] Specifically, the multispectral filter array structure covers the sensor pixels with a filter layer having specific transmission band characteristics, separating the incident light according to wavelength and distributing it to different pixels to achieve spatial sampling of spectral information. The metasurface spectral structure utilizes subwavelength-scale nano-optical elements to alter the phase, amplitude, or polarization state of light, performing complex encoding of the incident light field and reconstructing spectral information through computational decoding. The three-channel and multispectral image dual-mode structure simultaneously outputs conventional RGB images and high-resolution multispectral data, using the high spatial resolution of RGB to assist multispectral data, improving the visual effect and detail of color reproduction. The spectral and polarization joint structure records the polarization state of light while acquiring spectral signals, using polarization information to distinguish between diffuse reflection and specular reflection, thereby effectively suppressing surface reflection interference and obtaining the true spectral characteristics of objects. The spectral and depth or angle joint structure integrates three-dimensional spatial geometric information, using depth or viewpoint data to correct spectral sampling errors caused by surface tilt or distance changes, assisting in the construction of high-precision three-dimensional spectral models. The spectral, polarization, and depth joint structure simultaneously acquires spectral, polarization, and depth information, achieving comprehensive perception from material composition and surface state to spatial structure, supporting more complex scene understanding and analysis. Event or time-triggered structures are based on data acquisition triggered by drastic changes in light intensity or external signals, rather than fixed-frequency exposure. They are suitable for capturing high-speed motion or transient optical phenomena, improving acquisition efficiency in dynamic scenes.

[0067] As an optional embodiment, the parameter data includes at least one of the following: device identifier, module identifier, batch identifier, mode identifier, exposure time, gain, black level, temperature, sampling timing, region of interest information, superpixel layout information, pixel mapping index, main ray angle, and focus status.

[0068] Optionally, device metadata, serving as parameter data for the sensor structure, aims to provide the underlying physical environment and acquisition state constraints for decoding the encoded response data. Among these, device identifier, module identifier, batch identifier, and mode identifier constitute an identity index, used to accurately locate the corresponding sensor structure, ensuring the uniqueness and traceability of parameter calls. Exposure time, gain, black level, and temperature reflect the photoelectric signal intensity and environmental state at the moment of sensor structure acquisition, serving as the basis for black level subtraction, flat field correction, and noise suppression. Sampling timing records the temporal dimension information of photon capture by the sensor structure, supporting event triggering or dynamic scene processing. Region of interest information defines the effective imaging range, while superpixel layout information and pixel mapping index define the spatial sampling structure, directly determining the decoding algorithm's reconstruction logic from original pixels to physical channels. The principal ray angle reflects the incident light angle characteristics, used to correct microlens crosstalk or metasurface dispersion deviation. Focus status is associated with image sharpness and depth of field, assisting in judging data validity or triggering retesting. These parameters collectively constitute the context environment required for decoding, enabling multi-branch processing modules to achieve accurate mapping and consistent quality control of spectral, color, and task branches based on the same set of metadata and response files.

[0069] Specifically, device identifier, module identifier, batch identifier, and mode identifier are index keys used to uniquely identify the camera hardware identity, production batch, and current operating mode. This ensures accurate loading of response file parameters matching the specific sensor hardware configuration, guaranteeing traceability and consistency in data processing. Exposure time refers to the length of time the sensor receives light signals; gain is the amplification factor of the electrical signal; black level is the sensor's basic output value in the absence of light; and temperature is an indicator of the sensor's operating environment. These parameters collectively describe the photoelectric signal intensity and environmental conditions during acquisition, used to correct noise and baseline drift caused by temperature and signal intensity. Sampling timing records the specific time point at which each pixel captures photons, rather than simple continuous frames. This is crucial for capturing high-speed moving objects or responding to external events, supporting data processing with non-uniform time intervals. Region of interest information defines the effective pixel range in the image that needs to be processed, helping to reduce computation and focus on key areas. Superpixel layout refers to the arrangement of multiple physical pixels into a logical unit to capture specific modal information, while pixel mapping index establishes the correspondence between physical pixel positions and spectral or physical channels. These two directly determine how the decoding algorithm restores the original signal into physically meaningful data. The principal ray angle describes the angle between the light ray entering the lens and the optical axis. It is used to correct image edge distortion and color deviation caused by reduced microlens focusing efficiency due to oblique light incidence or inconsistent metasurface dispersion. Focus status reflects the current image sharpness and depth of field, used to assess the effectiveness of image quality. For example, it may indicate unusable data or trigger refocusing in cases of severe defocusing to ensure the accuracy of subsequent decoding results.

[0070] As an optional embodiment, Figure 2 This is a schematic diagram of the overall architecture of a sensor data processing system workflow provided by an optional embodiment of the present invention, as shown below. Figure 2 As shown, the target object (i.e., the object under test) first enters the encoding front end; the encoding front end can be an MSFA, RGB+MSI, metasurface, or other composite structure. The encoding front end outputs raw data and metadata (i.e., encoded response data and device metadata); simultaneously, the system calls the response file corresponding to the current task configuration. The raw data, metadata, and relevant parameters in the response file work together in the subsequent processing chain. The system then enables one or more branches of spectral output, color output, task output, and physical modality output according to the task configuration, and performs quality gating on the branch results, finally generating the result package.

[0071] and Figure 3 This is a schematic diagram of a process for generating one or more outputs from the same acquired data in a sensor data processing system, according to an optional embodiment of the present invention. Figure 3As shown, after acquiring raw data and metadata, the system calls the mappings, response parameters, and quality rules in the response file to perform pre-correction on the data and construct a local window or decoding tensor. Subsequently, based on the task configuration, it enables one or more of the following branches: spectral output branch, color output branch, task output branch, and physical mode output branch. After quality gating and encapsulation of the results from the enabled branches, a result package is generated.

[0072] According to an embodiment of the present invention, a sensor data processing method embodiment is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0073] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 4 A hardware block diagram of a computer terminal for implementing a sensor data processing method is shown. Figure 4 As shown, the computer terminal 40 may include one or more processors (shown as 402a, 402b, ..., 402n in the figure) (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 404 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 4 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 40 may also include... Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown.

[0074] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 40. As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0075] The memory 404 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the sensor data processing method in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 404, thereby implementing the sensor data processing method of the aforementioned application program. The memory 404 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 404 may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 40 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0076] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 40.

[0077] Figure 5 This is a flowchart illustrating a sensor data processing method provided according to an embodiment of the present invention, as shown below. Figure 5 As shown, the method includes the following steps:

[0078] Step S501: Obtain the encoded response data and device metadata of the object under test. The encoded response data is collected through the encoding front end, which represents the sensor structure that can be described and called by the response file module. The device metadata is the parameter data of the sensor structure.

[0079] Step S502: Call the stored response file from the response file module. The response file includes various parameters and rules required for the data processing process. Multiple output branch units share the parameters in the response file.

[0080] Step S503: Based on the encoded response data, device metadata, and parameters retrieved from the response file, one or more output branch units are enabled according to the preset task configuration to obtain the initial data processing result. The output branch units include at least one of the following: spectral output branch unit, color output branch unit, task output branch unit, and physical mode output branch unit.

[0081] Step S504: Perform quality assessment on the initial data processing results, and generate the target data processing results based on the quality assessment results.

[0082] This technical solution constructs a platform-level computational imaging sensing architecture. First, it acquires coded response data and device metadata of the object under test through an encoding front-end. The encoding front-end encompasses various sensor structures that can be described by response files, such as multispectral filter arrays or metasurface structures. Then, it calls the stored response file from the response file module. This file contains core data such as channel mapping, calibration parameters, model parameters, and quality rules, which can be shared and accessed by multiple subsequent output branch units. Based on the task configuration, one or more of the spectral, color, task, or physical mode output branches are activated. The data is decoded using shared parameters to generate initial data processing results, and the physical mode results can be used to correct the results of other branches. Next, consistency checks and quality judgments are performed on the initial results. The validity of the results is determined based on confidence or validity indicators, or a retest is triggered. Finally, the qualified target data processing results and related data are packaged into a unified result package. This process achieves a complete closed loop from data acquisition, parameter sharing, multi-branch decoding to quality gating, significantly improving cross-task reusability, result consistency, and system scalability.

[0083] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0084] Through the above description of the embodiments, those skilled in the art can clearly understand that the sensor data processing method according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0085] Embodiments of the present invention may provide a computer device. Optionally, in this embodiment, the computer device may be located in at least one of a plurality of network devices in a computer network. The computer device includes a memory and a processor.

[0086] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the sensor data processing method and apparatus in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned sensor data processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0087] The processor can invoke information and applications stored in the memory via a transmission device to perform the following steps: acquiring coded response data and device metadata of the object under test, wherein the coded response data is acquired through a coding front-end, the coding front-end characterizes the sensor structure that can be described and invoked by the response file module, and the device metadata consists of parameter data of the sensor structure; invoking a stored response file from the response file module, wherein the response file includes various parameters and rules required for the data processing process, and multiple output branch units share the parameters in the response file; based on the coded response data, device metadata, and parameters invoked from the response file, enabling one or more output branch units according to a preset task configuration to obtain an initial data processing result, wherein the output branch unit includes at least one of the following: a spectral output branch unit, a color output branch unit, a task output branch unit, and a physical mode output branch unit; performing a quality assessment on the initial data processing result, and generating a target data processing result based on the quality assessment result.

[0088] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a non-volatile storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0089] Embodiments of the present invention also provide a non-volatile storage medium. Optionally, in this embodiment, the aforementioned non-volatile storage medium can be used to store the program code executed by the sensor data processing method provided in the above embodiments.

[0090] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0091] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: acquiring coded response data and device metadata of the object under test, wherein the coded response data is acquired through a coding front-end, the coding front-end characterizes the sensor structure that can be described and called by the response file module, and the device metadata is parameter data of the sensor structure; calling a stored response file from the response file module, wherein the response file includes various parameters and rules required for the data processing process, and multiple output branch units share the parameters in the response file; based on the coded response data, device metadata, and parameters obtained from the response file, enabling one or more output branch units according to a preset task configuration to obtain an initial data processing result, wherein the output branch unit includes at least one of the following: a spectral output branch unit, a color output branch unit, a task output branch unit, and a physical mode output branch unit; performing a quality judgment on the initial data processing result, and generating a target data processing result based on the quality judgment result.

[0092] Embodiments of the present invention also provide a computer program product, including a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it can: acquire coded response data and device metadata of the object under test, wherein the coded response data is collected through a coding front-end, the coding front-end characterizes the sensor structure that can be described and called by the response file module, and the device metadata is parameter data of the sensor structure; call a stored response file from the response file module, wherein the response file includes various parameters and rules required for the data processing process, and multiple output branch units share the parameters in the response file; based on the coded response data, device metadata, and parameters obtained from the response file, enable one or more output branch units according to a preset task configuration to obtain an initial data processing result, wherein the output branch unit includes at least one of the following: a spectral output branch unit, a color output branch unit, a task output branch unit, and a physical mode output branch unit; perform quality judgment on the initial data processing result, and generate a target data processing result based on the quality judgment result.

[0093] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0094] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0095] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The system embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0096] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0097] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0098] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0099] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A sensor data processing system, characterized in that, include: The data acquisition module is used to acquire the coded response data and device metadata of the object under test. The coded response data is acquired through a coding front-end, which represents the sensor structure that can be described and called by the response file module. The device metadata is the parameter data of the sensor structure. A response file module is used to store and call response files, wherein the response file includes at least one parameter and rule required for the data processing process, and multiple output branch units share the parameters in the response file for calling; A multi-branch processing module is used to enable one or more output branch units based on the encoded response data, the device metadata, and parameters called from the response file, according to a preset task configuration, to obtain the initial data processing result. The output branch units include at least one of the following: spectral output branch unit, color output branch unit, task output branch unit, and physical mode output branch unit. The quality gating module is used to determine the quality of the initial data processing results and generate the target data processing results based on the quality determination results.

2. The system according to claim 1, characterized in that, The multi-branch processing module includes: A receiving unit is configured to receive the encoded response data and the device metadata; The calling unit is used to call the response file to obtain the parameters corresponding to the task configuration; The startup unit is used to enable the corresponding output branch unit according to the task configuration, and process the encoded response data based on the device metadata and the parameters corresponding to the task configuration to obtain the initial data processing result. The processing result of the physical mode output branch unit is also used to correct the processing results of the spectral output branch unit, the color output branch unit, or the task output branch unit.

3. The system according to claim 1, characterized in that, The spectral output branch unit is used to output at least one of the following: reconstructed spectrum, spectrogram, low-dimensional spectral features, spectral angle, reconstruction quality score, and metamerism risk warning; the color output branch unit is used to output at least one of the following: true color map, display color map, color coordinate data, color confidence level, comprehensive color difference or color correction map; the task output branch unit is used to output at least one of the following: material category, defect area, status level, control marker, result code, and confidence level; the physical mode output branch unit is used to output at least one of the following: polarization map, depth map, angle map, event features, phase features, and wavefront features.

4. The system according to claim 1, characterized in that, It also includes a version maintenance module, which is used to update the response file module and the multi-branch processing module based on the quality judgment result, and redistribute the updated response file module and the updated multi-branch processing module through local writing, firmware update or network distribution.

5. The system according to claim 1, characterized in that, The response file includes fields of at least one of the following types: The identity field includes at least one of the following: the device identifier of the sensor structure, the module identifier of the sensor structure, the batch identifier of the sensor structure, the type of the sensor structure, and the version of the sensor structure; The geometric field includes at least one of the following: the mapping relationship between pixels and channels in the sensor structure, the filter array layout of the sensor structure, the superpixel layout of the sensor structure, and the region of interest relationship corresponding to the task configuration; The spectral field includes at least one of the following: the spectral response matrix required for the task configuration, the quantum efficiency required for the task configuration, the dark current required for the task configuration, the crosstalk required for the task configuration, and the gain parameter required for the task configuration; The color field includes at least one of the following: color conversion parameters required by the task configuration, lighting set required by the task configuration, color correction parameters required by the task configuration, and metamerism correction parameters required by the task configuration; The physical modal field includes at least one of the following: the polarization required for the task configuration, the depth required for the task configuration, the angle required for the task configuration, the initial calibration parameters required for the task configuration, and the joint response tensor required for the task configuration; The task field includes at least one of the following: the version of the task model corresponding to the task configuration, the tag system required by the task configuration, and the feature normalization rule required by the task configuration; The quality field includes at least one of the following: quality threshold, rejection trigger rule, retest trigger rule, and confidence fusion parameter.

6. The system according to claim 1, characterized in that, The sensor structure includes at least one of the following: a multispectral filter array structure, a metasurface spectral structure, a three-channel and multispectral image dual-mode structure, a spectral and polarization combined structure, a spectral and depth or angle combined structure, a spectral and polarization and depth combined structure, and an event or time-series triggered structure.

7. The system according to claim 1, characterized in that, The parameter data includes at least one of the following: device identifier, module identifier, batch identifier, mode identifier, exposure time, gain, black level, temperature, sampling timing, region of interest information, superpixel layout information, pixel mapping index, main ray angle, and focus status.

8. A sensor data processing method, characterized in that, The sensor data processing system according to any one of claims 1 to 7 comprises: The encoded response data and device metadata of the object under test are obtained. The encoded response data is collected through an encoding front-end, which represents the sensor structure that can be described and called by the response file module. The device metadata is the parameter data of the sensor structure. The stored response file is called from the response file module, wherein the response file includes various parameters and rules required for the data processing process, and multiple output branch units share the parameters in the response file. Based on the encoded response data, the device metadata, and the parameters retrieved from the response file, one or more output branch units are activated according to the preset task configuration to obtain the initial data processing result. The output branch unit includes at least one of the following: spectral output branch unit, color output branch unit, task output branch unit, and physical mode output branch unit. The initial data processing results are assessed for quality, and the target data processing results are generated based on the quality assessment results.

9. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the non-volatile storage medium to perform the sensor data processing method of claim 8.

10. A computer device, characterized in that, include: Memory and processor The memory stores computer programs; The processor is configured to execute a computer program stored in the memory, wherein when the computer program is executed, the processor performs the sensor data processing method of claim 8.

Images