Visual reconstruction method and system for preprocessing vehicle-mounted camera image by intelligent agent

Abstract

The application discloses a kind of vehicle-mounted camera image visual reconstruction method and system of intelligent agent preprocessing, belong to image data processing and visual reconstruction field.Geometric clean, window fixed pollution or geometric evidence unavailable state of pixel is formed according to the motion parameters of the vehicle and the moment of rolling shutter exposure line by line, for geometric clean pixel, intelligent agent maintains permissible pollution parameter set, total output error bound and has tried to measure group, according to which to select and execute waveguide measurement group;With joint waveguide loss observation shrinkage clean reference and film layer branch, form physical clean evidence by branch separation and Jacobian rank verification;Reconstruction operator generates candidate value, forms pollution output error bound and total output error bound by optical forward model and influence matrix.Error bound meets budget, write in candidate value, otherwise continue to measure or output same frame original value, while outputting state mask and authentication error bound field.The application can distinguish protective window pollution and scene change, and image fusion is weighted according to error bound.

Description

Technical Field

[0001] This invention relates to the field of image data processing and visual reconstruction technology, specifically to a visual reconstruction method and system for vehicle-mounted camera images preprocessed by an intelligent agent. Background Technology

[0002] Vehicle-mounted cameras typically have transparent protective windows that capture road environment images for image fusion and environmental modeling. When rainwater, oil films, or mud adhere to the surface of the protective window, they alter the transmission, scattering, and point diffusion characteristics of incident light, causing the image to become blurry, hazy, or partially obscured. To mitigate this image degradation, existing solutions generally determine the degradation state based on consecutive frame images or additional sensor information, and then invoke the corresponding image processing operators; some solutions also employ control programs or intelligent agents to schedule pollution measurements and operator invocation. While these solutions can improve image quality under certain conditions, they still have the following shortcomings: First, image changes caused by fixed window contamination, distant scene content, vehicle movement, and rolling shutter line-by-line exposures may exhibit similar characteristics. When vehicle movement is insufficient or local matching is unreliable, it is difficult to determine whether image components are attached to the protective window based solely on inter-frame stability, thus lacking a reliable basis for subsequent contamination judgment and its corresponding reconstruction range.

[0003] Second, when using additional sensor information to confirm contamination, candidate measurements are usually selected in a predetermined order or based on contamination classification results, and are not directly related to the reconstruction error of the current pixel. New measurements may not address the main uncertainties affecting reconstruction acceptance, making it difficult to determine whether they will enable the reconstruction candidate value to meet the writing conditions and whether further measurements are necessary.

[0004] Third, using a single pollution inversion value, confidence score, or average performance evaluation to reconstruct the results makes it difficult to cover other pollution states compatible with actual observations, and it is also difficult to uniformly account for pollution degradation, image noise, model errors, and re-projection residuals. When there are multiple solutions in the observations or model biases, there is a lack of verifiable evidence as to whether the reconstructed candidate values ​​can be written into the output image.

[0005] Fourth, existing solutions typically only output the processed image without simultaneously indicating the acceptance status and error range of each pixel. Subsequent image fusion or environmental modeling struggles to distinguish between the reconstructed values ​​and the original observations retained after acceptance failures, making it difficult to determine the appropriate usage for different pixels.

[0006] The above issues indicate that before vehicle-mounted camera images can be used for visual reconstruction, a reliable basis needs to be established to distinguish between protective window contamination and scene changes, and to ensure that subsequent measurements, reconstruction acceptance, and output status representation are interconnected.

[0007] Therefore, there is a need for a visual reconstruction method and system that can distinguish between protective window contamination and scene changes before reconstruction, enable physical measurements to respond to pixel states and acceptance results, and submit only vehicle camera images that meet the acceptance conditions and carry state and error limits after preprocessing by an intelligent agent. Summary of the Invention

[0008] To address the aforementioned technical problems, the present invention adopts the following technical solution: On one hand, the present invention provides a visual reconstruction method for vehicle-mounted camera images preprocessed by an intelligent agent, comprising: S1: Acquire the image sequence from the vehicle camera, the vehicle's motion parameters, and the rolling shutter's line-by-line exposure time to form the pixel's geometric cleanliness state, window-fixed contamination state, or geometric evidence unavailable state; S2: For pixels in a geometrically clean state, the agent maintains a set of permissible contamination parameters initialized with a defined parameter domain, a total output error bound, and a set of attempted measurement group identifiers. It simulates the parameter set and post-measurement total output error bound corresponding to the results within the range of each candidate guided wave measurement group. From the measurement groups, if the current total output error bound is finite, the group with the largest lower bound of positive reduction is selected; if there is no finite value, the group with the smallest and finite worst-case post-measurement total output error bound is selected, and the selected group is executed. S3: Clean reference branch and film branch of the joint guided wave loss observation error domain shrinkage parameter set returned by the selected guided wave measurement group; physical property clean evidence is formed only when the updated parameter set is non-empty and bounded, the existing clean reference branch passes the loss domain separation verification, the existing film branch passes the normalized Jacobian rank verification, and the upper bound of the contamination degree is lower than the contamination threshold. S4: For pixels that are geometrically clean and have evidence of physical cleanliness, the reconstruction operator is invoked to form reconstruction candidate values. The updated parameter set is converted into a residual degradation bound through the optical forward model. The contamination output error bound and the total output error bound are formed through the region-pixel influence matrix and the operator local sensitivity propagation. S5: Write permission is granted and reconstruction candidate values ​​are written only when the contamination output error boundary and the total output error boundary do not exceed the contamination error budget and the total error budget, respectively; preprocessing ends when geometric cleanliness is not achieved in S1; if physical property cleanliness evidence is not achieved in S3 or write permission is not granted in S5, and there are unattended measurement groups, the measurement budget is not exhausted, and the preprocessing deadline has not been reached, the agent maintains the original value of the same frame and returns to S2; otherwise, preprocessing ends and the original value is written; after write permission is granted or preprocessing is completed, the output image, the registered reconstruction state mask, and the authentication error boundary field are provided for image fusion weighted by boundary.

[0009] On the other hand, the present invention also provides a visual reconstruction system for vehicle-mounted camera images preprocessed by an intelligent agent, including an image evidence formation module, an intelligent agent preprocessing module, a cleanliness authentication module, a reconstruction acceptance module, and a state output module. The five modules respectively execute the aforementioned S1 to S5: the image evidence formation module forms three geometric evidence states, sends the geometric cleanliness state to the intelligent agent preprocessing module, and sends the other two states to the state output module; the intelligent agent preprocessing module maintains a parameter set, a total output error bound, and a set of attempted values, simulates results within the measurement range, and selects measurements; the cleanliness authentication module shrinks the two types of parameter branches, sends the formed physical property cleanliness evidence, cleanliness authentication, and updated parameter set to the reconstruction acceptance module; the reconstruction acceptance module forms candidate values, residual degradation bounds, two types of output error bounds, and write permissions, and sends the candidate values, write permissions, and total output error bounds to the state output module; the state output module outputs the candidate values ​​or original values ​​of the same frame, as well as the registered state mask and authentication error bound field accordingly. When physical property cleanliness evidence or write permissions are not formed but are still measurable, the corresponding state is fed back to the intelligent agent preprocessing module and returns to S2.

[0010] Furthermore, the in-range results of the candidate guided wave measurement group are divided into certified candidate units and write-rejected units, with only the former used to evaluate the worst-case post-measurement error; the parameter set includes a clean reference and finite-dimensional film branch; write permissions can also verify batch consistency and re-entry residuals. When no write permission is formed and the total output error bound is finite, reversible spatially encoded photothermal detection can be performed to shrink the parameter set through differential observation, and crosstalk and state transition errors are included in the model error bound.

[0011] Compared with the prior art, the present invention has the following beneficial effects: First, this invention generates a predicted displacement of scene points based on the vehicle's motion parameters and the rolling shutter's line-by-line exposure time. The residual between the predicted and measured displacements forms a geometrically clean state, a window-fixed contamination state, or a geometrically unavailable state. If the motion excitation is insufficient, the matching is unreliable, or the timing and depth conditions are not met, a geometric cleanliness judgment is not made. Therefore, image shifts caused by the vehicle's motion in distant scenes, time differences caused by rolling shutter line-by-line exposures, and fixed components in the protective window coordinates enter their respective state branches. Pixels with insufficient geometric evidence do not proceed to subsequent property authentication and reconstruction.

[0012] Second, this invention enables the agent to read the current total output error bound or its unmarked finite value, the attempted set, and the in-range results of the candidate guided wave measurement groups. When a finite error bound exists, the agent selects the measurement group with the largest lower bound of the positive reduction on the certification candidate unit; when no finite error bound exists, the agent selects the measurement group with the smallest finite and worst-case post-measurement total output error bound. Measurement returns are used to shrink the contaminated parameter set, and processing continues by the same certification and reconstruction acceptance chain. Thus, the selection of the guided wave measurement group is directly related to its potential for write permission, and the iteration termination is limited by the attempted set, measurement budget, and preprocessing deadline.

[0013] Third, this invention retains clean reference branches and film branches compatible with joint guided wave observations, forming evidence of clean physical properties only when the updated parameter set is non-empty and bounded, and the existing branches pass loss domain separation or Jacobian rank verification respectively. Then, it propagates the residual degradation of all existing contaminated states to an upper bound, and uses contaminated output error, noise output error, model error, and re-projection observation residuals together for write permission determination. Thus, write permission is established on the worst-case error upper bound of all existing states; when there are unresolved branches, no finite upper bound, or acceptance conditions are not met, reconstructed candidate values ​​are not written into the output image.

[0014] Fourth, this invention directly controls the submission of reconstruction candidate values ​​by granting write permissions. When a geometrically clean state, physical property cleanliness evidence, or write permission has not been formed and measurement can no longer continue, the original observation values ​​of the same input frame are output, and the output image, reconstruction state mask, and authentication error boundary field are spatially registered. Thus, each output pixel has a source state that can be directly read by subsequent image fusion, and pixels with written reconstruction candidate values ​​simultaneously have corresponding authentication error boundaries, enabling image fusion to select inputs according to source states and determine fusion weights according to the authentication error boundaries. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a visual reconstruction method for vehicle-mounted camera images preprocessed by an intelligent agent, as provided in an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram of the state transition during the preprocessing of the intelligent agent in an embodiment of the present invention.

[0017] Figure 3 This is a schematic diagram of the architecture of a visual reconstruction system for vehicle-mounted camera images preprocessed by an intelligent agent, provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0019] Combination Figure 1 This embodiment provides a visual reconstruction method for vehicle-mounted camera images preprocessed by an intelligent agent, including the following steps: S1: Forming pixel geometric evidence. Acquire the image sequence from the vehicle's camera, the vehicle's motion parameters, and the rolling shutter's line-by-line exposure times to form the pixel's geometric cleanliness state, window-fixed contamination state, or geometric evidence unavailable state.

[0020] Image frames carry frame identifiers, acquisition times, exposure, and gain; the line-by-line exposure time base provides the exposure time of each pixel row relative to the acquisition time. Vehicle motion parameters are provided by the pose source within the shared effective time covering adjacent frames. First, pixels are back-projected into camera coordinate rays using camera intrinsics and distortion parameters. Then, vehicle pose is interpolated according to the pixel row exposure times. Finally, the effective depth obtained from synchronized binoculars, range sensors, or multi-frame triangulation is used to project the same scene point onto adjacent frames, obtaining the predicted horizontal and vertical displacements. , Each data source must have a consistent clock and calibration version; when the exposure time is not covered by the effective pose, the depth is unavailable, the projection goes out of bounds, or the iteration does not converge, the geometric evidence becomes unavailable.

[0021] Local matching is performed within a limited search window around the predicted location to obtain the measured horizontal and vertical displacements. , When the difference between the best and second-best matches does not reach the calibrated confidence threshold, the measured displacement is not replaced by zero displacement or predicted displacement, resulting in a state where geometric evidence is unusable. Let... , The displacement error scales in two strictly positive directions are obtained from the pollution-free motion calibration sequence, and are pixels. dimensionless residual for: ; When the predicted displacement amplitude is below the excitation threshold, a state where geometric evidence is unavailable is formed; when it is not below the threshold... Pixels in the geometrically clean state are defined as those not exceeding the residual threshold, while those exceeding the residual threshold are defined as those in the window-fixed contaminated state. The two thresholds are obtained by adding the measurement margin to the smallest resolvable predicted displacement and the joint upper bound of the residual in the uncontaminated motion calibration sequence, respectively. Pixels equal to the excitation threshold enter the residual judgment stage, while those equal to the residual threshold are classified as geometrically clean. Only pixels in the geometrically clean state are selected for subsequent guided wave measurements; pixels in other states maintain their original values ​​in the current frame.

[0022] During specific calibration, the camera, pose source, and depth source are mapped to the same hardware time reference, and images are acquired under a clean, uncontaminated protective window and a known motion trajectory. The line-by-line exposure time is the product of the frame reference time and the line number and exposure interval; pose translation uses linear interpolation, and attitude is calculated using normalized quaternion short-arc interpolation. Error samples in two directions are formed by the difference between the actual projected position and the locally matched position, and the upper bounds of the absolute errors covering the effective temperature, exposure, and motion range are used as the basis for these errors. , The calibration samples are sorted in ascending order of predicted displacement amplitude. The excitation threshold is set to the value where the lower bound of the predicted displacement is first strictly greater than the upper bound of the matching error; the residual threshold is set to the value of the uncontaminated samples. The upper bound is added along with clock, pose, depth, and matching error margins. Recalibration is performed when camera intrinsics, progressive time base, or time synchronization version are changed; geometric evidence becomes unavailable when calibration samples cannot cover the current operating conditions.

[0023] When the exposure, gain, or image processing chain of adjacent frames is inconsistent, conversion to a common image amplitude domain is only possible if both frames can be converted using the same monotonic response calibration and the conversion error has been incorporated. , Matching is performed only when necessary. Depth sources are selected according to pre-registration priority, without averaging depths with different times or calibration versions; two simultaneously valid depth intervals that do not intersect form an unusable state. All inputs and failure exits are determined by pre-registration rules and are not subject to on-site judgment by operators.

[0024] S2: The agent selects the guided wave measurement group. For pixels in a geometrically clean state, the agent maintains a set of permissible contamination parameters initialized with a defined parameter domain, a total output error bound, and a set of attempted measurement group identifiers. It simulates the parameter set and post-measurement total output error bound corresponding to the results within the range of each candidate guided wave measurement group. From the measurement groups, if the current total output error bound is finite, the agent selects the one with the largest lower bound of positive reduction; if there is no finite value, the agent selects the one with the smallest and finite worst-case post-measurement total output error bound, and executes the selected group.

[0025] Combination Figure 2 The intelligent agent is a finite state control program executed by the processor, rather than a refactoring operator name. The program includes a pixel decision record, a candidate action register, a state transition configuration, and a request-return interface: the record stores the technical state and the return value of the previous action; the register stores the guided wave measurement group and photothermal detection configuration; the transition configuration stores candidate filtering, action selection, parallel elimination, failure diversion, and termination rules; the interface sends measurement requests and writes the return value with an identity field to the corresponding record. The intelligent agent selects the next measurement action based on the current total output error bound and the return value of the previous action, without changing the physical property cleanliness evidence, error bound, or the conditions for writing permission. Figure 2The registered measurement actions shown are candidate guided wave measurement groups in the main process; photothermal detection configuration is only included as a candidate action when no write permission has been formed, the total output error bound is finite, and the conditions of reversible detection domain, stimulus budget, and deadline are met. After the action returns and is verified, the agent sends the joint observation and its identity field to the clean authentication module, requests an update to the parameter set and the formation of authentication, and reads the physical property clean evidence status returned by the module; after the physical property clean evidence is formed, the agent issues a reconstruction and acceptance request and waits for the write permission to return; if the write permission has not been formed, the next measurement action is selected only if there are still executable unattended actions.

[0026] During construction, an independent decision record is established in memory for each pixel to be processed, the verified action registration table and transition configuration are loaded, and the waveguide and photothermal hardware interfaces, measurement count counters, and preprocessing cutoff timers are registered. S1 writes geometric evidence, and each action return triggers a state transition until the reconstructed or original value-preserving final state is reached. The program can process multiple pixels in a time-sharing manner and merge requests from the same partition and measurement group; the parameter set, error bound, attempt set, and termination state of each pixel are saved separately, and the return is written back by frame, pixel, and request sequence number.

[0027] by The preprocessing action number representing the pixel decision state is indicated by the superscript. Indicates the action sequence number rather than the exponent; Indicates the first The geometric evidence state at the time of the next action, This represents the state of parameter sets aggregated by associated partitions but for which cross-partition common constraints have not yet been applied. Indicates the first The current total output error bound at the next action, with Indicates the first The measurement batch identifier corresponding to this action, with Indicates returning to the previous action, This represents the set of actions that are still within the measurement budget and can be executed before the preprocessing deadline. This represents a deterministic state transition mapping that determines the next action based on the candidate action set, fixed parallel elimination rules, and failure handling rules. The defining relationship is: ; During a cold start, the parameter set is taken from the limited parameter domain of the current calibration version, and the total output error bound is marked as not yet having formed a finite value, with the attempted set being empty. The total output error bound adopts three states: not yet having formed a finite value, finite, and no finite upper bound; the latter two non-finite states both enter the no finite value selection branch of S2, and are not replaced by zero or a null pointer.

[0028] The output of equation (2) The current action field is written to the pixel decision record and executed by the processor; the joint observation, error code, updated parameter set, or permission status obtained after the action is completed are used as... Returning to the previous state transition provides a clear input.

[0029] At each new frame, the agent waits for S1 to generate geometric evidence. It enters the measurement selection state when geometry is clean; it enters the final state of preserving the original value when the window is fixed and contaminated or geometric evidence is unavailable. It also enters this final state if there are no untried registered actions that can be completed before the preprocessing deadline; otherwise, it determines the measurement action based on the current total output error bound and the predicted post-measurement error bound, issues a request containing the frame, pixel, partition, measurement group, calibration version, and request sequence number, and waits for a response.

[0030] Upon arrival, the system verifies the request fields, measurement batch, and calibration version. Successfully verified joint observations are used for parameter set shrinking; if an error, timeout, or verification fails, the corresponding action is recorded in the attempted set. If a return exception occurs, or if physical property cleanliness evidence or write permission is not formed, and if there are still untried actions, the remaining measurement count is greater than 0, and the preprocessing deadline has not yet arrived, the measurement selection is returned; otherwise, the system enters the final state of retaining the original value. When write permission is formed, the reconstructed candidate values, reconstructed state, and authentication error bound are output in the same version, and the system enters the reconstructed final state. All non-terminating states have a definite successor.

[0031] Each candidate guided wave measurement group has a fixed ordered channel table, joint range, measurement error domain, calibration version, upper bound on completion time, and upper bound on energy consumption. The joint results within the range are divided into physically achievable result units according to the quantization interval and error domain. The parameter set and post-measurement total output error bound are updated according to each result unit. Units that can form evidence of physical property cleanliness and have a finite post-measurement total output error bound are included in the certification candidate unit set. subscript Indicates candidate guided wave measurement group, subscript The first category indicates the candidate category for authentication; the rest are classified as write-reject units. Results within the range but not intersecting with the model image are still retained as model conflict write-reject units. Only out-of-range results that cannot form a complete joint return are excluded. Results within the resource limit that have not yet been proven are treated as write-reject units.

[0032] Specifically, the joint possible observation domain of each loss channel in the candidate guided wave measurement group is divided into multi-dimensional result units according to the quantization interval and measurement error domain. The result units located within the joint range are regarded as physically reachable result units, and are respectively assigned to the certification candidate unit or the rejection unit according to the previous rules.

[0033] The candidate group registration form stores the ordered channels, guided wave modes, wavelength, spatial code, transmit power range, receive gain, integration time, joint range, quantization step size, upper bound of completion time, upper bound of energy consumption, forward model version, and applicable temperature range for each group. Groups with fewer channels than the dimension of the film parameters, whose spatial response matrix cannot be proven to be full column rank, or whose any field is missing or whose independent verification fails, will not be included in the action set. The joint output of the candidate group must be obtained in the registration order; if channels are missing or the order is incorrect, the entire group will return an error to avoid combining advantageous channels from different batches into a single observation.

[0034] For candidate groups and pixels ,by Representation of result unit The bounded post-measurement total output error. Current. When finite, the lower bound of the reduction on the certification candidate unit. for: ; This indicates that the infimum of the set of authentication candidate units is taken. Only comparisons are performed. Non-empty and The group of unattended measurements that are strictly greater than 0 was selected. The largest. Currently. When there are no finite values, the difference between infinite values ​​is not calculated, but rather... This represents the total output error bound after the worst-case certification candidate test, where This indicates that the set of candidate authentication units is bounded supremum; the choice is finite. The smallest. When tagged, the one with the shorter upper bound of completion time, the lower upper bound of energy consumption, and the smaller value is selected in that order. The write-rejection unit does not participate in the calculation of the two indicators. When the actual return falls into the write-rejection unit, it only triggers continued measurement or retains the original value, and does not make the unproven result a writable result.

[0035] A trial key is formed using the frame identifier, candidate guided wave measurement group identifier, and calibration version. After execution or return of an error, it is recorded in the attempted set for this frame. When the calibration version changes, a new trial key is formed. The old key and its return are retained until the end of the frame and marked as version mismatch; they are not deleted, renamed, or consumed by the current version. The new version regenerates all physically reachable result units, without using the advantageous classifications of the old version. Pixels in the same group within the same partition share the same measurement batch. Returns are only consumed when the start epoch, frame, partition, group, version, request sequence number, and batch identifier are all the same, and the pixel is still in a test state. When a pixel is associated with multiple partitions, each batch, influence matrix version, and common effective time are recorded in ascending order of partition identifier to form a composite identifier. Changes in any component invalidate the original acceptance result. There are upper limits on the number of candidate groups, result units, interval subdivision times, and action times. The preprocessing deadline is registered at the beginning of each frame's processing; when any upper limit is reached or the deadline is reached, the original values ​​for the same frame are maintained.

[0036] The root unit of the joint range can be recursively bisected according to the channel with the largest dimensionless width; the subdivision of the unit that can be proven to be a certification candidate or a write-rejection unit stops, and the unresolved unit that reaches the unit upper limit is downgraded to a write-rejection unit. Records of certification candidates, write-rejection units, unresolved units, and failure results are all retained until the end of the frame; write-rejection and unresolved results do not participate in the selection index calculation, but are not deleted from the complete result record. In this way, it is not necessary to enumerate all code values ​​of each channel, and unfavorable results are not deleted. When multiple requests contend for the same waveguide hardware, requests that are the same in both partition and measurement group are merged first, and the rest are sorted according to the preprocessing deadline, partition identifier, and group identifier; the hardware serves only one active batch at a time. Requests that are not expected to be completed before the preprocessing deadline will return a timeout. Shared scheduling only changes the execution order, and does not change the group selected by equation (3) or allow cross-batch consumption return.

[0037] Parallel actions are eliminated sequentially based on the upper bound of completion time, the upper bound of energy consumption, and the action identifier. The state transition configuration, candidate action registration table, and calibration version jointly generate a verification value; configurations are invalid if fields are missing or verifications are inconsistent. Before deployment, the sequence of registered initial states and action returns is replayed; the selected actions, state updates, and termination outputs for the same configuration version must be consistent. The agent's actions are jointly determined by the state transition configuration, candidate action registration table, calibration version, and action returns.

[0038] The control program may employ conditional branching, state transition tables, event callbacks, or other equivalent structures. Processors, memory, timers, and communication may utilize general-purpose embedded hardware and software. When using different programming languages, operating systems, thread models, or processors, the aforementioned state fields, action registration, transition order, parallel elimination, return verification, failure handling, termination conditions, and write permission consumption relationships must still be maintained.

[0039] S3: Update the parameter set and form evidence of clean physical properties. Based on the clean reference branch and film branch in the parameter set returned by the joint guided wave loss observation error domain returned by the selected guided wave measurement group; evidence of clean physical properties is formed only if the updated parameter set is non-empty and bounded, the surviving clean reference branch is verified by loss domain separation, the surviving film branch is verified by normalized Jacobian rank, and the upper bound of the contamination level is below the contamination threshold.

[0040] Each candidate guided wave measurement group contains independent loss channels with a dimension no less than that of the film branching parameters. These channels are formed by multiple guided wave modes and multiple wavelengths. Orthogonal spatial codes can be used to excite guard window partitions and synchronously demodulate them, or multiple edge-coupled and probe channels can be used to obtain distinguishable responses. The partition-measurement matrix must be able to prove full column rank even with measurement uncertainty. The reference code slot is used to estimate ambient light and receiver chain bias, and at least one wavelength is located in the water absorption band. If the spatial code is not full rank, the reference power range contains 0, the channel is saturated or missing, or the decoding residual exceeds the calibration domain, the entire group is returned as unusable, and advantageous channels are not spliced.

[0041] The raw detection values ​​from each channel are first divided into a joint observation error domain based on the quantization interval, the code slot offset interval, the reference power interval, and the decoding residual interval. When using the power ratio, the positive reference power interval is used as the denominator; when using decibel loss, the interval is transformed according to the monotonicity of the negative logarithm. All endpoints are rounded outward, thus the joint observation error domain covers the corresponding quantization and measurement uncertainties.

[0042] A waveguide hardware implementation uses a transparent protective window as the planar waveguide medium. A switchable wavelength transmitter, coupling structure, and receiver detector are placed at the edge to ensure that each registered mode satisfies the total internal reflection propagation condition. The outer surface film layer alters the propagation loss through an evanescent field. Different coupling paths can be configured for each spatial partition, or they can be distinguished using linearly independent spatial codes. The uncontaminated reference code slot, the dark code slot, and the effective excitation code slot use the same receiving gain and integration time. Joint observation is unavailable when the corrected received power is not positive, the reference power fails, or ambient light crosstalk exceeds the calibration range.

[0043] The permissible contamination parameter set includes two types of branches. The clean reference branch is formed from clean reference samples, and the film layer branch is formed from finite-dimensional property parameter vectors. The clean reference branch stores the joint loss error domain and the upper bound of the contamination level. The film layer vector includes the real part of the complex refractive index at the reference wavelength, the extinction coefficient, the film thickness, and the parameterized dispersion coefficients. The number of parameterized dispersion coefficients and the basis functions are fixed during model registration. One implementation uses dimensionless first-order dispersion coefficients. With a strictly positive reference wavelength Real part of refractive index at Based on, wavelength Take the real part of the refractive index at the point The variation of the extinction coefficient across measurement wavelengths is achieved using a normalized absorption basis function derived from measurements of the reference film and registered with the model version. At this point, the film vector is... Extinction coefficient at reference wavelength, film thickness and It consists of four components. When using higher-order dispersion basis functions, the number of coefficients must be fixed before candidate group registration, and the number of independent loss channels must not be less than the dimension of the film vector. The defined parameter domain is jointly determined by the protective window material, the measurement band, the property range of the reference sample, and the effective calibration domain of the device.

[0044] by Indicates the partitioning of the protective window space, in order to Indicates the measurement batch of this partition, in Indicates partition A permissible state of contamination; when When it belongs to the clean reference branch, its content is a record of the combined clean reference loss state and the upper bound of the contamination level. When it belongs to the membrane layer branch, its content is the aforementioned finite-dimensional vector of physical property parameters. Indicates the parameter set before the update, Indicates according to The branch generates a forward model that jointly predicts the loss interval, in order to This indicates the first channel with the same channel order and dimensions as the joint predicted loss interval. Batch joint observation error domain, update parameter set for: ; In equation (4), the symbol To indicate a set membership relationship, the curly braces and colon indicate that states satisfying the condition after the colon are selected from the parameter set before the update. This represents the intersection of sets in the same joint loss coordinate space. To represent the empty set, This indicates that the intersection of sets is not empty. The condition for the intersection of sets to be non-empty is: [The state is missing here, likely due to an error in the original text]. The generated joint prediction interval is compatible with all corresponding observation intervals simultaneously, without substituting local intersection of any channel for joint compatibility. In implementation, the outward rounding model image of the interval boxes is used for exclusion judgment, and the retained boxes are subdivided to a finite depth. When the outer envelope is empty, the original set is determined to be empty by the original set being contained within the outer envelope. A non-empty outer envelope only indicates that exclusion is not yet possible; a witness with the same parameter must be obtained and the parameter set before updating and all original observation constraints must be substituted back before the corresponding connected region is confirmed to be non-empty. Boxes without witnesses by the time the number of interval boxes or subdivision depth is reached, or by the preprocessing deadline, remain as unresolved branches, neither confirmed as non-empty nor deleted; the existence of unresolved branches prevents the formation of clean evidence. If the value is empty, proceed directly to failure handling.

[0045] The joint witnessing mechanism prioritizes subdividing interval boxes based on the longest dimensionless edge, and performs damped Gaussian-Newton iterations starting from the center of the interval box and the calibration node falling within the box. The damping coefficient is halved sequentially from 1. The first update uses the upper bound of the sum of squares of the residuals of all normalized components not exceeding the current iteration value; if no such update is performed, the corresponding initial point is marked as an iteration failure. After each update, each parameter component is truncated to the closed interval of the corresponding component in the current interval box. The component residual is the distance from the predicted loss to the corresponding observation interval divided by the measurement error scale of the channel; if it falls within the observation interval, the component is 0. Candidate points must be substituted back into the original interval box, the parameter set before the update, and each valid observation constraint; only when all conditions are met is it registered as a witness. The number of iterations, the number of damping attempts per iteration, the number of initial points, the subdivision depth, and the number of interval boxes are all finite; if the solver claims convergence but the substitution fails, it is still marked as unresolved.

[0046] An interval lookup table constrained by waveguide physics or a continuously differentiable interval model can be used. A model is constructed using a clean reference component and a reference membrane whose properties and thickness are measured independently. The model is divided into a construction group and a verification group based on the reference membrane entity; duplicate frames of the same entity must not cross groups. Each model unit stores the input effective domain, the predicted loss interval, and the model error bound formed by the uncovered residuals from independent verification. If the verification record cannot be covered, the corresponding error bound is expanded or the effective domain is shrunk and re-verified; if it still cannot be covered, the model unit is not published.

[0047] When using an interval lookup table, the measured loss interval is stored channel by channel on a finite property parameter grid. Piecewise multilinear or tensor product spline interpolation with outward rounding is used to obtain the predicted interval within the grid. Measurement uncertainty, fitting residuals, and independent validation residuals are added to the corresponding channels. The model is only used within the property, temperature, and optical path configuration domains covered by the sample and is not used for error-free extrapolation. The clean reference branch separately stores a lower bound on the distance between the loss domain of the clean sample and the predicted domain of the membrane sample that has reached the contamination threshold; if this lower bound is not positive, the clean reference branch cannot be independently certified.

[0048] The film branching makes the waveguide loss and physical properties dimensionless according to strictly positive scales. and Let these represent the measurement error scale and parameter domain scale diagonal matrices, respectively, where the subscripts are... and Label the measurement domain and parameter domain separately; Represents a strictly diagonal matrix The inverse matrix, Let represent the Jacobian matrix of the joint waveguide loss forward model with respect to the film branch property parameter vector. The resulting dimensionless normalized measurement Jacobian matrix is ​​denoted as . The calculation relationship is as follows: ; Equation (5) Dimensionless, and written into the membrane branch verification record. Each diagonal element is used as a measurement error scale with strictly positive values ​​for the corresponding loss channel. Each diagonal element of the parameter is taken as the width of the parameter domain that is strictly positive for the corresponding physical property parameter; the two sets of scales remain unchanged during the verification of the same parameter set as the calibration version is registered. For a normalized Jacobian matrix with no less than the number of rows and columns, a minimum singular value that is strictly greater than 0 is equivalent to full column rank. According to the singular value perturbation inequality, for each reserved interval box, the minimum singular value of the interval Jacobian center matrix minus the Frobenius norm of the interval radius matrix is ​​used as the conservative lower bound of the minimum singular value; the interval Jacobian can be proven to be full column rank only when the lower bound is greater than 0. If any scale is 0, the derivative does not exist, or the lower bound cannot be proven to be positive, it is not passed. Before the candidate group is registered, the interval Jacobian is formed using the reference film calibration data of the actual device; the group with the number of independent channels less than the parameter dimension is not registered. The clean reference branch does not execute equation (5), but verifies whether there is a positive separation margin between its joint loss error domain and the predicted loss domain of all registered film layers with a contamination degree not lower than the threshold. When both types of branches exist simultaneously, each must pass the corresponding verification; when only one type of branch exists, only that type of verification is performed; when neither type exists, the updated parameter set is empty, and no physical property cleanliness evidence is formed.

[0049] by Representing state The dimensionless contamination degree is formed by the decrease in transmittance, the increase in dot width, and the film thickness relative to their respective allowable dimensions. Indicates spatial partitioning After the first The updated upper limit of pollution level after batch measurement is calculated as follows: ; Only Non-empty bounded, each surviving branch passes the corresponding identifiability verification and Evidence of cleanliness is formed when the water is strictly below the contamination threshold. Equivalent to the threshold, with unresolved branches, or without an attainable upper bound does not constitute evidence. The permissible scale is determined by the protective window imaging tolerance and the allowable degradation during subsequent processing; the loss in the water absorption band is used to constrain the film thickness and properties of the transparent water film, and cleanliness evidence is not formed independently by a single water absorption channel.

[0050] Sequentially acquired loss channels do not assume an absolutely constant contamination state. During calibration, the reference film is repeatedly acquired within the measurement set completion time, and the changes in physical properties, temperature, and channel sampling time differences form a state change interval. During runtime, the exposure time parameter set is propagated outward to the measurement time, intersected with the observation, and then projected back to the exposure time according to the same change boundary. If the change boundary has no finite value, the common effective time is empty, or it loses its identifiability after propagation, the set is unusable for the current frame. This processing ensures that subsequent measurements still correspond to the same input frame, without reducing the parameter set based on a static assumption.

[0051] An assembleable measurement configuration divides the protective window into eight partitions, using two polarization mode families and four wavelengths to form eight ordered loss channels. Wavelengths can include 850 nm, 940 nm, 1310 nm, and 1450 nm. Each partition is configured with addressable coupling paths, and after installation, a spatial response matrix is ​​formed using partition-by-partition reference responses. This configuration is registered only if the matrix, including measurement uncertainty, can demonstrate full column rank and a positive separation margin between the target partition response and the non-target partition response. When using this configuration, candidate groups are registered based on actual device calibration results.

[0052] In this example, silicon photodetectors can be used for the 850 nm and 940 nm channels, and indium gallium arsenide (IGaAs) detectors can be used for the 1310 nm and 1450 nm channels. Each channel is synchronously decoded after transimpedance amplification and analog-to-digital conversion. After installation, the lower bound of the minimum singular value is verified using the interval Jacobian of the actual device, and the loss domain separation margin is verified using clean components and threshold contamination film samples. If either of these cannot be proven to be positive, the coupling position is adjusted, the parameter dimension is reduced, the effective domain is narrowed, or the partition is marked as unusable. Candidate groups are registered only when the measured results meet the registration conditions.

[0053] S4: Forming reconstruction candidate values ​​and error bounds. For pixels that are geometrically clean and have evidence of physical cleanliness, the reconstruction operator is invoked to form reconstruction candidate values. The updated parameter set is converted into residual degradation bounds through the optical forward model. The contamination output error bound and the total output error bound are formed through the region-pixel influence matrix and operator local sensitivity propagation.

[0054] The optical forward model sequentially processes clean image local patches through convolution of finite-support point spread functions determined by physical property parameters, transmittance multiplication, and stray light superposition to obtain predicted observation local patches under protective window contamination. The transmittance interval is formed by the response ratio of a uniform field reference to a reference film; the point spread function interval is formed by point source or knife-edge calibration; and the stray light interval is formed by dark field and uniform illumination records. The model is registered according to protective window partitioning, physical property parameter grids, incident angle, image channels, and temperature. The model is unusable if the input exceeds the effective domain or if independent verification records cannot be covered by the model interval and error bounds.

[0055] The specific calculation order of the optical forward model is as follows: For each clean image region, perform interval convolution using the point spread function kernel corresponding to the current contamination state, then multiply by the transmittance interval of the same state, and finally add the stray light interval. The point spread function kernel elements must be non-negative and the kernel and interval must contain 1; the transmittance interval must be non-negative and bounded; the image boundary adopts the registered mirror expansion rule. When the true value of the clean image is unknown, first use the sensor code value range as the initial interval, then form the observation interval using the original observation values ​​and noise boundaries, and then shrink the clean input interval in reverse under the constraints of physical property parameters and observations. If the clean input interval is empty, contains an unrecoverable saturation range, or the model's inverse image has no finite bounds, the model is unusable, and a favorable true value is not generated from neighboring pixels.

[0056] During model construction, reference membrane entities and scene entities are constructed separately into a construction group and a validation group, and repeated frames of the same entity do not cross groups. Each mesh cell stores transmittance, point spread function, stray light range, and independent validation uncovered residuals. Operating temperature, incident angle, or physical property parameters that exceed the mesh are not extrapolated. In this way, the inputs, computational relationships, effective domain, and failure states of the residual degradation boundary can all be recalculated from the calibration records.

[0057] The influence support set of pixels is determined according to the region-pixel influence matrix, and the residual degradation bound of each partition is propagated using the local Lipschitz constant of the reconstruction operator. The region-pixel influence matrix is ​​obtained by applying a distinguishable reference response partition by partition and recording pixel changes, with non-negative coefficients. Indicates partition For pixels The upper bound of the residual degradation contribution. The support set representing the impact of contributions reaching the truncation threshold is... Represents the partition residual degradation boundary and the pixel residual degradation boundary. for: ; In equation (7), Indicates the satisfaction Sum of contributions from all partitions, membership relationship The meaning of equation (4) is used. This equation accumulates the contributions of the partitions that exist simultaneously on the same pixel, and each contribution forms a unique account with the partition and the pixel. It is the non-negative upper bound coefficient of the residual degradation contribution, not the mixing proportion, and its sum is not required to be 1; the same physical change enters only one partition account, and mutually exclusive substitution paths take the maximum value of the upper endpoints of each branch before forming the account without repeated summation; tail terms Take the sum of the endpoints of contributions from partitions not individually covered. Contributions from outside the support set are not deleted but aggregated into tail items. If any partition boundary or tail term has no finite upper bound, then the total output error bound has no finite value.

[0058] Obtained through partition-by-partition calibration: A distinguishable reference response is applied sequentially to only one partition. Under fixed target, pose, and exposure conditions, the pixel changes relative to a clean reference image are recorded. A non-negative influence coefficient is formed using the upper bound of pixel change and the upper bound of the model output change for that partition, incorporating registration and repeatability errors. The effects of different partitions on the same pixel are recorded separately; changes already entered into one partition's account are not included in the final entry. Partitions without separate coverage are not defaulted to 0, but are included in the final entry. .

[0059] by This represents the upper bound of the local Lipschitz constant under the current reconstruction operator version, configuration, input neighborhood, and output norm, in order to Represents pixels The pollution output error limit, where the subscript Indicates the contamination error category, and the contamination output error boundary is: ; Linear filtering operators It can be obtained from the corresponding induced norm; finite-layer reconstructed networks It can be obtained by propagating layer by layer from the norms induced by each linear transformation and the upper bound of the slope of the activation function, covering jumpers, pruning, and upsampling. For the modified linear unit at the non-differentiable point, its global Lipschitz upper bound of 1 is used, independent of the derivative at that point. It cannot be proven to be finite. The operator does not enter the candidate write chain. One implementation without training weights uses a 3×3 mean filter; under the maximum norm, the sum of the absolute values ​​of the nine coefficients is 1, therefore the operator coefficients directly determine the behavior. .

[0060] When using a reconstructed network, the operator file, weight summary, normalization, pruning, input neighborhood, output norm, and result are all considered. Create records of the same version; if the input of related branches cannot prove that execution will not be performed, take the upper bound of all possible branches. Recalculate after changing any record field. The resulting finite upper bound is used by formula (8).

[0061] by Represents pixels At the current exposure, gain, and temperature, the sensor noise amplitude limits are defined as follows: Represents pixels The sum of the non-repeating parts of the optical forward model error, influence matrix error, tail term output error, numerical propagation margin, and sensitivity estimation margin, where the subscripts are... This represents the total model error; Represents pixels The total output error bound, where the subscript The total output error category is represented by the following calculation relationship: ; Equation (9) uses the same output norm and image amplitude dimensions for all terms. The noise boundary is formed by the upper envelope of repeated sampling of the dark field and uniform field, quantization step size, black level drift, and exposure error; each error account records the source, effective domain, and version. When the source cannot be separated or any account has no finite value, It is denoted as having no finite upper bound. The result of equation (9) is written into the pixel decision state and used by S2 for next action selection and S5 for acceptance.

[0062] The optical model error is taken as the maximum uncovered residual relative to the prediction interval of the independent verification record; the influence matrix error is taken as the upper bound of the response generated by repeated positioning of the calibration target point and attitude perturbation; the tail term output error is taken as... The numerical margin is the upper bound of the machine error for rounding outwards. Each item has a unique source identifier, and contributions that have entered the noise boundary or other accounts shall not be added again. When the reconstruction operator operates in the normalized irradiance domain and outputs sensor code values, all items are first converted to the output code value domain using the calibrated monotonic transformation and its error bound before executing equation (9).

[0063] To verify whether candidate values ​​can interpret the same input frame, Represents pixels and the local block formed by operator input support, to and Representing the candidate local block and the actual observed local block respectively, in Indicates according to the influence support set The joint pollution state vector is arranged according to the partition identifiers in the vector. This indicates the current pixel decision state when entering S4. The joint parameter set formed after considering common effective time, the same grid, the same calibration version, the same composite batch, and all cross-regional original measurement constraints, is used to... This represents an optical forward model that maps candidate local patches and joint contamination states to predict observed local patches. This represents the local block output norm calculated according to the pre-registered pixel and channel order, in order to This represents the upper bound of the retrospective observation residuals obtained under all combined pollution conditions, and its calculation relationship is as follows: ; In equation (10) Using the supremum meaning of equation (3), the result is Write the reconstruction acceptance record and consume it by equation (11). The common pixel grid uses the discrete pixel coordinates of the input frame. Each partition is mapped to the grid through the same calibrated version of the region-pixel influence matrix. No interpolation splicing is performed on the material property parameter vectors of different partitions. The joint vector is formed by constraining the Cartesian product of the parameter sets of each associated partition to the common effective time, the same composite batch, and all cross-partition original constraints; Each component is a separate partition and independent of the others. The same state witness is a fixed assignment to the joint vector, not a requirement that different partitions take the same property value. The joint witness must back-substitute all original constraints partition by partition; non-empty or pairwise intersection of partitions does not replace the joint witness. Common valid time being empty, batch or influence matrix versions being inconsistent, etc. If the upper bound is empty or cannot be calculated finitely, the write-back acceptance fails. The upper bound is determined by prioritizing the longest dimensionless side and bisecting the existing joint interval box at a finite depth, then propagating outwards. If a finite upper bound cannot be formed by reaching the number of nodes or the depth, or by reaching the preprocessing deadline, no write permission is granted.

[0064] When calculating equation (10), transmission, convolution, and stray light interval propagation are performed on each surviving joint branch using the same physical property state. Then, the upper endpoint of the residual norm is calculated, and finally, the maximum value is taken on all branches. When candidate values ​​undergo quantization, clipping, or color space conversion, the corresponding error is added according to the actual imaging reverse path before re-projection. This process only verifies the consistency between the candidate values ​​and the original observations, without reselecting favorable parameters that reduce the residuals.

[0065] S5: Write permission is granted. A write permission is granted and a reconstruction candidate value is written only if the contamination output error boundary and the total output error boundary do not exceed the contamination error budget and the total error budget, respectively. Preprocessing ends if a geometrically clean state is not formed in S1. If physical property cleanliness evidence is not formed in S3 or a write permission is not granted in S5, and if there are unattended measurement groups, the measurement budget is not exhausted, and the preprocessing deadline has not arrived, the agent maintains the original value of the same frame and returns to S2; otherwise, preprocessing ends and the original value is written. After the write permission is granted or the preprocessing is completed, the output image, the registered reconstruction state mask, and the authentication error boundary field are provided for image fusion weighted by boundary.

[0066] In an implementation method that simultaneously enables re-entry and batch verification, with This indicates the current geometric evidence state upon entering S5. Derived binary geometry cleanup markers; When in a geometrically clean state ,otherwise .by Indicates a mark indicating the cleanliness of physical properties; when the evidence is established. ,otherwise .by Indicates the current measurement batch identifier. The composite batch identifier, consisting of each associated partition batch, influence matrix version, and common effective time, is frozen when candidate values ​​and boundaries are formed. This indicates a valid identifier for submission obtained using the same field structure; , Let these represent the contamination error budget and the total error budget, which have the same dimensions as the output image, respectively. Represents the noise amplitude boundary under the same local block and norm, with This is a conjunction indicator function that returns 1 if all conditions within the parentheses are true, and 0 otherwise; write permission. for: ; Equation (11) assigns each threshold endpoint to the pass side; unavailable evidence, lack of finite values ​​for the boundary, or batch mismatch all cause The agent can consume the reason for failure to determine whether to return to S2, but cannot change the permission conditions or the current result. Let... , and Let the candidate value, the original observation value of the same input frame, and the output value be respectively, then: ; Equation (12) Pixels written to the output image . Write candidate values ​​and record the reconstructed state at the appropriate time; The original value of the same frame is written and the original value or measurement unavailable state is recorded. It is not replaced by pixels or generated values ​​from other frames. The output image, status mask, and authentication error boundary use the same frame identifier, size, and pixel order, and are committed in a versioned transaction. If the batch identifier or output version changes upon commit, buffer verification fails, or the preprocessing deadline is reached, the candidate transaction is not visible to consumers, and the original value of the same frame is used to form a safe output.

[0067] Specifically, each frame sets up an active output slot and an inactive output slot. The inactive slot simultaneously stores the output image, state mask, error boundary field, frozen composite batch identifier, and verification digest. Only when all three are written and the frame identifier, size, number of pixels, and batch at the time of submission are consistent, does the inactive slot become the active version through a version comparison and exchange. If the comparison fails or the batch changes, the candidate slot is discarded, and the original value output is generated from the immutable input frame buffer. Consumers only read the same set of three components that have been submitted, avoiding partial updates across frames or batches.

[0068] The authentication error boundary field is recorded as a non-negative finite field at the reconstructed pixels. In other states, an unavailable flag is recorded. Image fusion first verifies the frame identifier and size of the three-piece set; for reconstructed pixels, a strictly positive scaling factor can be used. Forming dimensionless weights: ; The dimension of is the reciprocal of the image amplitude, obtained from the reference error bound and the corresponding target weight inverse solution given by the downstream task, and registered together with the fusion task and the image amplitude representation. Equation (13) yields Write the fusion weight field of the reconstructed pixel; retain the original value or consume the unavailable pixel as the original observation without using equation (13) to assign the reconstructed confidence weight.

[0069] The above process has a defined feedback and termination relationship. If S3 fails to generate evidence of physical property cleanliness or S5 fails to generate write permission, the current measurement group is added to the attempted set. If there are still unattended measurement groups that meet the conditions of S2 and can be measured, updated, and accepted before the preprocessing deadline, then return to S2 and continue to maintain the original value of the same frame; otherwise, enter the final state of retaining the original value in the current frame. Late returns, errors, timeouts, or version mismatches do not change the final state. The decision state is re-established for each new frame, and the permission of the previous frame is not directly copied as the permission of the current frame.

[0070] Furthermore, for pixels that have not formed a write permission and have a finite total output error bound, a candidate pulse with a positive and largest lower bound of the total output error bound reduction on the writable detection result is selected from the spatially encoded photothermal detection pulses in the reversible detection domain. If no positive lower bound exists, photothermal detection is stopped and the original value of the same frame before stimulation is output. The differential observation interval before and after stimulation is compared with the uncontaminated crosstalk response interval. When they intersect, the corresponding result is classified as a write-rejection detection result. Only when the two do not intersect, the predicted response of the differential observation constraint state transition model is used to form the obtained parameter set. If the obtained parameter set is not empty and still forms evidence of physical cleanliness and a finite total output error bound after re-verification, it is classified as a writable detection result. Otherwise, it is classified as a write-rejection detection result. The write-rejection detection result is not included in the lower bound and the original value of the same frame before stimulation is output. S4 to S5 are executed with the parameter set obtained from the writable detection result, and the width of the crosstalk response interval and the image amplitude bound after the state transition error bound is propagated through the optical forward model are included in the model error bound.

[0071] The photothermal action follows a sequence of freeze, acquisition, verification, and submission. Before execution, the original frame, parameter set version, and measurement batch are frozen. Pulses are applied according to the registered area of ​​effect, energy, duration, and sampling time, and differential observations of the image, waveguide loss, and guard window temperature are obtained. The reversible detection domain is calibrated within the allowable temperature and pulse range using a reference film with traceable properties and thickness, free from contamination. A pulse is only registered when the film thickness, coverage area, and waveguide baseline recover to the repeatability range after repeated pulses, and the guard window does not exceed the material's allowable temperature. Detection stops and the original frozen value is output if any input exceeds the domain, sampling is missing, or the upper limit for recovery is unattainable.

[0072] The photothermal state transition model preserves image differences, waveguide loss differences, and temperature difference intervals according to physical property parameter grids, pulse energy grids, initial temperature, and ordered sampling times. The construction group and independent validation group are divided according to the reference membrane entity; heating, cooling, and repeated acquisition records for the same entity are kept in the same group. Adjacent grids are interpolated using piecewise multilinear interpolation with outward rounding, and pulse energy error, temperature measurement error, sampling time error, and independent validation residuals are included. The model is unusable if the input exceeds the grid or if the validation record cannot be covered by the expanded prediction interval, and the parameter set is not changed according to the pollution category label.

[0073] by The identifier representing the current photothermal pulse, in order to Representing state The ordered differential response range of the image, guided wave, and temperature under this pulse, with This represents the actual difference observation interval with the same component order and dimensions. The pulse action is unavailable when the uncontaminated crosstalk response interval is empty or unbounded; otherwise, first... Comparing the two response intervals, if they intersect or completely overlap, it indicates that non-contamination responses cannot be ruled out, and they are directly included in the write-rejection results, without forming a photothermal detection post-parameter set. A photothermal detection post-parameter set is formed only when the two responses do not intersect. ; In equation (14), superscript The term "photothermal pulse" indicates the photothermal pulse and does not represent the power exponent. The intersection in the formula is the model compatibility judgment between the predicted response of the state transition model and the measured difference, which is different from the judgment between the measured difference and the pollution-free crosstalk interval. An empty state indicates a model conflict; when the state is not empty, the same state witness must still be obtained, all measurement constraints and equation (14) must be substituted back before the stimulus, and S3 and S4 must be re-executed. Only results that re-form physical property cleanliness evidence and finite total output error bounds are classified as writable detection results; other results are rejected results. The state transition model can be constructed by interpolating the measured joint difference intervals on the finite parameter, pulse energy, and temperature grids through outward rounding. Construction and verification are grouped according to the reference membrane entity; if the model goes out of the grid, the verification record cannot be covered by the prediction interval, or the model has no finite bounds, the parameters are not excluded and the model is returned as unusable.

[0074] by This represents the upper bound of the width of the pollution-free crosstalk response interval converted to the dimensions of the output image. This represents the image amplitude boundary after the state transition error propagates through the optical forward model, with... Indicates pulse The calculation relationship of the model error bound after execution is as follows: ; In equation (15), superscript Similarly, it represents the photothermal pulse identifier, but does not represent the power exponent; each term adopts the same norm and dimension as in equation (9). Only cover non-polluting irritant crosstalk. Only the state transition model error after deducting the crosstalk is covered; when the sources are not mutually exclusive, a single merging upper bound is used, and if duplication or omission still cannot be ruled out, it is recorded as no finite value. In the photothermal detection branch, with In the alternative (9) Recalculate The result is then fed into the write permission determination of equation (11). The photothermal action and the guided wave action share a finite measurement budget and a preprocessing cutoff time. The action stops when the boundary of the stimulus budget, the reversible domain, or the effective domain of the model is reached, and no image during the stimulus period is submitted.

[0075] The differential results within the range of candidate photothermal pulses are also divided into writable and unwritable results. The reduction lower bound is calculated only for the results that are first judged by crosstalk, then formed by Equation (14) into a non-empty parameter set with common witnesses, and then obtained by S3 and S4 to obtain evidence of physical property cleanliness and a finite total output error bound. The unwritable result does not participate in the lower bound calculation but is retained as a determined original value output branch. When there is no positive lower bound, the actual result falls into the unwritable state, the preprocessing cutoff time is reached, or the pulse energy exceeds the registration interval, the frozen original value of the same frame before stimulation is output.

[0076] To obtain the aforementioned models and thresholds, a unified construction, verification, and release procedure can be followed for calibration. The construction group and the independent verification group are isolated by reference entity, forming a joint waveguide loss model, an optical forward model, a region-pixel influence matrix, a sensor noise bound, an operator sensitivity bound, and a photothermal state transition model, respectively. The model records input and output fields and units, effective domain, interval coefficients or lookup tables, mutually exclusive error accounts, dependent versions, and failure codes. When independent verification observations are not covered by the predicted interval, the corresponding error account is expanded or the effective domain is shrunk before re-verification; models that still cannot cover the interval or whose error accounts have no finite values ​​are not released. The residual threshold, excitation threshold, contamination threshold, and two error budgets are obtained from the uncontaminated motion calibration, the guard window imaging tolerance, and the downstream image task tolerance, respectively, and are used for corresponding determinations.

[0077] The complete chain can be verified using samples of uncontaminated parts, permissible residual film, and water film, oil film, or dust exceeding the contamination threshold. Each sample records the true physical properties and uncertainty, geometric tri-state, whether the parameter set includes the true state, branch identifiability, selected measurement group, write permission, output value, and certification error bound; the output error adopts the same norm as Equation (9) and is referenced by the reconstruction result of the clean protective window in the same scene and orientation. This record is used to verify the coverage of the true state, the coverage of the certification bound, and the original value output in the same frame when failure occurs.

[0078] Combination Figure 3 On the other hand, the present invention provides a visual reconstruction system for vehicle-mounted camera images preprocessed by an intelligent agent, comprising: The image evidence forming module is used to form a geometrically clean state, a window-fixed contamination state, or a geometrically unavailable state based on the image sequence of the vehicle-mounted camera, the vehicle's motion parameters, and the rolling shutter line-by-line exposure time. The geometrically clean state is provided to the intelligent agent preprocessing module, and the window-fixed contamination state or the geometrically unavailable state is provided to the state output module. The intelligent agent preprocessing module is used to receive the geometric cleanliness state, maintain the permissible contamination parameter set initialized with the defined parameter domain, the total output error bound state, and the set of attempted measurement group identifiers; simulate parameter set constraints and post-measurement total output error bounds for the results within the range of each candidate guided wave measurement group; from the measurement groups, if the current total output error bound is finite, select the candidate unit with the largest lower bound of positive reduction on the authentication candidate unit; if there is no finite value, select the worst-case post-measurement total output error bound that is finite and smallest, issue a measurement request for the selected measurement group, and receive a return; if physical property cleanliness evidence has not been formed or write permission has not been formed, but measurement is still possible, maintain the original value of the same frame and continue selection; The cleanliness certification module is used to shrink the clean reference branch and film branch in the allowable contamination parameter set by the joint guided wave loss observation error domain returned by the selected guided wave measurement group. It forms physical property cleanliness evidence only when the updated parameter set is non-empty and bounded, the existing clean reference branch passes the loss domain separation verification, the existing film branch passes the normalized Jacobian rank verification, and the upper bound of the contamination degree is lower than the contamination threshold. Cleanliness certification is formed only for pixels that are in a geometrically clean state and have physical property cleanliness evidence. The reconstruction acceptance module is used to receive cleaning certification and updated parameter sets, form reconstruction candidate values, residual degradation limits, contamination output error limits and total output error limits. When the two output error limits do not exceed the contamination error budget and the total error budget respectively, a write permission is formed, and the reconstruction candidate values, the write permission and the total output error limits are provided to the status output module. The status output module is used to output reconstruction candidate values ​​when the write permission is valid, output the original values ​​of the same frame when the state is not geometrically clean, there are no untried measurement groups, the measurement budget is exhausted, or the preprocessing deadline is reached, and output the registered reconstruction status mask and authentication error boundary.

[0079] The five modules are top-level modules at the same level and can be executed by the same or multiple processors. When the image evidence formation module forms a window-fixed contamination or geometric evidence unavailable state, it directly sends this state to the state output module to form the original value output of the same frame; when a geometrically clean state is formed, it sends it to the agent preprocessing module. The agent preprocessing module sends a measurement request to the pilot wave excitation and receiving interface and sends the complete joint return and its identity field to the cleanliness authentication module; the cleanliness authentication module sends the cleanliness authentication and updated parameter set to the reconstruction acceptance module; the reconstruction acceptance module sends the reconstruction candidate values, write permission, and corresponding total output error bound to the state output module, and feeds back the error bound state to the agent preprocessing module when no write permission has been formed but measurement is still possible. When the updated parameter set is empty or no physical property cleanliness evidence has been formed, if there are still unattended measurement groups, the measurement budget is not exhausted, and the preprocessing deadline has not yet arrived, the cleanliness authentication module feeds back the parameter set state to the agent preprocessing module. Each record must at least include the frame, pixel, partition, valid time, batch, and calibration version; the receiver must verify these before consumption.

[0080] During system deployment, the five modules can be deployed on the same onboard processor or distributed across multiple processors that can complete processing before the preprocessing deadline; deployment changes do not alter the modules or their recording interfaces. Cross-processor recording uses a fixed byte order and unit, and includes version, length, and checksum; if packets are lost, duplicated, or the version is mismatched, the system will not proceed to the next module. The agent controls the waveguide or photothermal hardware only through the registered request-return interface; the measurement hardware does not directly modify the permission or output buffer.

[0081] The system interfaces include the vehicle-mounted camera and progressive exposure timing interface, the vehicle motion interface, the guided wave excitation and reception interface, the processor, and the memory; when photothermal backoff is enabled, it also includes the controlled pulse and protective window temperature interface. The memory stores the candidate group registration table, calibration model, pixel decision state, original value buffer for the same frame, parameter interval boxes, and the output set. Before execution, the storage requirements are calculated based on the image size, number of buffer slots, number of interval boxes, and queue limit; if insufficient, a resource shortage is returned before receiving the frame, and the original value buffer is not overwritten. The upper bound of the completion time of each candidate action is determined by actual measurement on the target platform; the agent only executes actions that can complete measurement, updating, acceptance, and submission before the preprocessing deadline.

[0082] The calibration model, candidate actions, and time upper bounds are loaded with the release package. The release package saves the hardware identifier, calibration version, model valid domain, error account, candidate groups, state transitions, number of measurements and upper bound of the interval box, preprocessing deadline, and verification values. When the hardware or software version changes, the completion time and model coverage are re-verified; if dependencies are inconsistent, the old package is not used.

[0083] This forms a closed loop from geometric evidence, state dependency measurement, cleanliness certification, refactoring acceptance to state output.

[0084] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A visual reconstruction method for vehicle-mounted camera images preprocessed by an intelligent agent, characterized in that, include: S1: Acquire the image sequence from the vehicle camera, the vehicle's motion parameters, and the rolling shutter's line-by-line exposure time to form the pixel's geometric cleanliness state, window fixed contamination state, or geometric evidence unavailable state; S2: For pixels in a geometrically clean state, the agent maintains a set of permissible contamination parameters initialized with a defined parameter domain, a total output error bound, and a set of attempted measurement group identifiers. It simulates the parameter set and post-measurement total output error bound corresponding to the results within the range of each candidate guided wave measurement group. From the measurement groups, if the current total output error bound is finite, the group with the largest lower bound of positive reduction is selected; if there is no finite value, the group with the smallest and finite worst-case post-measurement total output error bound is selected, and the selected group is executed. S3: Shrink the clean reference branch and film branch of the parameter set according to the joint guided wave loss observation error domain returned by the selected guided wave measurement group; form physical property clean evidence only when the updated parameter set is non-empty and bounded, the existing clean reference branch passes the loss domain separation verification, the existing film branch passes the normalized Jacobian rank verification, and the upper bound of the contamination degree is lower than the contamination threshold. S4: For pixels that are geometrically clean and have evidence of physical cleanliness, the reconstruction operator is invoked to form reconstruction candidate values. The updated parameter set is converted into a residual degradation bound through the optical forward model. The contamination output error bound and the total output error bound are formed through the region-pixel influence matrix and the operator local sensitivity propagation. S5: Write permission is granted and reconstruction candidate values ​​are written only when the contamination output error bound and the total output error bound do not exceed the contamination error budget and the total error budget, respectively. Preprocessing ends when S1 does not form a geometrically clean state; if S3 does not form evidence of physical property cleanliness or S5 does not form write permission, and if there are unattended measurement groups, the measurement budget is not exhausted, and the preprocessing deadline has not arrived, the agent maintains the original value of the same frame and returns to S2; otherwise, preprocessing ends and the original value is written. After the write permission is formed or preprocessed, the output image, the registered reconstructed state mask, and the authentication error boundary field are used for image fusion and weighted by boundary.

2. The method according to claim 1, characterized in that, In S1, a geometric evidence unavailable state is formed when a reliable measured displacement is not obtained; when a reliable measured displacement is obtained, a predicted displacement is formed based on the exposure time of the row where the pixel is located, which is synthesized from the scene parallax component and the scroll shutter row tilt component; when the amplitude of the predicted displacement is lower than the excitation threshold, a geometric evidence unavailable state is formed; when it is not lower than the excitation threshold, the horizontal and vertical components of the difference between the measured displacement and the predicted displacement are dimensionless according to the corresponding calibration error scale, and the residual is formed by the sum of the squares of the two dimensionless differences; when the residual does not exceed the residual threshold, a geometric clean state is formed; when it is greater than the residual threshold, a window fixed contamination state is formed.

3. The method according to claim 1, characterized in that, In S2, the joint possible observation domain of each loss channel in the candidate guided wave measurement group is divided into multi-dimensional result units according to the quantization interval and measurement error domain. The result units located within the joint range of the measurement group are determined as physically reachable result units. The corresponding joint prediction loss constrains the current allowable contamination parameter set. After constraint, S3 and S4 are executed to form physically reachable result units with physical property cleanliness evidence and finite total output error bounds. The rest are write-reject units. When the current total output error bound is finite, the lower bound of the difference between the current total output error bound and the constrained total output error bound in all certified candidate units is used as the reduction amount. When there is no finite value, the upper bound of the constrained total output error bound in all certified candidate units is used as the worst value. Write-reject units do not participate in the calculation of the reduction amount and the worst value. When the actual joint observation falls into a write-reject unit, the original value of the same frame is maintained during the failure handling period.

4. The method according to claim 1, characterized in that, The clean reference branch is formed by a clean reference sample, and the film branch is formed by a finite-dimensional property parameter vector, which includes the real part of the complex refractive index at the reference wavelength, the extinction coefficient, the film thickness, and the parameterized dispersion coefficient. The loss domain separation verification verifies whether the joint loss error domain of the clean reference branch and the predicted loss domain of the registered membrane branch with a contamination level not lower than the threshold have a positive separation margin; the normalized Jacobian rank verification verifies whether the normalized measurement Jacobian matrix of the membrane branch can be proven to have full column rank and a lower bound of the minimum singular value greater than 0 on each reserved interval box of the updated parameter set.

5. The method according to claim 1, characterized in that, Each candidate guided wave measurement group contains independent loss channels with a dimension no less than that of the film branching parameters. The channels are formed by multiple guided wave modes and multiple wavelengths. The measurement group excites different guard window spatial partitions and demodulates the loss response synchronously through orthogonal spatial code sequences, or acquires the loss response through multiple coupling channels and probe channels at the edge of the guard window. The partition-measurement matrix formed by the excitation and acquisition response is full rank, and the reference code slot is used to form an ambient light crosstalk estimate. At least one wavelength is located in the water absorption band.

6. The method according to claim 1, characterized in that, In S4, the influence support set of the pixel is determined according to the region-pixel influence matrix, and the residual degradation bounds corresponding to all update parameters in the influence support set are propagated by the local Lipschitz constant of the reconstruction operator to form the contaminated output error bound. The upper bound of the residual degradation contribution of the influence support set outside the spatial partition is propagated through the local Lipschitz constant to obtain the output amplitude bound as the tail term output error bound. The contamination output error bound, the noise output error bound, and the model error bound including the tail term output error bound are added to form the total output error bound under the same output norm and image amplitude dimension. The reconstruction candidate values ​​are back-projected through the same batch of optical forward models to form the upper bound of the back-projection observation residual. The write permission is formed only when the upper bound of the back-projection observation residual does not exceed the noise amplitude bound under the same observation norm.

7. The method according to claim 1, characterized in that, A trial key is formed by the frame identifier, candidate guided wave measurement group identifier, and calibration version. Measurement groups that have been executed or returned an error are recorded in the attempted set in this frame and will not re-enter the candidate set in this frame unless the measurement group configuration or calibration version is changed. The untried measurement groups are those that have been registered, not recorded in the attempted set, and can be completed before the preprocessing deadline. Pixels mapped to the same protective window space partition and selected with the same measurement group share a measurement batch. The measurement group is executed once within the batch, and its return is consumed only by pixels with the same batch identifier that are still in the state of being under test. If the measurement batch identifier attached to each error boundary is different from the valid identifier at the time of submission, no write permission is formed. If an error occurs, timeout occurs, batch mismatch occurs, physical property cleanliness evidence cannot be formed, or write permission cannot be formed, and the aforementioned unattended measurement group exists, return to S2; otherwise, retain the original value of the same frame.

8. The method according to claim 1, characterized in that, For pixels that have not formed a write permission and have a finite total output error bound, a candidate pulse with a positive and largest lower bound of the total output error bound reduction in the writable detection result is selected from the spatially encoded photothermal detection pulses in the reversible detection domain. If no positive lower bound exists, photothermal detection is stopped and the original value of the same frame before stimulation is output. The differential observation interval before and after stimulation is compared with the uncontaminated crosstalk response interval. If they intersect, the corresponding result is classified as a write-rejection detection result. Only when the two do not intersect, the predicted response of the differential observation constraint state transition model is used to form the obtained parameter set. If the obtained parameter set is not empty and still forms evidence of physical cleanliness and a finite total output error bound after re-verification, it is classified as a writable detection result. Otherwise, it is classified as a write-rejection detection result. The write-rejection detection result is not included in the lower bound and the original value of the same frame before stimulation is output. Execute S4 to S5 using the parameter set obtained from the writable detection results, and include the crosstalk response interval width and the image amplitude boundary after the state transition error boundary is propagated through the optical forward model into the model error boundary.

9. A visual reconstruction system for vehicle-mounted camera images preprocessed by an intelligent agent, characterized in that, It includes an image evidence formation module, an agent preprocessing module, a clean authentication module, a reconstruction acceptance module, and a status output module; The image evidence forming module is used to form a geometrically clean state, a window-fixed contamination state, or a geometrically unavailable state based on the image sequence of the vehicle camera, the vehicle's motion parameters, and the rolling shutter line-by-line exposure time. The geometrically clean state is provided to the intelligent agent preprocessing module, and the window-fixed contamination state or the geometrically unavailable state is provided to the state output module. The intelligent agent preprocessing module is used to receive the geometric cleanliness state, maintain the permissible contamination parameter set initialized with the defined parameter domain, the total output error boundary state, and the set of attempted measurement group identifiers; and simulate parameter set constraints and post-measurement total output error boundaries for the results within the range of each candidate guided wave measurement group. From the measurement group, if the current total output error bound is finite, select the candidate unit with the largest lower bound of positive reduction on the certification candidate unit; if there is no finite value, select the worst-case post-measurement total output error bound that is finite and smallest, send a measurement request for the selected measurement group and receive a response. If evidence of physical property cleanliness has not been formed or write permission has not been formed, and measurement is still possible, maintain the original value of the same frame and continue to select; The clean certification module is used to shrink the clean reference branch and film branch in the allowable contamination parameter set by the joint guided wave loss observation error domain returned by the selected guided wave measurement group. It forms physical property clean evidence only when the updated parameter set is non-empty and bounded, the existing clean reference branch passes the loss domain separation verification, the existing film branch passes the normalized Jacobian rank verification, and the upper bound of the contamination degree is lower than the contamination threshold. Clean certification is formed only for pixels that are in a geometrically clean state and have physical property clean evidence. The reconstruction acceptance module is used to receive cleaning certification and updated parameter set, form reconstruction candidate value, residual degradation limit, pollution output error limit and total output error limit. When the two output error limits do not exceed the pollution error budget and the total error budget respectively, a write permission is formed, and the reconstruction candidate value, the write permission and the total output error limit are provided to the status output module. The status output module is used to output the reconstruction candidate value when the write permission is valid, output the original value of the same frame when the state is not geometrically clean, there is no unattended measurement group, the measurement budget is exhausted, or the preprocessing deadline is reached, and output the registered reconstruction status mask and authentication error boundary.

10. The system according to claim 9, characterized in that, The intelligent agent preprocessing module includes guided wave measurement and photothermal detection actions. The photothermal detection action selects a spatially encoded photothermal detection pulse with the largest positive lower bound of the error reduction in the writable detection result when no write permission is formed for the pixel and the total output error bound is finite. Photothermal detection stops when no positive lower bound exists. First, the differential observation interval before and after stimulation is compared with the uncontaminated crosstalk response interval. If they intersect, they are classified as write-reject detection results. Only when they do not intersect are the parameter sets formed by the predicted responses of the differential observation-constrained state transition model. If the obtained parameter sets are non-empty and pass the physical property cleanliness evidence and finite total output error bound verification, they are classified as writable detection results. Other results are classified as write-reject detection results and are not included in the lower bound. The reconstruction acceptance module includes the crosstalk response interval width and the image amplitude bound after propagation of the state transition error bound through the optical forward model into the model error bound and re-forms a write permission. The state output module outputs reconstruction candidate values ​​only when the write permission is valid; otherwise, it outputs the original observation values ​​of the same input frame before stimulation.

Images