A semantic-physical co-modulation traffic evidence image restoration module and method

Abstract

This application relates to a semantic-physical co-modulation traffic evidence image restoration module and method. The method includes: acquiring degraded traffic images and corresponding degradation category information collected by an evidence collection terminal at the edge of a closed traffic hub; extracting physical degradation prior information such as differential Gaussian decomposition components, gradient magnitude maps, dark channel transmission maps, illumination separation components, and rain line direction response maps through a degradation feature-semantic co-modulation module to generate a degradation feature prior map and a task-adaptive physical candidate map; inputting the degraded traffic image, degradation feature prior map, and task-adaptive physical candidate map into the backbone of a conditional neural network, and achieving cross-fusion of semantic information and spatial features through a semantic-physical co-gated attention mechanism; and training the restored image end-to-end based on the predicted channel field through Euler iterative output. This application achieves highly robust image restoration, improving the recognition accuracy and reliability of traffic evidence targets such as license plates, container numbers, hazardous chemical markings, and vehicle outlines.

Description

Technical Field

[0001] This application relates to the fields of intelligent traffic image processing and computer vision, and in particular to a semantic-physical co-modulation traffic evidence image restoration module and method. Background Technology

[0002] With the development of intelligent transportation, intelligent security, and intelligent evidence collection technologies, the demand for video image evidence collection in transportation hub scenarios is constantly increasing. In closed or semi-closed traffic areas such as highway toll stations, tunnel entrances and exits, port gates, logistics parks, underground parking lots, and hazardous chemical parks, edge evidence collection terminals typically need to continuously collect vehicle traffic images for extended periods and identify and collect evidence of targets such as license plate characters, vehicle body markings, container numbers, hazardous chemical signs, vehicle outlines, and traffic trajectories. Due to the common occurrence of complex weather, dynamic lighting, and high-frequency vehicle movement in these scenarios, the collected traffic images are prone to various forms of degradation, thus affecting the accuracy of subsequent identification, detection, and evidence collection results.

[0003] In real-world engineering scenarios, the degradation types vary significantly depending on the traffic environment. For example, at tunnel entrances and exits, the drastic changes in ambient brightness as vehicles enter or exit can easily lead to localized overexposure, strong reflections, and headlight glare. At port gates or logistics parks, the effects of water vapor evaporation, sea fog, and rain can cause images to appear fogged, have low contrast, and be obscured by raindrops. At tollbooth ramps at night, low illumination, sensor noise, vehicle motion blur, and compression distortion are often present simultaneously. Furthermore, these degradation phenomena often do not occur in isolation but rather alternate or overlap within the same work cycle, making traffic evidence image restoration tasks complex, variable, and requiring high real-time performance.

[0004] Currently, numerous deep learning-based image restoration methods have been proposed to address issues such as image dehazing, deraining, denoising, low-light enhancement, and motion blur recovery. Most existing methods employ dedicated network models designed for specific degradation scenarios, such as constructing separate deraining, dehazing, or night-time enhancement networks, and using different models to handle the corresponding degradation type. While this approach can achieve good restoration results under specific degradation conditions, it has significant limitations in practical traffic evidence collection deployments.

[0005] First, multi-model solutions require maintaining multiple sets of network weight files and inference branches simultaneously. The system typically also needs to add a pre-degradation classifier to enable model routing. When misclassification occurs, the incorrect recovery model will output distorted images, affecting subsequent tasks such as license plate recognition, object detection, and trajectory analysis, and in severe cases, even leading to the loss of critical forensic information. Furthermore, multi-model deployment significantly increases the storage footprint, computing power consumption, and operational complexity of edge devices, hindering long-term stable operation in resource-constrained edge forensics terminals.

[0006] Secondly, while some unified restoration networks can achieve multi-degradation processing through task labels, most methods rely primarily on data-driven training, using only discrete task numbers to distinguish degradation categories, lacking physical prior constraints for different degradation mechanisms. For example, in rain removal tasks, rain lines typically exhibit directionality and high-frequency sparseness; in dehazing tasks, image degradation is closely related to transmittance, atmospheric light distribution, and dark channel characteristics; and in low-light enhancement tasks, illumination separation and local brightness compensation are involved. Existing unified networks, lacking explicit modeling of these degradation mechanisms, are prone to problems such as overly smoothed details, texture loss, edge artifacts, or incomplete restoration of distant structures in complex degradation scenarios, making it difficult to meet the requirements of image realism and structural integrity in traffic evidence collection scenarios.

[0007] In addition, some recent studies have begun to introduce visual language models or text conditional mechanisms to enhance the network's ability to perceive different degradation scenarios by generating degraded descriptions as network recovery conditions. However, most existing methods rely on large online visual language models to generate text descriptions in real time, which still has significant limitations in engineering deployment. On the one hand, large visual language models typically require high computing power and memory overhead, making them difficult to deploy directly on edge forensics devices. On the other hand, in closed private networks, security intranets, or environments without external network access, online model invocation is easily affected by network interruptions, communication delays, and data security restrictions, resulting in risks such as poor inference stability, insufficient real-time performance, and leakage of private data, making it difficult to meet the real-time processing requirements of high frame rate traffic videos.

[0008] Furthermore, while some existing methods incorporate textual semantic conditions, they typically only impose global constraints on image features at the loss function level, lacking deep interaction between semantic information and the network's internal spatial features. This results in semantic conditions being unable to effectively participate in multi-scale feature modulation. For example, existing methods struggle to provide targeted local feature guidance for different degradation modes such as rain line direction, contrast decay in fog, and increased brightness at night, thus limiting their robustness in recovery under complex scenes.

[0009] Regarding the aforementioned technologies, the inventors believe that existing traffic image restoration technologies still have the following problems: difficulty in simultaneously considering the processing capabilities of multiple degradation scenarios and edge deployment efficiency under a unified network framework; lack of prior constraints that incorporate degradation physical mechanisms; insufficient depth of utilization of semantic conditions; and strong dependence on large online models. Summary of the Invention

[0010] To address the technical challenges of existing traffic image restoration techniques, such as difficulty in simultaneously handling multiple degradation scenarios and improving edge deployment efficiency within a unified network framework; lack of prior constraints incorporating degradation physical mechanisms; insufficient depth of semantic condition utilization; and strong dependence on large online models, this application provides a semantic-physical co-modulation traffic evidence image restoration module and method.

[0011] The semantic-physical co-modulation traffic evidence image restoration method provided in this application adopts the following technical solution: Firstly, a semantic-physical co-modulation method for restoring traffic evidence images includes the following steps: S1. Obtain degraded traffic images collected by the evidence collection terminal at the edge of the closed traffic hub, and obtain the discrete index of the degradation category or the on-site condition code corresponding to the degraded traffic images. S2. Input the degraded traffic image into the degradation feature-semantic co-modulation module to calculate the corresponding physical degradation prior information. The physical degradation prior information includes at least one or more of the following: differential Gaussian decomposition components, Sobel gradient magnitude map, local variance map, dark channel transmission map, Retinex illumination separation components, rain line direction template response map, noise level estimation map, and edge confidence map. S3. Based on the discrete index of the degradation category or the field condition code, perform task-selective mask weighted fusion on each physical degradation prior information to generate a degradation feature prior map, and simultaneously generate a task-adaptive physical candidate map corresponding to the current degradation category. S4. Based on the discrete index of the degradation category, read the corresponding fixed text phrase from the preset semantic phrase library, and generate the corresponding semantic embedding vector through the text encoder in the pre-trained image-text alignment model to obtain the normalized semantic anchor. S5. The degraded traffic image, the task adaptive physical candidate map, and the degraded feature prior map are concatenated along the channel dimension and input into the conditional neural network backbone. In the bottleneck layer of the conditional neural network backbone, the semantic anchor is fused with the spatial features in a cross-attention manner through a semantic-physical co-gated attention mechanism to obtain semantic modulation features. S6. Based on the semantic modulation features, predict the image recovery channel field, and generate the recovered image through single-step or multi-step Euler iteration; S7. Based on the channel field loss, reconstruction loss, structural similarity loss, and preprocessing auxiliary loss, the conditional neural network is jointly trained end-to-end to obtain a traffic evidence collection image restoration model.

[0012] By adopting the above technical solutions, the physical prior information extracted by the degradation feature-semantic co-modulation module effectively guides the network to perform local feature recovery for different degradation modes such as rain, fog, low light, and motion blur, avoiding over-smoothing of details or artifacts. Through task-selective mask weighted fusion, the physical prior strength can be adaptively adjusted according to the degradation category to achieve unified processing of multiple degradation scenarios. The semantic anchors generated by the pre-trained image-text alignment model are integrated into spatial features with a semantic-physical co-gated attention mechanism, enabling the network to obtain semantic guidance at the internal multi-scale feature level, enhancing the perception ability of rain line directionality, contrast in fog, and brightness distribution at night. The Euler iterative recovery based on the channel field further maintains the continuity of image structure and the integrity of details, improving the recognition accuracy of traffic evidence targets such as license plates, container numbers, vehicle markings, and hazardous chemical signs, while improving the system's evidence collection reliability and real-time processing capability in complex environments such as highways, tunnels, ports, and nighttime.

[0013] Optionally, in step S3, the gain coefficients corresponding to each physical degradation prior information and the intensity weights corresponding to the task-adaptive physical candidate maps are generated by inputting the learnable degradation embedding vectors corresponding to the degradation categories into the multilayer perceptron network, so that different degradation categories correspond to different physical prior enhancement magnitudes and candidate map intensities, thereby achieving adaptive optimization of physical prior parameters.

[0014] By adopting the above technical solution, the enhancement magnitude of each physical prior information and the intensity weight of candidate maps can be adaptively adjusted according to different degradation categories, realizing dynamic optimization of physical prior parameters. This allows the network to effectively utilize physical priors in multiple degradation scenarios without manual parameter tuning, guiding the recovery process of different degradation modes such as rain, fog, noise, and low light, improving image detail fidelity and structural integrity, while enhancing the system's adaptability and recovery robustness to complex traffic environments.

[0015] Optionally, the text encoder in the pre-trained image-text alignment model keeps its parameters frozen during both the model training and inference phases. The semantic anchor is pre-computed offline during the model initialization phase and registered as a non-trainable cached tensor. During the inference phase, it is called by looking up the table based on the discrete index of the degradation category.

[0016] By adopting the above technical solutions, the offline pre-computation and caching mechanism of semantic anchors ensures low latency and stability in the inference process, while avoiding the impact of network interruption or computing power limitation on system performance. It provides reliable semantic modulation capabilities in multi-degradation scenarios, enhances the network's feature perception capabilities for complex traffic images such as rain, fog and low light, and improves the structural integrity of the restored image and the accuracy of evidence information.

[0017] Optionally, the semantic-physical co-gated attention mechanism includes: Map the spatial feature map to the query vector Q; The semantic anchors are mapped to key vector K and value vector V, respectively; Attention weights are obtained by performing a scaled dot product operation between the query vector Q and the key vector K. The attention weights are used to weight the value vector V to generate semantic modulation residuals; The semantic modulation residual is projected by a 1×1 convolution and then broadcast and superimposed onto the spatial feature map to achieve conditional modulation of spatial features by semantic information.

[0018] By adopting the above technical solutions, efficient interaction and fusion of semantic information and spatial features can be achieved within the network. The semantic-physical co-gated attention mechanism enables the semantic anchor to finely modulate the features of each spatial location, enhancing the network's local perception capabilities of rain line direction, contrast in fog, and low-light details. At the same time, the original spatial structure is maintained through residual superposition, avoiding information loss or excessive smoothing, thereby improving the detail fidelity, structural integrity, and recognition accuracy of traffic evidence targets in image restoration.

[0019] Optionally, the physical degradation prior information in step S2 includes rain-specific physical priors and fog-specific physical priors; The rain-removal-specific physical priors include a rain line direction response map generated by a three-directional difference template, a fine stripe enhancement map generated by white hat morphological operations, and a sparse high-frequency response map. The dehazing-specific physical priors include dark channel maps, atmospheric light estimation maps, original transmission maps, guided filtering refined transmission maps, and color attenuation prior maps. Each dedicated physical prior is selectively activated for the task based on the degradation category mask.

[0020] By adopting the above technical solutions, the rain-de-sighting prior effectively captures the directionality of rain lines and high-frequency stripe features, while the fog-de-sighting prior provides transmittance and illumination constraints, enabling the network to focus on relevant features for recovery under their respective tasks. The task-selective activation mechanism avoids interference from non-target degradation, achieving unified processing of multiple degradation scenarios, improving image detail fidelity and structural integrity, and enhancing the recognizability and reliability of traffic evidence images in complex environments such as rain and fog.

[0021] Optionally, the evidence collection terminal at the edge of the closed transportation hub is deployed at highway toll station ramps, tunnel entrances and exits, port gates, hazardous chemical industrial park entrances and exits, logistics park entrances and exits, or underground parking lot entrances and exits. The restored image is used for license plate recognition, container number recognition, vehicle marking recognition, hazardous chemical marking recognition, vehicle contour extraction, or vehicle trajectory verification. The conditional neural network backbone adopts an encoder-decoder structure. In the encoding stage, degradation features are extracted through multi-scale convolutional modules. In the decoding stage, image detail information is restored through skip connections. The semantic-physical co-gated attention mechanism is embedded in the bottleneck layer.

[0022] By employing the above technical solutions, high-precision image restoration for evidence collection can be achieved in typical traffic hub scenarios. The encoder-decoder structure, combining multi-scale convolution and skip connections, effectively extracts degradation features while preserving image details. The semantic-physical co-gated attention mechanism embedded in the bottleneck layer further enhances semantic modulation capabilities, enabling the network to adaptively recover from complex degradations such as rain, fog, low light, and motion blur. The overall effect is a significant improvement in the recognition accuracy of license plates, container numbers, vehicle markings, hazardous materials signs, and vehicle outlines, providing reliable image support for traffic evidence collection.

[0023] Optionally, the Euler iteration in step S6 includes: Explicitly integrate and update the current image state based on the predicted channel field: xt+1 = xt + Δt·vθ(xt,t,c) Where xt represents the image state at step t, Δt represents the integration step size, vθ represents the channel field predicted by the conditional neural network, and c represents the degradation condition information.

[0024] By employing the above technical solution, utilizing the channel field predicted by a conditional neural network, and continuously updating the image state through Euler explicit integral iteration, the image is gradually restored. This effectively eliminates the degradation effects of rain, fog, and motion blur while maintaining structural continuity and detail integrity. Multi-step iteration further enhances restoration accuracy, significantly improving both the visual quality and the completeness of evidence information in the final output image, thereby increasing the reliability of license plate, container number, and vehicle outline recognition in traffic evidence collection scenarios.

[0025] Secondly, a semantic-physical co-modulation traffic evidence image restoration system includes an image acquisition module, which is used to acquire degraded traffic images and corresponding degradation category information; It includes a physics prior generation module, which is used to generate a degradation feature prior map and a task-adaptive physics candidate map; It includes a semantic anchor generation module, which is used to generate corresponding semantic anchors based on the degradation category; It includes a semantic-physical fusion module, which is used to fuse semantic information and spatial features through a semantic-physical co-gated attention mechanism; It includes an image restoration module, which is used to predict and restore the channel field and output the restored image; It includes a training optimization module, which is used to train a conditional neural network based on a joint loss function.

[0026] By adopting the above technical solutions, the image acquisition module can acquire degraded traffic images and corresponding degradation category information in real time, providing accurate input conditions for subsequent processing; the physical prior generation module uses various physical operators to extract degradation features and provides refined physical guidance to the network through task-adaptive candidate maps, enhancing the recovery capability of degradations such as rain, fog, low light, and motion blur; the semantic anchor generation module calculates semantic embeddings offline, providing reliable semantic conditions without the need for online large model inference, ensuring efficient and low-latency operation of the system; the semantic-physical fusion module achieves deep interactive modulation of semantic information and spatial features through a semantic-physical co-gated attention mechanism, enabling the network to obtain targeted guidance at the internal multi-scale feature level; the image restoration module achieves gradual image restoration based on the predicted channel field through Euler iteration, maintaining structural continuity and detail integrity; the training and optimization module uses a joint loss function to train the network end-to-end, ensuring that the network has robustness and high restoration accuracy in multiple degradation scenarios, improving the recognizability and reliability of traffic evidence images, and providing high-quality image support for key targets such as license plates, container numbers, vehicle markings, hazardous chemical signs, and vehicle outlines.

[0027] Thirdly, an electronic device includes a memory, a processor, and a computer program stored in the memory, wherein when the processor executes the computer program, it implements the semantic-physical co-modulation traffic evidence image restoration method according to any one of claims 1 to 8.

[0028] Fourthly, a computer-readable storage medium storing a computer program thereon, characterized in that: when the computer program is executed by a processor, it implements the semantic-physical co-modulation traffic evidence image restoration method according to any one of claims 1 to 8.

[0029] By adopting the above technical solution, the semantic-physical co-modulation traffic evidence image restoration method can be implemented in software on a general-purpose computing device through a computer-readable storage medium. Users can complete the entire process of degraded image acquisition, physical prior generation, semantic modulation fusion and image restoration by executing the stored program without additional hardware. It takes into account both flexible deployment and efficient operation, and can ensure the integrity of image details and structure in multiple degradation scenarios, thereby improving the recognition accuracy of evidence targets such as license plates, box numbers, vehicle markings and vehicle outlines and the overall reliability of the system.

[0030] In summary, this application includes at least one of the following beneficial technical effects: 1. This application uses a semantic-physical co-modulation mechanism to deeply integrate physical degradation priors with semantic conditions, thereby achieving adaptive image restoration for complex traffic degradation such as rain, fog, low light, and motion blur. This effectively improves the recognition accuracy and verification reliability of traffic evidence targets such as license plates, container numbers, vehicle markings, hazardous chemical signs, and vehicle outlines. 2. This application employs a task-selective physical prior and semantic anchor mechanism, enabling the unified conditional neural network to achieve highly robust restoration in multiple degradation scenarios without manual parameter tuning, while maintaining the continuity of image structure and the integrity of details through Euler iteration; 3. This application provides a modular system and readable storage medium implementation scheme, which balances edge deployment efficiency and low-latency operation. It does not rely on large online visual language models and can stably and in real-time complete traffic evidence image restoration tasks in closed traffic hubs, private networks, or high frame rate video scenarios. Attached Figure Description

[0031] Figure 1 This is an interface structure diagram of the DFSCM module in an embodiment of this application, showing the interface relationship between the degraded image, the degraded index / operation code, the bottleneck feature and the prior map, the candidate map and the modulation feature.

[0032] Figure 2 This is an internal structure diagram of the DFSCM module in an embodiment of this application, showing the data flow between the degenerate feature primitive library, physical candidate map, offline semantic anchor cache, and semantic-physical co-gating.

[0033] Figure 3 This is a structural diagram of the prior map generation of degradation features in 3DFSCM embodiment of this application, showing the various physical operator groups, mask activation mechanism, preprocessing candidate map and 30-channel prior stitching process.

[0034] Figure 4 This is a diagram of the DFSCM semantic-physical co-gated attention structure in an embodiment of this application, illustrating the query, key-value generation, and spatial feature modulation process.

[0035] Figure 5This is a framework diagram of the traffic evidence collection image restoration system according to an embodiment of this application. Detailed Implementation

[0036] The following is in conjunction with the appendix Figure 1-5 This application will be described in further detail.

[0037] This application discloses a method for restoring traffic evidence images using semantic-physical co-modulation.

[0038] Includes the following steps: S1. Obtain degraded traffic images collected by the evidence collection terminal at the edge of the closed traffic hub, and obtain the discrete index of the degradation category or the on-site condition code corresponding to the degraded traffic images. S2. Input the degraded traffic image into the degradation feature-semantic co-modulation module to calculate the corresponding physical degradation prior information. The physical degradation prior information includes at least one or more of the following: differential Gaussian decomposition components, Sobel gradient magnitude map, local variance map, dark channel transmission map, Retinex illumination separation components, rain line direction template response map, noise level estimation map, and edge confidence map. S3. Based on the discrete index of the degradation category or the field condition code, perform task-selective mask weighted fusion on each physical degradation prior information to generate a degradation feature prior map, and simultaneously generate a task-adaptive physical candidate map corresponding to the current degradation category. S4. Based on the discrete index of the degradation category, read the corresponding fixed text phrase from the preset semantic phrase library, and generate the corresponding semantic embedding vector through the text encoder in the pre-trained image-text alignment model to obtain the normalized semantic anchor. S5. The degraded traffic image, the task adaptive physical candidate map, and the degraded feature prior map are concatenated along the channel dimension and input into the conditional neural network backbone. In the bottleneck layer of the conditional neural network backbone, the semantic anchor is fused with the spatial features in a cross-attention manner through a semantic-physical co-gated attention mechanism to obtain semantic modulation features. S6. Based on the semantic modulation features, predict the image recovery channel field, and generate the recovered image through single-step or multi-step Euler iteration; S7. Based on the channel field loss, reconstruction loss, structural similarity loss, and preprocessing auxiliary loss, the conditional neural network is jointly trained end-to-end to obtain a traffic evidence collection image restoration model.

[0039] Reference Figure 5 A semantic-physical co-modulation traffic evidence image restoration system includes an image acquisition module, which is used to acquire degraded traffic images and corresponding degradation category information; This includes a physics prior generation module, which generates a degenerate feature prior map and a task-adaptive physics candidate map. This includes a semantic anchor generation module, which generates corresponding semantic anchors based on the degradation category; It includes a semantic-physical fusion module, which is used to fuse semantic information and spatial features through a semantic-physical co-gated attention mechanism; It includes an image restoration module, which is used to predict and restore the channel field and output the restored image; It includes a training optimization module, which is used to train the conditional neural network based on the joint loss function.

[0040] This application aims to solve at least the following technical problems: 1) Unified restoration of various degradation scenarios in the evidence collection scenario at the edge of closed transportation hubs, eliminating the difficulties of parallel operation and maintenance of multiple models and the risk of incorrect routing; 2) Without relying on the constraints of online large visual language models, it provides dual constraints at both the physical prior level and the degenerate semantic level for the unified recovery network. 3) By using the Degenerate Feature-Semantic Co-modulation Module (DFSCM), physical-driven preprocessing and prior feature extraction are completed before the image enters the deep backbone, and degenerate semantic modulation is completed at the bottleneck layer of the network, reducing the learning burden of pure data-driven methods on spatial structures such as evidence characters, rain line direction, and fog scattering gradient. 4) By using a semantic-physical co-gating mechanism, the degraded semantic vector hints from offline caching are deeply integrated into network bottleneck features, thereby enhancing the ability of semantic conditions to substantially guide multi-scale features. 5) Meets the engineering deployability requirements for scenarios with limited computing power and network isolation, such as edge computing and private network deployment.

[0041] During implementation, refer to Figures 1-4 The traffic evidence image restoration method provided in this application includes the following steps: Step 1: Obtain the degraded image and corresponding degeneracy category discrete index ; Step 2: Calculate the 30-channel prior feature map P using the degenerate feature primitive library in the Degenerate Feature-Semantic Co-modulation Module (DFSCM); and task-adaptive preprocessing candidate graph No external database required; Step 3: Read fixed English phrases from the pre-stored phrase table based on the degradation category index d, and compute and cache the 512-dimensional normalized semantic embeddings offline using the frozen CLIP text encoder. ; Step 4 , , The input tensor is concatenated along the channels to form a 36-channel input tensor, which is then fed into the U-Net backbone with time-step conditions. Simultaneously, at the bottleneck layer, a semantic-physical co-gated attention mechanism (MDCross) is used to... Deeply integrated into spatial features; Step 5: Network Prediction of Channel Fields The image is recovered by single-step or multi-step Euler sampling. Step 6: Train the network end-to-end using the combined channel field loss, reconstruction loss, structural similarity loss, and preprocessing auxiliary loss.

[0042] This application also provides a traffic evidence image restoration system, including a memory and a processor, wherein the memory stores a program, and the processor executes the program to implement the above steps.

[0043] This application also provides a computer-readable storage medium for storing a program that performs the above-described methods.

[0044] 4. Degradation Feature-Semantic Co-modulation Module (DFSCM) 4.1 Module Positioning, Interfaces, and Engineering Significance In the forensic scenario of a closed transportation hub, the same frame of image often exhibits degradation features that can be physically modeled: rain images show fine line structures distributed along specific directions, the contrast of distant scenes in fog images follows the atmospheric scattering equation, noise images show abnormal energy accumulation in the spectral ring region, and the white balance deviation between color channels in low-light images can be quantified by the gray world hypothesis. The engineering starting point of the DFSCM module is to explicitly express the degradation-related deterministic information using low-cost physical operators before the deep network backbone begins large-scale matrix operations. This information is fed into the network in the form of a 30-channel degradation feature prior map, and offline semantic anchors are injected at the bottleneck layer through a semantic-physical co-gating mechanism. This reduces the learning pressure of the pure data-driven approach on the degradation-specific spatial structure and improves the generalization stability when the degradation intensity changes.

[0045] Another engineering advantage of this module is its complete differentiability and the fact that it requires no additional training data: all physical operators are fixed kernel function operations or analytical formulas, with only degenerate embeddings. and prior gain coefficient These are learnable parameters with a very small number of parameters, which do not increase the storage and quantization adaptation burden of inference deployment.

[0046] 4.2 Symbol Conventions Let the batch size be The input degraded image is The degradation category index is . The standard deviation is expressed as Gaussian convolution kernel, This represents depthwise convolution (grouped by channel). Indicates to The result of grayscale conversion by taking the average of the three channels.

[0047] 4.3 Physical Operator Set 4.3.1 Difference Gaussian Decomposition (DoG) The input images are respectively... and Gaussian smoothing is performed to obtain the low-frequency components. and high-frequency edge enhancement components : It reflects the image's illumination and low-frequency texture trends. Sensitive to high-frequency degradation components such as rain lines and noise, it provides frequency separation priors for the network. In areas with high noise levels (… When ), for The weights are suppressed to avoid transmitting excessively strong high-frequency noise signals to the network.

[0048] 4.3.2 Sobel gradient field and local variance plot Apply a Sobel horizontal kernel to the grayscale image and vertical core The gradient magnitude map is obtained. : Local variance plots are used to estimate local statistics using a 7×7 mean filter: These two aspects are particularly important for deblurring tasks in engineering: the gradient magnitude of motion-blurred images is significantly lower than that of clear images, and the local variance map can effectively locate the blurred boundary region, allowing the auxiliary network to focus on restoring high-frequency details in these regions.

[0049] 4.3.3 Dark Channel Transmission Map Estimation and Dehazing Prior Based on the dark channel prior of He et al. (a technique known in the art), the dark channel of the degraded image is calculated: in A 15×15 local window is used. Atmospheric light is estimated from the brightest 0.1% of pixels in the dark channel. The transmittance is estimated as follows: Through The guide filter refines the transmittance of the guide map. (Filter radius) Regularization coefficient The color decay prior (CAP) is: The above transmission diagram Refined transmission image The dehazing-specific prior consists of 5 channels: atmospheric light quantity ASCII and color attenuation prior. This prior is only available in... (Defog task) is activated with a mask to avoid interfering with other tasks.

[0050] 4.3.4 Retinex Illumination Separation and Low-Light Enhancement Prior Using single-scale Retinex (SSR, a well-known technique in the field): Illumination Components (1-channel mean) and reflectivity components (3 channels) Captures the spatial distribution of brightness attenuation in low-light images. Additional candidate images for adaptive Gamma correction are also provided. And candidate maps for histogram equalization based on channel-wise cumulative distribution function (CLAHE class). The above four channels constitute a low-light enhancement-specific prior, which is only used in... Activated at that time.

[0051] 4.3.5 Rain line direction response and rain-out prior Rain streaks are typically distributed nearly vertically or obliquely in images. This invention designs a three-directional differential template to capture the rain streak response: White hat morphological operations (white hat = original image - opening operation) extract fine bright stripes from an image: Sparse high-frequency maps are extracted using an adaptive thresholding method based on the Laplacian response: The three items, totaling three channels, constitute a rain-removal-specific prior, which is only applicable to... Activated at that time.

[0052] 4.3.6 Multidimensional noise level estimation This invention estimates noise levels from three complementary dimensions: 1) Local variance noise map (shared calculation with variance map): 2) Median Absolute Deviation (MAD) Estimation: For high-frequency residuals Calculate after taking the average: 3) Spectral Ring Energy Ratio: Perform a two-dimensional discrete FFT on the grayscale image to calculate the high-frequency energy band (radial frequency > 0.35). ) and low-frequency energy band (radial frequency <0.15) The ratio of ) In engineering scenarios, this 3D noise estimation exhibits complementary sensing sensitivity to noise from different sources (sensor thermal noise, compressed quantization noise, and shot noise). The 3-channel noise prior is activated when d∈{0,1,2}, and the gain is multiplied by 1.3 when σn=50 to enhance the prior weights in noisy scenarios.

[0053] 4.3.7 Marginal Confidence Plot (Defuzzy Prior) The consistency of gradient directions is calculated based on the Sobel gradient field to measure the sharpness of local edges: Additional candidate images for lightweight sharpening The three items, totaling three channels, constitute the deblurring process. Exclusive priors, only Activated at that time.

[0054] 4.4 Task Adaptive Preprocessing Candidate Graph One of the key design features of DFSCM is outputting a physics-driven preprocessing candidate graph corresponding to the degradation category. As an "intermediate reference" between the degraded and clean images, it is directly incorporated into the network input: The mask The candidate graphs are defined as follows: 1) Denoising candidate image: Guided filtering results by hierarchical weighting based on noise intensity index. in This is the result of lightweight guided filtering denoising.

[0055] 2) Candidate images for rain removal ( (for learnable strength weights) 3) Dehazing candidate images: 4) Deblurring candidate images: 5) Candidate images for low-light enhancement: In the project deployment, As an initial guess of physical constraints, it is used in conjunction with the degenerate graph and prior features. Figure 1 By incorporating the data into the network, the network performs residual refinement based on physical preprocessing for specific degradation types, rather than learning from the entire degraded image. This is particularly beneficial when the number of training samples is limited or the degradation distribution is shifted.

[0056] 4.5 Concatenation and Gain Control of Prior Feature Maps The prior information for each task is weighted by a mask and concatenated to obtain a 30-channel prior feature map: in (6 channels), Gray World Residual Defined as: Prior gain coefficient Learnable degenerate embeddings Through two layers of MLP computation, the network is able to learn the optimal magnitude scaling of prior features for each degradation category end-to-end.

[0057] 4.6 Semantic-Physical Co-gated Attention Mechanism (MDCross) 4.6.1 Module Positioning and Engineering Significance In practical engineering, CLIP text semantic anchors carry high-level degradation expectation information such as "dehazing should restore contrast and depth of field" and "de-raining should eliminate stripes and retain sharp edges." If such semantics are only constrained outside the loss function (such as alignment loss), they cannot directly affect the computation path of features within the network. Therefore, this invention uses MDCross as a semantic-physical co-gated attention mechanism within DFSCM, rather than listing it as a separate, isolated technical module.

[0058] The design goal of this mechanism is to inject a 512-dimensional degenerate semantic embedding vector into the spatial feature map at the network bottleneck in a way that minimizes computational cost, so that the modulation of the deepest features has semantic awareness capabilities without increasing inference latency and parameter storage.

[0059] 4.6.2 Module Structure MDCross operates at network bottlenecks using a cross-attention mechanism: generating queries from spatial feature maps and key-value pairs from semantic vectors. Let the bottleneck feature map be... The degenerate semantic vector is The number of attention heads is Dimensions per head .

[0060] 4.6.2.1 Query Generation Using 1×1 convolution from spatial feature maps Generate query: Reshape it into Each spatial location corresponds to a query vector.

[0061] 4.6.2.2 Key and Value Generation From semantic vectors Generate keys and values ​​through a linear transformation, while maintaining a spatial dimension of 1: Remodeling That is, each head corresponds to a single vector key / value.

[0062] 4.6.2.3 Attention Calculation and Feature Modulation The relevance score between each spatial location and the semantic vector is calculated using a scaled dot product: Attention-weighted semantic modulation residuals are obtained that are spatially uniform. After 1×1 convolution projection, the image is broadcast to full resolution and added to the residual of the original feature map: 4.6.2.4 Design Advantage Analysis The computational complexity of MDCross is It is independent of spatial resolution. Given that the feature map resolution of the bottleneck layer has typically been reduced to H / 16×W / 16, even using standard self-attention already incurs low overhead. MDCross further compresses the key-value dimension into a single semantic vector, reducing the attention computation to negligible levels while maintaining semantic modulation efficiency.

[0063] From an engineering deployability perspective, MDCross adds only three linear transformation matrices as parameters. common One parameter, in the basic channel (Bottleneck dimension) is about 1MB in size, which can be ignored.

[0064] 5. Network Backbone and Overall Forward Process 5.1 Input Concatenation Degraded image Preprocessing candidate images output by DFSCM and 30-channel prior feature map Along the way Daowei splicing: 5.2 Time Step Embedding and Conditional Fusion Time step After sine and cosine position encoding, it is mapped to a temporal embedding vector by two layers of MLP: The temporal embedding vector is injected into each ResBlock via FiLM affine modulation to control the scaling and translation of features at different time steps, adapting to the changing characteristics of the channel field at different time steps.

[0065] 5.3 U-Net Encoding / Decoding Structure The backbone employs a 4-scale symmetric U-Net. Each coding layer contains two ResBlocks with temporally conditional FiLM modulation and one linear attention layer, with spatial downsampling achieved through convolutions with a stride of 2 or learnable downsampling. The decoding layers are symmetrically configured, with skip connections concatenating the corresponding coding layer feature maps along the channels before inputting them into the decoding ResBlock.

[0066] ResBlock forward (FiLM modulation): Linear attention (computational complexity) Suitable for high-resolution coding layers): 5.4 MDCross Fusion of the Bottleneck Layer At the deepest bottleneck of U-Net, the feature maps are processed by double ResBlock and linear attention, and then fed into the semantic-physical co-gated attention mechanism in DFSCM for semantic fusion: The introduction of the semantic-physical co-gated attention mechanism enables the bottleneck features to obtain global modulation of the degenerate semantic conditions at the maximum receptive field, ensuring that each subsequent decoding scale carries semantic-aware information during feature decompression.

[0067] 5.5 Channel Field Prediction and Euler Sampling Network final output channel field Random sampling during training Constructing intermediate interpolation states: The target channel is When reasoning, use Euler Iterative Recovery: In the single-step (NFE=1) case, when K=1, the output is direct. This meets the low-latency inference requirements of edge devices.

[0068] 5.6 Training Objective Function The loss function in this section is used to simultaneously constrain the dynamic prediction, pixel fidelity, structural consistency, and rationality of the DFSCM physical candidate image of the restoration network. The loss terms are not independent substitutes for each other, but rather jointly constrain the restored image from different levels: the channel field loss ensures the stability of the streaming sampling path, the reconstruction loss ensures the overall pixel similarity, the structural similarity loss ensures local contrast and texture structure, and the preprocessing auxiliary loss limits the physical candidate image to only serve as an intermediate reference and prevents it from overfitting the clean image.

[0069] 5.6.1 Channel Field Loss The channel field loss is used to constrain the channel directions predicted by the network at random time steps to be consistent with the target channels from the degraded image to the sharp image. This loss enables the model to learn not only the final restored result, but also the trend of gradual transition from the degraded state to the sharp state, thereby ensuring the output stability during single-step or multi-step Euler sampling.

[0070] 5.6.2 Reconstruction Losses The reconstruction loss directly measures the pixel-level difference between the restored image and the true sharp image, and is used to constrain the low-frequency consistency of overall brightness, color, and texture. In traffic evidence collection scenarios, this loss helps to maintain the basic visual content of license plate areas, container number areas, vehicle markings, and road boundaries without shifting.

[0071] 5.6.3 Structural Similarity Loss Structural similarity loss is used to compensate for the insufficient perception of local structure by simple pixel errors. It focuses on constraining the relationship between local mean, variance and covariance, so that the restored result maintains more stable contrast and structural continuity at character edges, lane lines, vehicle outlines and distant boundaries.

[0072] Local mean and variance are estimated using 3×3 average pooling, and position-by-position SSIM is calculated. ): 5.6.4 Preprocessing Auxiliary Losses The preprocessing auxiliary loss is used to constrain the physical candidate map output by DFSCM to maintain a "moderately corrected" intermediate state. This loss avoids the candidate map directly fitting the clear map and undertaking the entire restoration task of the backbone network, while also preventing the candidate map from deviating too far from the degraded map and introducing unreliable priors to subsequent networks.

[0073] The candidate images output by DFSCM are encouraged to approximate the degraded images, serving as a suitable physical reference rather than an overfit to the clean images. 5.6.5 Total Loss Function The total loss function sums the above constraints according to their weights. In the early stages of training, the weights of reconstruction loss and channel field loss can be increased to ensure convergence stability. In the later stages of training, the weights of structural similarity loss can be appropriately increased to enhance the details of evidence characters, edges, and textures. The weights of preprocessing auxiliary loss are kept small to stabilize DFSCM candidate images without suppressing the learning ability of the backbone network.

[0074] In one embodiment, =1.0, =1.0, = 0.12, It can be set to 0 or a small positive number at different stages.

[0075] Example 1 (Single-task inference from an edge forensics terminal) The tollbooth ramp evidence-collecting camera is connected to an edge evidence-collecting terminal, which has built-in weights trained according to this invention. The system sets a degradation category index d=3 (rain removal) based on current operating conditions (such as rainfall sensor signals and time period), and sends the camera output frames to DFSCM to calculate the rain line direction response Rrain and white-hat priors. Generate candidate images for rain removal The three-channel rain removal prior is stitched together and fed into the backbone network for single-step inference (NFE=1). The output rain removal image is then transmitted to the video platform. The overall inference latency meets the real-time frame rate requirements.

[0076] Example 2 (Multi-degenerate batch mixed training) During training, each batch contains paired training data of multiple degradation categories (rain, fog, noise, low light). DFSCM adaptively activates the corresponding physical operator based on the degradation category index of each sample, and uses a masking mechanism to ensure that the dedicated prior channels for each degradation category do not contaminate each other. A semantic-physical co-gated attention mechanism obtains semantic vectors from the CLIP embeddings of the corresponding degradation phrases, applying degradation-aware modulation to the feature map at the bottleneck layer, enabling the same network to effectively distinguish the feature space distributions of different degradation categories. The optimizer uses Adam (… =0.9, = 0.99), learning rate 10 -4 The total number of training iterations was 700,000.

[0077] Example 3 (Offline Verification at Port Gates) Videos from port gates or hazardous chemical industrial park entrances and exits are batch-processed on mobile workstations without network connectivity. The system batch-assigns degradation indexes based on weather conditions, visibility estimates, and compression bitrate. This invention runs entirely offline and does not require access to external language model services, thus meeting security audit requirements.

[0078] Example 4 (Hardware Form) The system includes one or a combination of an edge computing device (embedded GPU / NPU) or a server GPU. Storage media include solid-state drives, embedded flash memory, or network-attached storage. The network of this invention can be quantized using INT8 on the edge device, and the physical operator portion of DFSCM can be pre-optimized using a fixed-core fusion approach to further reduce inference power consumption.

[0079] Beneficial effects of the technical solution of this application 1. Synergy between physical priors and deep learning: DFSCM completes the extraction of physical priors before the network backbone, reducing the learning burden of deep networks on degenerate-specific spatial structures (rain line directionality, fog gradient, noise spectrum distribution), and has stronger generalization stability when training samples are limited or degenerate distributions drift. 2. Semantic Conditional Deep Fusion: The semantic-physical co-gated attention mechanism in DFSCM anchors CLIP degenerate semantics at the bottleneck maximum receptive field and integrates them into spatial features. Compared with outer-loop semantic loss, it has a more direct feature-level modulation capability and effectively reduces feature confusion between different degenerate categories. 3. No online large model dependency: Degenerate semantic phrases are CLIP encoded once during model initialization and cached as non-trainable tensors. Inference runs offline throughout, meeting the network isolation requirements of private networks and edge deployments. 4. Single deployment, unified multi-task: A single set of weights handles seven types of degradation (noise level 3, rain removal, fog removal, deblurring, and low-light enhancement), eliminating the risk of multi-model routing errors; 5. Scalable sampling steps: NFE=1 meets real-time scenarios, NFE=2 and above can further improve recovery quality when computing power allows, and the same weight supports deployment forms with different computing power budgets.

[0080] The implementation principle of the semantic-physical co-modulation traffic evidence image restoration module and method in this application embodiment is as follows: First, the degradation feature-semantic co-modulation module extracts physical degradation priors such as differential Gaussian decomposition components, Sobel gradient magnitude map, local variance map, dark channel transmission map, Retinex illumination separation components, rain line direction response map, and noise level estimation map. These priors can describe the physical characteristics of degradation such as rain, fog, low light, and noise. Then, according to the degradation category index or the on-site working condition code, the physical priors are weighted and fused by task-selective masking to generate a degradation feature prior map and a task-adaptive physical candidate map, thereby realizing adaptive physical guidance in multiple degradation scenarios. Simultaneously, semantic anchors are generated from the pre-trained image-text alignment model. Through a semantic-physical co-gated attention mechanism, semantic information and spatial features are cross-fused at the bottleneck layer of the conditional neural network backbone. This enables the network to acquire semantic modulation capabilities at the internal multi-scale feature level, thereby accurately guiding the restoration of rain line direction, contrast in fog, and brightness distribution at night. Based on semantic modulation features, the image restoration channel field is predicted, and the image state is gradually updated through Euler iteration to achieve high-fidelity restoration of structural continuity and detail integrity. The entire network adopts end-to-end joint training, including channel field loss, reconstruction loss, structural similarity loss, and preprocessing auxiliary loss, so that the system has robustness and high restoration accuracy in different degradation scenarios, thereby effectively improving the recognition accuracy and image usability of traffic evidence targets.

[0081] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for restoring traffic evidence images using semantic-physical co-modulation, characterized in that, Includes the following steps: S1. Obtain degraded traffic images collected by the evidence collection terminal at the edge of the closed traffic hub, and obtain the discrete index of the degradation category or the on-site condition code corresponding to the degraded traffic images. S2. Input the degraded traffic image into the degradation feature-semantic co-modulation module to calculate the corresponding physical degradation prior information. The physical degradation prior information includes at least one or more of the following: differential Gaussian decomposition components, Sobel gradient magnitude map, local variance map, dark channel transmission map, Retinex illumination separation components, rain line direction template response map, noise level estimation map, and edge confidence map. S3. Based on the discrete index of the degradation category or the field condition code, perform task-selective mask weighted fusion on each physical degradation prior information to generate a degradation feature prior map, and simultaneously generate a task-adaptive physical candidate map corresponding to the current degradation category. S4. Based on the discrete index of the degradation category, read the corresponding fixed text phrase from the preset semantic phrase library, and generate the corresponding semantic embedding vector through the text encoder in the pre-trained image-text alignment model to obtain the normalized semantic anchor. S5. The degraded traffic image, the task adaptive physical candidate map, and the degraded feature prior map are concatenated along the channel dimension and input into the conditional neural network backbone. In the bottleneck layer of the conditional neural network backbone, the semantic anchor is fused with the spatial features in a cross-attention manner through a semantic-physical co-gated attention mechanism to obtain semantic modulation features. S6. Based on the semantic modulation features, predict the image recovery channel field, and generate the recovered image through single-step or multi-step Euler iteration; S7. Based on the channel field loss, reconstruction loss, structural similarity loss, and preprocessing auxiliary loss, the conditional neural network is jointly trained end-to-end to obtain a traffic evidence collection image restoration model.

2. The semantic-physical co-modulation traffic evidence image restoration method according to claim 1, characterized in that: In step S3, the gain coefficients corresponding to each physical degradation prior information and the intensity weights corresponding to the task adaptive physical candidate map are generated by inputting the learnable degradation embedding vector corresponding to the degradation category into the multilayer perceptron network, so that different degradation categories correspond to different physical prior enhancement magnitudes and candidate map intensities, thereby achieving adaptive optimization of physical prior parameters.

3. The traffic evidence image restoration method based on semantic-physical co-modulation according to claim 1, characterized in that: The text encoder in the pre-trained image-text alignment model keeps its parameters frozen during both the model training and inference phases. The semantic anchor is pre-computed offline during the model initialization phase and registered as a non-trainable cached tensor. During the inference phase, it is called by looking up the table based on the discrete index of the degradation category.

4. The semantic-physical co-modulation traffic evidence image restoration method according to claim 1, characterized in that: The semantic-physical co-gated attention mechanism includes: Map the spatial feature map to the query vector Q; The semantic anchors are mapped to key vector K and value vector V, respectively; Attention weights are obtained by performing a scaled dot product operation between the query vector Q and the key vector K. The attention weights are used to weight the value vector V to generate semantic modulation residuals; The semantic modulation residual is projected by a 1×1 convolution and then broadcast and superimposed onto the spatial feature map to achieve conditional modulation of spatial features by semantic information.

5. The traffic evidence image restoration method based on semantic-physical co-modulation according to claim 1, characterized in that: The physical degradation prior information in step S2 includes rain-specific physical priors and fog-specific physical priors. The rain-removal-specific physical priors include a rain line direction response map generated by a three-directional difference template, a fine stripe enhancement map generated by white hat morphological operations, and a sparse high-frequency response map. The dehazing-specific physical priors include dark channel maps, atmospheric light estimation maps, original transmission maps, guided filtering refined transmission maps, and color attenuation prior maps. Each dedicated physical prior is selectively activated for the task based on the degradation category mask.

6. The traffic evidence image restoration method based on semantic-physical co-modulation according to claim 1, characterized in that: The evidence collection terminals at the edge of the closed transportation hubs are deployed at highway toll station ramps, tunnel entrances and exits, port gates, hazardous chemical industrial park entrances and exits, logistics park entrances and exits, or underground parking lot entrances and exits. The restored image is used for license plate recognition, container number recognition, vehicle marking recognition, hazardous chemical marking recognition, vehicle contour extraction, or vehicle trajectory verification. The conditional neural network backbone adopts an encoder-decoder structure. In the encoding stage, degradation features are extracted through multi-scale convolutional modules. In the decoding stage, image detail information is restored through skip connections. The semantic-physical co-gated attention mechanism is embedded in the bottleneck layer.

7. The semantic-physical co-modulation traffic evidence image restoration method according to claim 1, characterized in that: The Euler iteration in step S6 includes: Explicitly integrate and update the current image state based on the predicted channel field: xt+1 = xt + Δt·vθ(xt,t,c) Where xt represents the image state at step t, Δt represents the integration step size, vθ represents the channel field predicted by the conditional neural network, and c represents the degradation condition information.

8. A traffic evidence image restoration system based on semantic-physical co-modulation, characterized in that, It includes an image acquisition module, which is used to acquire degraded traffic images and corresponding degradation category information; It includes a physics prior generation module, which is used to generate a degradation feature prior map and a task-adaptive physics candidate map; It includes a semantic anchor generation module, which is used to generate corresponding semantic anchors based on the degradation category; It includes a semantic-physical fusion module, which is used to fuse semantic information and spatial features through a semantic-physical co-gated attention mechanism; It includes an image restoration module, which is used to predict and restore the channel field and output the restored image; It includes a training optimization module, which is used to train a conditional neural network based on a joint loss function.

9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program, it implements the semantic-physical co-modulation traffic evidence image restoration method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the traffic evidence image restoration method of semantic physical co-modulation as described in any one of claims 1 to 8.

Images