Intelligent electronic police dynamic scene imaging clear method

Abstract

The application discloses a kind of intelligent electronic police dynamic scene imaging clear method, first, obtain electronic police dynamic scene video frame and clean, get clear image.Secondly, using the average method of frame insertion obtains fuzzy image and gradient diagram, establishes data set.Then, EADN network is constructed, and edge information and multi-scale information are used to realize fuzzy image clearification, and loss function is involved to train network.Finally, the network is tested to verify the network effect, and the output is the clear image of deblurring.The application enhances the ability of network learning nonlinear feature, generates more accurate attention map, reduces data redundancy, accelerates network convergence, so that the restored image is clearer.

Description

Technical Field

[0001] This invention relates to the field of image processing, specifically to a method for enhancing the clarity of dynamic scene imaging for intelligent electronic police systems. Background Technology

[0002] Rapid economic development has led to continuous improvements in transportation infrastructure. However, with the ever-increasing number of cars, modern society faces significant challenges to traffic safety, making the detection and recording of traffic violations crucial. Intelligent transportation systems, such as electronic traffic enforcement cameras, have greatly helped alleviate traffic congestion and improve traffic management.

[0003] Although camera technology is now highly advanced, traffic enforcement cameras often face complex scenarios with multiple moving targets, such as traffic light intersections. This inevitably leads to blurry images captured by these cameras. The blurriness is primarily dynamic scene blur, including large-scale blur caused by camera shake and non-uniform blur caused by the different motion states of multiple objects. Blurry images have become a major obstacle to traffic enforcement. Furthermore, dynamic scene blur differs from general blur types; it is mainly caused by different convolution kernels performing convolution at different locations in the image. This makes the restoration of blurred images in dynamic scenes a highly underdetermined problem, and previously used deblurring methods have not been very effective. Therefore, ensuring that traffic enforcement cameras obtain clear images in dynamic scenes is crucial for reducing traffic accidents and recording violations. Summary of the Invention

[0004] This invention addresses the problem of dynamic image blurring caused by high-speed targets or complex scenes during electronic traffic enforcement (ECE) violation capture. It provides a method for enhancing the clarity of dynamic scene images in intelligent ECE systems, resolving large-scale blurring and non-uniform motion blur issues arising during dynamic imaging. To achieve the above objective, this invention employs the following technical solution:

[0005] A method for enhancing the clarity of dynamic scene images in intelligent electronic police systems includes the following steps:

[0006] S1: Acquire dynamic scene video frames of electronic police cameras and clean them using the Laplacian operator to obtain clear video frames, i.e., clear images.

[0007] S2: Obtain blurred images using the frame interpolation averaging method, and build a dataset together with the gradient maps of the clear and blurred images. The dataset includes a training set and a test set.

[0008] S3: Construct the Edge Deblurring Network EADN, which includes a strong edge extraction network and a multi-level deblurring network. It utilizes edge information and multi-scale information to achieve the sharpening of blurred images.

[0009] S4: Calculate the loss to train EADN. The loss function includes perceptual loss, edge loss, and frequency domain content loss.

[0010] S5: Test the trained network to verify its performance. The input is the images in the test set, and the output is the clear, deblurred image.

[0011] This invention utilizes OpenCV to clean video frames. The specific method is as follows:

[0012] The input video frames are converted to grayscale, then the gradient map is extracted using the Laplacian operator, and its variance is calculated as the return value x. x is then compared with a pre-set threshold. The comparison is performed, where 'a' represents the average variance of the gradient map of the blurred image, and 'b' represents the average variance of the gradient map of the sharp image. This invention selects the same number of blurred and sharp images for average variance calculation. If... The video frame is then marked as a clear image; otherwise, it is marked as a blurry image and placed in the blurry image category. This invention only selects clear images for subsequent processing. Since image size and scene affect the returned gradient value, this invention calculates a suitable threshold range using multiple publicly available dynamic scene blur datasets. The specific calculation formula is as follows:

[0013]

[0014] Where N represents the number of images, var indicates variance calculation, laplace indicates gradient extraction, and I... blur and I sharp These represent blurred and sharp images, respectively.

[0015] The dataset is created as follows:

[0016] 1. Group the images marked as clear according to their scene content, with images from the same scene forming a group. Based on the principle that the number of images in each group must not exceed a threshold T, a total of P groups of clear images are obtained.

[0017] 2. Let the i-th group of images be S. i ={F0,F1,…,F j ,…,F N-1}, 1≤N≤T, 0≤j≤N-1, where, F N-1Let S' represent the Nth image in the i-th image group. Then, the i-th image group is processed using frame interpolation averaging to obtain a blurred image. The specific steps of frame interpolation averaging are as follows: 7 frames are inserted between two adjacent clear images using the RIFE algorithm (Real-Time Intermediate Flow Estimation), achieving 8-fold interpolation. Therefore, the i-th image group generated after frame interpolation can be represented as: S′ i ={F′0,F′1,…,F′ j×8 ,…,F′ (N-1)×8 The original clear image is F. j =F′ j×8 ,0≤j≤N-1. Therefore, when discarding the original image group S i Given the first and last three frames, the set of clear images in the i-th group can be obtained as: Q i ={q j |q j =F j+3 =F′ (j+3)×8}, 0≤j≤N-7, where q j This represents the j-th image in the i-th group of images.

[0018] 3. The corresponding set of blurred images can be represented as: B i ={b j |b j =G j+3}, 0≤j≤N-7, where b j That is, G j+3 Representing the j-th image in the i-th group of blurred images, its generation process can be described as taking the original sharp image F... j+3 After frame interpolation, for S′ i The corresponding clear image F′ (j+3)×8 The average of the sharp images of the region and its adjacent spatial domains is calculated by adding them together. The specific formula is as follows:

[0019]

[0020] Where L represents the length of the selected spatial domain.

[0021] 4. Use the Sobel operator to extract gradient maps from the acquired blurred image set, obtaining the gradient map set E. i Therefore, the obtained dataset includes pairs of blurred and sharp images, as well as gradient maps of the blurred images, which can be represented as: D = {Q} i B i E i The dataset obtained in this invention includes two parts: a training set and a test set.

[0022] This invention selects a convolutional neural network as the network structure for completing the image restoration task. This invention constructs an EADN (Edge And Deblurring Network) network and designs a Hybrid-activated DCT Adaptive Module (HDAM) and a Cross-stage Adaptive Fusion Module (CAFM) to achieve image restoration.

[0023] The EADN network consists of two networks: a strong edge extraction network and a multi-level deblurring network. This invention specifically designs a parallel structure for the multi-level deblurring network and the strong edge extraction network. The strong edge extraction network uses a U-Net structure as its main structure, with HDAM groups added to both the encoder and decoder to obtain important information. Each HDAM group consists of multiple HDAM blocks. The strong edge extraction network can utilize the acquired edge information to guide the multi-level deblurring network to achieve the deblurring task through feature fusion connections and a CAFM module, and can also supplement the missing fine structural information in the blurred image.

[0024] Unlike strong edge extraction networks, the multi-level deblurring network employs a fusion structure of U-Net and multi-scale architecture. Each scale uses a U-Net structure, with the input image obtained by downsampling the input image at a higher scale by a factor of two, and multiple HDAM groups are added for feature extraction. This invention specifically designs a double downsampling operation at the largest scale, while omitting downsampling at the smallest scale, thereby reducing the overall computational cost of the network while enhancing its deblurring performance. Furthermore, adjacent scales in the multi-level deblurring network achieve information flow between different stages through upsampling fusion connections and cross-scale fusion connections, supplementing the higher-level scale with information from the smaller scale. The HDAM and CAFM modules will be described in detail below.

[0025] The HDAM module employs a dual-branch structure for image feature extraction. The input image x is fed into a 3×3 convolutional layer and a 5×5 convolutional layer, with a normalization layer added after each convolutional layer. The normalized results are then activated using ReLU and ELU to enhance the network's ability to learn nonlinear features. The specific calculation process is shown in equation (3).

[0026]

[0027] Here, concat represents fusion along the channel dimension, δ and ε represent ReLU and ELU activation functions respectively, and f3 and f5 represent 3×3 and 5×5 convolutional layers respectively.

[0028] The information is then sequentially input into the DCA (DCT Channel Attentional Block) and DSA (DCT Spatial Attentional Block) of the HDAM module to focus on important features in both the channel and spatial dimensions. This invention specifically replaces the pooling layers in traditional channel and spatial attention with Discrete Cosine Transform (DCT). This modification changes the implementation of the attention mechanism from the time domain to the frequency domain, thereby obtaining richer and more effective image information.

[0029] In the DCA module, the input X∈R is first... C×H×W Divide the channel into blocks, i.e., [X] 0 ,X 1 ,…,X p ,…,X q-1 ],X p ∈R C′×H×W p∈{0,1,…,q-1}, Where C, H, and W represent the number of channels, height, and width of the input X, respectively. Then, for X... p The formula for performing two-dimensional DCT on height and width is as follows:

[0030]

[0031] Where: : represents no calculation in the channel dimension, N is the number of points in the original signal, which can be H and W in equation (4), and k is an integer. Freq p Let C' represent the compressed C′-dimensional vector. Therefore, the entire compressed vector can be represented as:

[0032] Freq = concat(Freq) 0 ,Freq 1 ,…,Freq q-1 (5)

[0033] Here, concat represents fusion at the channel level.

[0034] Therefore, the entire DCA calculation process can be represented as:

[0035] U C =X·s=X·σ(F dense2 (δ(F dense1 (Freq))) (6)

[0036] Where σ represents the Sigmoid activation function, F dense1 and F dense2These represent the first and second fully connected layers, respectively.

[0037] In the DSA module, the output U from the DCA module C The formula for performing a one-dimensional DCT along the channel dimension is:

[0038]

[0039] Therefore, the entire DSA calculation process can be represented as:

[0040] U S =U C ·S=U C ·σ(f7(M)) (8)

[0041] Where f7 represents a 7×7 convolutional layer, U S This is the output of the DSA module.

[0042] The CAFM module will receive edge information Y from the same size. edge Characteristics of stage n, Y n and the characteristics of stage n-1 Y n-1 Adaptive fusion is used to enhance the network's ability to learn effective features and improve deblurring performance. The entire computation process of the CAFM module can be represented as:

[0043]

[0044] in, This represents the LeakyReLU activation function.

[0045] This invention selects a convolutional neural network jointly optimized by perceptual loss, edge loss, and frequency domain content FFT-content loss. The formula for calculating the joint loss function is as follows:

[0046] L=αL perceptual +βL edge +γL fftc (10)

[0047] Where α, β, and γ are the weights of the perceptual loss, edge loss, and FFT-content loss, respectively.

[0048] Perceptual loss measures the overall difference between the features of the generated image and the corresponding real, sharp image. The features used to calculate the perceptual loss in this invention can be extracted from a pre-trained VGG19 network. The inputs are a real, sharp image and a restored image. The perceptual loss uses the VGG19 feature layer to calculate the difference between the generated data and the original real data. The loss function formula is:

[0049]

[0050] Among them, I B and I S These are a blurry image and a clear image, respectively. W represents the feature map of the e-th convolution before the d-th max-pooling layer in the VGG19 network. d,e and H d,e G(I) represents the width and height of the feature map obtained from the VGG19 network, respectively. B ) x,y This represents the pixel value at point (x, y) of the deblurred image output by the network.

[0051] The Sobel operator is selected to establish an edge loss function, which measures the difference in edge features between the initially restored image and the clear image. The loss function takes the ground truth image and the deblurred image as inputs. The loss function is calculated by plotting the image gradients, thus improving the module's performance. The loss function formula is as follows:

[0052]

[0053] in, This represents the Sobel operator, where W and H represent the width and height of the input image, respectively.

[0054] Since the network learns features in the frequency domain, this invention specifically designs an FFT-content loss function. Unlike traditional content loss, this invention first performs a Fourier transform on the input image, and then calculates the loss function, which is formulated as follows:

[0055]

[0056] Here, fft represents the Fourier transform.

[0057] Therefore, the present invention has the following beneficial effects:

[0058] 1. This invention constructs a new dataset for the application scenarios of electronic police systems. This new dataset consists of blurred images, sharp images, and their gradient images. To obtain realistic and reliable blurred and sharp images, a novel image cleaning method and blurred image synthesis method are designed. Sharp dynamic video frames are obtained through gradient threshold cleaning and used as sharp images in the dataset. Based on obtaining the sharp video frames and performing frame interpolation, the interpolated sharp video frames are integrated in the spatial domain to obtain a series of corresponding blurred images. Then, gradient maps are extracted from the blurred images, ultimately constructing the new dataset.

[0059] 2. This invention designs a novel deblurring network—the EADN network—specifically for the characteristics of blurred images in dynamic scenes. Unlike previous deblurring networks that used a single image as input, resulting in limited image information and unsatisfactory restoration effects, this invention uses a new dataset as input, specifically adding a gradient map as a new input to guide the deblurring process and supplement edge information. To this end, this invention employs a parallel structure of a strong edge extraction network and a multi-level deblurring network. The strong edge extraction network guides the restoration process of the multi-level deblurring network and supplements missing details in the restored image, resulting in a clearer image.

[0060] 3. This invention features a specially designed multi-level deblurring network that combines a multi-scale network with a U-Net fusion structure. This approach combines the advantages of both multi-scale networks and U-Net. By designing the network structure for each scale, the network parameters are reduced while information from different scales of the image is extracted. This information is then fused through a cross-scale adaptive fusion module, enhancing the network's ability to learn nonlinear features.

[0061] 4. This invention designs a Hybrid-activated Discrete Cosine Transform Adaptive Module (HDAM), which, through a dual-branch structure, acquires feature information at different scales and uses both ReLU and ELU activation functions to enhance the network's ability to learn nonlinear features. Furthermore, DCA and DSA modules are designed after the dual-branch structure. Unlike traditional attention mechanisms, this module utilizes discrete cosine transform to capture important reconstruction information in the channel and spatial dimensions in the frequency domain, generating a more accurate attention map.

[0062] 5. This invention designs a Cross-stage Adaptive Fusion Module (CAFM) to replace the traditional simple fusion on the channel. By fusing information from different streams, including edge information, information from low scale and information from the scale of the module, and adaptively integrating them, effective information is preserved, data redundancy is reduced, network convergence is accelerated, and a clear image is effectively reconstructed.

[0063] 6. This invention specifically designs an FFT-content loss function for the network structure, transforms the generated restored image and clear image to the frequency domain, and then optimizes the network performance by measuring the difference in content between the restored image and the clear image. Attached Figure Description

[0064] Figure 1 Diagram of the EADN network structure;

[0065] Figure 2 This is a diagram of the upsampling fusion connection structure;

[0066] Figure 3(a) shows the HDAM module structure;

[0067] Figure 3(b) is a DCA structure diagram;

[0068] Figure 3(c) is a diagram of the DSA structure;

[0069] Figure 4 This is a diagram of the CAFM structure.

[0070] Figure 5 This is a flowchart illustrating a specific implementation method of the present invention;

[0071] Figure 6 This is a network framework diagram. Detailed Implementation

[0072] The present invention will be further described below through embodiments and in conjunction with the accompanying drawings.

[0073] Example:

[0074] This embodiment describes the specific network structure and sub-module organization of a dynamic scene imaging sharpening method for intelligent electronic police, as shown in the attached figure. Figure 1-4 As shown, this invention includes acquiring the dataset, building the network framework, and outputting the processing results. Notably, this invention does not use common datasets but instead constructs its own. The dataset construction method includes acquiring and cleaning dynamic video frames and generating blurred images and their gradient maps. The steps for acquiring and cleaning dynamic video frames are as follows:

[0075] For various application scenarios of electronic traffic enforcement cameras, such as traffic light intersections, highway intersections, and areas with various crowds, a certain amount of dynamic scene video frames are collected. First, a certain amount of dynamic scene video frames are collected for these various application scenarios. Second, the Laplacian operator is used to clean these video frames, that is, to remove blurry or incomplete video frames and retain clear video frames.

[0076] This invention utilizes OpenCV to clean video frames. The specific method is as follows:

[0077] The input video frames are converted to grayscale, then the gradient map is extracted using the Laplacian operator, and its variance is calculated as the return value x. x is then compared with a pre-set threshold. The comparison is performed, where 'a' represents the average variance of the gradient map of the blurred image, and 'b' represents the average variance of the gradient map of the sharp image. This invention selects the same number of blurred and sharp images for variance calculation. If... The video frame is then marked as a clear image; otherwise, it is marked as a blurry image and placed in the blurry image category. This invention only selects clear images for subsequent processing. Since image size and scene affect the returned gradient value, this invention calculates a suitable threshold range using multiple publicly available dynamic scene blur datasets. The specific calculation formula is as follows:

[0078]

[0079] Where N represents the number of images, var indicates variance calculation, laplace indicates gradient extraction, and I... blur and I sharp These represent blurred and sharp images, respectively.

[0080] The specific steps for generating blurred images and their gradient maps, and building a dataset, are as follows:

[0081] 1. Group the images marked as clear according to their scene content, with images from the same scene forming a group. Based on the principle that the number of images in each group must not exceed a threshold T, a total of P groups of clear images are obtained.

[0082] 2. Let the i-th group of images be S. i ={F0,F1,…,F j ,…,F N-1}, 1≤N≤T, 0≤j≤N-1, where, F N-1 Let S' represent the Nth image in the i-th image group. Then, the i-th image group is processed using frame interpolation averaging to obtain a blurred image. The specific steps of frame interpolation averaging are as follows: 7 frames are inserted between two adjacent clear images using the RIFE algorithm (Real-Time Intermediate Flow Estimation), achieving 8-fold interpolation. Therefore, the i-th image group generated after frame interpolation can be represented as: S′ i ={F′0,F′1,…,F′ j×8 ,…,F′ (N-1)×8 The original clear image is F. j =F′ j×8 ,0≤j≤N-1. Therefore, when discarding the original image group S i Given the first and last three frames, the set of clear images in the i-th group can be obtained as: Q i ={q j |q j =F j+3 =F′ (j+3)×8}, 0≤j≤N-7, where q j This represents the j-th image in the i-th group of images.

[0083] 3. The corresponding set of blurred images can be represented as: B i ={b j |b j =G j+3}, 0≤j≤N-7, where b j That is, G j+3 Representing the j-th image in the i-th group of blurred images, its generation process can be described as taking the original sharp image F... j+3 After frame interpolation, the sharp image F′ corresponding to S′ is... (j+3)×8 The average of the sharp images of the region and its adjacent spatial domains is calculated by adding them together. The specific formula is as follows:

[0084]

[0085] Where L represents the length of the selected spatial domain, which is set to 16.

[0086] 4. Use the Sobel operator to extract gradient maps from the acquired blurred image set, obtaining the gradient map set E. i Therefore, the obtained dataset includes pairs of blurred and sharp images, as well as gradient maps of the blurred images, which can be represented as: D = {Q} i B i E i The dataset obtained in this invention includes two parts: a training set and a test set. The test set contains 1,000 pairs of blurred-sharp image pairs, and the training set contains 3,000 basic blurred-sharp image pairs. Then, the dataset undergoes image enhancement processing, including but not limited to rotation, flipping, translation, and scaling transformation. Therefore, the training set ultimately contains 15,000 pairs of blurred-sharp image pairs.

[0087] This invention selects a convolutional neural network as the network structure for completing the image restoration task. For example... Figure 1 As shown, this invention constructs an Edge And Deblurring Network (EADN) and designs a Hybrid-activated DCT Adaptive Module (HDAM) and a Cross-stage Adaptive Fusion Module (CAFM) to achieve image restoration.

[0088] The EADN network consists of two networks: a strong edge extraction network and a multi-level deblurring network. This invention specifically designs a parallel structure for the multi-level deblurring network and the strong edge extraction network. The strong edge extraction network uses a U-Net structure as its main structure, with HDAM groups added to both the encoder and decoder to obtain important information. Each HDAM group consists of multiple HDAM blocks. The strong edge extraction network can utilize the acquired edge information to guide the multi-level deblurring network to achieve the deblurring task through feature fusion connections and a CAFM module, and can also supplement the missing fine structural information in the blurred image.

[0089] Unlike strong edge extraction networks, the multi-level deblurring network employs a fusion structure of U-Net and multi-scale architecture. Each scale uses a U-Net structure, with the input image obtained by downsampling the input image at a higher scale by a factor of two, and multiple HDAM groups are added for feature extraction. Downsampling is a common operation in U-Net, expanding the receptive field and acquiring more comprehensive contextual information by reducing the image size. Downsampling has achieved significant results in image deblurring at large scales. However, downsampling at smaller scales severely reduces the amount of information that can be acquired. Therefore, this invention specifically designs a system that performs downsampling twice at the largest scale and no downsampling at the smallest scale, thereby reducing the overall computational cost of the network while enhancing its deblurring performance. Figure 2 As shown, in a multi-level deblurring network, information flow between different stages is achieved through upsampling fusion connections and cross-level fusion connections between adjacent scales, supplementing information from smaller scales to higher-level scales. The HDAM and CAFM modules will be introduced in detail below.

[0090] As shown in Figure 3(a), the HDAM module employs a dual-branch structure to extract image features. The input image x is fed into a 3×3 convolutional layer and a 5×5 convolutional layer, respectively, with a normalization layer added after each convolutional layer. The normalized results are then activated using ReLU and ELU to enhance the network's ability to learn nonlinear features. The specific calculation process is shown in Equation (3):

[0091]

[0092] Here, concat represents fusion along the channel dimension, δ and ε represent ReLU and ELU activation functions respectively, and f3 and f5 represent 3×3 and 5×5 convolutional layers respectively.

[0093] The information is then sequentially input into the DCA (DCT Channel Attentional Block) and DSA (DCT Spatial Attentional Block) of the HDAM module to focus on important features in both the channel and spatial dimensions. This invention specifically replaces the pooling layers in traditional channel and spatial attention with Discrete Cosine Transform (DCT). This modification changes the implementation of the attention mechanism from the time domain to the frequency domain, thereby obtaining richer and more effective image information.

[0094] As shown in Figure 3(b), in the DCA module, the input X∈R is first... C×H×W Divide the channel into blocks, i.e., [X] 0 ,X 1 ,…,X p ,…,X q-1 ],X p ∈R C′×H×W …,p∈{0,1,…,q-1}, Where C, H, and W represent the number of channels, height, and width of the input X, respectively. Then, for X... p The formula for performing two-dimensional DCT on height and width is as follows:

[0095]

[0096] Where: : represents no calculation in the channel dimension, N is the number of points in the original signal, which can be H and W in equation (4), and k is an integer. Freq p Let C' represent the compressed C′-dimensional vector. Therefore, the entire compressed vector can be represented as:

[0097] Freq = concat(Freq) 0 ,Freq 1 ,…,Freq q-1 (5)

[0098] Here, concat represents fusion at the channel level.

[0099] Therefore, the entire DCA calculation process can be represented as:

[0100] U C =X·s=X·σ(F dense2 (δ(F dense1 (Freq))) (6)

[0101] Where σ represents the Sigmoid activation function, F dense1 and Fdense2 These represent the first and second fully connected layers, respectively.

[0102] As shown in Figure 3(c), in the DSA module, the output U from the DCA module... C The formula for performing a one-dimensional DCT along the channel dimension is:

[0103]

[0104] Therefore, the entire DSA calculation process can be represented as:

[0105] U S =U C ·S=U C ·σ(f7(M)) (8)

[0106] Where f7 represents a 7×7 convolutional layer, U S This is the output of the DSA module.

[0107] The specific structure of the CAFM module is shown in the attached figure. Figure 4 As shown, it will come from edge information Y of the same size. edge Characteristics of stage n, Y n and the characteristics of stage n-1 Y n-1 Adaptive fusion is used to enhance the network's ability to learn effective features and improve deblurring performance. The entire computation process of the CAFM module can be represented as:

[0108]

[0109] in, This represents the LeakyReLU activation function. Specifically, since no edge information is introduced into the intermediate-scale network layers, the CAFM module in the intermediate-scale network layers is changed to a two-input block, i.e., the edge information input is removed, while the rest remains unchanged.

[0110] The EADN network downsamples a 1280×720 blurred image and its gradient map by a factor of two, then randomly crops it into 256×256 image blocks. The number of channels is set to 3, and these blocks are used as input to a multi-level deblurring network and a strong edge extraction network. For the strong edge extraction network, an encoder-decoder structure is used. Specifically, the 256×256×3 image blocks are processed through a 3×3 convolutional layer to extract features, resulting in 64 feature maps. These feature maps are then normalized and activated. Next, the feature maps are input to an HDAM group consisting of three HDAM blocks for feature extraction, focusing on effective features and extracting clean image information. Then, the images pass through two identical 3×3 convolutional layers with a stride of 2 and the HDAM group to complete downsampling and feature extraction. The resulting feature maps undergo two pixel shuffling operations and two HDAM groups for upsampling, ultimately outputting a strong edge image.

[0111] For the multi-level deblurring network, the network consists of a fusion of U-Net and a multi-scale network. The specific steps are as follows: A 256×256×3 image is downsampled twice to obtain 128×128×3 and 64×64×3 images. These three images serve as inputs to the network at three scales, with the smallest image used as the input for the first scale and the largest as the input for the third scale. For the first scale, the 64×64×3 image undergoes feature extraction through a convolutional layer with a stride of 1 and a kernel size of 3×3, resulting in a 64-feature map. This feature map is then processed by an HDAM group to extract further features, ultimately yielding a preliminary restored image for this scale. For the second scale, the 128×128×3 image is first convolved to obtain a 64-feature map. Then, a cross-scale adaptive fusion block is used to fuse the preliminary restored image from the first scale. This is followed by feature extraction through an HDAM group. Next, downsampling is achieved through a 3×3 convolution with a stride of 2, and a cross-scale adaptive fusion block is used to fuse features from the first scale. Finally, this is processed by an HDAM group for feature extraction. Then, after pixel shuffling and HDAM group upsampling, the preliminary restored image at the second scale is obtained. For the third scale, a 64 feature map is obtained first, then the preliminary restored image at the second scale and edge features of the same size are fused across orders, followed by feature extraction through an HDAM group. Next, downsampling, multi-stream feature fusion, and important information extraction are completed through two 3×3 convolutions with a stride of 2, two cross-order adaptive fusion blocks, and two HDAM groups. Then, image reconstruction is achieved through two more pixel shuffling operations, one cross-order adaptive fusion block, and two HDAM groups. Finally, strong edge images are fused, and the fused feature map is subjected to two convolution operations to obtain the final restored image.

[0112] This invention selects a convolutional neural network jointly optimized by perceptual loss, edge loss, and FFT-content loss. The formula for calculating the joint loss function is as follows:

[0113] L=αL perceptual +βL edge +γL fftc (10)

[0114] Where α, β, and γ are the weights of the perceptual loss, edge loss, and FFT-content loss, respectively, and their values ​​are set to α = 5, β = 12, and γ = 5. The VGG19 network is widely used for image feature extraction. The features used in this invention to calculate the loss function can all be extracted from the pre-trained VGG19 network. The distribution difference between the generated data and the original data is calculated through the VGG19 network.

[0115] Perceptual loss measures the overall difference between the features of the generated image and the corresponding real, sharp image. The inputs are the real, sharp image and the restored image. The perceptual loss uses the VGG19 feature layer to calculate the difference between the generated data and the original real data. The loss function formula is:

[0116]

[0117] Among them, I B and I S These are a blurry image and a clear image, respectively. W represents the feature map of the e-th convolution before the d-th max-pooling layer in the VGG19 network. d,e and H d,e G(I) represents the width and height of the feature map obtained from the VGG19 network, respectively. B ) x,y This represents the pixel value at point (x, y) of the deblurred image output by the network.

[0118] The Sobel operator is selected to establish an edge loss function, which measures the difference in edge features between the initially restored image and the clear image. The loss function takes the real image as input and the deblurred image as output. The loss function is calculated by plotting the image gradient, thus optimizing the edge feature extraction module to help extract more edge details and improve module performance. The loss function formula is as follows:

[0119]

[0120] in, This represents the Sobel operator, where W and H represent the width and height of the input image, respectively.

[0121] Since the network learns features in the frequency domain, this invention specifically designs an FFT-content loss function. Unlike traditional content loss, this invention first performs a Fourier transform on the input image, and then calculates the loss function, thereby optimizing network performance in the frequency domain. The loss function formula is:

[0122]

[0123] Here, fft represents the Fourier transform.

[0124] The flowchart of the specific implementation method of this invention is as follows: Figure 5 As shown, the network framework diagram of the present invention is as follows: Figure 6 As shown, the EADN algorithm deblurs the blurred image and outputs a clear image.

[0125] Training and testing process of the network: In this invention, the number of HDAM blocks in each HDAM group is set to 3 in the strong edge extraction network and the multi-level deblurring network. Furthermore, during training, the input image is downsampled by a factor of 2 and then randomly cropped into 256×256 image blocks for training. This invention uses the Adam optimizer and selects the mini-batch stochastic gradient descent method. The initial learning rate of the network is set to 2×10⁻⁶. -4 Set the batch size to 1 and the epoch to 4×10. 5 Using a cosine annealing strategy, the initial learning rate was stably reduced to 1×10. -6 Experiments showed that the model obtained in this way performed better on the test set, without a significant difference in experimental validity.

[0126] To verify the effectiveness of the present invention, SSIM (structural similarity) and PSNR (peak signal-to-noise ratio) were selected to calculate the difference between the restored image and the real clear image. These two values ​​were used as evaluation indicators to verify the effectiveness of the model.

Claims

1. A method for enhancing the clarity of dynamic scene imaging in intelligent electronic police systems, characterized in that, Includes the following steps: S1: Acquire dynamic scene video frames of electronic police and clean them using the Laplacian operator to obtain clear video frames, i.e., clear images; S2: Obtain a blurred image from a clear image using the frame interpolation averaging method; The dataset is constructed by combining gradient maps of blurred, sharp, and blurred images. The dataset includes a training set and a test set. S3: Construct the Edge Deblurring Network EADN, which includes a strong edge extraction network and a multi-level deblurring network; The EADN constructs a structure in parallel of a multi-level deblurring network and a strong edge extraction network; The strong edge extraction network adopts the encoder-decoder network U-Net structure as the main structure. Both the encoder and decoder parts are equipped with a Hybrid Activated Discrete Cosine Transform Adaptive Module (HDAM) group, which consists of multiple HDAM blocks. The strong edge extraction network utilizes the edge information of the gradient map through the feature fusion connection and the cross-order adaptive fusion module CAFM module to guide the multi-level deblurring network to achieve the deblurring task and supplement the missing fine structural information in the blurred image. The multi-level deblurring network adopts a fusion structure of encoder-decoder network U-Net and multi-scale structure. Each scale adopts the U-Net structure. Its input image is obtained by downsampling the input image at a higher scale by a factor of two, and multiple HDAM groups are added for feature extraction. Downsampling is performed twice at the largest scale, while no downsampling operation is performed at the smallest scale. In a multi-level deblurring network, information flow between different stages is achieved through upsampling fusion connections and cross-order fusion connections between two adjacent scales. The HDAM structure is a dual-branch structure for extracting image features. The input image x is fed into two convolutional layers, and a normalization layer is added after the convolutional layers. The normalized results are activated using the ReLU activation function and the ELU activation function, respectively. Then, they are sequentially fed into the Discrete Cosine Channel Attention Block (DCA) and the Discrete Cosine Spatial Attention Block (DSA) in the HDAM. S4: Set the loss function and calculate the loss to train EADN. The loss function includes perceptual loss, edge loss and frequency domain content FFT-content loss. S5: Test the trained EADN to verify the network performance. Its input is the images in the test set, and the output is the clear image after blurring.

2. The method for enhancing dynamic scene imaging of intelligent electronic police according to claim 1, characterized in that, The specific cleaning process described in S1 is as follows: The input video frame is converted to grayscale, and then the gradient map is extracted using the Laplacian operator and its variance is calculated as the return value x. Compare x with a pre-set threshold The comparison is performed, where a is the average variance of the gradient map of the blurred image in the existing public dataset, and b is the average variance of the gradient map of the sharp image in the existing public dataset. The same number of blurred and sharp images are selected for the average variance calculation. like The video frame will be marked as a clear image if it is not clear, otherwise it will be marked as a blurry image.

3. The method for enhancing dynamic scene imaging of intelligent electronic police according to claim 2, characterized in that, The specific process for establishing the dataset described in S2 is as follows: S2.

1. Group the clear images according to the scene content, with the same scene as one group. Based on the principle that the number of images in each group should not exceed the set threshold T, a total of P groups of clear images are obtained. S2.2, Let the i-th group of images be S i ={F0,F1,…,F j ,…,F N-1 }, 1≤N≤T, 0≤j≤N-1, where, F N-1 Let S' represent the Nth image in the i-th image group. Then, the i-th image group is processed using the frame interpolation averaging method: seven frames are inserted between two adjacent clear images using the Real-Time Intermediate Stream Estimation (RIFE) algorithm, achieving eight-fold interpolation. The i-th image group generated after interpolation is represented as: S′ i ={F′0,F′1,…,F′ j×8 ,…,F′ (N-1)×8 } Among them, the clear image is F j =F′ j×8 ,0≤j≤N-1, when discarding S i Given the first and last three frames, the set of clear images in the i-th group is: Q i ={q j |q j =F j+3 =F′ (j+3)×8 }, 0≤j≤N-7, where q j This represents the j-th clear image in the i-th image group; S2.3, The set of fuzzy images is represented as: B i ={b j |b j =G j+3 }, 0≤j≤N-7, where b j That is, G j+3 Representing the j-th image in the i-th group of blurred images, its generation process is described as... i The corresponding clear image F′ in ′ (j+3)×8 The clear images of the region and its adjacent spatial domains are summed and then averaged. S2.

4. Use the Sobel operator to extract gradient maps from the acquired set of blurred images, obtaining the gradient map set E. i The dataset consists of pairs of blurred and sharp images, as well as gradient maps of the blurred images, represented as: D = {Q} i B i E i The dataset is divided into training and testing sets.

4. The method for enhancing dynamic scene imaging of intelligent electronic police according to claim 3, characterized in that, In the DCA, the input X∈R is first... C×H×W Divide the channel into blocks, i.e., [X] 0 ,X 1 ,…,X p ,…,X q-1 ],X p ∈R C′×H×W p∈{0,1,…,q-1}, Among them, X p Let X be the p-th image, where C, H, and W represent the number of channels, height, and width of the input X, respectively. p Performing a two-dimensional discrete cosine transform (DCT) on the height and width yields Freq. p , representing the compressed C′-dimensional vector, the entire compressed vector is represented as: Freq=concat(Freq 0 ,Freq 1 ,…,Freq q-1 ) Here, concat represents fusion at the channel level; The entire DCA calculation process is represented as follows: U C =X·s=X·σ(F dense2 (δ(F dense1 (Freq))) Where σ represents the Sigmoid activation function, δ is the ReLU activation function, and F dense1 and F dense2 These represent the first and second fully connected layers, respectively. In the DSA module, the output U from the DCA module C Performing a one-dimensional DCT along the channel dimension yields M. The entire DSA calculation process can be represented as follows: THE S =U C ·S=U C ·σ(f7(M)) Where f7 represents a 7×7 convolutional layer, U S This is the output of the DSA module.

5. The method for enhancing dynamic scene imaging of intelligent electronic police according to claim 4, characterized in that, The CAFM module will receive edge information Y from the same size. edge Characteristics of stage n, Y n and the characteristics of stage n-1 Y n-1 Adaptively integrated, the entire calculation process of the CAFM module can be represented as follows: f Fusion =G×Y n +(1-G)×Y n-1 in, f1 represents the LeakyReLU activation function, and f3 represents a 1×1 convolutional layer and a 3×3 convolutional layer, respectively.

6. A method for enhancing dynamic scene imaging of intelligent electronic police according to any one of claims 1 to 5, characterized in that, The loss function described in S4 is composed of perceptual loss, edge loss, and frequency domain content FFT-content loss, and its calculation formula is as follows: L=αL perceptual +βL edge +γL fftc Where α, β, and γ are the weights of the perceptual loss, edge loss, and FFT-content loss, respectively; The perceptual loss is calculated using the VGG19 feature layer to determine the difference between generated and real data. The loss function formula is as follows: Among them, I B and I S These are a blurry image and a clear image, respectively. W represents the feature map of the e-th convolution before the d-th max-pooling layer in the VGG19 network. d,e and H d,e G(I) represents the width and height of the feature map obtained from the VGG19 network, respectively. B ) x,y This represents the pixel value at point (x, y) of the deblurred image output by the network. The edge loss function is established using the Sobel operator. The input is the real image and the output is the deblurred image. The loss function is calculated by calculating the image gradient. The formula for the loss function is as follows: in, Represents the Sobel operator; The FFT-content loss function involves performing a Fourier transform on the input image and then calculating the loss function. The calculation formula is as follows: Here, fft represents the Fourier transform.

Images