Automobile data recorder image resolution improving method based on computer vision

Abstract

The invention relates to the technical field of automobile data recorder optimization, and discloses an automobile data recorder picture resolution improving method based on computer vision. The method comprises the following steps: obtaining an original video frame sequence shot by an automobile data recorder, and carrying out denoising and calibration processing on the original video frame sequence to obtain a preprocessed video frame sequence; performing feature extraction on the preprocessed video frame sequence to obtain key feature data; constructing a resolution improvement hierarchical model comprising an initial model and an optimization model based on the key feature data; and determining constraint conditions of the two models, inputting the key feature data into the models, carrying out iterative updating solution according to the constraint conditions, and outputting a high-resolution video frame sequence. According to the method, hardware upgrading is not needed, through systematic algorithm processing, time sequence association and detail features of a video frame sequence are fully utilized, the image resolution can be stably improved, key information of a driving scene can be clearly presented, the practical value of the automobile data recorder is improved, and the method is suitable for various driving scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of driving recorders, and particularly to a driving recorder picture resolution improvement method based on computer vision. BACKGROUND

[0002] In the context of increasing demand for vehicle intelligence and traffic safety, driving recorders have become a standard equipment for vehicles. The video recorded by driving recorders plays a key role in traffic accident liability identification, road anomaly event tracing, and vehicle surrounding environment monitoring. However, due to multiple factors such as vehicle installation space, equipment cost control, and complex driving environment, most driving recorders on the market have obvious limitations in hardware configuration, especially the performance of image sensors, which is difficult to compare with professional camera equipment, resulting in low resolution of the recorded original video frame sequence.

[0003] In daily driving scenarios, low-resolution video pictures often cause a series of practical problems. For example, in night or low-light environments, the pictures taken by driving recorders are prone to be blurred, and the license plate numbers of vehicles in front, the appearance features of pedestrians on the roadside, and the specific content of traffic signs cannot be clearly presented. When the vehicle is driving at high speed, the recorded video frames may also produce trailing, edge blurring, and other phenomena due to insufficient dynamic capture capability of the equipment, further exacerbating the loss of picture information. These problems make it difficult for driving recorders to provide effective visual evidence in critical moments, reducing their practical value in traffic safety management and vehicle security fields.

[0004] To improve the picture quality of driving recorders, the industry has attempted to solve the problem by upgrading hardware configuration, such as using higher-pixel image sensors and optimizing lens optical structures. However, such solutions significantly increase the production cost of the equipment, and are constrained by vehicle power supply capacity and installation space. Hardware upgrades often face bottlenecks and are difficult to be widely applied in various vehicle models. In addition, some technical solutions use simple image interpolation algorithms to enlarge low-resolution pictures, which can improve picture size to some extent, but cannot truly restore lost detail information, and may even cause new problems such as picture jaggedness and increased noise, making it difficult to meet the requirements of actual applications for picture clarity.

[0005] With the rapid development of computer vision and deep learning technology, the image resolution improvement scheme based on algorithm optimization has gradually become a research hotspot. Such scheme does not need to rely on hardware upgrade, and can realize the repair and enhancement of picture details through software algorithm processing on the original video frame. However, the application of the related technology in the scene of the driving recorder still has some problems: most of the existing algorithms only process single frame images, and do not fully consider the time sequence correlation of the video frame sequence of the driving recorder, resulting in problems such as frame jitter and poor coherence of the processed video picture; at the same time, the traditional resolution improvement model often uses a single network structure, which is difficult to balance picture detail recovery and processing efficiency, and the model generalization ability is insufficient when facing complex and changeable light and motion state in the driving scene, so it cannot stably output high-quality high-resolution video frame sequence. Therefore, a technical scheme is needed to efficiently and stably improve the resolution of the video picture in the scene of the driving recorder. SUMMARY

[0006] The purpose of the present application is to provide a driving recorder picture resolution improvement method based on computer vision to solve the problems raised in the background art.

[0007] To achieve the above purpose, the present application provides a driving recorder picture resolution improvement method based on computer vision, which comprises: obtaining an original video frame sequence shot by a driving recorder, and performing denoising and calibration processing on the original video frame sequence to obtain a preprocessed video frame sequence; performing feature extraction processing on the preprocessed video frame sequence to obtain key feature data; constructing a resolution improvement hierarchical model based on the key feature data, wherein the resolution improvement hierarchical model comprises an initial model and an optimization model; determining the constraint conditions of the initial model and the optimization model, inputting the key feature data into the initial model and the optimization model, and iteratively updating and solving according to the constraint conditions to obtain a high-resolution video frame sequence.

[0008] Preferably, the step of performing denoising and calibration processing on the original video frame sequence to obtain a preprocessed video frame sequence comprises: performing brightness equalization processing on the original video frame sequence to obtain equalization data; introducing a motion estimation method and calculating the interframe difference degree; determining the difference degree between different frames and determining the change amplitude of the difference degree until the change amplitude of the difference degree tends to be stable, outputting the frame number corresponding to the difference degree and taking it as the key frame number; selecting a corresponding frame as a reference frame in the equalized data based on the number of key frames, determining a similarity distance of each of the equalized data to the reference frame, assigning each of the equalized data to a reference frame with the minimum similarity distance, and repeating the grouping process until the change of the reference frame is less than a preset threshold, to obtain the preprocessed video frame sequence.

[0009] Preferably, the step of performing feature extraction processing on the preprocessed video frame sequence to obtain key feature data comprises: performing state estimation on the preprocessed video frame sequence as an input of a feature extractor and establishing a dash cam video model based on feature extraction; determining a first time update equation of the dash cam video model based on feature extraction; determining a first state update equation of the dash cam video model based on feature extraction; predicting a first current value from the preprocessed video frame at the previous time, substituting the first current value into the first state update equation for correction to obtain a first current correction value, and substituting the first current correction value into the first time update equation to predict the preprocessed video frame at the next time to obtain high-frequency feature data and low-frequency feature data.

[0010] Preferably, the step of constructing a resolution promotion hierarchical model based on the key feature data comprises: selecting a convolutional neural network and a generative adversarial network to form a hybrid resolution promotion system, taking the low-frequency feature data as a basic resolution reference value, and distributing the high-frequency feature data in the hybrid resolution promotion system to obtain distribution data; constructing a resolution promotion hierarchical model based on the distribution data, the resolution promotion hierarchical model comprising an initial model and an optimized model.

[0011] Preferably, the step of distributing the high-frequency feature data in the hybrid resolution promotion system to obtain distribution data comprises: taking the high-frequency feature data as the input power of the hybrid resolution promotion system, the high-frequency feature data being the difference between the output feature of the preprocessed video frame sequence after being input into the hybrid resolution promotion system and the basic resolution reference value, performing state estimation on the input power of the hybrid resolution promotion system as a prediction measurement of the feature extractor, and establishing a hybrid resolution promotion system power distribution model based on feature extraction; determining a second time update equation of the hybrid resolution promotion system power distribution model based on feature extraction; determining a second state update equation of the hybrid resolution promotion system power distribution model based on feature extraction; predicting a second current value according to the high-frequency feature data of the previous moment, substituting the second current value into the second state update equation to correct, to obtain a second current correction value, and substituting the second current correction value into the second time update equation to predict the high-frequency feature data of the next moment; calculating a filter coefficient; determining a distribution principle based on the filter coefficient, and distributing the predicted high-frequency feature data of the next moment according to the distribution principle to obtain distributed data.

[0012] Preferably, the initial model minimizes the blur degree of the video frame. The optimization model maximizes the clarity benefit of the video frame.

[0013] Preferably, the constraint condition of the initial model includes: video frame brightness constraint; video frame contrast constraint; video frame noise constraint; The constraint condition of the optimization model includes: system resolution balance constraint; video frame output constraint.

[0014] Preferably, the step of inputting the key feature data into the initial model and the optimization model and iteratively updating and solving according to the constraint condition to obtain the high-resolution video frame sequence includes: inputting the key feature data into the initial model to analyze the blur degree to obtain a preliminary resolution improvement result; inputting the preliminary resolution improvement result into the optimization model and modifying the optimization model to obtain an optimization modification model; The optimization modification model calculates the detail enhancement demand of the hybrid resolution improvement system using actual clarity data, inputs the data corresponding to the preliminary resolution improvement result into a convolutional neural network and a generative adversarial network system respectively, and iteratively solves under the constraint condition to obtain a high-resolution video frame sequence.

[0015] Preferably, the method further includes: selecting a matching video segment matching the complexity of the current video scene from historical video data; analyzing the change of the feature data of the matching video segment in time sequence to determine the proportional gain action point of the matching video segment; taking the time difference from the starting point to the proportional gain action point of each matching video segment as the time delay of each matching video segment; Determine the gain matching degree of each matched video segment by comparing the change of the proportional gain data sequence before the proportional gain action point with the change of the feature data sequence after the proportional gain action point; Determine the proportional gain weight of the matched video segment by combining the time delay amount, the gain matching degree, and the difference between the complexity of the matched video segment and the current video scene; Determine the proportional gain coefficient of the current video scene by weighting the proportional gain coefficients of all the matched video segments corresponding to the current video scene through the proportional gain weight; and input the proportional gain coefficient and the feature data into a resolution enhancement controller to control the resolution enhancement intensity by the resolution enhancement controller.

[0016] Preferably, the method further comprises: Obtain the difference between the video frame to be enhanced and the current video frame, set a preliminary adjustment method, and calculate a preliminary enhancement angle; Obtain a preset enhancement path and a video model, set a simulation algorithm, optimize the preliminary enhancement angle, and generate an actual enhancement angle for actual enhancement; Drive the resolution enhancement system to adjust to the actual enhancement angle, perform the enhancement operation on the video frame to be enhanced, set a blind area compensation method after the enhancement is completed, calculate a blind area compensation angle, judge the details of the blind area, determine an enhancement posture, and perform compensation enhancement; Record the enhancement data of the video frame to be enhanced, set an adjustment optimization method, optimize the adjustment coefficient used for calculating the enhancement angle, and apply the optimized adjustment coefficient to the calculation of the next enhancement angle.

[0017] Compared with the prior art, the present application has the following advantages: In the preprocessing link, the original video frame sequence is subjected to denoising and calibration processing, so that random noise points, salt and pepper noise points and other interference information in the picture can be accurately eliminated, and picture distortion caused by factors such as device installation angle deviation and lens optical distortion can be corrected, so that the preprocessed video frame sequence has a more stable quality basis. Compared with the simple image filtering denoising method in the traditional scheme, the preprocessing process of the present method is more targeted, and the denoising parameters and calibration strategies can be adaptively adjusted according to the characteristics of the picture recorded by the driving recorder, so that the effective details of the picture can be retained while the influence of interference factors on subsequent processing is minimized, thereby laying a reliable data foundation for subsequent feature extraction and model construction.

[0018] In the feature extraction link, key feature data is extracted from the preprocessed video frame sequence through targeted processing means, which can accurately capture the edge information, texture details and inter-frame timing correlation features in the picture. Unlike traditional algorithms that only focus on local features of a single frame image, the key feature data extracted by this method not only contains static detail features within a single frame picture, but also covers dynamic correlation features between adjacent video frames, so that the subsequent model can fully utilize the timing information of the video sequence and avoid problems such as frame discontinuity and jitter after processing, ensuring the smoothness and stability of the high-resolution video frame sequence.

[0019] By constructing a resolution enhancement hierarchical model, the synergistic effect of the initial model and the optimization model is achieved, which realizes efficient utilization and accurate processing of key feature data. The initial model can quickly analyze and process the key feature data, and output video frame sequences with basic resolution enhancement effect; the optimization model further optimizes the details, edge blur and other problems in the initial processing results, and realizes deep recovery and enhancement of picture details through multi-level feature learning and information fusion. This hierarchical model structure can balance processing efficiency and processing effect, avoiding the disadvantages of single model in balancing speed and quality in complex scenes, so that the model can still output high-quality pictures stably when facing changing light conditions and vehicle motion states in driving scenes.

[0020] In the model solving process, by determining the constraint conditions of the initial model and the optimization model and iteratively updating the solution, the model parameters can be continuously optimized to improve the fitting ability of the model to key feature data. The setting of constraint conditions can guide the model to follow the visual rules of driving scenes in the processing process, avoid false details that do not conform to the actual scene, and ensure that the output high-resolution video frame sequence has clear detail information and conforms to the visual perception habits of the human eye. The iterative updating solution enables the model to adaptively adjust the processing strategy according to the characteristics of different video frame sequences, enhancing the generalization ability of the model. Whether it is a complex scene such as night low light or high-speed driving, or a regular urban road driving scene, the model can achieve stable resolution enhancement effect. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The working principle diagram of the driving recorder picture resolution enhancement method based on computer vision described in the present application; Figure 2 The flowchart for denoising and calibration processing of the original video frame sequence; Figure 3 The flowchart for feature extraction of the preprocessed video frame sequence. DETAILED DESCRIPTION

[0022] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0023] With reference to Figure 1 The present application provides a computer vision-based resolution improvement method for a driving recorder, which comprises the following steps: acquiring an original video frame sequence captured by the driving recorder; performing denoising and calibration processing on the original video frame sequence to obtain a preprocessed video frame sequence; performing feature extraction processing on the preprocessed video frame sequence to obtain key feature data; constructing a resolution improvement hierarchical model comprising an initial model and an optimization model based on the key feature data; determining constraint conditions of the initial model and the optimization model and inputting the key feature data into the initial model and the optimization model to iteratively update and solve according to the constraint conditions; and finally obtaining a high-resolution video frame sequence. The method combines hierarchical modeling and multi-stage optimization, effectively maintains the authenticity and time sequence consistency of the video content while improving the image resolution.

[0024] Embodiment 1: With reference to Figure 2 When performing denoising and calibration processing on the original video frame sequence, first, brightness equalization processing is implemented to eliminate the influence of changes in lighting conditions. This processing uses an adaptive histogram equalization algorithm to adjust the brightness distribution of pixels frame by frame, enhances the contrast of the image by calculating the gray value distribution of the local region and remapping the pixel value, and at the same time limits excessive enhancement to avoid noise amplification. After this step, an equalized data sequence with consistent brightness features is output. The introduction of a motion estimation method to calculate the inter-frame difference degree is a key step in subsequent processing. A motion estimation model based on the optical flow method is used to calculate the pixel displacement vector field between adjacent video frames. The degree of inter-frame change is quantified by analyzing the motion trajectory of corresponding pixel points between consecutive frames. This model takes into account the comprehensive influence of global motion vectors and local motion vectors, and finally generates a difference degree matrix containing motion amplitude and direction information. To determine the difference degree between different frames and monitor its change amplitude, a time sequence analysis mechanism needs to be established. The first and second difference values of the difference degree sequence are calculated by a sliding window algorithm. When the difference values in consecutive windows remain within a set threshold range, it is determined that the change amplitude tends to be stable. At this time, the number of video frames corresponding to the stable interval is recorded as the key frame number. This value dynamically reflects the severity of the video content change.

[0025] The reference frames are selected from the equalized data based on the number of key frames using a hierarchical sampling strategy. First, the video sequence is divided into several paragraphs in chronological order, then the initial reference frames are randomly selected in each paragraph according to the content complexity, and the complexity is evaluated by calculating the intra-frame edge density and texture feature variance to ensure that the selected reference frames can represent the visual features of the paragraph. The similarity distance of each equalized data to the reference frame is determined using a multi-feature fusion measurement method, which combines color histogram correlation, structural similarity index and motion feature consistency, etc. multi-dimensional indicators to construct a comprehensive distance function to calculate the visual similarity between each frame and each reference frame. The distance function is combined by weighting each feature component to adapt to the comparison requirements of different scene types. Each equalized data is assigned to the reference frame with the smallest similarity distance to implement the dynamic clustering process, and the improved k-means clustering algorithm is used for frame grouping. The algorithm recalculates the center position of the reference frame in each iteration and updates the ownership relationship of each frame. By introducing a momentum factor, the convergence speed is accelerated and the local optimal solution is avoided.

[0026] The grouping process is repeated until the change of the reference frame is less than the preset threshold. The convergence judgment condition needs to be set. The sum of the Euclidean distances of the reference frame features in this iteration and the last iteration is calculated. When this value is less than the set threshold for three consecutive iterations, the clustering process is terminated. Finally, the preprocessed video frame sequence with stable grouping is output. These grouped frame sequences provide structured input data for subsequent feature extraction. The brightness equalization is the basic step to ensure the uniformity of the input data. The motion estimation and difference analysis provide the quantitative basis for the timing changes. The determination of the number of key frames realizes the adaptive sampling of the video content. The dynamic clustering grouping establishes the semantic association relationship between the frames.

[0027] Embodiment 2: see Figure 3When performing feature extraction processing on the pre-processed video frame sequence, the pre-processed video frame sequence is taken as an input of a feature extractor for state estimation, the feature extractor adopts an architecture combining a deep convolutional neural network and a long short-term memory network, the convolutional layer is responsible for extracting spatial features including edge, texture and shape local patterns, and the recurrent layer captures inter-frame temporal dependence and models motion features, through this hybrid network structure, a feature extraction-based dashboard video model is established, which can process spatial information and temporal evolution law at the same time. A first time update equation of the feature extraction-based dashboard video model is determined, which describes the transition rule of the hidden state in the time dimension, a linear dynamic system is used to model the temporal variation of the state variable, wherein the state transition matrix is learned through the training data, which can accurately reflect the evolution trend of the video features between consecutive frames, and the equation also contains a process noise term to simulate unmodeled dynamic characteristics. A first state update equation of the feature extraction-based dashboard video model is determined, which establishes the mapping relationship between the hidden state and the observation value, the high-dimensional hidden state is projected into the measurement space through the observation matrix, the equation considers the observation noise and distortion factors existing in the actual video acquisition process, an additive noise model is used to represent the observation uncertainty, and the robustness of the state estimation is ensured.

[0028] The first current value is predicted according to the pre-processed video frame at the previous moment, the prediction process uses the state transition matrix and an extrapolation algorithm to calculate the prior state estimation, the first current value is substituted into the first state update equation for correction, the correction process uses a Kalman filter algorithm to calculate the Kalman gain and optimize the posterior state estimation, and a first current correction value is obtained, which integrates the prior knowledge and the current observation information. The first current correction value is substituted into the first time update equation to predict the pre-processed video frame at the next moment, a complete feature prediction sequence is constructed through multiple iteration calculations, the state covariance matrix and the Kalman gain parameters are updated in the iteration process, and finally the high-frequency feature data and the low-frequency feature data are separated, the high-frequency component mainly contains detailed texture and edge information, and the low-frequency component carries the main structure and brightness information of the image.

[0029] A hybrid resolution enhancement system is selected by combining convolutional neural network and generative adversarial network. The convolutional neural network adopts a multi-scale architecture design, including feature extraction layer, up-sampling layer and reconstruction layer, which can gradually enhance the image details. The generative adversarial network includes generator and discriminator components. The generator is responsible for synthesizing high-resolution images, and the discriminator evaluates the authenticity of the generated images. Both of them improve the output quality through adversarial training. The low-frequency feature data is used as the basic resolution reference value, which provides the basic structure and global illumination information of the image. The high-frequency feature data is distributed in the hybrid resolution enhancement system. The distribution strategy is based on the feature saliency map, and more high-frequency resources are allocated to the areas with high visual importance. Through this adaptive allocation mechanism, the optimized allocation data is obtained. Based on the allocation data, a hierarchical resolution enhancement model is constructed. The model adopts a two-level architecture design. The initial model focuses on basic resolution reconstruction, and the image size is enlarged and the basic details are restored through the convolutional neural network. The optimization model is responsible for high-level detail enhancement, and the realistic texture details are synthesized by using the generative adversarial network. The two models work together to realize progressive resolution enhancement. In the feature extraction stage, the state space modeling is used to accurately separate the feature components of different frequencies. In the system construction stage, the differential processing path is designed according to the feature characteristics, and the hierarchical model formed can fully exert the advantages of different network architectures, providing an effective processing framework for subsequent resolution optimization.

[0030] Taking a specific urban intersection dashcam scene as an example, the dashcam shoots 720p resolution video at 30fps. The scene contains multi-directional traffic flow, traffic signal changes, pedestrians crossing zebra crossings and other complex dynamic elements. The preprocessed video frame sequence has been denoised and calibrated, presenting continuous frame images with balanced brightness and corrected jitter. These frames are used as input data for the feature extraction stage. The feature extractor uses the VGG-16 network architecture as the spatial feature extraction backbone, combined with a bidirectional LSTM network to handle the temporal relationship. The convolutional layer extracts feature maps of different abstraction levels from each frame, including edge responses, texture patterns and shape contours, and the recurrent layer analyzes the evolution law of the feature maps between consecutive frames to establish a feature extraction-based dashcam video dynamic model. When determining the first time update equation of the video model, a linear dynamic system is used to model the time sequence transfer relationship of the hidden state. The state variable contains a 128-dimensional feature vector representing the abstract features of each frame, and the state transition matrix is trained based on a large amount of driving video data, which can accurately describe the dynamic characteristics of feature changes in typical traffic scenes.

[0031] The first state update equation is established to map the hidden state to the observation space, and the observation matrix is designed as a learnable parameter matrix, whose numerical expression is optimized through training. The equation considers the observation noise factors such as motion blur and illumination change in the actual recording process of the driving recorder, and uses a Gaussian noise model to represent the observation uncertainty. The first current value at the t-th time is predicted according to the preprocessed video frame at the t-1-th time. The prediction process uses a state transition matrix and an extrapolation algorithm to calculate a prior state estimate, and the prediction value is substituted into the first state update equation for correction. The correction process uses a Kalman filter algorithm to calculate the optimal Kalman gain and update the posterior state estimate to obtain the first current corrected value. The first current corrected value is substituted into the first time update equation to predict the preprocessed video frame features at the t+1-th time. Through multi-step iteration calculation, a complete feature prediction sequence is constructed, and the state covariance matrix and Kalman gain parameters are continuously updated during the iteration process. Finally, high-frequency feature data and low-frequency feature data are separated. In this intersection scene, high-frequency data mainly includes traffic signal details, license plate text, pedestrian facial features, and other fine elements, while low-frequency data carries vehicle outlines, road structures, building overall shapes, and other basic information. A hybrid resolution enhancement system composed of a convolutional neural network and a generative adversarial network is selected. The convolutional neural network uses the ESPCN real-time super-resolution architecture, which includes three core modules: feature extraction, sub-pixel convolution, and reconstruction. The generator of the generative adversarial network uses the SRGAN architecture, which includes multiple residual blocks and up-sampling layers, and the discriminator uses a deep convolutional network to determine the authenticity of the image.

[0032] The low-frequency feature data is used as the basic resolution reference value, which provides the basic geometric structure and spatial layout information of the scene. The high-frequency feature data is distributed in the hybrid resolution enhancement system based on visual saliency detection results, and more high-frequency resources are allocated to important areas such as traffic signs and vehicle license plates. Based on the allocated data, a hierarchical resolution enhancement model is constructed. The model uses a dual-path processing architecture. The initial model uses a convolutional neural network path to achieve 4 times resolution enhancement, focusing on restoring the basic details of the image. The optimized model uses a generative adversarial network path to further refine the texture features, especially enhancing the color saturation of traffic signals and the clarity of license plate characters. In the feature extraction stage, the detailed information required for 1080p resolution is successfully separated, and the hybrid system reasonably allocates processing resources. The hierarchical model gradually improves the image quality. The final output video maintains real-time while significantly improving the visual recognition of key traffic elements, providing higher quality visual data for subsequent traffic scene analysis.

[0033] In the embodiment 3, the high-frequency feature data obtained after feature extraction of the pre-processed video frame sequence is regarded as the input power of the hybrid resolution enhancement system, and the power value is obtained by calculating the difference between the feature map output by the feature extraction network and the low-frequency basic reference value, reflecting the enhancement requirement strength of the detailed information in the video frame sequence. This quantization method converts the loss degree of visual details into the energy value that needs to be processed by the system. When the input power of the hybrid resolution enhancement system is used as the prediction measurement of the feature extractor for state estimation, a power allocation model based on feature extraction needs to be established, which converts the power allocation problem into an optimal estimation problem in the state space. The hidden state variable represents the demand degree of different processing units in the system for high-frequency features, and the observation variable corresponds to the total amount of actual allocatable high-frequency feature data. When determining the second time update equation of the power allocation model of the hybrid resolution enhancement system based on feature extraction, the equation describes the evolution rule of the power demand state in the time dimension. A linear Gaussian model is used to model the state transition process, and the state transition matrix is trained by analyzing historical video data, which can accurately capture the dynamic influence mode of video content changes on power demand.

[0034] When determining the second state update equation of the power allocation model of the hybrid resolution enhancement system based on feature extraction, the equation establishes the observation relationship between the power demand state and the actual allocatable power. Considering the constraints of system processing capacity and resource allocation efficiency, the hidden state is mapped to the measurable power output space through the observation matrix, while considering the noise interference in the observation process. When predicting the second current value according to the high-frequency feature data at the previous moment, the state extrapolation method is used to calculate the prior state estimation, and the second current value is substituted into the second state update equation for correction. The extended Kalman filtering algorithm is used to process the possible nonlinear characteristics of the system in the correction process, and the Kalman gain matrix is calculated to optimize the posterior state estimation, obtaining a more accurate second current correction value. When the second current correction value is substituted into the second time update equation to predict the high-frequency feature data at the next moment, the time sequence profile of power demand is constructed through multi-step prediction. The error covariance matrix and the uncertainty information of state estimation are updated during the prediction process, and these prediction results provide an important basis for subsequent resource allocation decisions.

[0035] The minimum mean square error criterion is used to calculate the filter coefficients, and the optimal filter parameters are solved by optimizing the objective function. The filter coefficient matrix reflects the evaluation results of the importance of different frequency components in the system. The diagonal elements represent the confidence weight of the corresponding frequency band, and the non-diagonal elements represent the coupling relationship between different frequency components. When determining the allocation principle based on the filter coefficients, a weighted allocation strategy is used to direct the high-frequency feature data to different processing paths. The allocation principle follows the importance priority and balanced consideration criteria. More enhancement resources are allocated to areas with high visual saliency, while maintaining the overall image consistency. According to the allocation principle, the predicted high-frequency feature data at the next time is allocated. The allocation process uses a dynamic resource scheduling algorithm that monitors the working state and resource utilization of each processing unit in real time. Through the feedback adjustment mechanism, the allocation effect is optimized, and the optimized allocation data is obtained.

[0036] The quantitative evaluation of the allocation effect uses the following formula:

[0037] Wherein: represents the allocation effect score, is the total number of processing units, represents the resource utilization rate of the unit, represents the processing priority coefficient of the unit, is the system resource adjustment factor. This formula simulates the marginal effect of resource allocation through an exponential decay function, which can effectively evaluate the overall effect of different allocation strategies. The entire power allocation process forms a closed-loop optimization system, accurately predicts future power demand through state estimation, optimizes the allocation strategy through filter coefficients, and ultimately realizes the reasonable allocation of high-frequency feature data among different processing units.

[0038] In the embodiment 4, the initial model is set to minimize the blurriness of the video frames, and the target is achieved by calculating the variance of the image gradient amplitude. In the implementation, Laplace convolution operation is performed on each frame of image, and the dispersion degree of the output value is counted. The higher the value, the more serious the image blurring degree. The optimization process aims to minimize the index. The optimization model aims to maximize the clarity gain of the video frames. The clarity gain is quantified by combining multiple quality evaluation indexes, including edge sharpness measurement, local contrast evaluation and texture richness analysis, and finally forms a comprehensive scoring system to guide the optimization direction. The constraint conditions of the initial model include three main aspects: video frame brightness constraint requires that the average brightness value of the processed image remains within ±15% of the original frame brightness, preventing overexposure or brightness distortion in the enhancement process; video frame contrast constraint limits the variance rate of the gray scale histogram within 10%, maintaining the dynamic range characteristics of the original image; video frame noise constraint uses high-frequency energy threshold control to limit the increase of high-frequency coefficients after DCT transformation to below 20% of the original energy, effectively suppressing noise amplification.

[0039] The constraint conditions of the optimization model include system resolution balance constraint, which ensures that the resolution improvement amplitude of different regions of the image is consistent, and the balance degree is quantified by calculating the standard deviation of the resolution ratio between regions, and the standard deviation is required to be less than a set threshold; the video frame output constraint limits the structural similarity index between the output frame and the input frame to be not less than 0.85, maintaining the structural consistency of the enhanced image and the original content. When the key feature data is input into the initial model for blurriness analysis, a blind deconvolution algorithm is used to estimate the point spread function and construct the corresponding inverse filter. The point spread function is estimated by frequency spectrum analysis to evaluate the blurring characteristics, and then a preliminary resolution improvement result is generated, which includes basic resolution improvement but lacks detail enhancement. When the preliminary resolution improvement result is input into the optimization model for correction, a clarity gain feedback mechanism is introduced to dynamically adjust the model parameters. The correction amount is calculated by analyzing the difference between the preliminary result and the ideal clarity target, and an optimization correction model is generated, which can adaptively adjust the processing intensity according to the actual clarity requirement.

[0040] The optimization correction model calculates the demand of hybrid resolution enhancement system for detail enhancement using actual definition data, and determines the proportion of computing resources needed by establishing the mapping relationship between definition index and processing resources. The proportion increases with the increase of definition requirement. The data corresponding to the preliminary resolution enhancement result is input into the convolutional neural network and the generative adversarial network system respectively. The convolutional neural network is responsible for the enhancement and reconstruction of local features, and adopts multi-scale feature fusion technology to restore details. The generative adversarial network focuses on the improvement of global consistency and visual realism, and generates more natural texture details through adversarial training. Taking the maximum running benefit in the whole life cycle as the objective function, the function comprehensively considers the balance of processing quality, computing efficiency and resource consumption, and iteratively solves under the condition of meeting all constraint conditions. The iteration process uses the optimization algorithm based on Lagrange multiplier to process multiple constraint conditions, and finds the optimal solution by constantly adjusting the multiplier value.

[0041] Taking a 1280x720 resolution video of a driving recorder as an example, which contains typical elements such as road scenes, vehicles and pedestrians. Referring to Table 1, the changes of key parameters in the iterative solution process are shown.

[0042] Table 1: Parameter changes in the resolution enhancement iteration process

[0043] After 20 iterations, all parameters meet the requirements of the constraint conditions, and finally a high-quality video frame sequence with a resolution of 1920x1080 is obtained. The sequence significantly improves the visual detail performance while maintaining the authenticity of the original content.

[0044] In embodiment 5, the matching video segments are selected from the historical video database, which have similar complexity to the current urban road scene. The complexity of the current scene is evaluated by calculating the weighted sum of edge pixel density and optical flow vector amplitude. The selected matching video segments have similar traffic flow and object movement patterns. The time series feature data of the matching video segments are analyzed using a sliding window change point detection algorithm. This algorithm monitors the statistical property of the feature sequence and marks the points of interest when the mean value of the feature in three consecutive windows changes more than two standard deviations. These points of interest usually correspond to the appearance or disappearance of important objects in the scene. The time delay is calculated as the time difference between the start point of each matching video segment and the point of interest. This delay reflects the time interval required for the system to respond to the scene change, which is usually associated with the time of sudden lane change or pedestrian crossing in the traffic scene. The dynamic time warping algorithm is used to compare the feature data sequence before the point of interest and the sequence after the point of interest. This algorithm aligns the two sequences non-linearly and calculates their cosine similarity. The similarity value is used as the gain matching degree of each matching video segment. The higher the matching degree, the closer the response characteristics of the historical scene and the current scene. The time delay, gain matching degree, and complexity difference between the matching video segment and the current video scene are combined to calculate the gain weight using the weighted geometric mean method. The gain matching degree occupies the main weight coefficient, the complexity difference is used as the adjustment factor, and the time delay is used as the decay coefficient to affect the final weight value.

[0045] The proportional gain coefficient of all matching video segments corresponding to the current scene is weighted and fused by proportional gain weight, and the weighting process adopts a weighted average method based on confidence, which gives higher confidence level to high weight video segments. The final proportional gain coefficient reflects the enhancement experience of historical scenes. The proportional gain coefficient and real-time feature data are input into the resolution enhancement controller, which generates a resolution enhancement intensity instruction using a fuzzy PID control algorithm. The proportional gain coefficient is used as the adjustment parameter of the proportional term, the rate of change of the feature data is used as the input of the differential term, and the cumulative error of the feature data is used as the input of the integral term. The difference between the video frame to be enhanced and the current video frame is obtained by using multi-scale structural similarity calculation, which compares the structural differences of the two images at different resolution levels. According to the difference value, the preliminary enhancement angle is calculated, which represents the direction of the resolution enhancement intensity to be applied. After obtaining the preset enhancement path and video model, the simulation algorithm is set. This algorithm tests the impact of different enhancement angles on video quality in a virtual environment. Through quality evaluation feedback, the optimal angle is selected, and the actual enhancement angle is generated, which considers the balance between processing efficiency and output quality. The resolution enhancement system is driven to adjust to the actual enhancement angle for enhancement operation. The system adjusts the filter parameters of the convolutional neural network and the generation strategy of the generative adversarial network according to the angle value, and performs directional enhancement on the video frame to be enhanced, focusing on improving the detail clarity of key areas such as moving vehicle license plates and pedestrian faces. After the enhancement is completed, a blind area compensation method is set. By analyzing the energy distribution in the frequency domain, the frequency range that is not fully enhanced is identified, and the blind area compensation angle is calculated to specifically strengthen the processing intensity of these frequency bands. A morphological detection algorithm is used to judge the spatial distribution characteristics of the detail missing area.

[0046] When performing compensation enhancement, the regional allocation of processing resources is adjusted according to the spatial distribution of the blind area. More computing resources are allocated to areas with severe detail missing. Iterative enhancement is used to gradually improve the visual quality of these areas until the detail performance of all areas reaches a balanced state. The enhancement data of the video frame to be enhanced includes the quality index change before and after processing, resource consumption data, and enhancement effect evaluation. These data are stored in the historical database to enrich the video segment samples for subsequent matching, providing a basis for continuous learning for the system. An adjustment optimization method is set to update the adjustment coefficient in the enhancement angle calculation using a reinforcement learning algorithm. By building a state-action-reward model, the coefficient mapping relationship is optimized. The optimized adjustment coefficient is applied to the calculation of the next enhancement angle, forming a closed-loop optimization system for continuous improvement.

[0047] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and illustrative figures, it should be apparent that the scope of the present application is not limited to these specific embodiments.

[0048] While the embodiments of the application have been shown and described herein, it will be understood by those skilled in the art that many changes, modifications, substitutions and alterations to these embodiments can be made without departing from the principles and spirits of the application, and it is intended that the scope of the application be limited solely by the scope of the appended claims and the equivalents thereof.

Claims

1. A method for improving the resolution of a dashcam image based on computer vision, characterized in that, include: The original video frame sequence captured by the dashcam is obtained, and the original video frame sequence is denoised and calibrated to obtain a preprocessed video frame sequence. Feature extraction processing is performed on the preprocessed video frame sequence to obtain key feature data; A hierarchical resolution enhancement model is constructed based on the key feature data. The hierarchical resolution enhancement model includes an initial model and an optimized model. The constraints of the initial model and the optimized model are determined, the key feature data are input into the initial model and the optimized model, and the solution is iteratively updated according to the constraints to obtain a high-resolution video frame sequence.

2. The method for improving the resolution of a dashcam image based on computer vision according to claim 1, characterized in that, The step of denoising and calibrating the original video frame sequence to obtain a preprocessed video frame sequence includes: The original video frame sequence is subjected to brightness equalization processing to obtain equalized data; Motion estimation is introduced and inter-frame differences are calculated. Determine the degree of difference between different frames and the magnitude of the change in the degree of difference until the magnitude of the change in the degree of difference tends to stabilize. Output the number of frames with the corresponding degree of difference and use it as the number of key frames. Based on the number of keyframes, a corresponding frame is selected from the equalization data as a reference frame. The similarity distance from each equalization data to the reference frame is determined. Each equalization data is assigned to the reference frame with the smallest similarity distance, and the grouping process is repeated until the change of the reference frame is less than a preset threshold to obtain a preprocessed video frame sequence.

3. The method for improving the resolution of a dashcam image based on computer vision according to claim 1, characterized in that, The step of performing feature extraction processing on the preprocessed video frame sequence to obtain key feature data includes: The preprocessed video frame sequence is used as input to the feature extractor for state estimation and a dashcam video model based on feature extraction is established. Determine the first-time update equation for the feature extraction-based dashcam video model; Determine the first state update equation for the feature extraction-based dashcam video model; The first current value is predicted based on the preprocessed video frame of the previous moment. The first current value is substituted into the first state update equation for correction to obtain the first current corrected value. The first current corrected value is then substituted into the first time update equation to predict the preprocessed video frame of the next moment to obtain high-frequency feature data and low-frequency feature data.

4. The method for improving the resolution of a dashcam image based on computer vision according to claim 3, characterized in that, The steps for constructing a resolution-enhanced hierarchical model based on the key feature data include: A hybrid resolution enhancement system is selected, consisting of a convolutional neural network and a generative adversarial network. The low-frequency feature data is used as the basic resolution reference value, and the high-frequency feature data is allocated within the hybrid resolution enhancement system to obtain the allocated data. A resolution enhancement hierarchical model is constructed based on the allocated data. The resolution enhancement hierarchical model includes an initial model and an optimized model.

5. The method for improving the resolution of a dashcam image based on computer vision according to claim 4, characterized in that, The step of allocating the high-frequency feature data within the hybrid resolution enhancement system to obtain allocated data includes: The high-frequency feature data is used as the input power of the hybrid resolution enhancement system. The high-frequency feature data is the difference between the output features of the preprocessed video frame sequence after adding the hybrid resolution enhancement system and the basic resolution reference value. The input power of the hybrid resolution enhancement system is used as the predictive measurement of the feature extractor for state estimation, and a power allocation model of the hybrid resolution enhancement system based on feature extraction is established. Determine the second time update equation for the power allocation model of the hybrid resolution enhancement system based on feature extraction; Determine the second state update equation for the power allocation model of the hybrid resolution enhancement system based on feature extraction; The second current value is predicted based on the high-frequency feature data of the previous moment. The second current value is then substituted into the second state update equation for correction to obtain the second current corrected value. The second current corrected value is then substituted into the second time update equation to predict the high-frequency feature data of the next moment. Calculate the filter coefficients; Based on the filtering coefficients, the allocation principle is determined, and the high-frequency feature data of the next time step is allocated according to the allocation principle to obtain the allocated data.

6. The method for improving the resolution of a dashcam image based on computer vision according to claim 1, characterized in that, The initial model is to minimize the blur of the video frames; The optimization model aims to maximize the sharpness gain of video frames.

7. The method for improving the resolution of a dashcam image based on computer vision according to claim 1, characterized in that, The constraints of the initial model include: Video frame brightness constraints; Video frame contrast constraint; Video frame noise constraint; The constraints of the optimization model include: System resolution balance constraints; Video frame output constraints.

8. The method for improving the resolution of a dashcam image based on computer vision according to claim 1, characterized in that, The step of inputting the key feature data into the initial model and the optimized model and iteratively updating and solving according to the constraints to obtain a high-resolution video frame sequence includes: The key feature data is input into the initial model for ambiguity analysis to obtain preliminary resolution improvement results; The preliminary resolution improvement result is input into the optimization model and the optimization model is corrected to obtain the optimized correction model; The optimized correction model uses actual resolution data to calculate the detail enhancement requirements of the hybrid resolution enhancement system. The data corresponding to the initial resolution enhancement results are input into the convolutional neural network and generative adversarial network systems respectively. With the goal of maximizing the operating efficiency over the entire life cycle, iterative solutions are performed under the constraints to obtain a high-resolution video frame sequence.

9. The method for improving the resolution of a dashcam image based on computer vision according to claim 1, characterized in that, Also includes: Select matching video segments from historical video data that match the complexity of the current video scene; Analyze the temporal feature data changes of the matched video segments to determine the proportional gain application point of the matched video segments; The time difference between the start point of each matched video segment and the point of action of the proportional gain is used as the time delay of each matched video segment. By comparing the changes in the proportional gain data sequence before and after the proportional gain action point, the gain matching degree of each matching video segment is determined. By combining the time delay, gain matching degree, and the difference in complexity between the matched video segment and the current video scene, the proportional gain weight of the matched video segment is determined. The proportional gain coefficient of the current video scene is determined by weighting all matching video segments corresponding to the current video scene using proportional gain weighting. The proportional gain coefficient and feature data are then input into the resolution enhancement controller, which controls the resolution enhancement intensity.

10. The method for improving the resolution of a dashcam image based on computer vision according to claim 9, characterized in that, Also includes: Obtain the difference between the video frame to be enhanced and the current video frame, set the initial adjustment method, and calculate the initial enhancement angle; Obtain the preset lifting path and video model, set the simulation algorithm, optimize the initial enhancement angle, and generate the actual enhancement angle for actual lifting. The drive resolution enhancement system is adjusted to the actual enhancement angle, and the video frame to be enhanced is enhanced. After the enhancement is completed, a blind spot compensation method is set, the blind spot compensation angle is calculated, the blind spot details are judged, the enhancement posture is determined, and compensation enhancement is performed. Record the enhancement data of the video frame to be enhanced, set the adjustment and optimization method, optimize the adjustment coefficient used to calculate the enhancement angle, and apply the optimized adjustment coefficient to the next calculation of the enhancement angle.

Images