Vehicle monitoring system and method based on Internet of Vehicles

Abstract

The invention discloses a vehicle monitoring system and method based on the Internet of Vehicles, and relates to the technical field of vehicle monitoring, and the method comprises the steps: obtaining continuous video frames of a vehicle-mounted camera, and calculating an exposure parameter difference index of adjacent frames to recognize an illumination change point; using the change point as a starting point to expand backwards to construct a local time window; screening and removing abnormal frames in the window to obtain a screened video frame sequence; and generating a compensation frame for an abnormal frame position in the screened video frame sequence through space-time interpolation, sorting and outputting a final video sequence according to a timestamp, and realizing vehicle monitoring based on the sequence. Therefore, under the illumination sudden change scene that the vehicle enters or drives out of the tunnel, the continuous video frames can be screened and compensated, the video frame rate is smoothed, the influence of exposure sudden change on the vehicle behavior recognition accuracy of the Internet of Vehicles platform is reduced, misjudgment is reduced, and the accuracy of vehicle state monitoring and safety decision making is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle monitoring, and in particular to a vehicle monitoring system and method based on an Internet of Vehicles. Background Art

[0002] In vehicle monitoring through the Internet of Vehicles, by relying on V2X communication technology, various vehicle perception data, such as speed, positioning, camera, radar, etc., are uploaded to the cloud or edge server in real time, and combined with intelligent algorithms to dynamically monitor and analyze the vehicle's operating status, driving behavior and surrounding environment; it is not only used for traditional driving records, but also widely used in scenarios such as intelligent traffic management, driving behavior analysis, accident warning and automatic driving assistance, to achieve remote supervision and risk prevention and control of the vehicle throughout its life cycle.

[0003] However, existing vehicle monitoring via the Internet of Vehicles (IoV) still faces a significant technical challenge in highway tunnel scenarios. Due to the dramatic changes in lighting inside and outside tunnels, onboard cameras rapidly adjust their exposure parameters when entering or exiting tunnels. This transient exposure adjustment often causes frame rate fluctuations or even frame dropouts, resulting in discontinuous video streams. Furthermore, cloud-based driving behavior recognition algorithms rely on continuous, high-quality video frames. Sudden exposure changes and frame rate anomalies can easily lead to misjudgments, such as misidentifying normal driving as abnormal, compromising accurate vehicle status monitoring and safety decision-making. Summary of the Invention

[0004] The purpose of the present invention is to solve the above-mentioned problems and provide a vehicle monitoring system and method based on the Internet of Vehicles.

[0005] In a first aspect of the present invention, a vehicle monitoring method based on the Internet of Vehicles is first proposed, the method comprising:

[0006] Obtain continuous video frames from the onboard camera while the vehicle is driving, and calculate the exposure difference index based on the exposure parameters of the current video frame and the previous video frame to identify the lighting change points;

[0007] Starting from the illumination change point, extend the preset number of frames backward to construct a local time window;

[0008] Each video frame in the local time window is screened, abnormal frames are screened out and removed, and a filtered video frame sequence is obtained;

[0009] For the positions corresponding to abnormal frames in the filtered video frame sequence, compensation frames are generated through the spatiotemporal interpolation method, and the output is reordered according to the timestamp to output the final video frame sequence, and the vehicle is monitored based on the output final video frame sequence.

[0010] Optionally, the exposure parameter is any one of the exposure time, exposure value, and exposure gain of the vehicle-mounted camera;

[0011] The steps for calculating the exposure difference index based on the exposure parameters of the current video frame and the previous video frame to identify the illumination change point are as follows:

[0012] Calculate the absolute difference in exposure parameters between the current video frame and the previous video frame, and divide the absolute difference by the maximum value of the exposure parameters between the current video frame and the previous video frame to obtain an exposure difference index;

[0013] The exposure difference index is compared with a preset threshold. If the exposure difference index is not less than the preset threshold, the corresponding current video frame is recorded as an illumination change point.

[0014] Optionally, starting from the illumination change point and extending backward by a preset number of frames, the step of constructing a local time window is:

[0015] Obtain the actual vehicle speed corresponding to the illumination change point, and calculate the total number of frames in the local time window based on the vehicle's actual speed and the benchmark reference speed. The calculation formula is: Where N is the total number of frames in the local time window, N min is the minimum frame number of the window, ensuring that the window will not be smaller than this number; V is the actual driving speed, V ref is the reference speed, N base is the benchmark frame number, the total number of frames in the window when the vehicle speed is equal to the reference speed;

[0016] Starting from the video frame corresponding to the illumination change point, N-1 video frames are taken backward to construct a local time window.

[0017] Optionally, the steps of screening each video frame within the local time window to screen out abnormal frames and remove them are as follows:

[0018] Calculating each frame of image within the time window, calculating an exposure excess index and an exposure-vehicle coupling anomaly index for each frame of image, normalizing the exposure excess index and the exposure-vehicle coupling anomaly index, removing the unit, and mapping them to a value range of 0-1, and performing a weighted summation of the normalized exposure excess index and exposure-vehicle coupling anomaly index to obtain an anomaly value for each frame of image;

[0019] The outlier value is compared with a preset outlier threshold, and each frame of the image in the time window is screened according to the comparison result to screen out the outlier frames and remove them.

[0020] Optionally, the step of calculating the exposure excess index is:

[0021] For each frame image in the local time window, extract the brightness histogram H of the frame i And normalize it to get the probability distribution h i,k , where k∈[1,m] represents the grayscale level; h i,k Represents the probability distribution of the current frame at gray level k;

[0022] Get the brightness histogram of the previous and next frames of the current frame and normalize it to the probability distribution h i-1,k and h i+1,k ;

[0023] Calculate the average value of the histogram of the adjacent frames

[0024] Calculate the brightness center of the current frame, that is, the average brightness μ i ;

[0025] And calculate the brightness center of the average histogram of the adjacent frames before and after

[0026] Calculate the standard deviation of the brightness distribution of the current frame, that is, the brightness width σ i ; Calculate the standard deviation of the brightness distribution of adjacent frames before and after, that is, the brightness width

[0027] Calculate the KL divergence between the current frame and the histograms of the previous and next adjacent frames. The calculation formula is: Where, is the KL divergence between the current frame and the adjacent average frame histogram;

[0028] Calculate the exposure excess index using the following formula:

[0029] Optionally, the steps for calculating the exposure and vehicle coupling anomaly index are:

[0030] For each frame image in the local time window, obtain the exposure parameter E of the i-th frame i and vehicle speed V i ;Exposure parameter E i Usually any one of exposure time, exposure value or exposure gain;

[0031] Calculate the rate of change of exposure parameters ΔE i , ΔE i =E i -E i-1 ; ΔE i is the change of exposure parameter of frame i; E i and E i-1Represent the exposure parameters of the i-th video frame and the i-1-th video frame respectively;

[0032] Calculate the rate of change of vehicle speed ΔV i , ΔV i =V i -V i-1 , ΔV i V is the change of exposure parameter of frame i; i and V i-1 Represent the vehicle speed corresponding to the i-th video frame and the i-1-th video frame respectively;

[0033] The rate of change of exposure parameter ΔE i and the rate of change of vehicle speed ΔV i Compared with 0, if the rate of change of exposure parameters ΔE i and the rate of change of vehicle speed ΔV i If both are greater than 0 or less than 0 at the same time, or equal to 0 at the same time, the exposure and vehicle coupling abnormality index is recorded as 0; if the exposure parameter change rate ΔE i and the rate of change of vehicle speed ΔV i If the values ​​are not greater than 0, less than 0, or equal to 0 at the same time, the exposure and vehicle coupling anomaly index is recorded as 1.

[0034] Optionally, the outlier value is compared with a preset outlier threshold, and each frame of the image in the time window is screened according to the comparison result to screen out the outlier frame and remove it. The steps are:

[0035] If the outlier value is not less than the preset outlier threshold, the corresponding frame image in the time window is abnormal, and the corresponding frame image in the time window is discarded;

[0036] If the outlier value is smaller than the preset outlier value threshold, the corresponding frame image in the time window is positive, and the corresponding frame image in the time window is retained.

[0037] In a second aspect of the present invention, a vehicle monitoring system based on the Internet of Vehicles is provided, the system comprising:

[0038] Exposure Difference Module: This module obtains continuous video frames from the vehicle's onboard camera while the vehicle is driving, and calculates the exposure difference index based on the exposure parameters of the current video frame and the previous video frame to identify the lighting change points;

[0039] Window construction module: Starting from the illumination change point, it extends backward by a preset number of frames to construct a local time window;

[0040] Screening module: Screens each video frame within the local time window, filters out abnormal frames and removes them to obtain a screening time series;

[0041] Monitoring module: For the positions corresponding to abnormal frames in the filtered time series, compensation frames are generated through the spatiotemporal interpolation method, the output is reordered according to the timestamp, the final video frame sequence is output, and the vehicle is monitored based on the output final video frame sequence.

[0042] Beneficial effects of the present invention:

[0043] The present invention proposes a vehicle monitoring system and method based on the Internet of Vehicles, which obtains continuous video frames from an on-board camera during vehicle driving, and calculates an exposure difference index based on the exposure parameters of each video frame and the previous video frame to identify the illumination change point; starting from the illumination change point, a preset number of frames are extended backward to construct a local time window; each video frame in the local time window is screened, abnormal frames are screened out and removed to obtain a screening time window; for the positions corresponding to the abnormal frames in the screening time window, compensation frames are generated by a spatiotemporal interpolation method, and the output is reordered according to the timestamp to output a final video frame sequence, and the vehicle is monitored based on the output final video frame sequence; in this way, in scenarios with sudden illumination changes such as vehicles entering or exiting tunnels, continuous video frames can be screened and compensated, the video frame rate can be smoothed, the impact of sudden exposure changes on the accuracy of vehicle behavior recognition on the Internet of Vehicles platform is reduced, misjudgment is reduced, and the accuracy of vehicle status monitoring and safety decision-making is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The present invention will be further described below with reference to the accompanying drawings.

[0045] Figure 1 is a flow chart of a vehicle monitoring method based on the Internet of Vehicles;

[0046] Figure 2 This is a framework diagram of a vehicle monitoring system based on the Internet of Vehicles. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] The embodiment of the present invention provides a vehicle monitoring method based on the Internet of Vehicles. Figure 1 , Figure 1 A flow chart of a vehicle monitoring method based on the Internet of Vehicles provided in an embodiment of the present invention. The method includes the following steps:

[0049] Obtain continuous video frames from the onboard camera while the vehicle is driving, and calculate the exposure difference index based on the exposure parameters of the current video frame and the previous video frame to identify the lighting change points;

[0050] Starting from the illumination change point, extend the preset number of frames backward to construct a local time window;

[0051] Each video frame in the local time window is screened, abnormal frames are screened out and removed, and a filtered video frame sequence is obtained;

[0052] For the positions corresponding to abnormal frames in the filtered video frame sequence, compensation frames are generated through the spatiotemporal interpolation method, and the output is reordered according to the timestamp to output the final video frame sequence, and the vehicle is monitored based on the output final video frame sequence.

[0053] According to an embodiment of the present invention, a vehicle monitoring method based on the Internet of Vehicles is provided. Through the above-mentioned method, when the on-board camera enters or exits a tunnel, the continuous video frames captured by the on-board camera can be screened and abnormal frames can be eliminated to achieve a smooth video frame rate, and reduce the impact of sudden exposure changes on the accuracy of the Internet of Vehicles platform in identifying the behavior of the vehicle during driving, thereby reducing misjudgments and reducing the impact on the accurate monitoring of the vehicle status and safety decision-making.

[0054] In one embodiment, continuous video frames from a vehicle-mounted camera are acquired while the vehicle is traveling, and an exposure difference index is calculated based on exposure parameters of a current video frame and a previous video frame to identify illumination change points;

[0055] In one implementation, the exposure parameter is any one of an exposure time, an exposure value, and an exposure gain of the vehicle-mounted camera;

[0056] The steps for calculating the exposure difference index based on the exposure parameters of the current video frame and the previous video frame to identify the illumination change point are as follows:

[0057] Calculate the absolute difference in exposure parameters between each video frame and the previous video frame, and divide the absolute difference by the maximum exposure parameter between the current video frame and the previous video frame to obtain the exposure difference index. The calculation formula is: Where, EDI i is the exposure difference index between the i-th video frame and the i-1-th video frame, where i is a positive integer, E i-1 is the exposure parameter of the i-1th video frame, E i is the exposure parameter of the i-th video frame; ∈ is the minimum value to prevent the denominator from being 0;

[0058] The exposure difference index is compared with a preset threshold. If the exposure difference index is not less than the preset threshold, it indicates that the vehicle's ambient lighting conditions have changed significantly. At this time, the corresponding video frame is recorded as the lighting change point, which is used for subsequent abnormal exposure analysis and frame rate compensation processing of the video frame.

[0059] It should be noted that the exposure parameter can be the camera's exposure time, exposure time in milliseconds, exposure value, or exposure gain; the specific one to be used depends on the API of the onboard camera; it is not limited or elaborated here; in addition, the denominator is max(E i+1 ,E i )+∈ is to avoid proportional distortion when the overall exposure is high or low; in addition, by calling the device interface of the on-board camera or the synchronous metadata channel of the video stream, the exposure parameters corresponding to each frame are extracted in real time, including any one of exposure time, exposure value or exposure gain. This exposure parameter does not require additional image processing and comes directly from the automatic exposure control module inside the camera. Through the timestamp synchronized with the vehicle video acquisition, it is ensured that the exposure parameters are strictly one-to-one corresponding to the corresponding video frames, thereby constructing a complete time-series exposure parameter sequence. As input data, this sequence can drive the calculation of the exposure difference index in real time with extremely low computational load, providing an accurate data basis for subsequent detection of illumination change points, and ensuring that the algorithm can run efficiently and reliably on the vehicle edge side.

[0060] It should be noted that the above method can determine the nodes where the light intensity changes suddenly when the vehicle enters or exits a tunnel. The camera will continuously adjust the exposure parameters in a short period of time to adapt to the new lighting conditions. However, the quality of the video frames during this adjustment process is often unstable. If the video frames of the adjustment process are uploaded to the vehicle monitoring cloud, the driving behavior recognition algorithm on the cloud may cause incorrect recognition results based on the uploaded video frame sequence. Therefore, taking the illumination change point as the starting point, the local time window is formed by expanding several frames backward to adjust the video frame to ensure that the video frame during the exposure parameter adjustment process is available, thereby reducing the probability of incorrect recognition results of the driving behavior recognition algorithm on the cloud based on the uploaded video frame sequence. At the same time, calibration with illumination change points helps to accurately define abnormal areas: avoid unnecessary global processing of the entire video, save computing resources, and improve the real-time performance and efficiency of the algorithm.

[0061] It should be noted that this approach enables rapid, lightweight, efficient, and adaptive detection of abnormal video exposure changes caused by sudden changes in external lighting conditions during vehicle operation, such as those caused by entering or exiting a tunnel, strong backlight, or shadows. By calculating the normalized difference in exposure parameters between adjacent frames, distortion in absolute differences caused by overall camera exposure levels can be effectively avoided, allowing for more accurate capture of relative exposure changes. Furthermore, compared to pixel-level full-image histogram analysis or deep neural network detection, calculations based on exposure metadata consume virtually no computing resources and offer excellent real-time performance, making them ideally suited for in-vehicle edge computing environments, particularly those in transit. The timely identification of illumination change points not only serves as a trigger for subsequent video anomaly detection, outlier frame removal, and timing compensation, but also enables local pre-processing of the video stream before uploading it to the cloud. This fundamentally reduces frame rate fluctuations, image flickering, and misidentification of behavior caused by drastic lighting changes, thereby improving the continuity, stability, and effectiveness of video surveillance data in connected car environments. This method significantly reduces the interference and errors caused by sudden changes in lighting to downstream algorithms, such as driving behavior analysis and road event detection.

[0062] In one embodiment, starting from the illumination change point and extending backward by a preset number of frames, the steps of constructing a local time window are:

[0063] Obtain the actual vehicle speed corresponding to the illumination change point, and calculate the total number of frames in the local time window based on the vehicle's actual speed and the benchmark reference speed. The calculation formula is: Where N is the total number of frames in the local time window, N min is the minimum frame number of the window, ensuring that the window will not be smaller than this number; V is the actual driving speed, V ref is the reference speed, N base is the benchmark frame number, the total number of frames in the window when the vehicle speed is equal to the reference speed;

[0064] Starting from the video frame corresponding to the illumination change point, N-1 video frames are taken backward to construct a local time window.

[0065] It should be noted that the actual vehicle speed corresponding to the illumination change point is acquired in real time via the vehicle's OBD interface, CAN bus, or high-precision positioning system and synchronously transmitted to the vehicle's local edge computing unit or the cloud. The baseline reference speed and baseline frame rate are preset parameters based on typical road scenarios, such as highways, tunnels, and urban roads. The minimum window frame rate is used to prevent the window from being too small at high speeds or in extreme scenarios, resulting in insufficient adaptation to illumination changes. Furthermore, the advantage of using this speed-adaptive time window calculation method to construct the local time window is that it effectively addresses the impact of illumination changes on video data continuity at different vehicle speeds. At low speeds, vehicles take longer to enter or exit tunnels, bridges, and other environments, and the duration of illumination changes is also longer. Therefore, the window frame rate needs to be increased to fully capture this change. At high speeds, illumination changes occur more quickly, so the window frame rate needs to be reduced accordingly, ensuring that key changes are captured while reducing computational and storage overhead.

[0066] It should be noted that in scenarios with drastic lighting changes, such as vehicles traveling through tunnels, the camera's exposure adjustment process may not be instantaneous; it may take some time to stabilize. The number of video frames corresponding to this time period actually varies at different vehicle speeds. At lower vehicle speeds, the vehicle spends longer within the exposure adjustment region, requiring more frames for the camera to adapt to the exposure. Therefore, a larger time window is needed to fully cover the entire exposure adjustment process. Meanwhile, when a vehicle travels at high speeds, it quickly exits the illumination change region, naturally reducing the number of frames involved in the exposure adjustment process. Continuing to capture a large number of frames would not only introduce irrelevant data, but also waste computational resources, impacting real-time performance. Therefore, by obtaining the actual speed at the corresponding illumination change point and combining it with a reference speed and a baseline frame count to dynamically calculate the total number of frames in the local time window, we can effectively balance algorithm integrity and real-time performance. This method adapts to the vehicle's varying driving states, ensuring that the entire exposure adjustment process is fully preserved at low speeds and avoiding redundant processing at high speeds. This improves stability, adaptability, and processing efficiency across various speeds, thereby enhancing the accuracy of video restoration and the reliability of vehicle state recognition, resulting in improved practicality.

[0067] In one embodiment, each video frame within the local time window is screened to identify and remove abnormal frames, thereby obtaining a screened video frame sequence;

[0068] In one implementation, the steps of screening each video frame within the local time window, screening out abnormal frames and removing them are as follows:

[0069] Calculate each frame of image within the local time window, calculate the exposure excess index and exposure and vehicle coupling abnormality index of each frame of image, and normalize the exposure excess index and exposure and vehicle coupling abnormality index, remove the unit, and map them to the range of 0-1. Then perform weighted summation of the normalized exposure excess index and exposure and vehicle coupling abnormality index to obtain the abnormal value of each frame of image; the calculation formula of the abnormal value is: Where vb is the abnormal index, DR and DE are the normalized exposure exceeding standard index and exposure and vehicle coupling abnormal index, respectively;

[0070] The outliers are compared with the preset outlier threshold, and each frame of the image in the time window is screened according to the comparison results to filter out the abnormal frames and remove them.

[0071] It should be noted that commonly used normalization methods include Min-Max normalization and Z-Score standardization. The specific selection depends on the actual situation and is not limited or elaborated on. In addition, under normal circumstances, the weight values ​​of the normalized exposure excess index and the exposure and vehicle coupling abnormality index are the same, and the sum of the weights is 1.

[0072] In one embodiment, the steps for calculating the exposure excess index are:

[0073] For each frame image in the local time window, extract the brightness histogram H of the frame i And normalize it to get the probability distribution h i,k , where k∈[1,m] represents the grayscale level, for example, 256 levels; h i,k Represents the probability distribution of the current frame at gray level k;

[0074] Get the brightness histogram of the previous and next frames of the current frame and normalize it to the probability distribution h i-1,k and h i+1,k ;

[0075] Calculate the average value of the histogram of the adjacent frames

[0076] Calculate the brightness center of the current frame, that is, the average brightness μ i ,

[0077] And calculate the brightness center of the average histogram of the adjacent frames before and after

[0078] Calculate the standard deviation of the brightness distribution of the current frame, that is, the brightness width σ i , Calculate the standard deviation of the brightness distribution of adjacent frames before and after, that is, the brightness width

[0079] Calculate the KL divergence between the current frame and the histograms of the previous and next adjacent frames. The calculation formula is: Where, is the KL divergence between the current frame and the previous and next adjacent average frame histograms;

[0080] Calculate the exposure excess index using the following formula:

[0081] It should be noted that the data involved in the calculation of the above-mentioned exposure excess index is mainly obtained through the video image sequence collected in real time by the on-board camera. Specifically, first, during the driving process of the vehicle, video frames are continuously collected, the brightness information of each frame is extracted through the image processing module, and the corresponding brightness histogram is generated based on the brightness value; in order to ensure that the comparison of the brightness distribution has temporal consistency, the histogram data of the current frame and its adjacent frames are extracted synchronously. In addition, the timestamp and frame sequence information of the video acquisition are also recorded synchronously for accurate matching of adjacent frames to avoid errors caused by frame rate fluctuations. This process does not require reliance on external sensors and is completely based on the image data of the on-board camera itself. The algorithm module automatically completes the extraction of brightness features, normalization of probability distribution, and calculation of mathematical indicators, thereby achieving accurate detection of exposure anomalies.

[0082] It should be noted that the Exposure Index is designed to comprehensively quantify the degree of exposure anomaly in the current frame, thereby accurately reflecting whether the frame has an exposure anomaly, especially in scenes with drastic lighting changes, such as vehicle tunnels. First, by calculating the absolute difference in the average brightness of the current frame and its preceding and succeeding frames, it captures sudden changes in the overall image brightness, reflecting the rapid changes in brightness levels during exposure adjustment. Second, by calculating the absolute difference in the standard deviation of the brightness of the current frame and its preceding and succeeding frames, it reveals changes in the width of the brightness distribution—that is, the degree of brightness concentration or diffusion—which is particularly important for identifying the size and changes of overexposed or underexposed areas. Finally, the KL divergence is introduced as a metric to evaluate the difference in the probability distribution between the brightness histogram of the current frame and the average brightness histogram of the adjacent frames. A larger KL divergence indicates a more significant difference in the brightness distribution, indicating anomalies in the image's detail and overall lighting structure. The Exposure Index, calculated by combining these three metrics, provides a more detailed and multi-dimensional assessment of image exposure anomalies, capable of identifying underexposure, overexposure, or uneven exposure caused by sudden changes in lighting during the camera's automatic exposure adjustment process. A higher exposure index indicates that the current frame experiences not only a dramatic change in brightness level but also a significant shift in brightness distribution compared to the preceding and following frames. This strong abnormality signal typically occurs when the camera's exposure adjustment has not yet stabilized. Therefore, a higher exposure index indicates a higher likelihood of abnormality in the corresponding video frame. This provides a scientific basis for subsequent frame removal and interpolation compensation, ensuring the stability and continuity of the video sequences used in the monitoring system and improving the accuracy and safety of driving environment perception.

[0083] In one implementation, the overexposure index effectively captures abnormal fluctuations in exposure parameter adjustment by comprehensively considering the differences between the current frame and the preceding and following frames in terms of the brightness center of gravity, brightness distribution width, and overall brightness histogram shape. This multi-dimensional quantification method not only identifies frames with over- or underexposure due to sudden changes in the lighting environment, but also reveals subtle changes in the exposure adjustment process, such as uneven brightness distribution or localized exposure anomalies. Using the overexposure index to screen abnormal frames helps eliminate distorted frames caused by unresponsive or over-adjusted automatic exposure adjustments, preventing these abnormal frames from interfering with subsequent visual analysis and vehicle monitoring systems. Furthermore, the abnormal information reflected by this index is crucial for improving the continuity and stability of image sequences, particularly when vehicles enter tunnels, pass through areas of strong light, or experience drastic lighting changes. This ensures the quality of video data output by the camera, improves the cloud's perception of the surrounding environment, reduces false positives and monitoring blind spots, and thus enhances driving safety. In addition, the calculation of this index is entirely based on the image brightness distribution characteristics, without the need for additional hardware support, making it easy to deploy in real time on the vehicle side, achieving rapid response and local processing, and reducing the pressure on cloud computing and communication bandwidth. Overall, the application of the exposure excess index not only improves the vehicle monitoring system's adaptability to complex lighting environments, but also enhances the system's reliability and safety level in actual road scenarios.

[0084] In one embodiment, the steps for calculating the exposure and vehicle coupling anomaly index are:

[0085] For each frame image in the local time window, obtain the exposure parameter E of the i-th frame i and vehicle speed V i ;Exposure parameter E i Usually any one of exposure time, exposure value or exposure gain;

[0086] Calculate the rate of change of exposure parameters ΔE i , ΔE i =E i -E i-1 ; ΔE i is the change in exposure parameter of the i-th frame, indicating the increase in exposure parameter compared with the previous frame; E i and E i-1 Represent the exposure parameters of the i-th video frame and the i-1-th video frame respectively;

[0087] Calculate the rate of change of vehicle speed ΔV i , ΔV i =V i -V i-1 , ΔV i V is the change in exposure parameter of the i-th frame, indicating the change in vehicle speed compared with the previous frame;i and V i-1 Represent the vehicle speed corresponding to the i-th video frame and the i-1-th video frame respectively;

[0088] The rate of change of exposure parameter ΔE i and the rate of change of vehicle speed ΔV i Compared with 0, if the rate of change of exposure parameters ΔE i and the rate of change of vehicle speed ΔV i If both are greater than 0 or less than 0 at the same time, or equal to 0 at the same time, the exposure and vehicle coupling abnormality index is recorded as 0; if the exposure parameter change rate ΔE i and the rate of change of vehicle speed ΔV i If the values ​​are not greater than 0, less than 0, or equal to 0 at the same time, the exposure and vehicle coupling anomaly index is recorded as 1.

[0089] It's important to note that during the calculations described above, exposure parameter data, including exposure time, exposure value, or exposure gain, is typically extracted from the image metadata automatically captured by the vehicle's camera. This data is updated in real time by the camera's automatic exposure control module. Vehicle speed data is obtained from the vehicle's CAN bus or GPS system, synchronously acquired and timestamped by the vehicle's data acquisition unit to ensure a one-to-one correspondence with the corresponding video frame. This synchronized acquisition method ensures the accuracy and timing consistency of exposure parameter and speed data, making the calculation of the coupling anomaly index more reliable and effectively reflecting the dynamic relationship between the two.

[0090] It should be noted that the Exposure-Vehicle Coupling Anomaly Index reflects the synchronization and rationality between changes in camera exposure parameters and vehicle speed, measuring whether exposure adjustments are consistent with changes in the vehicle's driving state. Typically, changes in vehicle speed directly affect the ambient lighting conditions surrounding the camera. For example, when a vehicle accelerates into a tunnel, the light rapidly dims, and the camera's exposure parameters should be increased accordingly to compensate for the insufficient light. Conversely, when the vehicle decelerates or exits a tunnel, the exposure parameters should be gradually decreased. If the rate of change of the exposure parameters aligns with the rate of change of vehicle speed—increasing, decreasing, or remaining constant—this indicates that exposure adjustment is well coupled with the vehicle's motion and is normal, indicating a low likelihood of image anomalies in the corresponding frame. Conversely, if the rate of change of the exposure parameters aligns with or is inconsistent with the rate of change of vehicle speed, this indicates that exposure adjustment is failing to effectively respond to changes in vehicle speed, indicating an exposure anomaly. The corresponding frame is likely to be anomaly, with over- or underexposure, potentially impacting subsequent image processing and driving behavior analysis. For example, if the camera exposure unexpectedly decreases during acceleration, causing the image to darken, this could lead to a false interpretation of the vehicle entering a tunnel or a sudden drop in light, potentially triggering a safety alert or determining abnormal driving behavior. Uploading these abnormal frames to the cloud for analysis could lead to misjudgments and reduce the reliability of driver assistance. Therefore, by calculating the exposure and speed coupling anomaly index, we can effectively identify and eliminate abnormal frames caused by improper exposure adjustment, improving data quality and the accuracy of driving behavior monitoring.

[0091] In one implementation, analyzing the Exposure-Vehicle Coupling Anomaly Index provides the following benefits for vehicle monitoring: It provides real-time information on whether camera exposure adjustments are appropriately responding to changes in vehicle speed, helping to identify image quality issues caused by abnormal exposure parameters, such as overexposure or underexposure, thereby avoiding visual information loss due to sudden changes in image brightness. Secondly, the index helps eliminate or flag anomalous frames resulting from abnormal exposure-vehicle speed coupling, improving the overall quality and reliability of video data and ensuring that subsequent driving behavior analysis, environmental perception, and safety assessments are based on stable and accurate image information. Furthermore, by promptly detecting abnormal exposure-vehicle speed mismatches, the monitoring system can assist in diagnosing camera or exposure control module faults, facilitating more accurate and efficient maintenance and debugging. In summary, the Exposure-Vehicle Coupling Anomaly Index not only improves vehicle monitoring video quality but also enhances the system's sensitivity and responsiveness to abnormalities, providing solid data support for intelligent driving and safety management.

[0092] In one embodiment, the outlier value is compared with a preset outlier threshold, and each video frame in the local time window is screened based on the comparison result. The steps of screening out the outlier frames and removing them are as follows:

[0093] If the outlier value is not less than the preset outlier threshold, the corresponding frame image in the time window is abnormal, and the corresponding frame image in the time window is discarded;

[0094] If the outlier value is smaller than the preset outlier value threshold, the corresponding frame image in the time window is positive, and the corresponding frame image in the time window is retained.

[0095] It should be noted that the outliers are compared one by one with the pre-set outlier threshold. If the outlier value of a certain frame is greater than or equal to the threshold, it means that there is an obvious abnormality in the frame, such as severe overexposure, mismatch between exposure adjustment and vehicle speed change, abnormal image brightness fluctuation, etc., which may affect subsequent driving behavior analysis, target recognition or cloud-based intelligent judgment. The frame will be automatically marked as an abnormal frame and removed from the valid data. On the contrary, if the outlier value of a certain frame is less than the threshold, it means that the frame image is within the normal range, the lighting change is reasonable, the exposure adjustment is highly matched with the vehicle status, and it does not affect the image quality. The frame will be retained. Through this precise threshold-based screening mechanism, poor-quality frames caused by sudden environmental changes, abnormal equipment adjustment or noise can be effectively filtered out, thereby greatly improving the validity and accuracy of the data, and providing high-quality image input for subsequent intelligent recognition, driving behavior analysis and risk warning.

[0096] In one embodiment, for the positions corresponding to abnormal frames in the filtered video frame sequence, compensation frames are generated by a spatiotemporal interpolation method, and the output is reordered according to the timestamp to output a final video frame sequence, and the vehicle is monitored based on the output final video frame sequence.

[0097] In one implementation, after outlier frames are removed, the original video frame sequence exhibits temporal discontinuities between frames, known as inter-frame temporal discontinuities. These discontinuities compromise the temporal integrity of the video stream, hindering video-based continuity analysis such as target tracking, vehicle status detection, and driving behavior monitoring. To repair these discontinuities, the cloud uses a spatiotemporal interpolation method based on adjacent normal frames to generate compensation frames. Specifically, to identify the positions of outlier frames in the filtered video frame sequence, the cloud first locates the specific positions of these outlier frames in the video time series. Then, for each outlier frame, the cloud searches for the two closest normal frames forward and backward along its time axis. These two normal frames, even if not immediately adjacent to the outlier frame, serve as a reference for compensation. Next, a spatiotemporal interpolation method is used to calculate a compensation frame for the outlier frame using the image content of the two normal frames and the time interval between them. This spatiotemporal interpolation method not only considers the weighting ratio of the temporal dimension but also incorporates pixel variations in the spatial dimension, such as using optical flow to estimate pixel motion, to infer the expected image content of the outlier frame, thereby generating a compensation frame that closely resembles the true frame. The generated compensation frames replace the missing abnormal frames, ensuring the continuity and integrity of the video sequence. Subsequently, all normal and compensated frames are rearranged in timestamp order to form a continuous, complete, and smooth final video frame sequence, avoiding image discontinuity caused by missing abnormal frames. Based on this sequence, the vehicle monitoring system can stably and accurately perform tasks such as vehicle identification, behavior analysis, and environmental perception. For example, if within a 20-frame time window, frames 10 and 15 are identified as abnormal, frames 9 and 11 are used to generate a compensation image for frame 10, and frames 14 and 16 are used to generate a compensation image for frame 15. These compensation frames are then inserted into the corresponding positions and output in sequence, ensuring the continuity and integrity of the monitoring video stream. This approach not only effectively overcomes single-frame anomalies caused by drastic lighting changes and exposure anomalies, but also significantly improves the vehicle monitoring system's adaptability to video quality and fault tolerance to abnormal situations, ensuring stable and accurate monitoring results.

[0098] Based on the same inventive concept, the embodiment of the present invention also provides a vehicle monitoring system based on the Internet of Vehicles. Figure 2 , Figure 2 A framework diagram of a vehicle monitoring system based on the Internet of Vehicles provided in an embodiment of the present invention, the system includes:

[0099] Exposure Difference Module: This module obtains continuous video frames from the vehicle's onboard camera while the vehicle is driving, and calculates the exposure difference index based on the exposure parameters of the current video frame and the previous video frame to identify the lighting change points;

[0100] Window construction module: Starting from the illumination change point, it extends backward by a preset number of frames to construct a local time window;

[0101] Screening module: Screens each video frame within the local time window, filters out abnormal frames and removes them to obtain a screening time series;

[0102] Monitoring module: For the positions corresponding to abnormal frames in the filtered time series, compensation frames are generated through the spatiotemporal interpolation method, the output is reordered according to the timestamp, the final video frame sequence is output, and the vehicle is monitored based on the output final video frame sequence.

[0103] According to an embodiment of the present invention, a vehicle monitoring system based on the Internet of Vehicles is provided. Through the above-mentioned method, when the on-board camera enters or exits a tunnel, the continuous video frames captured by the on-board camera can be filtered and abnormal frames can be eliminated to achieve a smooth video frame rate, and the impact of sudden exposure changes on the accuracy of the Internet of Vehicles platform in identifying the vehicle's behavior during driving is reduced, thereby reducing misjudgments and reducing the impact on the accurate monitoring of the vehicle status and safety decision-making.

[0104] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A vehicle monitoring method based on the Internet of Vehicles, characterized in that: The following steps are involved: Obtain continuous video frames from the onboard camera while the vehicle is driving, and calculate the exposure difference index based on the exposure parameters of the current video frame and the previous video frame to identify the lighting change points; Starting from the illumination change point, extend the preset number of frames backward to construct a local time window; Each video frame in the local time window is screened, abnormal frames are screened out and removed, and a filtered video frame sequence is obtained; For the positions corresponding to abnormal frames in the filtered video frame sequence, compensation frames are generated through the spatiotemporal interpolation method, and the output is reordered according to the timestamp to output the final video frame sequence, and the vehicle is monitored based on the output final video frame sequence.

2. The vehicle monitoring method based on the Internet of Vehicles according to claim 1, characterized in that: The exposure parameter is any one of the exposure time, exposure value, and exposure gain of the vehicle-mounted camera; The steps for calculating the exposure difference index based on the exposure parameters of the current video frame and the previous video frame to identify the illumination change point are as follows: Calculate the absolute difference in exposure parameters between the current video frame and the previous video frame, and divide the absolute difference by the maximum value of the exposure parameters between the current video frame and the previous video frame to obtain an exposure difference index; The exposure difference index is compared with a preset threshold. If the exposure difference index is not less than the preset threshold, the corresponding current video frame is recorded as an illumination change point.

3. The vehicle monitoring method based on the Internet of Vehicles according to claim 1, characterized in that: Starting from the illumination change point and extending backward by a preset number of frames, the steps to construct a local time window are: Obtain the actual vehicle speed corresponding to the illumination change point, and calculate the total number of frames in the local time window based on the vehicle's actual speed and the benchmark reference speed. The calculation formula is: Where N is the total number of frames in the local time window, N min is the minimum frame number of the window, ensuring that the window will not be smaller than this number; V is the actual driving speed, V ref is the reference speed, N base is the benchmark frame number, the total number of frames in the window when the vehicle speed is equal to the reference speed; Starting from the video frame corresponding to the illumination change point, N-1 video frames are taken backward to construct a local time window.

4. The vehicle monitoring method based on the Internet of Vehicles according to claim 1, characterized in that: The steps for filtering each video frame within the local time window, filtering out abnormal frames and removing them are as follows: Calculate the exposure excess index and exposure-vehicle coupling anomaly index for each frame of image within the local time window, normalize the exposure excess index and exposure-vehicle coupling anomaly index, remove the unit, and map them to the range of 0-1. Then perform weighted summation of the normalized exposure excess index and exposure-vehicle coupling anomaly index to obtain the anomaly value of each frame of image; The outlier value is compared with a preset outlier threshold, and each frame of the image in the time window is screened according to the comparison result to screen out the outlier frames and remove them.

5. The vehicle monitoring method based on the Internet of Vehicles according to claim 4 is characterized in that: The calculation steps of the exposure excess index are as follows: For each frame image in the local time window, extract the brightness histogram H of the frame i And normalize it to get the probability distribution h i,k , where k∈[1,m] represents the grayscale level; h i,k Represents the probability distribution of the current frame at gray level k; Get the brightness histogram of the previous and next frames of the current frame and normalize it to the probability distribution h i-1,k and h i+1,k ; Calculate the average value of the histogram of the adjacent frames Calculate the brightness center of the current frame, that is, the average brightness μ i ; And calculate the brightness center of the average histogram of the adjacent frames before and after Calculate the standard deviation of the brightness distribution of the current frame, that is, the brightness width σ i ; Calculate the standard deviation of the brightness distribution of adjacent frames before and after, that is, the brightness width Calculate the KL divergence between the current frame and the histograms of the previous and next adjacent frames. The calculation formula is: Where, is the KL divergence between the current frame and the adjacent average frame histogram; Calculate the exposure excess index using the following formula:

6. The vehicle monitoring method based on the Internet of Vehicles according to claim 4 is characterized in that: The calculation steps of the exposure and vehicle coupling anomaly index are as follows: For each frame image in the local time window, obtain the exposure parameter E of the i-th frame i and vehicle speed V i ;Exposure parameter E i Usually any one of exposure time, exposure value or exposure gain; Calculate the rate of change of exposure parameters ΔE i , ΔE i =E i -E i-1 ; ΔE i is the change of exposure parameter of frame i; E i and E i-1 Represent the exposure parameters of the i-th video frame and the i-1-th video frame respectively; Calculate the rate of change of vehicle speed ΔV i , ΔV i =V i -V i-1 , ΔV i is the change of exposure parameter of frame i; V i and V i-1 Represent the vehicle speed corresponding to the i-th video frame and the i-1-th video frame respectively; The rate of change of exposure parameter ΔE i and the rate of change of vehicle speed ΔV i Compared with 0, if the rate of change of exposure parameters ΔE i and the rate of change of vehicle speed ΔV i If both are greater than 0 or less than 0 at the same time, or equal to 0 at the same time, the exposure and vehicle coupling abnormality index is recorded as 0; if the exposure parameter change rate ΔE i and the rate of change of vehicle speed ΔV i If the values ​​are not greater than 0, less than 0, or equal to 0 at the same time, the exposure and vehicle coupling anomaly index is recorded as 1.

7. The vehicle monitoring method based on the Internet of Vehicles according to claim 4 is characterized in that: The outlier value is compared with the preset outlier threshold, and each frame of the image in the time window is screened according to the comparison result. The steps of screening out the outlier frames and removing them are as follows: If the outlier value is not less than the preset outlier threshold, the corresponding frame image in the time window is abnormal, and the corresponding frame image in the time window is discarded; If the outlier value is smaller than the preset outlier value threshold, the corresponding frame image in the time window is positive, and the corresponding frame image in the time window is retained.

8. A vehicle monitoring system based on the Internet of Vehicles, used to implement the vehicle monitoring method based on the Internet of Vehicles according to any one of claims 1 to 7, characterized in that: The system comprises: Exposure Difference Module: This module obtains continuous video frames from the vehicle's onboard camera while the vehicle is driving, and calculates the exposure difference index based on the exposure parameters of the current video frame and the previous video frame to identify the lighting change points; Window construction module: Starting from the illumination change point, it extends backward by a preset number of frames to construct a local time window; Screening module: Screens each video frame within the local time window, filters out abnormal frames and removes them to obtain a screening time series; Monitoring module: For the positions corresponding to abnormal frames in the filtered time series, compensation frames are generated through the spatiotemporal interpolation method, the output is reordered according to the timestamp, and the final video frame sequence is output. The vehicle is monitored based on the output final video frame sequence.

Images