A bridge traffic monitoring system, method and storage medium based on video images

Abstract

The present application provides a bridge traffic monitoring system, method and storage medium based on video images, which relates to the field of traffic monitoring. S100, video acquisition: the video stream of the bridge area is continuously collected by high-definition cameras installed at key positions of the bridge; S200, preprocessing: noise filtering, contrast enhancement and inter-frame registration processing are performed on each frame of the image; S300, vehicle detection and tracking: S301, background modeling: background modeling is performed using a mixed Gaussian model (GMM) to distinguish between foreground and background. The present application extracts information from the image and, after algorithm calculation, can judge the traffic congestion of the bridge through the bridge vibration frequency, ambient light changes and vehicle light flashing patterns in the video image. Compared with the method of the number of cars in the image, these monitoring methods have better monitoring effects, and are not judged by a single factor, but by a comprehensive judgment, which further reduces the possibility of misjudgment and has a good monitoring effect.

Description

Technical Field

[0001] The present invention relates to the field of traffic monitoring, and in particular to a bridge traffic monitoring system, method and storage medium based on video images. Background Art

[0002] The traffic monitoring system makes the traffic information and traffic-related information on the road as complete and real-time as possible; the information exchange between traffic participants, traffic managers, vehicles, and road management facilities is real-time and efficient; the control center's control over the execution system is more efficient; the processing software system has the ability of self-learning and self-adaptation. At present, most bridge traffic systems use video images to monitor traffic, and most of them are determined based on the number of cars in each frame of the image. However, this method cannot judge whether the bridge is congested well. There are situations where there are many cars in the image, but the traffic is orderly and smooth. Therefore, this monitoring method does not have a good monitoring effect.

[0003] Therefore, we make improvements to this and propose a bridge traffic monitoring system, method and storage medium based on video images. Summary of the invention

[0004] The purpose of the present invention is to: for most of the existing bridge traffic systems, traffic monitoring is carried out by means of video images, and most of them are determined based on the number of cars in each frame of the image. However, since this method cannot judge whether the bridge is congested well, there may be a large number of cars in the image, but the traffic is orderly and smooth. Therefore, this monitoring method has a poor monitoring effect.

[0005] In order to achieve the above-mentioned purpose of the invention, the present invention provides a bridge traffic monitoring system based on video images to improve the above-mentioned problems.

[0006] The specific application is as follows:

[0007] The following steps are involved:

[0008] S100, video acquisition: HD cameras installed at key locations on the bridge continuously collect video streams of the bridge area;

[0009] S200, preprocessing: performing noise filtering, contrast enhancement and inter-frame registration processing on each frame of image;

[0010] S300, vehicle detection and tracking:

[0011] S301, Background modeling: Use the Gaussian mixture model (GMM) to model the background and distinguish between foreground and background. The GMM update formula can be expressed as ,in is the weight, is the learning rate, is the probability that the current pixel belongs to the background, is the time node;

[0012] S302, moving object segmentation: segmenting the moving vehicle from the background;

[0013] S400, vibration frequency extraction: Bridges vibrate naturally when vehicles pass through them. This vibration can reflect the traffic flow and vehicle type on the bridge. Heavier or more vehicles will cause the bridge to vibrate more. The vibration signal of the bridge is measured by sensors installed on the bridge, and then signal processing technology is used to analyze the spectral characteristics of the vibration signal. The current traffic conditions, such as whether there is congestion or whether there are heavy vehicles, can be identified through algorithms.

[0014] Region of Interest Selection: Determine the key parts of the bridge structure (the key parts are the middle section of the bridge in the image, which is the part of the bridge structure most susceptible to vibration) as the region of interest (ROI) for analysis;

[0015] Displacement measurement: estimate the small displacement of ROI by pixel displacement between consecutive frames;

[0016] FFT conversion: Use fast Fourier transform (FFT) to convert time domain signals into frequency domain signals;

[0017] The FFT conversion formula is: ,in is a time series, is the corresponding spectrum, is the total number of samples, It is a complex exponential term, which represents the combination of sine waves and cosine waves at different frequencies. It is used to calculate each spectrum sample, analyze the FFT results, and find the main frequency components. These frequency components reflect the main vibration mode of the bridge. The frequency corresponding to the identified frequency peak is the vibration frequency of the bridge.

[0018] S500, analysis of ambient light change patterns;

[0019] S600, vehicle light flickering law analysis;

[0020] S700, traffic condition assessment: The bridge vibration frequency, ambient light change pattern and vehicle light flashing law obtained by S400, S500 and S600 are compared with normal values. If any two or more of the data exceed 25% of the normal value, it is judged as a congested state, otherwise it is a smooth state.

[0021] As a preferred technical solution of the present application, S200 further includes the following steps:

[0022] S201, noise filtering: using an adaptive Wiener filter to remove random noise in the image;

[0023] S202, contrast enhancement: using histogram equalization to improve image contrast;

[0024] S203, inter-frame registration: using feature matching or optical flow method to ensure the correspondence between adjacent frames.

[0025] As a preferred technical solution of the present application, S500 further includes the following steps:

[0026] S501, brightness value statistics: calculate the average brightness value of the selected area in each frame;

[0027] S502, change rate calculation: Identify weather changes and vehicle passing by by comparing the change rates of brightness values ​​of consecutive frames.

[0028] As a preferred technical solution of the present application, S600 further includes the following steps:

[0029] S601, light brightness change detection: detect and record the brightness change of vehicle lights, determine the light area at the head or tail of the vehicle at night or in low light conditions, separate the light area from other backgrounds, and monitor the change of brightness value of the light area over time;

[0030] S602, PSD calculation: Analyze the frequency characteristics of light flicker through power spectral density (PSD), and use the periodogram method or Welch method to calculate the PSD of light brightness value. PSD is calculated by the periodogram method and Welch method.

[0031] As a preferred technical solution of the present application, the displacement measurement in S400 is specifically performed by calculating the position difference of the same feature point between adjacent frames to obtain the displacement vector of the point ,in It's time The position of the feature point.

[0032] As a preferred technical solution of the present application, in S501, a fixed ROI is selected to represent the lighting conditions of the entire scene, and the average brightness value within the ROI in each frame is calculated. ,in and is the height and width of the ROI, It's pixels The brightness value of the

[0033] As a preferred technical solution of the present application, in S502, the change rate calculation is specifically performed by comparing the change rate of the brightness value of consecutive frames to monitor the change of the ambient light. The change rate calculation formula is: ,in It's time The brightness value, is the time interval between two frames.

[0034] As the preferred technical solution of the present application, in S602, the frequency characteristics of light flickering can be identified through PSD analysis. The existence of a frequency peak indicates that the light has regular flickering behavior. The headlights and taillights of the vehicle will flicker regularly during driving, especially when braking or turning. These light signals can be used as an indication of traffic flow dynamics. Through the image sequence captured by the video surveillance system, the image processing technology and pattern recognition algorithm can be used to analyze the flickering of lights in the traffic flow. This regularity may be caused by the turning on of the vehicle's turn signal, the flashing of the emergency brake lights, etc. For example, when a large number of vehicles brake at the same time, the frequency of the taillights lighting up will increase significantly, which may be a sign of an obstacle or an accident ahead.

[0035] A bridge traffic monitoring system based on video images includes the following modules:

[0036] A video acquisition module, used to acquire video streams of the bridge area;

[0037] Data processing module, including pre-processing unit, vehicle detection and tracking unit, vibration analysis unit, illumination change analysis unit, and traffic condition assessment unit;

[0038] A storage module, used for storing various data in the system;

[0039] Network transmission module, used for remote monitoring and data uploading;

[0040] The power supply module is used to supply power to other modules.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] In the scheme of this application:

[0043] 1. By extracting information from the image and calculating it through an algorithm, the traffic congestion on the bridge can be judged through the bridge vibration frequency, ambient light changes, and vehicle light flashing patterns in the video image. Compared with the number of cars in the image, these monitoring methods have better monitoring effects, and are not judged by a single factor, but a comprehensive judgment, which reduces the possibility of misjudgment and has a good monitoring effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of a bridge traffic monitoring method based on video images provided in this application;

[0045] Figure 2A schematic diagram of a bridge traffic monitoring system based on video images provided in this application. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0047] As described in the background technology, most bridge traffic systems use video images to monitor traffic, and most of them are determined based on the number of cars in each frame of the image. However, this method cannot effectively determine whether the bridge is congested. There may be a large number of cars in the image, but the traffic is orderly and smooth. Therefore, this monitoring method does not have a good effect.

[0048] In order to solve this technical problem, the present invention provides a bridge traffic monitoring system, method and storage medium based on video images, which are applied to the field of traffic monitoring.

[0049] Specifically, please refer to Figure 1 The bridge traffic monitoring method based on video images specifically includes:

[0050] Follow these steps:

[0051] S100, video acquisition: HD cameras installed at key locations on the bridge continuously collect video streams of the bridge area;

[0052] S200, preprocessing: performing noise filtering, contrast enhancement and inter-frame registration processing on each frame of image;

[0053] S300, vehicle detection and tracking:

[0054] S301, Background modeling: Use the Gaussian mixture model (GMM) to model the background and distinguish between foreground and background. The GMM update formula can be expressed as ,in is the weight, is the learning rate, is the probability that the current pixel belongs to the background, is the time node;

[0055] S302, moving object segmentation: segmenting the moving vehicle from the background;

[0056] S400, vibration frequency extraction: Bridges vibrate naturally when vehicles pass through them. This vibration can reflect the traffic flow and vehicle type on the bridge. Heavier or more vehicles will cause the bridge to vibrate more. Sensors installed on the bridge (such as accelerometers) are used to measure the vibration signal of the bridge. Signal processing technology (such as Fourier transform) is then used to analyze the spectral characteristics of the vibration signal. Algorithms are used to identify the current traffic conditions, such as whether there is congestion or whether there are heavy vehicles.

[0057] Region of Interest Selection: Determine the key parts of the bridge structure (the key parts are the middle section of the bridge in the image, which is the part of the bridge structure most susceptible to vibration) as the region of interest (ROI) for analysis;

[0058] Displacement measurement: estimate the small displacement of ROI by pixel displacement between consecutive frames;

[0059] FFT conversion: Use fast Fourier transform (FFT) to convert time domain signals into frequency domain signals;

[0060] The FFT conversion formula is: ,in is a time series, is the corresponding spectrum, is the total number of samples, It is a complex exponential term, which represents the combination of sine waves and cosine waves at different frequencies. It is used to calculate each spectrum sample, analyze the FFT results, and find the main frequency components. These frequency components reflect the main vibration mode of the bridge. The frequency corresponding to the identified frequency peak is the vibration frequency of the bridge.

[0061] S500, analysis of ambient light change patterns; changes in ambient light can reflect the lighting conditions at different times of the day, which is somewhat related to traffic flow. For example, during peak hours in the morning and evening, the increase in vehicles may cause changes in lighting conditions in local areas. Using cameras to capture images and combining computer vision technology to analyze changes in light intensity can assist in determining traffic conditions. In addition, light sensors can also be used to directly monitor light intensity.

[0062] S600, vehicle light flickering law analysis;

[0063] S700, traffic condition assessment: The bridge vibration frequency, ambient light change pattern and vehicle light flashing pattern obtained by S400, S500 and S600 are compared with normal values ​​(it should be noted that the normal value is the average of the three data on the same bridge within a month without congestion, and needs to be adjusted according to actual conditions, so the specific value of the normal value is not disclosed). If any two or more data exceed 25% of the normal value, it is judged to be a congested state, otherwise it is a smooth state.

[0064] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0065] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions in the embodiments may be combined with each other.

[0066] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0067] Example 1, please refer to Figure 1 , a bridge traffic monitoring method based on video images, in S200, further comprising the following steps:

[0068] S201, noise filtering: Use adaptive Wiener filter to remove random noise in the image. The formula of Wiener filter is: ,in is the Fourier transform of the output image, is the Fourier transform of the input image, is the degradation function, is a constant, for complex conjugation;

[0069] S202, contrast enhancement: Use histogram equalization to improve image contrast. The cumulative distribution function (CDF) calculation formula of histogram equalization is: ,in Is grayscale The probability of

[0070] S203, inter-frame registration: using feature matching or optical flow method to ensure the correspondence between adjacent frames.

[0071] S500 further includes the following steps:

[0072] S501, brightness value statistics: calculate the average brightness value of the selected area in each frame;

[0073] S502, change rate calculation: Identify weather changes and vehicle passing by by comparing the change rates of brightness values ​​of consecutive frames.

[0074] S600 further includes the following steps:

[0075] S601, light brightness change detection: detect and record the brightness change of vehicle lights, determine the light area at the head or tail of the vehicle at night or in low light conditions, separate the light area from other backgrounds, and monitor the change of brightness value of the light area over time;

[0076] S602, PSD calculation: Analyze the frequency characteristics of light flicker through power spectral density (PSD), and use the periodogram method or Welch method to calculate the PSD of light brightness value. The PSD estimation formula of the periodogram method is: ,in is the brightness time series after FFT transformation, is the number of samples in the time series, The power spectral density PSD of the light brightness value is improved by the Welch method by windowing and overlapping the data segments, then estimating the periodogram for each segment, and finally taking the average value. The PSD is calculated by the periodogram method and the Welch method.

[0077] The displacement measurement in S400 is specifically performed by calculating the position difference of the same feature point between adjacent frames to obtain the displacement vector of the point. ,in It's time The location of the feature point.

[0078] In S501, a fixed ROI is selected to represent the lighting conditions of the entire scene, and the average brightness value within the ROI in each frame is calculated. ,in and is the height and width of the ROI, It's pixels The brightness value of the

[0079] In S502, the change rate calculation is specifically performed by comparing the change rates of the brightness values ​​of consecutive frames to monitor the change of the ambient light. The change rate calculation formula is: ,in It's time The brightness value, It is the time interval between two frames. According to the curve of brightness value changing over time, the influence of weather conditions (such as sunrise, sunset, cloudy day, etc.) on lighting can be judged. Sliding window averaging or other smoothing techniques can be used to reduce noise and highlight trends.

[0080] In S602, the frequency characteristics of light flickering can be identified through PSD analysis. The presence of frequency peaks indicates that the lights have regular flickering behavior. The headlights and taillights of vehicles will flicker regularly during driving, especially when braking or turning. These light signals can be used as an indication of traffic flow dynamics. Through the image sequences captured by the video surveillance system, image processing technology and pattern recognition algorithms can be used to analyze the flickering of lights in traffic. This regularity may be caused by the turning on of vehicle turn signals, the flashing of emergency brake lights, etc. For example, when a large number of vehicles brake at the same time, the frequency of taillights lighting up will increase significantly, which may be a sign of obstacles or accidents ahead.

[0081] Embodiment 2 further optimizes the bridge traffic monitoring system based on video images provided in Embodiment 1, and specifically includes the following modules:

[0082] A video acquisition module, used to acquire video streams of the bridge area;

[0083] Data processing module, including pre-processing unit, vehicle detection and tracking unit, vibration analysis unit, illumination change analysis unit, and traffic condition assessment unit;

[0084] A storage module, used for storing various data in the system;

[0085] Network transmission module, used for remote monitoring and data uploading;

[0086] The power supply module is used to supply power to other modules.

[0087] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection, an electrical connection, or communication with each other; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

Claims

1. A bridge traffic monitoring method based on video images, characterized in that: The following steps are involved: S100, video acquisition: HD cameras installed at key locations on the bridge continuously collect video streams of the bridge area; S200, preprocessing: performing noise filtering, contrast enhancement and inter-frame registration processing on each frame of image; S300, vehicle detection and tracking: S301, background modeling: using mixed Gaussian model GMM to perform background modeling and distinguish foreground from background; S302, moving object segmentation: segmenting the moving vehicle from the background; S400, vibration frequency extraction: Region of interest selection: determine the key parts of the bridge structure as the region of interest (ROI) for analysis; Displacement measurement: estimate the small displacement of the region of interest (ROI) by pixel displacement between consecutive frames; Fast Fourier Transform FFT conversion: Use Fast Fourier Transform FFT to convert time domain signals into frequency domain signals; The fast Fourier transform FFT conversion formula is: ,in is a time series, is the corresponding spectrum, is the total number of samples, It is a complex exponential term, which represents the combination of sine waves and cosine waves at different frequencies. It is used to calculate each spectrum sample, analyze the FFT results, and find the main frequency components. These frequency components reflect the main vibration mode of the bridge. The frequency corresponding to the identified frequency peak is the vibration frequency of the bridge. S500, analysis of ambient light change patterns; S600, vehicle light flickering law analysis; S500 further includes the following steps: S501, brightness value statistics: calculate the average brightness value of the selected area in each frame; S502, change rate calculation: identifying weather changes and vehicle passing by by comparing the change rates of brightness values ​​of consecutive frames; S600 further includes the following steps: S601, light brightness change detection: detect and record the brightness change of vehicle lights, determine the light area at the head or tail of the vehicle at night or in low light conditions, separate the light area from other backgrounds, and monitor the change of brightness value of the light area over time; S602, PSD calculation: The frequency characteristics of light flicker are analyzed by power spectral density PSD, and the PSD of light brightness value is calculated by periodogram method and Welch method. PSD is calculated by periodogram method and Welch method. In S602, the frequency characteristics of light flicker are identified by power spectral density PSD analysis. If there is a frequency peak, it indicates that the light has regular flickering behavior. S700, traffic condition assessment: The bridge vibration frequency, ambient light change pattern and vehicle light flickering pattern obtained from S400, S500 and S600 are compared with normal values. If any two or more of the data exceed 25% of the normal value, it is judged as a congested state, otherwise it is a smooth state. The normal value is the average of the three data of vibration frequency, vehicle light flickering pattern and ambient light change on the same bridge within one month without congestion.

2. A bridge traffic monitoring method based on video images according to claim 1, characterized in that: S200 also includes the following steps: S201, noise filtering: using an adaptive Wiener filter to remove random noise in the image; S202, contrast enhancement: using histogram equalization to improve image contrast; S203, inter-frame registration: using feature matching and optical flow method to ensure the correspondence between adjacent frames.

3. The bridge traffic monitoring method based on video images according to claim 2 is characterized in that: The displacement measurement in S400 is specifically performed by calculating the position difference of the same feature point between adjacent frames to obtain the displacement vector of the point. ,in It's time The position of the feature point.

4. The bridge traffic monitoring method based on video images according to claim 3 is characterized in that: In S501, a fixed region of interest ROI is selected to represent the lighting conditions of the entire scene, and the average brightness value in the region of interest ROI in each frame is calculated. ,in and is the height and width of the region of interest ROI, It's pixels The brightness value of the 5. The bridge traffic monitoring method based on video images according to claim 4 is characterized in that: In S502, the change rate calculation is specifically performed by comparing the change rates of the brightness values ​​of consecutive frames to monitor the change of the ambient light. The change rate calculation formula is: ,in It's time The brightness value, is the time interval between two frames.

6. A bridge traffic monitoring system based on video images, used to implement the bridge traffic monitoring method based on video images as claimed in claim 5, characterized in that: Includes the following modules: A video acquisition module, used to acquire video streams of the bridge area; Data processing module, including pre-processing unit, vehicle detection and tracking unit, vibration analysis unit, illumination change analysis unit, and traffic condition assessment unit; A storage module, used for storing various data in the system; Network transmission module, used for remote monitoring and data uploading; The power supply module is used to supply power to other modules.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images