A high-precision analysis method for video warning moments for intelligent driving evaluation

Abstract

The present invention discloses a high-precision analysis method for video warning moments for intelligent driving evaluation. The method uses a camera that supports hardware triggering, takes the Beidou second pulse signal as the reference signal, and uses FPGA frequency division to achieve high-frequency video warning signal acquisition. The video collected by the camera is converted into a picture frame sequence, and a higher-frequency picture frame sequence is obtained using a video differential algorithm. Template matching is performed on the new picture frame sequence, and the error in the warning time is evaluated. The present invention can be applied to intelligent driving evaluation projects that require video warning signal analysis. It has the function of obtaining high-sampling frequency sampling data by differentially using the sampling results of low-sampling frequency equipment without changing the hardware equipment, and the characteristics of reducing the warning time error.

Description

Technical Field

[0001] The present invention relates to the field of video warning signal evaluation, and in particular to a high-precision analysis method for video warning moments for intelligent driving evaluation. Background Art

[0002] Currently, video signal recognition technology based on template matching algorithms has become widely used. Video warning applications are increasingly being used in intelligent driving applications. Traditional video warning evaluation technologies, limited by the inherent frequency of video acquisition equipment, can only capture video at a fixed frequency, converting the video into a sequence of frame images with time intervals. The maximum detection time error is affected by the time interval between adjacent frames. Furthermore, because the video acquisition equipment is fixed and the sampling frequency cannot be changed, the error in warning signal recognition time is large.

[0003] This invention proposes a high-precision analysis method for video warning moments for intelligent driving assessment. This method uses industrial cameras to capture video warning information from the device under test. The captured video information is differentially processed, and template matching is performed using a mean absolute difference algorithm to achieve video signal warning. Traditional warning moment analysis methods are affected by the inherent frequency of the video and thus suffer from errors. This patent utilizes an interpolation method to reduce these errors without increasing hardware costs. Summary of the Invention

[0004] The purpose of this invention is to propose a high-precision analysis method for video warning moments for intelligent driving evaluation. First, a camera that supports hardware triggering is used, the Beidou second pulse is used as the reference signal, and FPGA frequency division is used to achieve high-frequency acquisition of video warning signals and obtain the accurate acquisition time. Second, the collected video warning images are processed using interpolation processing to obtain data information with a higher sampling frequency and the mean absolute difference algorithm is used for template matching to achieve warning signal recognition and warning moment analysis. The specific steps include:

[0005] Step 1: High-precision acquisition of video warning images based on hardware triggering

[0006] An industrial camera is used for video data acquisition. The camera's start signal is the first second pulse signal received, and the camera's trigger signal is the pulse signal after FPGA frequency division.

[0007] FPGA uses a high-stability quartz crystal as the clock source. The output clock signal of the high-stability quartz crystal is captured by the phase-locked loop in the microsecond timer. The FPGA accumulates the signal as the reference clock of the system and captures the rising edge of the Beidou PPS pulse signal as the start of the absolute whole second. When two PPS pulses are received consecutively, the UTC time corresponding to the previous PPS is added one second for time synchronization. At the same time, the internal accumulator counter of the FPGA is cleared to ensure that the signal edge of the internal second pulse of the FPGA is aligned with the signal edge of the Beidou PPS second pulse, thereby achieving time uniformity between the two and establishing a high-precision time base. At the same time, after receiving the rising edge of the PPS second pulse signal, the internal accumulator counter of the FPGA starts counting. When the count value reaches the set value, a trigger signal is sent to the industrial camera and the count value is cleared and counted again. After receiving the trigger signal, the camera starts to collect images. The collected images are all triggered by the same high-precision time base, and accurate timestamps can be obtained. For the FPGA clock frequency f F , the camera capture frame rate is f, the count value should be f F / f.

[0008] Step 2: High-precision analysis of warning moments based on image interpolation

[0009] Read the picture taken by the camera, the frame rate is known to be f, and convert it into a continuous image frame sequence S with a total number of frames L k (W,H), where 1≤k≤L, the recording start time is t0, then the time taken for each frame is The time of the k-th frame sequence is

[0010] Sub-step 1: Video interpolation processing

[0011] For a continuous image frame sequence S k (W,H) is interpolated to obtain a continuous image frame sequence S with a frame rate of f1 and a total number of frames L1 k1 (W,H), where 1≤k1≤L, the starting time is t0, and each frame takes The time of the k-th frame sequence is The specific steps include:

[0012] (1) Convert the original image frame into a grayscale image

[0013] Starting from k=1, select the kth and k+1th frames of the original image frame in turn. The grayscale value is solved by the R, G, B three-channel values ​​of the image frame to form a grayscale image G k*2-1 (W,H),G k*2+1 (W,H).

[0014] Gray=0.1140*R+0.5870*G+0.2989*B

[0015] (2)Interpolate to obtain the grayscale image of the new image frame

[0016] G k*2 (W, H) = (G k*2-1 (W, H) + G k*2+1 (W, H)) / 2

[0017] The original sequence of continuous image frames with a total number of frames L is transformed into a sequence of continuous grayscale images with a total number of frames 2*L - 1 through interpolation and grayscale processing.

[0018] Sub-step 2: Early warning image matching and recognition based on the sequence similarity detection matching algorithm

[0019] The traditional normalized product correlation algorithm uses an exhaustive method row by row and column by column for searching, which will result in a large number of repeated calculations for connected reference subgraphs with a large overlapping part, reducing the matching efficiency. To improve the real-time performance of template matching, the sequence similarity detection algorithm is used to perform fast matching on the searched graph.

[0020] The grayscale image of the early warning signal template to be matched is denoted as T(m, n), which is placed on the grayscale image G p (W, H) and translated. The area where the template covers the searched graph is called a subgraph. i, j are the coordinates of the lower left pixel point of the subgraph on the searched graph G p . The search range is: 1 ≤ i ≤ W - m + 1, 1 ≤ j ≤ H - n + 1, 1 ≤ p ≤ L * 2 - 1. The matching error at any point (u, v) in the subgraph is

[0021] e(u, v, i, j) = |G p (i + u, j + v) - T(u, v)|

[0022] Randomly extract n points (u1, v1), (u2, v2)…(u n , v n ) in the template and the subgraph, and calculate the partial error sum sequence of this subgraph as

[0023]

[0024] Compare e s (i, j, k) with the predefined sequence th(n). If e s (i, j, k) > th(k), then end the calculation here, record k, and calculate the error of the next subgraph. If e s (i, j, k) < th(k), then continue to calculate the error of the next extraction point in this subgraph until e s (i, j, k) > th(k) or k = n, and record the k value and the current subgraph coordinates (i, j).

[0025] Sub-step 3: Warning moment analysis

[0026] After calculating the k value of all sub-images, if there is a point where k=n, the system matches successfully and detects an alarm signal, then the current matching grayscale image serial number k is recorded. w , thus calculating the time when the warning message is issued as

[0027] Compared with existing technologies, this invention offers the following advantages: First, it uses hardware triggering to acquire warning signals, accurately capturing the image acquisition moment; second, it utilizes a differential method to obtain high-sampling-frequency images for image processing, reducing errors in warning moment analysis. This approach ensures the accuracy of video warning moment analysis through both hardware acquisition and signal processing, making it suitable for intelligent driving assessment projects requiring video warning signal analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flowchart of early warning signal identification.

[0029] Figure 2 It is the camera trigger signal waveform.

[0030] Figure 3 Video difference modification diagram, where the left and right images are two adjacent frames captured by the camera, and the middle image is the result of the difference between the left and right frames. DETAILED DESCRIPTION

[0031] Currently, video signal recognition technology based on template matching algorithms has become widely used. Video warning applications are increasingly being used in intelligent driving applications. Traditional video warning evaluation technologies, limited by the inherent frequency of video acquisition equipment, can only capture video at a fixed frequency, converting the video into a sequence of frame images with time intervals. The maximum detection time error is affected by the time interval between adjacent frames. Furthermore, because the video acquisition equipment is fixed and the sampling frequency cannot be changed, the error in warning signal recognition time is large.

[0032] The purpose of this invention is to propose a high-precision analysis method for video warning moments for intelligent driving evaluation. First, a camera that supports hardware triggering is used, Beidou second pulse is used as the reference signal, and FPGA frequency division is used to achieve high-frequency acquisition of video warning signals and obtain accurate acquisition time; secondly, the collected video warning images are processed by interpolation processing to obtain data information with a higher sampling frequency and the mean absolute difference algorithm is used for template matching to achieve recognition of warning signals and analysis of warning moments. The video warning moment analysis process is as follows: Figure 1 As shown, the specific steps include:

[0033] Step 1: High-precision acquisition of video warning images based on hardware triggering

[0034] An industrial camera is used for video data acquisition. The camera's start signal is the first second pulse signal received, and the camera's trigger signal is the pulse signal after FPGA frequency division.

[0035] FPGA uses a high-stability quartz crystal as the clock source. The output clock signal of the high-stability quartz crystal is captured by the phase-locked loop in the microsecond timer. The FPGA accumulates the signal as the reference clock of the system and captures the rising edge of the Beidou PPS pulse signal as the start of the absolute whole second. When two PPS pulses are received consecutively, the UTC time corresponding to the previous PPS is added one second for time synchronization. At the same time, the internal accumulator counter of the FPGA is cleared to ensure that the signal edge of the internal second pulse of the FPGA is aligned with the signal edge of the Beidou PPS second pulse, thereby achieving time uniformity between the two and establishing a high-precision time base. At the same time, after receiving the rising edge of the PPS second pulse signal, the internal accumulator counter of the FPGA starts counting. When the count value reaches the set value, a trigger signal is sent to the industrial camera and the count value is cleared and counted again. After receiving the trigger signal, the camera starts to collect images. The collected images are all triggered by the same high-precision time base, and accurate timestamps can be obtained. For the FPGA clock frequency f F , the camera capture frame rate is f, the count value should be f F / f. Trigger signal such as Figure 2 shown.

[0036] Step 2: High-precision analysis of warning moments based on image interpolation

[0037] Read the picture taken by the camera, the frame rate is known to be f, and convert it into a continuous image frame sequence S with a total number of frames L k (W,H), where 1≤k≤L, the recording start time is t0, then the time taken for each frame is The time of the k-th frame sequence is

[0038] Sub-step 1: Video interpolation processing

[0039] For a continuous image frame sequence S k (W,H) is interpolated to obtain a continuous image frame sequence S with a frame rate of f1 and a total number of frames L1 k1 (W,H), where 1≤k1≤L, the starting time is t0, and each frame takes The time of the k-th frame sequence is The specific steps include:

[0040] (1) Convert the original image frame into a grayscale image

[0041] Starting from k=1, select the kth and k+1th frames of the original image frame in turn. The grayscale value is solved by the R, G, B three-channel values ​​of the image frame to form a grayscale image G k*2-1 (W,H),G k*2+1 (W,H).

[0042] Gray=0.1140*R+0.5870*G+0.2989*B

[0043] (2) Interpolation solves the grayscale image of the new image frame

[0044] G k*2 (W,H)=(G k*2-1 (W,H)+G k*2+1 (W,H)) / 2

[0045] Through interpolation and grayscale processing, the original continuous image frame sequence with a total number of frames L is converted into a continuous grayscale image sequence with a total number of frames 2*L-1. The image interpolation effect is as follows Figure 3 shown.

[0046] Sub-step 2: Warning image matching and recognition based on sequence similarity detection and matching algorithm

[0047] Traditional normalized product correlation algorithms use an exhaustive row-by-column search method. This results in a large amount of repeated calculations for connected reference subgraphs with large overlaps, reducing matching efficiency. To improve the real-time performance of template matching, a sequence similarity detection algorithm is used to quickly match the searched graph.

[0048] The grayscale image of the warning signal template that needs to be matched is recorded as T(m,n), which is superimposed on the grayscale image G after differential processing. p (W,H) is translated, and the area of ​​the searched image covered by the template is called a sub-image. i,j is the pixel point at the lower left corner of the sub-image in the searched image G. p The search range is: 1≤i≤W-m+1, 1≤j≤H-n+1, ​​1≤p≤L*2-1. The matching error at any point (u,v) in the subgraph is

[0049] e(u,v,i,j)=|G p (i+u,j+v)-T(u,v)|

[0050] In the template and subgraph, we randomly select n points (u1, v1), (u2, v2)… (u n ,v n ), the partial error and sequence of the subgraph are calculated as

[0051]

[0052] e s(i, j, k) is compared with the pre-defined sequence th(n). If e s (i, j, k) > th(k), the calculation here ends, k is recorded, and the error of the next sub-graph is calculated. If e s (i, j, k) < th(k), the error of the next extraction point at this sub-graph is continuously calculated until e s (i, j, k) > th(k) or k = n, and the k value and the current sub-graph coordinates (i, j) are recorded.

[0053] Sub-step 3: Early warning time analysis

[0054] After calculating the k values at all sub-graphs, if there is a point where k = n, the system matches successfully, a warning signal is detected, and the serial number k of the current matched grayscale image is recorded w , thereby calculating the time when the warning message is sent as

[0055] Video early warning detection error analysis

[0056] The sampling frequency of the sampling device is f, and the number of image frames that can be collected per second is f. Without changing the hardware device, differential processing is performed on the sampled images, and the number of image frames obtained after processing is 2*f - 1. Then the sampling frequency corresponding to this result is 2*f. That is, sampling information at a higher sampling frequency is obtained without changing the hardware device.

[0057] Let the screen refresh rate of the measured terminal be 60Hz, and the presentation process of the warning information is about And because the template matching algorithm needs to reach the complete state of the warning information in terms of grayscale value to be accurately recognized. The sampling frequency of the video acquisition device is f, and the number of image frames that can be collected per second is f. Before the differential processing of the original image frame sequence, the time interval between adjacent image frames is The maximum time error Δt for video early warning signal recognition is

[0058]

[0059] Taking the sampling frequency of 40Hz as an example, substituting it into the above formula, the maximum time error can be obtained as 20.8ms, which does not meet the requirement that the maximum time measurement error is within 10ms. Therefore, differential processing is performed on the video to obtain a sampling result with a sampling frequency of 80Hz. The maximum time error is recalculated

[0060]

[0061] The maximum time error at this time is solved to be 8.3ms, which meets the requirement that the maximum time error is less than 10ms. Therefore, the technical solution of this scheme is reliable.

[0062] The high-precision analysis technology of video warning moments for intelligent driving evaluation proposed in this invention takes the Beidou second pulse signal as the benchmark and utilizes FPGA frequency division to realize high-frequency acquisition of warning signals and obtain precise acquisition moments. Secondly, the collected information is differentially processed to form new sampling information, and the new sampling data is used for template matching. Compared with traditional video signal warning technology, this technology has a smaller maximum time detection error, meets evaluation requirements, and can obtain sampling data with a higher sampling frequency without changing the hardware facilities of the sampling equipment.

Claims

1. A high-precision analysis method for video warning moments for intelligent driving evaluation, characterized by: First, a hardware-triggered camera is used, with the Beidou pulse per second signal as the reference signal. FPGA frequency division is used to achieve high-frequency acquisition of video warning signals and accurately obtain the acquisition time. Second, an interpolation method is used to process the collected video warning images to obtain data information with a higher sampling frequency. The mean absolute difference algorithm is used for template matching to achieve warning signal recognition and warning moment analysis. The specific steps include: Step 1: High-precision acquisition of video warning images based on hardware triggering An industrial camera is used for video data acquisition. The camera's start signal is the first second pulse signal received, and the camera's trigger signal is the pulse signal after FPGA frequency division. FPGA uses a high-stable quartz crystal as the clock source. The output clock signal of the high-stable quartz crystal is captured by the phase-locked loop in the microsecond timer. The FPGA accumulates the signal as the reference clock of the system and captures the rising edge of the Beidou PPS pulse signal as the start of the absolute whole second. When two PPS pulses are received consecutively, the UTC time corresponding to the previous PPS is added by one second for time synchronization. At the same time, the internal accumulator counter of the FPGA is cleared to ensure that the signal edge of the internal second pulse of the FPGA is aligned with the signal edge of the Beidou PPS second pulse, thereby achieving time uniformity between the two and establishing a high-precision time base. At the same time, the internal accumulator counter of the FPGA starts counting after receiving the rising edge of the PPS second pulse signal. When the count value reaches the set value, a trigger signal is sent to the industrial camera and the count value is cleared and counted again. After receiving the trigger signal, the camera starts to collect images. The collected images are all triggered by the same high-precision time base and can obtain accurate timestamps. For the FPGA clock frequency f F , the camera capture frame rate is f, the count value should be f F / f; Step 2: High-precision analysis of warning moments based on image interpolation Read the picture taken by the camera, the frame rate is known to be f, and convert it into a continuous image frame sequence S with a total number of frames L k (W,H), where 1≤k≤L, the recording start time is t0, then the time taken for each frame is The time of the k-th frame sequence is Sub-step 1: Video interpolation processing For a continuous image frame sequence S k (W,H) is interpolated to obtain a continuous image frame sequence S with a frame rate of f1 and a total number of frames L1 k1 (W,H), where 1≤k1≤L, the starting time is t0, and each frame takes The time of the k-th frame sequence is The specific steps include: (1) Convert the original image frame into a grayscale image Starting from k=1, select the kth and k+1th frames of the original image frame in turn; solve the grayscale value through the R, G, B three-channel values ​​of the image frame to form a grayscale image G k*2-1 (W,H),G k*2+1 (W,H); Gray=0.1140*R+0.5870*G+0.2989*B (2) Interpolation solves the grayscale image of the new image frame G k*2 (W,H)=(G k*2-1 (W,H)+G k*2+1 (W,H)) / 2 Through interpolation and grayscale processing, the original continuous image frame sequence with a total number of frames L is converted into a continuous grayscale image sequence with a total number of frames 2*L-1; Sub-step 2: Warning image matching and recognition based on sequence similarity detection and matching algorithm The traditional normalized product correlation algorithm uses an exhaustive search method row by row and column by column. For connected benchmark subgraphs with large overlaps, a large amount of repeated calculations will occur, reducing matching efficiency. To improve the real-time performance of template matching, a sequence similarity detection algorithm is used to quickly match the searched graph. The grayscale image of the warning signal template that needs to be matched is recorded as T(m,n), which is superimposed on the grayscale image G after differential processing. p (W,H) is translated, and the area of ​​the searched image covered by the template is called a sub-image. i,j is the pixel point at the lower left corner of the sub-image in the searched image G. p The search range is: 1≤i≤W-m+1, 1≤j≤H-n+1, ​​1≤p≤L*2-1; the matching error at any point (u,v) in the subgraph is e(u,v,i,j)=|G p (i+u,j+v)-T(u,v)| In the template and subgraph, we randomly select n points (u1, v1), (u2, v2)… (u n ,v n ), the partial error and sequence of the subgraph are calculated as Compare e s (i, j, k) with the predefined sequence th(n). If e s (i, j, k) > th(k), end the calculation here, record k, and calculate the error of the next subgraph. If e s (i, j, k) < th(k), continue to calculate the error of the next extraction point at this subgraph until e s (i, j, k) > th(k) or k = n, and record the value of k and the current subgraph coordinates (i, j); Sub-step 3: Warning moment analysis After calculating the k value of all sub-images, if there is a point where k=n, the system matches successfully and detects an alarm signal, then the current matching grayscale image serial number k is recorded. w , thus calculating the time when the warning message is issued as

Images