A method for identifying vehicle trajectories at night

Abstract

The present application discloses a method for identifying vehicle trajectories at night, comprising the following steps: preprocessing nighttime traffic video data to obtain multiple frames of a first nighttime image; filtering the first nighttime image to obtain a second nighttime image including a moving brightness region; constructing a parameter matrix for converting a two-dimensional matrix frame of daytime dual-headlights to three-dimensional coordinates; constructing multiple two-dimensional preselected frames containing bboxes in the second nighttime image based on the bbox size of the vehicle's daytime dual-headlights; converting the bbox center points within the two-dimensional preselected frames to frame center points in a three-dimensional coordinate system using a parameter matrix; and performing trajectory similarity matching on the obtained frame center points to eliminate ground reflection data. The present application has the following beneficial effects: it does not rely on a deep learning model, avoids the problem of feature extraction failure caused by insufficient nighttime illumination, can still operate stably in extreme scenarios such as high-speed driving in complete darkness, and significantly improves the reliability of the algorithm.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a method for identifying vehicle trajectories at night. Background Art

[0002] Vehicle trajectory recognition is a crucial component of intelligent transportation. Existing technologies primarily rely on visual recognition for this purpose. Currently, vision-based vehicle detection and tracking algorithms, such as YOLO and Deepsort, are primarily developed using deep learning. These algorithms accurately detect, classify, and continuously track vehicles in traffic videos or real-time camera footage. By analyzing vehicle trajectories, real-time traffic statistics can be generated, vehicle distribution in different areas can be analyzed, and average vehicle speeds can be estimated.

[0003] However, insufficient nighttime illumination can reduce camera image clarity, blurring key features like vehicle outlines and license plates, and even creating a "smearing" effect, hindering detection algorithms' ability to identify vehicle edges. This results in low accuracy in vehicle trajectory recognition using deep learning feature extraction in complete darkness, potentially rendering it impossible to identify vehicles. Summary of the Invention

[0004] One of the objectives of the present application is to provide a method for nighttime vehicle trajectory recognition that can address at least one of the drawbacks of the above-mentioned background technology.

[0005] To achieve at least one of the above purposes, the technical solution adopted in this application is: a method for identifying vehicle trajectories at night, comprising the following steps:

[0006] S100: Preprocessing nighttime traffic video data containing moving vehicles to obtain multiple frames of first nighttime images, each frame of the first nighttime image including multiple brightness regions preliminarily determined to be vehicle lights;

[0007] S200: filtering the first night image based on the optical flow trajectory prediction method and the IOU matching method to obtain a second night image including a moving brightness area;

[0008] S300: Constructing a parameter matrix for converting the two-dimensional matrix frame of the daytime running lights into three-dimensional coordinates based on the conversion relationship between the center point of the bbox of the vehicle's daytime running lights and the three-dimensional coordinate system;

[0009] S400: constructing multiple two-dimensional pre-selected boxes containing the bboxes in the second nighttime image based on the bbox size of the vehicle's daytime dual headlights; converting the bbox center points within the two-dimensional pre-selected boxes into box center points in a three-dimensional coordinate system using a parameter matrix;

[0010] S500: Performing trajectory similarity matching on the obtained frame center points to eliminate ground reflection data, thereby obtaining the movement trajectory of each vehicle at night.

[0011] Preferably, the acquisition of each frame of the first night image in step S100 includes the following process: extracting an RGB image from the traffic video data and converting it into an HSV image; performing V channel brightness comparison on the obtained HSV image to obtain a global mask and grayscale the original RGB image; based on the grayscale RGB image, obtaining multiple brightness areas through the connecting device principle, and applying rule constraints to the brightness areas to filter out clutter, thereby obtaining the first night image.

[0012] Preferably, the process of obtaining the brightness area through the connecting device principle is as follows: constructing a corresponding 8-neighborhood based on the coordinates of each pixel in the image; setting the connectivity conditions of the highlighted pixels, and traversing the 8-neighborhoods of all pixels through a breadth-first search algorithm to obtain the connected area of ​​the current highlighted pixel; counting the total number of pixels and the centroid of each connected area, and calculating the height and width of each connected area.

[0013] Preferably, the connectivity condition of the highlighted pixels is set to a pixel value equal to 255.

[0014] Preferably, the rule constraint for filtering out clutter in the brightness area is: when the total number of pixels in the connected area is within the range of (100, 10000) and the ratio of the height and width of the connected area is within the range of (0.5, 2), the connected area is preliminarily judged to be a car light.

[0015] Preferably, step S200 includes the following process: calculating the light fields of the two frames of the first night image before and after by the optical flow trajectory prediction method, and predicting the position of the brightness area of ​​the current frame; using the IOU matching method to compare the predicted highlight area of ​​the current frame with the actual detection area to calculate the superposition degree; updating the position according to the calculated superposition degree, and calculating the trajectory standard deviation of the centroid of each brightness area; retaining the brightness area whose trajectory standard deviation is greater than the set threshold, and deleting the remaining brightness areas to obtain the second night image.

[0016] Preferably, after the position prediction of the brightness area of ​​the current frame is completed by the optical flow trajectory prediction method, the error between the predicted result and the actual result is evaluated by the optical flow error EPE; the calculation formula of the optical flow error EPE is as follows:

[0017] ;

[0018] Among them, (u gt (x,y),v gt (x,y)) represents the real optical flow vector corresponding to the brightness area, (u pred (x,y),v pred(x,y)) represents the predicted optical flow vector corresponding to the brightness area.

[0019] Preferably, in step S300, the two-dimensional pixel coordinates (u, v) of the center point of the bbox and the three-dimensional coordinate system (X w , Y w , Z w ) is:

[0020] ;

[0021] Where d represents the depth of field parameter, K represents the camera intrinsic parameter matrix, and [R t] represents the camera extrinsic parameter.

[0022] Preferably, in step S400, the construction of the two-dimensional pre-selection frame includes the following process: constructing a detection frame according to the bbox size of the vehicle's daytime dual headlights; performing sliding window matching on the obtained detection frame on the second night image to obtain multiple pre-selection frames and retaining the pre-selection frame including the two headlights; among the retained pre-selection frames, screening out the optimal pre-selection frame with the shortest and equal distance from the bbox center point to the two headlight center points; deduplicating the screened pre-selection frames through the IOU matching method to obtain the required two-dimensional pre-selection frame.

[0023] Preferably, in step S500, the similarity matching is performed using the motion trajectory of the three-dimensional coordinates of the frame center points. If the x and y coordinates of the two frame center points maintain the same relative position and the z coordinates maintain the same height, they are considered to be the same vehicle trajectory; the similarity of the two frame center points is O 12 The calculation process is as follows:

[0024] ;

[0025] ;

[0026] ;

[0027] Among them, (x1(t), y1(t), z1(t)) and (x2(t), y2(t), z2(t)) represent the trajectory coordinates of the center points of the two boxes respectively; E xy (t) represents the relative position error in the horizontal direction, E z (t) represents the height direction error, s represents the scale factor, k represents the height scale factor, Represents the relative position error threshold, Indicates the height error threshold.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] (1) It does not rely on deep learning models, thus avoiding the problem of feature extraction failure caused by insufficient light at night. It can still run stably in extreme scenarios such as high-speed driving in complete darkness, significantly improving the reliability of the algorithm.

[0030] (2) Through a multi-stage interference filtering mechanism, interference sources such as irregular brightness areas, static backgrounds, side light clutter, and ground reflections are effectively eliminated, significantly improving vehicle detection accuracy and reducing false detections and missed detections.

[0031] (3) The optical flow method is combined with traditional algorithms such as coordinate transformation. Compared with the deep learning model, the computational complexity is lower, and it can quickly process video stream data to meet the needs of real-time vehicle flow detection. It is suitable for real-time statistics and analysis of traffic flow.

[0032] (4) It does not require the support of high-performance computing equipment and can be implemented based on ordinary cameras, which reduces hardware costs and facilitates large-scale promotion and application in areas such as traffic monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a schematic diagram of the overall workflow of this application.

[0034] Figure 2 This is a schematic diagram of an image of a frame in the nighttime traffic video data of this application.

[0035] Figure 3 For this application Figure 2 Schematic diagram of the corresponding first night image.

[0036] Figure 4 Schematic diagram of the previous frame image used in the optical flow trajectory prediction method in this application.

[0037] Figure 5 This is a schematic diagram of the next frame of image used in the optical flow trajectory prediction method in this application.

[0038] Figure 6 This is a schematic diagram of converting the bbox center point of the daytime running lights into three-dimensional coordinates in this application.

[0039] Figure 7 This is a schematic diagram of the case where there is only one optimal bbox corresponding to the image in this application.

[0040] Figure 8 This is a schematic diagram of the case where an image in this application corresponds to multiple optimal bboxes.

[0041] Figure 9 This is a schematic diagram of merging the midpoint trajectories of multiple bboxes of the same vehicle for this application. DETAILED DESCRIPTION

[0042] Below, the present application is further described in conjunction with specific implementation methods. It should be noted that, in the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.

[0043] In the description of this application, it should be noted that for directional words, such as the terms "center", "horizontal", "longitudinal", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and so on, indicating the orientation and position relationship are based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and cannot be understood as limiting the specific scope of protection of this application.

[0044] It should be noted that the terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0045] In this application, unless otherwise expressly specified or limited, terms such as "mounted," "connected," "connect," and "fixed" should be understood in a broad sense. For example, they may refer to connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on specific circumstances.

[0046] In this application, unless otherwise expressly specified or limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features being in contact not directly but through another feature between them. Moreover, a first feature being "above," "above," and "above" a second feature may include the first feature being directly above or obliquely above the second feature, or may simply mean that the first feature is higher in level than the second feature. A first feature being "below," "below," and "below" a second feature may include the first feature being directly below or obliquely below the second feature, or may simply mean that the first feature is lower in level than the second feature.

[0047] The terms "comprises" and "having" and any variations thereof in the specification and claims of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units expressly listed, but may include other steps or units not expressly listed or inherent to such process, method, product or apparatus.

[0048] One of the preferred embodiments of this application is as follows: Figure 1 As shown, a method for identifying vehicle tracks at night includes the following steps:

[0049] S100: Preprocessing nighttime traffic video data containing moving vehicles to obtain multiple frames of first nighttime images, each frame of the first nighttime image including multiple brightness areas preliminarily determined to be vehicle lights.

[0050] S200: Filtering the first night image based on the optical flow trajectory prediction method and the IOU matching method to obtain a second night image including a moving brightness area.

[0051] S300: Constructing a parameter matrix for converting the two-dimensional matrix frame of the daytime running lights into three-dimensional coordinates based on a conversion relationship between the center point of the bbox of the vehicle's daytime running lights and the three-dimensional coordinate system.

[0052] S400: Construct multiple two-dimensional pre-selected boxes containing the bboxes in the second night image according to the bbox size of the vehicle's daytime dual headlights; and convert the bbox center points in the two-dimensional pre-selected boxes into box center points in a three-dimensional coordinate system using a parameter matrix.

[0053] S500: Performing trajectory similarity matching on the obtained frame center points to eliminate ground reflection data, thereby obtaining the movement trajectory of each vehicle at night.

[0054] It can be understood that the technical solution of the present application systematically solves the problem of nighttime vehicle detection through phased and multi-dimensional processing. In the image preprocessing stage, after converting the RGB image into the HSV color space, a global mask is generated based on the brightness comparison of channel V to accurately identify irregular brightness areas, and they are filtered out through binarization processing to avoid background stray light interfering with vehicle feature extraction, effectively improving the problem of nighttime imaging blur. In the region differentiation stage, through the optical flow method trajectory prediction and IOU matching update technology, static and moving areas are dynamically distinguished based on pixel motion features, effectively avoiding the risk of "smear" or static objects on the roadside being misjudged as vehicles, and improving the accuracy of target detection. In the interference filtering stage, the daytime dual headlight information is used to construct a pre-selection box, combined with the two-dimensional to three-dimensional coordinate conversion and trajectory similarity matching, the side lights, clutter and ground reflection interference are eliminated from the spatial dimension and motion trajectory level to ensure that the final detection result only contains the real vehicle trajectory, overcoming the problem of deep learning failing due to feature ambiguity at night.

[0055] In simple terms, the technical solution of this application features high robustness, precise anti-interference capabilities, efficient real-time performance, and low deployment cost. Its high robustness is reflected in the fact that the entire recognition process does not rely on deep learning models, avoiding the problem of feature extraction failure caused by insufficient nighttime illumination. It can still operate stably in extreme scenarios such as completely dark highways, significantly improving algorithm reliability. Its precise anti-interference capability is reflected in the multi-stage interference filtering mechanism, which effectively eliminates interference sources such as irregular brightness areas, static backgrounds, sidelight clutter, and ground reflections, significantly improving vehicle detection accuracy and reducing false detections and missed detections. Its efficient real-time performance is reflected in the combination of optical flow and traditional algorithms such as coordinate transformation. Compared to deep learning models, it has lower computational complexity and can quickly process video stream data, meeting the needs of real-time vehicle flow detection and being suitable for real-time traffic flow statistics and analysis. Its low deployment cost is reflected in the fact that it does not require high-performance computing equipment and can be implemented based on ordinary cameras, reducing hardware costs and facilitating large-scale application in fields such as traffic monitoring. For ease of understanding, the various steps of this application are described in detail below.

[0056] In this embodiment, the acquisition of each frame of the first nighttime image in step S100 includes the following process: Figure 2 As shown in , RGB images are extracted from traffic video data and converted into HSV images; the V channel brightness of the obtained HSV images is compared to obtain a global mask and the original RGB image is grayed out. Figure 3 As shown, based on the grayscale RGB image, multiple brightness areas are obtained through the connecting vessel principle, and regular constraints are applied to the brightness areas to filter out clutter, thereby obtaining the first night image.

[0057] It is understandable that the specific process of converting an RGB image into an HSV image is a well-known technology for those skilled in the art. For ease of understanding, a brief description will be given below.

[0058] First, perform min / max calculations on the pixels of the current RGB image:

[0059] max= max(R, G, B); min= min(R, G, B).

[0060] Then, based on the obtained min / max, the brightness V, saturation S, and hue H corresponding to the HSV image are calculated according to the standard definition.

[0061] V=max.

[0062] .

[0063] .

[0064] The obtained hue H is limited to 0 to 360°, that is, H=H%360°.

[0065] Then the binary mask B0 is obtained by thresholding the brightness V.

[0066] .

[0067] When B0 = 255, it indicates the foreground; when B0 = 0, it indicates the background. The specific value of the brightness V threshold can be selected according to the actual needs of those skilled in the art, and V>200 is a preferred value.

[0068] Finally, the grayscale value G is calculated using the brightness V, saturation S, and an adjustable weight k: G = V × (1 - S × k); k ranges from 0 to 1. When k approaches 0, the grayscale image consists almost entirely of brightness V, and the influence of saturation S is minimized. When k approaches 1, highly saturated areas tend to be significantly darkened, highlighting low-saturation areas. After this transformation, the output first nighttime image consists only of highlights.

[0069] Specifically, the process for obtaining brightness regions using the connected component principle is as follows: Based on the coordinates (i, j) of each pixel p in the image, a corresponding 8-neighborhood N8(p) is constructed. Connectivity conditions for highlighted pixels are set, and the 8-neighborhood N8(p) of all pixels is traversed using the breadth-first search algorithm (BFS(N)) to obtain the connected regions of the currently highlighted pixel. The total number of pixels and the centroid of each connected region are counted, and the height and width of each connected region are calculated.

[0070] It can be understood that the 8-neighborhood N8(p) of each pixel point p is the adjacent pixel positions in the upper, lower, left, right, and four diagonal directions; it can be expressed as follows: .

[0071] It is also understandable that the connectivity condition for the highlighted pixels can be set to a pixel value equal to 255. When traversing pixels using the breadth-first search algorithm BFS(N), unmarked neighbor pixels are accessed each time until the entire connected area is traversed. This can be specifically expressed by the following expression:

[0072] .

[0073] Among them, N represents the pixel position, N=true means the position is connected, Indicates moving to the neighboring pixel position.

[0074] The total number of pixels in the connected area count, centroid x , centroid y ) and the calculation formulas for the height height and width width of the connected area are:

[0075] .

[0076] ; .

[0077] ; .

[0078] in, Indicates the connected area judged as "valid"; The i coordinate represents the brightness area, Indicates the j-coordinate of the brightness region.

[0079] Specifically, after obtaining multiple brightness regions through the connected device principle, a rule for filtering out clutter can be formulated for the brightness regions. Specifically, when the total number of pixels in the connected region is within the range of (100, 10000) and the ratio of the height to the width of the connected region is within the range of (0.5, 2), the connected region is preliminarily judged to be a car light, that is, Figure 3 The bright area shown in . For ease of understanding, it can be expressed by the following expression:

[0080] .

[0081] As can be seen from the above content, step S100 is mainly for image preprocessing. In order to address the interference of irregular brightness areas in the image, the global threshold and constraint rules are used to perform image binarization preprocessing, and the brightness areas in the background that are not related to the vehicle are initially filtered out, reducing the impact of the complex background on subsequent vehicle recognition, making the effective vehicle-related information in the image more prominent, and laying the foundation for accurate counting.

[0082] In this embodiment, step S200 includes the following process: Figure 4 and Figure 5 As shown in the figure, the light field of the first night image of the two frames before and after is calculated by the optical flow trajectory prediction method, and the position U of the brightness area of ​​the current frame is predicted. The IOU matching method is used to compare the highlight area A predicted in the current frame with the real detection area B to calculate the superposition IOU. The position is updated according to the calculated superposition IOU, and the trajectory standard deviation σ of the center of mass of each brightness area is calculated. x . The trajectory standard deviation σ x The brightness areas greater than the set threshold are retained, and the remaining brightness areas are deleted to obtain the second night image.

[0083] It is understandable that the optical flow trajectory prediction method constructs the basic equation by assuming brightness consistency: I x u+I y v=-I t ; Among them, I x and I y Represents the gradient of image brightness I in the x and y directions, , ;I t represents the rate of change of brightness I over time, ; u and v represent the speed of the pixel in the x and y directions, that is, the optical flow vector; u=dx / dt, v=dy / dt.

[0084] According to the basic equation of the optical flow trajectory prediction method, the expression of the predicted position U of the brightness area of ​​the current frame can be obtained as follows: .

[0085] Among them, (x n ,y n ) represents the centroid coordinates of the predicted position of the brightness area of ​​the current frame, (x0, y0) represents the centroid coordinates of the initial position, u i and v i Represents the speed of the pixel in the i-th frame in the x and y directions respectively.

[0086] The expression of superposition IOU is: .

[0087] Trajectory standard deviation σ x The calculation formula is: ; Among them, (x i ,y i ) represents the centroid coordinates of the ith brightness region, and Respectively represent the average x-axis and y-axis coordinates of the centroid of all brightness areas.

[0088] After completing the trajectory standard deviation σ x After calculation of the trajectory standard deviation σ x The expression for screening movable brightness areas by comparison with the set threshold is:

[0089] .

[0090] The specific value of the threshold can be selected based on the actual needs of those skilled in the art. The standard deviation of the trajectory of the center of mass of the brightness area can be used to determine whether the movement is stable. Specifically, if the light moves with the vehicle, the trajectory fluctuates greatly, and the corresponding standard deviation is high. If the light is stationary or slightly shaking, the standard deviation is low, and it is eliminated.

[0091] In this embodiment, after the position U of the brightness area of ​​the current frame is predicted by the optical flow trajectory prediction method, the error between the predicted result and the actual result is evaluated by the optical flow error EPE; the calculation formula of the optical flow error EPE is as follows:

[0092] .

[0093] Among them, (u gt (x,y),v gt (x,y)) represents the real optical flow vector corresponding to the brightness area, (u pred (x,y),v pred (x, y) represents the predicted optical flow vector corresponding to the brightness region. The optical flow error EPE(x, y) measures the error between the predicted optical flow and the true optical flow, that is, it is used to evaluate the accuracy of the optical flow trajectory prediction results. The smaller the error, the more reliable the prediction result.

[0094] In this embodiment, Figure 6 As shown, in step S300, the homography matrix is ​​obtained by using the projection equation of the center point of the daytime double headlight rectangular frame coordinates, that is, the center point of the bbox and the three-dimensional coordinate point. Because the center point is a single point, the depth of field can be converted from the two-dimensional coordinates. Among them, the two-dimensional pixel coordinates (u, v) of the bbox center point and the three-dimensional coordinate system (X w , Y w , Z w ) is:

[0095] .

[0096] .

[0097] .

[0098] Where d represents the depth of field parameter, i.e., the depth along the camera optical axis. The depth of field at the center of a single bbox is nonlinearly related to the coordinates (u, v). s represents the scaling function, and f represents a linear function. K represents the camera intrinsic parameter matrix, which can be given directly. [R t] represents the camera extrinsic parameters, which are obtained by substituting calibration data into the equation. Based on the above expression, the expression of the parameter matrix H can be obtained as: .

[0099] In this embodiment, Figure 7 and Figure 8 As shown, during step S400, the construction of a two-dimensional pre-selected box includes the following process: A detection box is constructed based on the bbox size of the vehicle's dual daytime headlights. The obtained detection box is subjected to sliding window matching on the second nighttime image to obtain multiple pre-selected boxes, and the pre-selected box containing the two headlights is retained. Among the retained pre-selected boxes, the optimal pre-selected box with the shortest and equal distance from the bbox center to the two headlight centers is selected, i.e., the optimal bbox. These selected pre-selected boxes are deduplicated using the IoU matching method to obtain the desired two-dimensional pre-selected box.

[0100] It should be noted that obtaining the second night image may only include a two-dimensional pre-selected frame, such as Figure 7 As shown, it may also include multiple two-dimensional pre-selected frames, such as shown in Figure 8. In the latter case, the multiple two-dimensional pre-selected frames may be the real double headlight frames of multiple vehicles, or they may be the real double headlight frames and the ground reflective frames. Therefore, it is necessary to extract the two-dimensional coordinates of the center point of each two-dimensional pre-selected frame in step S400, and then convert them into three-dimensional world coordinates for information dimensionality enhancement. Then, in step S500, the three-dimensional trajectories of the same vehicle are merged to eliminate the influence of the ground reflective frames to ensure the uniqueness of the vehicle trajectory.

[0101] In this embodiment, Figure 9 As shown, in step S500, the similarity matching is performed using the motion trajectory of the three-dimensional coordinates of the center points of the frames. If the x and y coordinates of the center points of the two frames keep the relative positions unchanged and the z coordinates keep the same height, they are considered to be the same vehicle trajectory. 12 The calculation process is as follows:

[0102] .

[0103] .

[0104] .

[0105] Among them, (x1(t), y1(t), z1(t)) and (x2(t), y2(t), z2(t)) represent the trajectory coordinates of the center points of the two boxes respectively; E xy (t) represents the relative position error in the horizontal direction, E z (t) represents the height direction error, s represents the scale factor, k represents the height scale factor, Represents the relative position error threshold, Indicates the height error threshold.

[0106] It is understandable that in the case where multiple two-dimensional pre-selection boxes may be the real double headlight frames of multiple vehicles, then as the vehicles travel, there will be relative position differences between the multiple vehicles, and then the trajectory coordinates of the center points of the corresponding multiple two-dimensional pre-selection boxes will change relative positions in the horizontal direction, based on which the current situation can be identified. For two two-dimensional pre-selection boxes with a small relative position error in the horizontal direction and a consistent vertical height, they can be considered to be the real double headlight frames and ground reflective frames of the same vehicle, and then the two two-dimensional pre-selection boxes can be merged to represent the motion trajectory of the same vehicle. For example Figure 9 As shown in the figure, there are three vehicles, namely vehicle 1, vehicle 2, and vehicle 3; each vehicle corresponds to two two-dimensional pre-selection boxes, and the two two-dimensional pre-selection boxes of each vehicle basically maintain the same horizontal relative position and the height tends to be consistent during the vehicle's driving process. Therefore, the center points of the two two-dimensional pre-selection boxes corresponding to each vehicle can be approximately merged to indicate the driving trajectory of the corresponding vehicle.

[0107] The above describes the basic principles, main features, and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-described embodiments. The above-described embodiments and the specification merely illustrate the principles of the present application. Various changes and improvements may be made to the present application without departing from the spirit and scope of the present application. These changes and improvements fall within the scope of the present application for which protection is sought. The scope of protection claimed by the present application is defined by the appended claims and their equivalents.

Claims

1. A method for identifying vehicle trajectories at night, characterized in that: The steps include: S100: Preprocessing nighttime traffic video data containing moving vehicles to obtain multiple frames of first nighttime images, each frame of the first nighttime image including multiple brightness regions preliminarily determined to be vehicle lights; S200: filtering the first night image based on the optical flow trajectory prediction method and the IOU matching method to obtain a second night image including a moving brightness area; S300: Constructing a parameter matrix for converting the two-dimensional matrix frame of the daytime running lights into three-dimensional coordinates based on the conversion relationship between the center point of the bbox of the vehicle's daytime running lights and the three-dimensional coordinate system; S400: constructing multiple two-dimensional pre-selected boxes containing the bboxes in the second nighttime image based on the bbox size of the vehicle's daytime dual headlights; converting the bbox center points within the two-dimensional pre-selected boxes into box center points in a three-dimensional coordinate system using a parameter matrix; S500: Performing trajectory similarity matching on the obtained frame center points to eliminate ground reflection data, thereby obtaining the movement trajectory of each vehicle at night.

2. The method for identifying vehicle tracks at night according to claim 1, wherein: The acquisition of each frame of the first nighttime image in step S100 includes the following process: Extract RGB images from traffic video data and convert them into HSV images; Compare the V channel brightness of the obtained HSV image to obtain a global mask and grayscale the original RGB image; Based on the grayscale RGB image, multiple brightness regions are obtained through the connecting vessel principle, and regular constraints are applied to the brightness regions to filter out clutter, thus obtaining the first night image.

3. The method for identifying vehicle tracks at night according to claim 2, wherein: The process of obtaining the brightness area through the connecting vessel principle is as follows: Construct the corresponding 8-neighborhood based on the coordinates of each pixel in the image; Set the connectivity condition of the highlighted pixel and traverse the 8-neighborhood of all pixels using the breadth-first search algorithm to obtain the connected area of ​​the current highlighted pixel; Count the total number of pixels and centroid of each connected region, and calculate the height and width of each connected region.

4. The method for identifying vehicle tracks at night according to claim 3, wherein: Set the connectivity condition for highlighted pixels to be equal to 255.

5. The method for identifying vehicle tracks at night according to claim 2, wherein: The rule constraints for filtering out clutter in the brightness area are: When the total number of pixels in the connected area is within the range of (100, 10000) and the ratio of the height to the width of the connected area is within the range of (0.5, 2), the connected area is preliminarily determined to be a car light.

6. The method for identifying vehicle tracks at night according to claim 1, wherein: Step S200 includes the following process: The light fields of the two previous and next frames of the first night image are calculated using the optical flow trajectory prediction method, and the position of the brightness area of ​​the current frame is predicted; The IOU matching method is used to compare the predicted highlight area of ​​the current frame with the actual detection area to calculate the degree of superposition; The position is updated according to the calculated superposition degree, and the trajectory standard deviation of the centroid of each brightness area is calculated; The brightness areas with trajectory standard deviation greater than the set threshold are retained, and the remaining brightness areas are deleted to obtain the second night image.

7. The method for identifying vehicle tracks at night according to claim 6, wherein: After the position prediction of the brightness area of ​​the current frame is completed by the optical flow trajectory prediction method, the error between the predicted result and the actual result is evaluated by the optical flow error EPE; the calculation formula of the optical flow error EPE is as follows: ； Among them, (u gt (x,y),v gt (x,y)) represents the real optical flow vector corresponding to the brightness area, (u pred (x,y),v pred (x,y)) represents the predicted optical flow vector corresponding to the brightness area.

8. The method for identifying vehicle tracks at night according to any one of claims 1 to 7, characterized in that: In step S300, the two-dimensional pixel coordinates (u, v) of the center point of the bbox are related to the three-dimensional coordinate system (X w , Y w , Z w ) is: ； Where d represents the depth of field parameter, K represents the camera intrinsic parameter matrix, and [R t] represents the camera extrinsic parameter.

9. The method for identifying vehicle tracks at night according to claim 1, wherein: In step S400, the construction of the two-dimensional pre-selection box includes the following process: Construct a detection frame based on the bbox size of the vehicle's daytime dual headlights; Perform sliding window matching on the second night image to obtain multiple pre-selected frames and retain the pre-selected frame including the two headlights; In the retained pre-selected boxes, select the optimal pre-selected box with the shortest and equal distances between the center point of the bbox and the center points of the two headlights; The filtered pre-selected boxes are deduplicated using the IOU matching method to obtain the required two-dimensional pre-selected boxes.

10. The method for identifying vehicle tracks at night according to claim 1, wherein: In step S500, the motion trajectory of the three-dimensional coordinates of the frame center points is used for familiarity matching. If the x and y coordinates of the two frame center points maintain the same relative position and the z coordinates maintain the same height, they are considered to be on the same vehicle trajectory; The similarity of the center points of the two boxes is O 12 The calculation process is as follows: ； ； ； Among them, (x1(t), y1(t), z1(t)) and (x2(t), y2(t), z2(t)) represent the trajectory coordinates of the center points of the two boxes respectively; E xy (t) represents the relative position error in the horizontal direction, E z (t) represents the height direction error, s represents the scale factor, k represents the height scale factor, represents the relative position error threshold, Indicates the height error threshold.

Images