Visual tracking prediction method and device for road traffic and storage medium

Abstract

The invention provides a visual tracking prediction method and device for road traffic and a storage medium, and relates to the technical field of trajectory prediction. The visual tracking prediction of the road traffic comprises the following steps: obtaining multiple frames of first observation images collected by a first camera before a target vehicle enters a blind area; obtaining road structure information of a road on which the target vehicle travels; searching a target driving track matched with the target vehicle from a plurality of preset driving tracks of a blind area in a historical database; and predicting the driving track of the target vehicle in the blind area according to the multiple frames of first observation images, the road structure information and the target driving track. According to the method, information of multiple dimensions, such as multiple frames of first observation images before the target vehicle enters the blind area, road structure information of a driving road, a matched target driving track in a historical database and the like, is fused, the actual traffic law is better met, and the driving track of the target vehicle in the blind area can be accurately and reliably predicted based on the information.

Description

Technical Field

[0001] This application relates to the field of trajectory prediction technology, and more specifically, to a visual tracking prediction method, device, and storage medium for highway traffic. Background Technology

[0002] In highways or urban tunnels, multiple cameras are typically used for segmented monitoring due to space constraints and varying lighting conditions. However, physical gaps exist between adjacent cameras, creating blind spots. When a vehicle enters a blind spot, the system cannot continuously acquire its image information, leading to interrupted target tracking and broken trajectory.

[0003] In related technologies, the trajectory of a vehicle in a blind spot is predicted based on the vehicle's historical trajectory data. However, this prediction method is difficult to adapt to complex traffic changes and has the problem of low prediction accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a visual tracking and prediction method, device, and storage medium for highway traffic, addressing the shortcomings of the prior art and solving the aforementioned technical problems.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a visual tracking and prediction method for highway traffic, the method comprising: Acquire multiple frames of first observation images captured by the first camera before the target vehicle enters the blind spot; Obtain the road structure information of the highway on which the target vehicle is traveling; From the multiple preset driving trajectories of the blind zone described in the historical database, find the target driving trajectory that matches the target vehicle; Based on the multiple first observation images, the road structure information, and the target driving trajectory, the driving trajectory of the target vehicle in the blind spot is predicted.

[0006] Optionally, predicting the trajectory of the target vehicle in the blind spot based on the multiple frames of the first observation image, the road structure information, and the target driving trajectory includes: The motion information of the target vehicle is extracted from the multiple frames of the first observation image; A prediction model is used to predict the driving trajectory in the blind spot based on the motion information, the road structure information, and the target driving trajectory.

[0007] Optionally, the step of employing a prediction model to predict the driving trajectory in the blind spot based on the motion information, the road structure information, and the target driving trajectory includes: Using the kinematic model, predictions are made based on the motion information to obtain motion information within the blind zone; Determine the road parameters within the blind zone based on road structure information; Using the aforementioned prediction model, multiple initial trajectory points are obtained by predicting the motion information within the blind zone and the target's driving trajectory. Using the aforementioned prediction model, multiple target trajectory points are selected from multiple initial trajectory points based on road parameters within the blind zone to obtain the driving trajectory within the blind zone.

[0008] Optionally, the step of employing a prediction model to predict the driving trajectory in the blind spot based on the motion information, the road structure information, and the target driving trajectory includes: Using the aforementioned prediction model, the driving trajectory in the blind zone is predicted based on the motion information, motion weight, target driving trajectory, trajectory weight, road structure information, and road weight within the blind zone.

[0009] Optionally, the method further includes: Based on the motion information and the motion information within the blind zone, calculate the motion residual components; Based on the number and matching degree of the target driving trajectories, determine the trajectory residual components; Based on the road parameters, preset road topology, and preset traffic rules within the blind zone, calculate the highway residual components; The confidence level of each target trajectory point in the driving trajectory in the blind zone is calculated based on the motion residual component, the trajectory residual component, and the road residual component.

[0010] Optionally, the method further includes: Acquire a second observation image of the target vehicle after it has driven out of the blind spot, captured by the second camera; The actual position of the target vehicle on the road after it leaves the blind spot is determined based on the second observation image; The kinematic model parameters are updated based on the actual position and the predicted end position indicated by the driving trajectory in the blind spot.

[0011] Optionally, updating the model parameters of the kinematic model based on the actual position and the predicted end position indicated by the driving trajectory in the blind spot includes: If the error between the actual position and the predicted end position is greater than a preset error, the model parameters of the kinematic model are updated based on the actual position.

[0012] Optionally, the method further includes: Based on the length of the blind spot, speed limit information, and minimum driving speed in the road structure information, calculate the passage time interval of the blind spot; Based on the passage time interval and the second observation image, the state of the target vehicle in the blind spot is predicted.

[0013] Secondly, embodiments of this application also provide a visual tracking and prediction device for highway traffic, comprising: a memory and a processor, wherein the memory stores a computer program executable by the processor, and the processor executes the computer program to implement the visual tracking and prediction method for highway traffic as described in any of the first aspects above.

[0014] Thirdly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when read and executed, implements the visual tracking and prediction method for highway traffic as described in any of the first aspects above.

[0015] Fourthly, embodiments of this application also provide a visual tracking and prediction device for highway traffic, the device comprising: The first acquisition module is used to acquire multiple frames of first observation images captured by the first camera before the target vehicle enters the blind spot; and to acquire road structure information of the road on which the target vehicle is traveling. The search module is used to search for a target driving trajectory that matches the target vehicle from multiple preset driving trajectories in the blind area of ​​the historical database; The prediction module is used to predict the driving trajectory of the target vehicle in the blind spot based on the multiple frames of the first observation image, the road structure information, and the target driving trajectory.

[0016] Optionally, the prediction module is specifically used to extract the motion information of the target vehicle based on the multiple frames of the first observation image; and to use a prediction model to predict the driving trajectory in the blind spot based on the motion information, the road structure information, and the target driving trajectory.

[0017] Optionally, the prediction module is specifically used to use the kinematic model to predict motion information within the blind zone based on the motion information; determine road parameters within the blind zone based on road structure information; use the prediction model to predict multiple initial trajectory points based on the motion information within the blind zone and the target driving trajectory; and use the prediction model to select multiple target trajectory points from the multiple initial trajectory points based on the road parameters within the blind zone to obtain the driving trajectory within the blind zone.

[0018] Optionally, the prediction module is specifically used to use the prediction model to predict the driving trajectory in the blind zone based on the motion information, motion weight, target driving trajectory, trajectory weight, road structure information, and road weight within the blind zone.

[0019] Optionally, the device further includes: The calculation module is used to calculate motion residual components based on the motion information and motion information within the blind zone; determine trajectory residual components based on the number and matching degree of the target driving trajectories; calculate highway residual components based on road parameters, preset road topology, and preset traffic rules within the blind zone; and calculate the confidence level of each target trajectory point in the driving trajectory within the blind zone based on the motion residual components, the trajectory residual components, and the highway residual components.

[0020] Optionally, the device further includes: The second acquisition module is used to acquire a second observation image of the target vehicle after it leaves the blind spot, captured by the second camera. The determination module is used to determine the actual position of the target vehicle on the road after it leaves the blind spot, based on the second observation image; An update module is used to update the model parameters of the kinematic model based on the actual position and the predicted end position indicated by the driving trajectory in the blind spot.

[0021] Optionally, the update module is specifically used to update the model parameters of the kinematic model based on the actual position if the error between the actual position and the predicted final position is greater than a preset error.

[0022] Optionally, the device further includes: The calculation module is also used to calculate the passage time interval of the blind spot based on the length of the blind spot, speed limit information and minimum driving speed in the road structure information; The prediction module is further configured to predict the state of the target vehicle in the blind spot based on the passage time interval and the second observation image.

[0023] The beneficial effects of this application are as follows: This application provides a visual tracking and prediction method for highway traffic. The method includes: acquiring multiple frames of first observation images captured by a first camera before a target vehicle enters a blind spot; acquiring road structure information of the road on which the target vehicle is traveling; searching for a target driving trajectory matching the target vehicle from multiple preset driving trajectories in the blind spot from a historical database; and predicting the driving trajectory of the target vehicle in the blind spot based on the multiple frames of first observation images, road structure information, and the target driving trajectory. By fusing multiple frames of first observation images of the target vehicle before entering the blind spot, road structure information of the road on which it is traveling, and the target driving trajectory matching the target vehicle in the historical database, the method better conforms to actual traffic patterns. Based on this information, accurate and reliable prediction of the driving trajectory of the target vehicle in the blind spot can be achieved. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating a visual tracking prediction method for highway traffic provided in this application embodiment. Figure 1 ; Figure 2 A flowchart illustrating a visual tracking prediction method for highway traffic provided in this application embodiment. Figure 2 ; Figure 3 A flowchart illustrating a visual tracking prediction method for highway traffic provided in this application embodiment. Figure 3 ; Figure 4 A flowchart illustrating a visual tracking prediction method for highway traffic provided in this application embodiment. Figure 4 ; Figure 5 A flowchart illustrating a visual tracking prediction method for highway traffic provided in this application embodiment. Figure 5 ; Figure 6 A schematic diagram of blind spots on a straight road provided in an embodiment of this application; Figure 7 A schematic diagram of the blind spot of a curved road provided in an embodiment of this application; Figure 8 A flowchart illustrating a visual tracking prediction method for highway traffic provided in this application embodiment. Figure 6 ; Figure 9A schematic diagram of the structure of a visual tracking and prediction device for highway traffic provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of a visual tracking and prediction device for highway traffic provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0027] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0028] In the description of this application, it should be noted that if the terms "upper", "lower", etc. appear to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship that the product of this application is usually placed in, it is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0029] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0031] This application provides a visual tracking prediction method for highway traffic, which is applied to a visual tracking prediction device for highway traffic. The device can be a server or a terminal device, and the terminal device can be any of the following: a computer, a laptop, a tablet computer, or a smartphone.

[0032] The following is an explanation of a visual tracking prediction method for highway traffic provided by an embodiment of this application.

[0033] Figure 1 A flowchart illustrating a visual tracking prediction method for highway traffic provided in this application embodiment. Figure 1 ,like Figure 1 As shown, the method includes: S101. Acquire multiple first observation images captured by the first camera before the target vehicle enters the blind spot.

[0034] In this embodiment of the application, the target vehicle is driving on a highway, and multiple cameras are set up in sections on the highway. There are blind spots between adjacent cameras. Before the target vehicle enters the blind spot, the first camera captures multiple consecutive first observation images of the target vehicle.

[0035] The system includes multiple cameras, including a highway entrance camera. When a target vehicle enters the monitoring area of ​​the highway entrance camera, a preset recognition algorithm is used to detect the target vehicle and bind its ID (Identity Document) to enable trajectory prediction and tracking within blind spots. Optionally, the preset recognition algorithm can be the YOLO algorithm (a real-time target detection algorithm).

[0036] S102. Obtain road structure information of the highway on which the target vehicle is traveling.

[0037] In practical applications, the road traveled by the target vehicle can be a tunnel.

[0038] In some implementations, road structure information such as blind spot length, road curvature, slope, and speed limit between adjacent cameras in the tunnel in which the target vehicle is traveling is obtained.

[0039] S103. From the multiple preset driving trajectories in the blind area of ​​the historical database, find the target driving trajectory that matches the target vehicle.

[0040] In some implementations, a trajectory matching model is used, based on the DTW (Dynamic Time Warping) algorithm, to find the target driving trajectory that matches the target vehicle from multiple preset driving trajectories in the blind spots of the historical database.

[0041] Specifically, the system queries the historical database for target vehicle driving trajectories that are the same model, within similar time periods, and under the same weather conditions. These target driving trajectories represent typical driving trajectories for the target vehicle. The multiple preset driving trajectories in the historical database were obtained through offline collection and statistical analysis.

[0042] S104. Based on multiple frames of the first observation image, road structure information, and the target driving trajectory, predict the driving trajectory of the target vehicle in the blind spot.

[0043] In this embodiment, based on multiple frames of first observation images, road structure information, and the target driving trajectory, the position information of the target vehicle at each time point in the blind spot is predicted. The driving trajectory of the target vehicle in the blind spot includes the position information of the target vehicle at multiple time points in the blind spot. The multiple time points have a sequential order.

[0044] It should be noted that the location information of the target vehicle at each point in time in the blind spot can be the longitude and latitude information of the target vehicle per second in the blind spot.

[0045] In summary, this application provides a visual tracking and prediction method for highway traffic. The method includes: acquiring multiple frames of first observation images captured by a first camera before a target vehicle enters a blind spot; acquiring road structure information of the road the target vehicle is traveling on; searching for a target driving trajectory matching the target vehicle from multiple preset driving trajectories in the blind spot from a historical database; and predicting the target vehicle's driving trajectory in the blind spot based on the multiple frames of first observation images, road structure information, and the target driving trajectory. By fusing multiple frames of first observation images before the target vehicle enters the blind spot, road structure information of the road it is traveling on, and the target driving trajectory matching the target trajectory in the historical database, the method better reflects actual traffic patterns. Based on this information, accurate and reliable prediction of the target vehicle's driving trajectory in the blind spot can be achieved.

[0046] Furthermore, the visual tracking prediction method for highway traffic provided in this application embodiment 2. improves prediction accuracy and robustness, and is especially suitable for scenarios with long blind spots or fluctuating traffic flow.

[0047] Optionally, Figure 2 A flowchart illustrating a visual tracking prediction method for highway traffic provided in this application embodiment. Figure 2 ,like Figure 2 As shown, the process of predicting the target vehicle's trajectory in the blind spot based on multiple frames of the first observation image, road structure information, and the target's driving trajectory in S104 above may include: S201. Extract the motion information of the target vehicle based on multiple frames of the first observation image.

[0048] In some implementations, motion information such as the target vehicle's position, speed, acceleration, and direction of travel is extracted from multiple frames of the first observation image.

[0049] S202. Using a prediction model, based on motion information, road structure information, and the target driving trajectory, predict the driving trajectory in the blind spot.

[0050] It is worth noting that the prediction model is based on a hybrid strategy, which integrates current traffic conditions, motion information, road structure information and target driving trajectory to obtain the driving trajectory in the prediction blind spot.

[0051] One method is to determine the current traffic status by using multiple cameras installed in sections along the highway.

[0052] Optionally, Figure 3 A flowchart illustrating a visual tracking prediction method for highway traffic provided in this application embodiment. Figure 3 ,like Figure 3 As shown, the process in S202 above, which uses a prediction model to predict the trajectory in the blind spot based on motion information, road structure information, and the target's trajectory, may include: S301. Using a kinematic model, motion information is used to predict motion information within the blind zone.

[0053] In some implementations, extended Kalman filtering and CV / CA models are used in the kinematic model to predict the motion information within the blind zone based on the target vehicle's position, velocity, acceleration, and direction of travel. The motion information within the blind zone can include the target vehicle's predicted position, predicted velocity, predicted acceleration, and predicted direction of travel.

[0054] It should be noted that the CV model is a constant velocity model, and the CA model is a constant acceleration model.

[0055] S302. Determine the road parameters within the blind spot based on the road structure information.

[0056] In this embodiment, the road structure information includes the road curvature, gradient, speed limit, and length of at least one blind spot of the entire highway traveled by the target vehicle. The entry of the target vehicle into a blind spot can be considered the current blind spot. Based on the location of this current blind spot, road parameters such as road length, curvature, gradient, and speed limit within the blind spot are determined from the road structure information.

[0057] S303. A prediction model is used to predict multiple initial trajectory points based on motion information within the blind spot and the target's driving trajectory.

[0058] In some implementations, the kinematic model outputs motion information within the blind zone, and the trajectory matching model outputs the target driving trajectory; the prediction model performs fusion prediction based on the motion information within the blind zone and the target driving trajectory to obtain the initial position information of the target vehicle at each time point in the blind zone.

[0059] Among them, multiple initial trajectory points are the initial position information of the target vehicle at multiple time points in the blind spot. The initial position information includes initial longitude information and initial latitude information.

[0060] S304. Using a prediction model, based on road parameters within the blind spot, multiple target trajectory points are selected from multiple initial trajectory points to obtain the driving trajectory within the blind spot.

[0061] In this embodiment, a prediction model is used to determine the area where the road is located in the blind spot based on road parameters. It then determines whether multiple initial trajectory points are within the area where the road is located in the blind spot. If an initial trajectory point is within this area, it is used as a target trajectory point; otherwise, it is removed. This process yields multiple target trajectory points, forming the driving trajectory within the blind spot.

[0062] It is important to note that multiple initial trajectory points refer to initial trajectory points at multiple time points, and multiple target trajectory points are obtained by filtering based on the initial trajectory points at multiple time points. Therefore, multiple target trajectory points are target trajectory points at multiple time points. These target trajectory points at multiple time points can contain target longitude and latitude information at a given second.

[0063] It's worth noting that set constraints for blind zone roads can be introduced based on road parameters within the blind zone. Examples include: maximum steering angle and minimum turning radius. Based on these blind zone road parameters, unreasonable paths can be filtered from multiple initial trajectory points to obtain multiple target trajectory points.

[0064] Optionally, the process of using a prediction model in S202 above to predict the driving trajectory in the blind spot based on motion information, road structure information, and the target driving trajectory may include: A predictive model is used to predict the driving trajectory in the blind spot based on motion information, motion weight, target driving trajectory, trajectory weight, road structure information, and road weight.

[0065] Among them, the prediction model can also be called the adaptive hybrid prediction model.

[0066] In some implementations, a prediction model is used to predict the driving trajectory in the blind spot by dynamically weighting and fusing motion information, motion weight, target driving trajectory, trajectory weight, road structure information, and road weight through a hybrid strategy.

[0067] In this embodiment, the prediction model's hybrid strategy employs a dynamic weighted fusion mechanism, which can be expressed as:

[0068] in, For kinematic models, For trajectory matching models, These are the road parameters within the blind spot.

[0069] It should be noted that, , , Adjustments can be made based on current traffic conditions, including vehicle speed, rate of change of acceleration, and traffic density. For example, when traveling at a constant speed on a high speed... Dominant, will Increased weight; low-speed complex road sections Weighting increased. , , The sum of is 1.

[0070] In summary, the adaptive hybrid prediction model, which integrates kinematic model, trajectory matching model and road structure information, and achieves optimal fusion through dynamic weighting mechanism, improves prediction accuracy.

[0071] Optionally, Figure 4 A flowchart illustrating a visual tracking prediction method for highway traffic provided in this application embodiment. Figure 4 ,like Figure 4 As shown, the method also includes: S401. Calculate the motion residual components based on the motion information and the motion information within the blind zone.

[0072] S402. Determine the trajectory residual components based on the number of target driving trajectories and the degree of matching.

[0073] Among them, the degree of matching of the target driving trajectory refers to the degree of matching between the target driving trajectory and the target vehicle.

[0074] S403. Calculate the highway residual components based on the road parameters, preset road topology, and preset traffic rules within the blind zone.

[0075] In practical applications, the area where the road is located in the blind zone is determined based on the road parameters within the blind zone. It is then determined whether the area where the road is located in the blind zone conforms to the preset road topology and preset traffic rules, thus obtaining the highway residual components.

[0076] S404. Calculate the confidence level of each target trajectory point in the driving trajectory in the blind spot based on the motion residual component, trajectory residual component and road residual component.

[0077] Among them, the driving trajectory in the blind spot is a predicted trajectory with an execution score.

[0078] In some implementations, a confidence assessment mechanism is used to calculate the confidence of each trajectory point in the driving trajectory in the blind spot based on the motion residual component, preset motion coefficient, trajectory residual component, preset trajectory coefficient, highway residual component, and preset highway coefficient.

[0079] It should be noted that the preset motion coefficient, preset trajectory coefficient, and preset road coefficient are all adjustable coefficients.

[0080] In the embodiments of this application, the confidence level of each trajectory point can be calculated using the following formula:

[0081] in, For the motion residual components, For trajectory residual components, For highway residual components, To preset motion coefficients, For preset trajectory coefficients, This is the preset highway coefficient.

[0082] In summary, the embodiments of this application introduce a confidence assessment mechanism, which integrates motion residual components, trajectory residual components and road residual components to output a predicted trajectory with a confidence score, thereby enhancing interpretability and security.

[0083] Optionally, Figure 5 A flowchart illustrating a visual tracking prediction method for highway traffic provided in this application embodiment. Figure 5 ,like Figure 5 As shown, the method may further include: S501. Acquire the second observation image of the target vehicle after it leaves the blind spot, captured by the second camera.

[0084] The first camera is the upstream camera, and the second camera is the downstream camera.

[0085] S502. Determine the actual position of the target vehicle on the road after it leaves the blind spot based on the second observation image.

[0086] It should be noted that the actual position of the target vehicle on the road after it leaves the blind spot can be its actual longitude and latitude.

[0087] S503. Update the model parameters of the kinematic model based on the actual position and the predicted end position indicated by the driving trajectory in the blind spot.

[0088] The predicted position at the end of the driving trajectory within the area essentially indicates the predicted position after the blind spot.

[0089] In addition, the model parameters of the trajectory matching model can be updated.

[0090] In some implementations, the error between the actual position and the predicted final position is calculated, and based on this error and a preset error, it is determined whether to update the model parameters of the kinematic model and the trajectory matching model.

[0091] Optionally, the process in S503 above of updating the model parameters of the kinematic model and the trajectory matching model based on the actual position and the predicted end position indicated by the driving trajectory in the blind spot may include: If the error between the actual position and the predicted position is greater than the preset error, the kinematic model parameters are updated based on the actual position.

[0092] In some implementations, if the error between the actual position and the predicted final position is greater than a preset error, it indicates that the predicted driving trajectory in the blind spot is not accurate enough. The model parameters of the kinematic model and the trajectory matching model are updated according to the actual position, so that the model parameters of the kinematic model and the trajectory matching model are learned based on the accurate actual position, which facilitates the improvement of the prediction accuracy of the kinematic model and the trajectory matching model.

[0093] Figure 6 This application provides a schematic diagram of blind spots on a straight road. Figure 7 A schematic diagram of a blind spot on a curved road provided in an embodiment of this application is shown below. Figure 6 and Figure 7 As shown, the blind spot area between two adjacent cameras differs for straight roads and curved roads. In this embodiment, the driving trajectory in the blind spot is predicted by combining road structure information, which makes the predicted driving trajectory in the blind spot more accurate.

[0094] Optionally, Figure 8 A flowchart illustrating a visual tracking prediction method for highway traffic provided in this application embodiment. Figure 6 ,like Figure 8 As shown, the method may further include: S601. Calculate the blind spot travel time interval based on the blind spot length, speed limit information, and minimum driving speed in the road structure information.

[0095] In some implementations, the longest travel time is calculated based on the length of the blind spot and the minimum driving speed. Calculate the shortest travel time based on the blind spot length and speed limit information. .

[0096] in, Where L is the blind zone length, This is the speed limit information; , This represents the minimum driving speed. The travel time interval can be expressed as... .

[0097] S602. Based on the passage time interval and the second observation image, predict the status of the target vehicle in the blind spot.

[0098] In this embodiment of the application, the travel time of the target vehicle from disappearing in the first observation image to reappearing in the second observation image is counted. If the travel time is less than If the speed is greater than 100 km / h, it means the target vehicle is speeding; if it is greater than 10 If the target vehicle is still not detected among the target vehicles, it indicates that the target vehicle has been involved in an accident or has stopped.

[0099] In conclusion, by setting reasonable passage time intervals, it can be used to detect abnormal behavior of target vehicles.

[0100] This application provides a visual tracking and prediction method for highway traffic. By fusing multi-dimensional information to construct a hybrid prediction model, it significantly improves the accuracy and robustness of vehicle position prediction in tunnel blind spots. Compared with existing technologies, this application effectively reduces the average prediction error in blind spots and can effectively handle complex scenarios such as sudden deceleration and lane keeping. Moreover, the introduction of a confidence assessment mechanism enables the system to have self-diagnostic capabilities, helping upper-layer applications to judge the usability of prediction results. It achieves highly reliable vehicle trajectory completion under purely visual conditions without the need for additional radar or geomagnetic sensors. It is suitable for upgrading existing monitoring systems, enabling low-cost, highly reliable target tracking under purely visual conditions without the need for additional radar or other hardware.

[0101] The following describes the visual tracking prediction device, electronic equipment, and storage medium for highway traffic used to implement the visual tracking prediction method for highway traffic provided in this application. For the specific implementation process and technical effects, please refer to the relevant content of the visual tracking prediction method for highway traffic mentioned above, which will not be repeated below.

[0102] Figure 9 A schematic diagram of the structure of a visual tracking and prediction device for highway traffic provided in an embodiment of this application is shown below. Figure 9 As shown, the device includes: The first acquisition module 101 is used to acquire multiple frames of first observation images captured by the first camera before the target vehicle enters the blind spot; and to acquire road structure information of the road on which the target vehicle is traveling. The search module 102 is used to search for a target driving trajectory that matches the target vehicle from multiple preset driving trajectories in the blind area of ​​the historical database; The prediction module 103 is used to predict the driving trajectory of the target vehicle in the blind spot based on the multi-frame first observation image, the road structure information, and the target driving trajectory.

[0103] Optionally, the prediction module 103 is specifically used to extract the motion information of the target vehicle based on the multiple frames of the first observation image; and to use a prediction model to predict the driving trajectory in the blind spot based on the motion information, the road structure information, and the target driving trajectory.

[0104] Optionally, the prediction module 103 is specifically used to use the kinematic model to predict motion information within the blind zone based on the motion information; determine road parameters within the blind zone based on road structure information; use the prediction model to predict multiple initial trajectory points based on the motion information within the blind zone and the target driving trajectory; and use the prediction model to select multiple target trajectory points from the multiple initial trajectory points based on the road parameters within the blind zone to obtain the driving trajectory within the blind zone.

[0105] Optionally, the prediction module 103 is specifically used to use the prediction model to predict the driving trajectory in the blind spot based on the motion information, motion weight, target driving trajectory, trajectory weight, road structure information, and road weight within the blind spot.

[0106] Optionally, the device further includes: The calculation module is used to calculate motion residual components based on the motion information and motion information within the blind zone; determine trajectory residual components based on the number and matching degree of the target driving trajectories; calculate highway residual components based on road parameters, preset road topology, and preset traffic rules within the blind zone; and calculate the confidence level of each target trajectory point in the driving trajectory within the blind zone based on the motion residual components, the trajectory residual components, and the highway residual components.

[0107] Optionally, the device further includes: The second acquisition module is used to acquire a second observation image of the target vehicle after it leaves the blind spot, captured by the second camera. The determination module is used to determine the actual position of the target vehicle on the road after it leaves the blind spot, based on the second observation image; An update module is used to update the model parameters of the kinematic model based on the actual position and the predicted end position indicated by the driving trajectory in the blind spot.

[0108] Optionally, the update module is specifically used to update the model parameters of the kinematic model based on the actual position if the error between the actual position and the predicted final position is greater than a preset error.

[0109] Optionally, the device further includes: The calculation module is also used to calculate the passage time interval of the blind spot based on the length of the blind spot, speed limit information and minimum driving speed in the road structure information; The prediction module 103 is further configured to predict the state of the target vehicle in the blind spot based on the passage time interval and the second observation image.

[0110] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0111] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more digital signal processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0112] Figure 10 This is a schematic diagram of the structure of a visual tracking and prediction device for highway traffic provided in an embodiment of this application, as shown below. Figure 10 As shown, the device includes: processor 201 and memory 202.

[0113] The memory 202 is used to store programs, and the processor 201 calls the programs stored in the memory 202 to execute the above method embodiments. The specific implementation and technical effects are similar, and will not be described in detail here.

[0114] Optionally, this application also provides a program product, such as a computer-readable storage medium, including a program that, when executed by a processor, performs the above-described method embodiments.

[0115] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0116] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0117] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0118] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0119] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A visual tracking and prediction method for highway traffic, characterized in that, The method includes: Acquire multiple frames of first observation images captured by the first camera before the target vehicle enters the blind spot; Obtain the road structure information of the highway on which the target vehicle is traveling; From the multiple preset driving trajectories of the blind zone described in the historical database, find the target driving trajectory that matches the target vehicle; Based on the multiple first observation images, the road structure information, and the target driving trajectory, the driving trajectory of the target vehicle in the blind spot is predicted.

2. The method according to claim 1, characterized in that, The step of predicting the trajectory of the target vehicle in the blind spot based on the multiple frames of the first observation image, the road structure information, and the target driving trajectory includes: The motion information of the target vehicle is extracted from the multiple frames of the first observation image; A prediction model is used to predict the driving trajectory in the blind spot based on the motion information, the road structure information, and the target driving trajectory.

3. The method according to claim 2, characterized in that, The step of employing a prediction model to predict the driving trajectory in the blind spot based on the motion information, the road structure information, and the target driving trajectory includes: Using a kinematic model, motion information within the blind zone is predicted based on the motion information. Determine the road parameters within the blind zone based on road structure information; Using the aforementioned prediction model, multiple initial trajectory points are obtained by predicting the motion information within the blind zone and the target's driving trajectory. Using the prediction model, multiple target trajectory points are selected from multiple initial trajectory points based on road parameters within the blind zone to obtain the driving trajectory within the blind zone.

4. The method according to claim 2, characterized in that, The step of employing a prediction model to predict the driving trajectory in the blind spot based on the motion information, the road structure information, and the target driving trajectory includes: Using the aforementioned prediction model, the driving trajectory in the blind zone is predicted based on the motion information, motion weight, target driving trajectory, trajectory weight, road structure information, and road weight within the blind zone.

5. The method according to claim 3, characterized in that, The method further includes: Based on the motion information and the motion information within the blind zone, calculate the motion residual components; Based on the number and matching degree of the target driving trajectories, determine the trajectory residual components; Based on the road parameters, preset road topology, and preset traffic rules within the blind zone, calculate the highway residual components; The confidence level of each target trajectory point in the driving trajectory in the blind zone is calculated based on the motion residual component, the trajectory residual component, and the road residual component.

6. The method according to claim 3, characterized in that, The method further includes: Acquire a second observation image of the target vehicle after it has driven out of the blind spot, captured by the second camera; The actual position of the target vehicle on the road after it leaves the blind spot is determined based on the second observation image; The kinematic model parameters are updated based on the actual position and the predicted end position indicated by the driving trajectory in the blind spot.

7. The method according to claim 6, characterized in that, The step of updating the kinematic model parameters based on the actual position and the predicted end position indicated by the driving trajectory in the blind spot includes: If the error between the actual position and the predicted end position is greater than a preset error, the model parameters of the kinematic model are updated based on the actual position.

8. The method according to claim 6, characterized in that, The method further includes: Based on the length of the blind spot, speed limit information, and minimum driving speed in the road structure information, calculate the passage time interval of the blind spot; Based on the passage time interval and the second observation image, the state of the target vehicle in the blind spot is predicted.

9. A visual tracking and prediction device for highway traffic, characterized in that, include: A memory and a processor, the memory storing a computer program executable by the processor, the processor executing the computer program to implement the visual tracking and prediction method for highway traffic as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when read and executed, implements the visual tracking and prediction method for highway traffic as described in any one of claims 1-8.

Images