Vehicle trajectory reconstruction method and device, server and computer storage medium

Abstract

The present application relates to vehicle intelligent management technology, provide a kind of vehicle trajectory reconstruction method, device, server and computer storage medium, the method comprises: obtaining the video stream that multiple cameras located in multiple different positions of target parking lot are shot;Vehicle identification is carried out to each video stream, and the position sequence of each vehicle in each video stream is obtained;According to the mapping relationship between the bird's eye view of target parking lot and the camera corresponding to each video stream, the local trajectory of the position sequence of each vehicle in each video stream in the bird's eye view is determined;The local trajectory of the same vehicle in all video streams is spliced, and the global trajectory of the same vehicle in all video streams is obtained.The present application realizes the reconstruction of vehicle trajectory under the premise of not relying on GPS.

Description

Technical Field

[0001] This invention relates to the field of vehicle intelligent management technology, and more specifically, to a vehicle trajectory reconstruction method, device, server, and computer storage medium. Background Technology

[0002] With the continuous development of intelligent parking systems, autonomous parking technology, and intelligent traffic management systems, vehicle trajectory reconstruction in parking lot environments is of great significance in applications such as vehicle dispatch management, parking space occupancy monitoring, parking behavior analysis, accident tracing, and intelligent guidance.

[0003] Traditional vehicle trajectory reconstruction in open road scenarios usually relies on GPS (Global Positioning System). However, parking lots are characterized by small spaces, building obstructions, multipath effects, and signal attenuation, which often result in unstable or even completely lost GPS signals. Therefore, traditional GPS-based vehicle trajectory reconstruction methods are difficult to apply to parking lots. Summary of the Invention

[0004] The purpose of this invention is to provide a vehicle trajectory reconstruction method, device, server, and computer storage medium.

[0005] The embodiments of the present invention can be implemented as follows:

[0006] In a first aspect, the present invention provides a vehicle trajectory reconstruction method, the method comprising:

[0007] Acquire video streams captured by multiple cameras located at different locations in the target parking lot;

[0008] Vehicle identification is performed on each video stream to obtain the position sequence of each vehicle in each video stream;

[0009] Based on the mapping relationship between the camera corresponding to each video stream and the bird's-eye view of the target parking lot, determine the local trajectory of the position sequence of each vehicle in each video stream in the bird's-eye view;

[0010] By stitching together the local trajectories of the same vehicle in all the video streams, the global trajectory of the same vehicle in all the video streams can be obtained.

[0011] In an optional implementation, the position sequence of each vehicle in each of the video streams includes the vehicle image coordinates of each vehicle in each of the multiple video frames of the video stream.

[0012] The step of determining the local trajectory of the position sequence of each vehicle in each video stream in the bird's-eye view based on the mapping relationship between the camera corresponding to each video stream and the bird's-eye view of the target parking lot includes:

[0013] Based on the mapping relationship, calculate the image coordinates of each vehicle in the position sequence of each vehicle in each video stream and map them to the vehicle world coordinates in the bird's-eye view;

[0014] By sequentially connecting the vehicle world coordinates corresponding to the position sequence of each vehicle in each video stream, the local trajectory of the position sequence of each vehicle in each video stream in the bird's-eye view is obtained.

[0015] In an optional implementation, the step of stitching together the local trajectories of the same vehicle in all the video streams to obtain the global trajectory of the same vehicle in all the video streams includes:

[0016] According to the chronological order of the shooting time of all video streams, the appearance features of each vehicle in each video stream are extracted sequentially.

[0017] Based on the appearance features of each vehicle in each of the video streams, a target video stream in which the same vehicle appears in all the video streams is determined;

[0018] The local trajectories of the target video streams of the same vehicle are stitched together according to the chronological order of shooting time to obtain the global trajectory of the same vehicle.

[0019] In an optional implementation, the step of stitching together the local trajectories of the target video streams of the same vehicle according to the chronological order of shooting time to obtain the global trajectory of the same vehicle includes:

[0020] For any vehicle, if the local trajectories in two adjacent target video streams are continuous, then the local trajectories in the two adjacent target video streams are directly spliced ​​together to obtain the merged trajectory of the two adjacent target video streams.

[0021] If the local trajectories in two adjacent target video streams are discontinuous, then the missing trajectory between the local trajectories in the two adjacent target video streams is determined.

[0022] Based on the missing trajectory, the local trajectories in two adjacent target video streams are completed to obtain the merged trajectory of the two adjacent target video streams;

[0023] The trajectories of adjacent target video streams are sequentially spliced ​​together to obtain the global trajectory of the same vehicle.

[0024] In an optional implementation, the step of determining the missing trajectory between local trajectories in two adjacent target video streams includes:

[0025] Obtain the reachable path between the cameras corresponding to two adjacent target video streams;

[0026] The estimated distance is calculated based on the time difference between the shooting times of the two adjacent target video streams and the preset vehicle speed.

[0027] The missing trajectory is determined from the length of the reachable path within a preset range of the estimated distance.

[0028] In an optional implementation, the method further includes:

[0029] For any target camera among the plurality of cameras, a target anchor point is selected within the field of view of the target camera;

[0030] Acquire a reference image captured by the target camera at the target anchor point;

[0031] Determine the target image coordinates of the target anchor point in the reference image;

[0032] Obtain the target world coordinates of the target anchor point in the bird's-eye view;

[0033] The mapping relationship of the target camera is determined based on the target image coordinates and the target world coordinates.

[0034] In an optional implementation, after the step of determining the mapping relationship of the target camera based on the target image coordinates and the target world coordinates, the method further includes:

[0035] Acquire verification images captured by the target camera targeting a preset target within the target parking lot;

[0036] Based on the image coordinates of the reference point on the preset target in the verification image and the mapping relationship, determine the world coordinates of the reference point in the bird's-eye view;

[0037] Calculate the error between the world coordinates of the reference point in the bird's-eye view and the pre-measured true coordinates of the reference point in the bird's-eye view;

[0038] If the error is greater than the calibration error, a new target anchor point is selected, and the mapping relationship of the target camera is re-determined based on the newly selected target anchor point.

[0039] In a second aspect, the present invention provides a vehicle trajectory reconstruction device, the device comprising:

[0040] The acquisition module is used to acquire video streams captured by multiple cameras located at different locations in the target parking lot;

[0041] The identification module is used to identify vehicles in each of the video streams and obtain the position sequence of each vehicle in each of the video streams.

[0042] The determining module is used to determine the local trajectory of the position sequence of each vehicle in each video stream in the bird's-eye view based on the mapping relationship between the camera corresponding to each video stream and the bird's-eye view of the target parking lot;

[0043] The stitching module is used to stitch together the local trajectories of the same vehicle in all the video streams to obtain the global trajectory of the same vehicle in all the video streams.

[0044] Thirdly, the present invention provides a server including a controller and a memory, the memory being used to store a program, and the controller being used to implement the vehicle trajectory reconstruction method as described in the first aspect when executing the program.

[0045] Fourthly, the present invention provides a computer storage medium having a computer program stored thereon, which, when executed by a controller, implements the vehicle trajectory reconstruction method as described in the first aspect.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] This invention identifies vehicles in video streams captured from multiple locations at a target parking lot, obtaining a sequence of vehicle positions in each video stream. Then, based on the mapping relationship between the cameras and a bird's-eye view of the target parking lot, the local trajectories of the position sequences in the bird's-eye view are determined. Finally, the local trajectories of the same vehicle from all video streams are stitched together to obtain the global trajectory of the same vehicle across all video streams. Because this invention utilizes video streams and mapping relationships to obtain local trajectories, and then stitches these local trajectories together to obtain the global trajectory, vehicle trajectory reconstruction is achieved without relying on GPS. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is an example diagram of the parking lot integrated management system provided in this embodiment.

[0050] Figure 2 This is a block diagram of the server provided in this embodiment.

[0051] Figure 3 This is a flowchart illustrating the vehicle trajectory reconstruction method provided in this embodiment.

[0052] Figure 4 This is a block diagram of the vehicle trajectory reconstruction device provided in this embodiment.

[0053] Icons: 10-Server; 11-Processor; 12-Memory; 13-Bus; 20-Data Acquisition Terminal; 30-Mobile Terminal; 40-Management Terminal; 100-Vehicle Trajectory Reconstruction Device; 110-Acquisition Module; 120-Identification Module; 130-Determination Module; 140-Stabbing Module. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0055] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0056] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0057] In the description of this invention, it should be noted that if terms such as "upper," "lower," "inner," or "outer" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of this invention is usually placed, they are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0058] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0059] It should be noted that, where there is no conflict, the features in the embodiments of the present invention can be combined with each other.

[0060] Please refer to Figure 1 , Figure 1 This is an example diagram of the parking lot integrated management system provided in this embodiment. Figure 1 The parking lot integrated management system includes a server 10, a data acquisition terminal 20, a mobile terminal 30, and a management terminal 40.

[0061] The data acquisition terminal 20 includes lane cameras, vehicle-to-infrastructure communication equipment, and collaborative control equipment, which can collect image data in the parking lot in real time.

[0062] Server 10 is the core device of the back-end data analysis platform. It processes and analyzes the data collected by the data collection terminal 20, and supports multi-dimensional applications such as safety supervision, intelligent operation, autonomous parking, and autonomous pick-up.

[0063] The mobile terminal 30 can be the car owner's mobile phone, which provides services such as reverse car finding and car location finding based on visual indoor positioning.

[0064] The management terminal 40 is a browser-based graphical management tool that provides a large visual screen for real-time monitoring of events within the venue.

[0065] based on Figure 1 This embodiment also mentions Figure 1 Please refer to the example diagram of the box for server 10. Figure 2 , Figure 2 This is a block diagram of the server 10 provided in this embodiment. The server 10 implements the vehicle trajectory reconstruction method of the aforementioned embodiment. The server 10 includes a processor 11, a memory 12 and a bus 13. The processor 11 and the memory 12 are connected through the bus 13.

[0066] The processor 11 can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the vehicle trajectory reconstruction method described above can be completed by the integrated logic circuits in the hardware of the processor 11 or by software instructions. The processor 11 can be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), etc.; it can also be a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Logic Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0067] The memory 12 is used to store the program for implementing the vehicle trajectory reconstruction method. The program can be a software function module stored in the memory 12 in the form of software or firmware or embedded in the OS (Operating System) of the server 10.

[0068] After receiving the execution instruction, the processor 11 executes the program to implement the vehicle trajectory reconstruction method of the aforementioned embodiment.

[0069] Based on the above Figure 1 and Figure 2 This embodiment also provides for Figure 1 and Figure 2 For the vehicle trajectory reconstruction method of server 10, please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the vehicle trajectory reconstruction method provided in this embodiment. The method includes the following steps:

[0070] Step S101: Acquire video streams captured by multiple cameras located at multiple different locations in the target parking lot.

[0071] In this embodiment, cameras can be pre-deployed in key areas such as corners and entrances / exits inside the target parking lot. One or more fixed or controllable cameras can be deployed at one location to collect video streams in real time. The video stream captured by each camera is a continuous video frame captured by the camera within its field of view.

[0072] Step S102: Perform vehicle identification on each video stream to obtain the position sequence of each vehicle in each video stream.

[0073] In this embodiment, target detection and tracking algorithms from computer vision can be used to process each frame of each video stream, identify vehicle objects, and establish identity associations between vehicle objects across consecutive video frames, thereby forming a time-varying position sequence for each vehicle in the video stream. The target detection algorithm can first utilize traditional algorithms such as thresholding, feature selection, and frame differencing with a fixed camera and background for coarse-grained target detection, and then employ deep learning algorithms (such as the YOLO series of target detection algorithms) for higher-precision target refinement to accurately locate the position sequence of each vehicle in each video stream. Target tracking can employ deep learning algorithms for multi-target tracking to adapt to situations where multiple vehicles are simultaneously captured by the same camera in a parking lot environment.

[0074] Step S103: Based on the mapping relationship between the camera corresponding to each video stream and the bird's-eye view of the target parking lot, determine the local trajectory of the position sequence of each vehicle in each video stream in the bird's-eye view.

[0075] In this embodiment, the mapping relationship is typically established in advance through calibration, representing the geometric transformation relationship between the camera image coordinate system and the bird's-eye view world coordinate system. For a video stream, the position sequence of each vehicle can be the vehicle's position in multiple video frames arranged chronologically, and the corresponding local trajectory is the path formed by connecting these vehicle positions in the bird's-eye view in chronological order. The vehicle position can be the location of a predetermined part of the vehicle in the video frame, such as the head, middle, etc.

[0076] Step S104: stitch together the local trajectories of the same vehicle in all video streams to obtain the global trajectory of the same vehicle in all video streams.

[0077] In this embodiment, for the same vehicle, the local trajectories of the vehicle in the video stream in which it appears are stitched together based on temporal sequence and spatial continuity, thereby reconstructing the complete driving path of the vehicle within the entire target parking lot area.

[0078] Understandably, depending on the vehicle's specific driving path and the camera's location, the same vehicle may appear in every video stream, or in one or more video streams.

[0079] The method described in this embodiment identifies vehicles in video streams captured from multiple different locations at the target parking lot, obtaining a sequence of vehicle positions in each video stream. Then, based on the mapping relationship between the camera and the bird's-eye view of the target parking lot, the local trajectory of the position sequence in the bird's-eye view is determined. Finally, the local trajectories of the same vehicle in all video streams are stitched together to obtain the global trajectory of the same vehicle across all video streams. Because this embodiment utilizes video streams and mapping relationships to obtain local trajectories, and then stitches these local trajectories together to obtain the global trajectory, vehicle trajectory reconstruction is achieved without relying on GPS.

[0080] In optional implementations, the mapping relationship between different cameras and the bird's-eye view varies due to differences in the installation location and configuration parameters of each camera. However, the method for determining the mapping relationship between each camera and the bird's-eye view is similar. This embodiment uses individual cameras to illustrate the implementation of their mapping relationship:

[0081] First, for any target camera among multiple cameras, select a target anchor point within the target camera's field of view;

[0082] In this embodiment, the target camera is any one of multiple cameras, whose field of view covers a local area of ​​the parking lot. A target anchor point refers to a physical point or marker within the target camera's field of view that has a fixed spatial location, is easily identifiable, and provides stable imaging, such as the intersection of road markings or the bottom of a parking post. To ensure the accuracy and robustness of the mapping relationship, target anchor points can be distributed across different areas within the target camera's field of view.

[0083] Secondly, obtain reference images captured by the target camera at the target anchor point;

[0084] In this embodiment, in order to improve the accuracy of the mapping relationship, the reference image can be an image taken when the target anchor point is unobstructed, the lighting conditions of the shooting environment are good, and the sharpness meets the preset sharpness threshold.

[0085] Third, determine the target image coordinates of the target anchor point in the reference image;

[0086] In this embodiment, the target image coordinates can be the coordinates of the target anchor point in the image coordinate system. The image coordinate system can be an image with the pixel at the top left corner of the reference image as the origin, the x-axis extending to the right, and the y-axis extending downward.

[0087] Fourth, obtain the target anchor point's world coordinates in the bird's-eye view;

[0088] In this embodiment, the target world coordinate system can be the coordinates of the target anchor point in the bird's-eye view coordinate system. The bird's-eye view coordinate system is a two-dimensional or three-dimensional coordinate system viewed from above on the parking lot ground, used to represent the position of objects on the ground. The bird's-eye view coordinate system can be with the center of the bird's-eye view of the target parking lot as the origin, with the x-axis extending to the left and right, and the y-axis extending upward and downward, respectively.

[0089] Finally, the mapping relationship between the target camera and the target image coordinates and the target world coordinates is determined.

[0090] In this embodiment, a coordinate transformation algorithm is used to establish a mapping function between the camera image space and the bird's-eye view space. This mapping function represents the mapping relationship between the camera and the bird's-eye view, and its form can be a matrix transformation model or a set of mathematical expressions used to convert image coordinates into corresponding world coordinates. Coordinate transformation algorithms include, but are not limited to, homography transformation and perspective projection transformation.

[0091] In an optional implementation, to ensure the accuracy of the determined mapping relationship meets the requirements, this embodiment also provides an implementation method for verifying the mapping relationship and optimizing the mapping relationship when the verification fails:

[0092] First, acquire verification images captured by the target camera targeting a preset target within the target parking lot;

[0093] In this embodiment, to ensure the accuracy of the verification results, the verification image can be captured in the same shooting environment as the reference image.

[0094] Secondly, based on the image coordinates and mapping relationship of the reference point on the preset target in the verification image, determine the world coordinates of the reference point in the bird's-eye view;

[0095] In this embodiment, a preset target refers to a physical object or marker with known world coordinates, such as a calibration target set on the ground of a parking lot or a fixed object with specific geometric features. A reference point on the preset target can be a single point on the preset target or multiple points located at different positions on the preset target. For example, if the preset target is a cuboid, the reference point can be one or more vertices of the cuboid.

[0096] Third, calculate the error between the world coordinates of the reference point in the bird's-eye view and the actual coordinates of the reference point in the bird's-eye view as measured in advance;

[0097] In this embodiment, the true coordinates of the reference point in the bird's-eye view can be the real-world coordinates of the reference point obtained in advance by a high-precision measuring device, including, but not limited to, a total station or a laser rangefinder. The error can be the Euclidean distance between the world coordinates of the reference point in the bird's-eye view and the true coordinates of the reference point in the bird's-eye view as measured in advance, or it can be the average deviation across multiple reference points.

[0098] Finally, if the error is greater than the calibration error, the target anchor point is reselected, and the mapping relationship of the target camera is re-determined based on the reselected target anchor point.

[0099] In this embodiment, the calibration error is used to determine whether the current mapping relationship meets the accuracy requirements of the actual application. Different calibration errors can be set depending on the accuracy requirements of the actual application; a smaller calibration error can be set for high accuracy requirements, and vice versa.

[0100] If the calculated error exceeds the calibration error, it indicates a significant deviation in the current mapping relationship, which cannot guarantee the spatial consistency of vehicle trajectory reconstruction. In this case, it is necessary to reselect target anchor points and redetermine the mapping relationship. Reselecting target anchor points can be done by choosing multiple target anchor points with a more reasonable distribution and more stable imaging, or by increasing the number of target anchor points. The methods for redetermining the mapping relationship have been described earlier and will not be repeated here.

[0101] After determining the mapping relationship, in order to map the location sequence to the vehicle trajectory in the bird's-eye view, this embodiment provides an implementation method:

[0102] First, based on the mapping relationship, the image coordinates of each vehicle in the position sequence of each vehicle in each video stream are mapped to the vehicle world coordinates in the bird's-eye view.

[0103] In this embodiment, the position sequence of each vehicle in the video stream refers to a series of position data formed by continuously or intermittently detecting the image coordinates of each identified vehicle in multiple video frames captured by a certain camera. The image coordinates are usually represented by two-dimensional coordinates (e.g., coordinates in a pixel coordinate system) to represent the position of the vehicle in the video frame. Accordingly, the position sequence of each vehicle in each video stream is a sequence formed by arranging the vehicle image coordinates of each vehicle in each of the multiple video frames of the video stream in chronological order of video capture time.

[0104] Secondly, the world coordinates of the vehicles corresponding to the position sequence of each vehicle in each video stream are connected sequentially to obtain the local trajectory of the position sequence of each vehicle in the bird's-eye view.

[0105] In this embodiment, the world coordinate points corresponding to the image coordinates of each vehicle in the position sequence are connected sequentially according to the time order of the video frames to obtain the local trajectory of the vehicle within the area covered by the video stream. The local trajectory can be a path representation of the vehicle's movement within the field of view of a certain camera, expressed in the form of world coordinates, which can reflect the vehicle's movement direction and path shape in the bird's-eye view.

[0106] In an optional implementation, in order to stitch together the local trajectories of the same vehicle to obtain the global trajectory, this embodiment provides an implementation method:

[0107] First, extract the appearance features of each vehicle in each video stream in chronological order of their capture time.

[0108] In this embodiment, the vehicle's appearance features can be visual attributes extracted from the video stream by computer vision algorithms and used to distinguish different vehicles, including but not limited to visual information such as vehicle color, model, license plate, vehicle logo, headlight shape, and window outline. The appearance features can be extracted using deep learning models such as convolutional neural networks, or using traditional image feature extraction algorithms.

[0109] Secondly, based on the appearance features of each vehicle in each video stream, the target video stream in which the same vehicle appears in all video streams is determined;

[0110] In this embodiment, similarity comparisons are performed across video streams based on appearance features to determine which video streams contain vehicles belonging to the same vehicle entity. The same vehicle is defined as the same actual vehicle captured sequentially by multiple cameras within the target parking lot. By setting a feature matching threshold, all video streams containing this vehicle can be filtered out, i.e., the target video stream.

[0111] Understandably, the selection of these target video streams is not only based on the similarity of appearance features, but also requires cross-validation in conjunction with the time window of vehicle appearance to exclude interfering vehicles that are similar in appearance but appear at inconsistent times.

[0112] Finally, the local trajectories of the target video streams of the same vehicle are stitched together according to the order of shooting time to obtain the global trajectory of the same vehicle.

[0113] In this embodiment, the stitching process must ensure the spatiotemporal consistency of the local trajectory, that is, the end position of the vehicle in the previous video stream and the start position of the vehicle in the next video stream have a reasonable transition relationship in time and space.

[0114] In an optional implementation, due to the influence of the vehicle's actual driving trajectory and the camera's location, when the same vehicle passes two cameras successively, the local trajectories in the corresponding adjacent video streams can be continuous or discontinuous. This embodiment provides methods for merging local trajectories for these two different situations:

[0115] For any given vehicle, considering the case of continuous local trajectories:

[0116] If the local trajectories in two adjacent target video streams are continuous, then the local trajectories in the two adjacent target video streams are directly spliced ​​together to obtain the merged trajectory of the two adjacent target video streams.

[0117] In this embodiment, if the local trajectories in two target video streams are continuous, it means that the spatial distance between the last world coordinate point of the vehicle trajectory in the previous target video stream and the first world coordinate point of the vehicle trajectory in the subsequent target video stream is less than a set threshold, and the time interval between them is within a reasonable range. If this continuity condition is met, the two local trajectories can be directly spliced ​​together to form a merged trajectory.

[0118] For any given vehicle, considering the case of local trajectory discontinuities:

[0119] First, if the local trajectories in two adjacent target video streams are discontinuous, then the missing trajectory between the local trajectories in the two adjacent target video streams is determined.

[0120] In this embodiment, if the local trajectories in the two target video streams are discontinuous, it means that there is a video coverage blind spot between the two target video streams or that the vehicle was not continuously captured while moving between the fields of view of the two cameras. In this case, it is necessary to determine the "missing trajectory" between these two local trajectories, that is, the path that the vehicle may have traveled in the blind spot. Factors affecting the determination of the missing trajectory include, but are not limited to: the physical path that the vehicle can travel between the two cameras, the distance that the vehicle may travel within the time difference between the capture of the two target video streams, and the geometric representation of the vehicle's travel path on the bird's-eye view.

[0121] To make the identified missing trajectories more reasonable and the final merged trajectory more closely match the actual situation, this embodiment combines physical path reachability with time-space constraints to provide a method for reasonably inferring the driving path (i.e., missing trajectory) of a vehicle that is not captured by the video stream between the fields of view of two cameras:

[0122] (1) Obtain the reachable path between the cameras corresponding to two adjacent target video streams;

[0123] In this embodiment, the reachable path can be the set of all legally traversable paths for a vehicle between the view boundaries of one camera and the view boundaries of another camera in the target parking lot, including, but not limited to, main roads, auxiliary roads, turning paths, etc. This path information can be pre-modeled and stored based on the bird's-eye view or map topology data of the target parking lot.

[0124] (2) Calculate the estimated distance based on the time difference between the shooting times of the two adjacent target video streams and the preset vehicle speed;

[0125] In this embodiment, the time difference can be the interval between the start time of the subsequent target video stream and the end time of the previous target video stream. The preset vehicle speed can be the average driving speed of the vehicle in the parking lot, or it can be dynamically adjusted according to the speed limit settings of different areas. The estimated distance represents the farthest distance that the vehicle can theoretically travel within the time period of this time difference.

[0126] (3) Determine the missing trajectory from the reachable paths whose lengths are within the preset range of the estimated distance.

[0127] In this embodiment, firstly, paths whose length falls within a preset range of the estimated distance are selected as candidate paths from all reachable paths. These candidate paths are those that the vehicle can actually complete in the time dimension. Secondly, if there is only one reachable path, it is determined as the missing trajectory. If there are multiple reachable paths, one of them is selected as the missing trajectory. The selection criteria can include logical judgments such as the path length being closest to the estimated distance or the path direction best matching the direction of the preceding and following local trajectories. The finally determined missing trajectory serves as a reasonable supplement to the gap between two local trajectories, used to complete and stitch together the local trajectories.

[0128] Secondly, based on the missing trajectory, the local trajectories in the two adjacent target video streams are completed to obtain the merged trajectory of the two adjacent target video streams.

[0129] In this embodiment, the missing trajectory is used to complete the two local trajectories in space and time, so that the gap between the two local trajectories is reasonably filled, thereby forming a complete merged trajectory.

[0130] Based on the above-mentioned merging process of local trajectories of two adjacent target video streams, in order to stitch together all the local trajectories of all target video streams of the same vehicle to obtain the global trajectory, one implementation method is as follows:

[0131] The merged trajectories of adjacent target video streams are stitched together sequentially to obtain the global trajectory of the same vehicle.

[0132] To perform the corresponding steps in the above embodiments and various possible implementations, an implementation method of the vehicle trajectory reconstruction device 100 is given below. Please refer to... Figure 4 , Figure 4 This is a block diagram of the vehicle trajectory reconstruction device provided in this embodiment. It should be noted that the basic principle and technical effects of the vehicle trajectory reconstruction device 100 provided by the present invention are the same as those of the corresponding embodiments described above. For the sake of brevity, this embodiment does not mention or specify these aspects.

[0133] The vehicle trajectory reconstruction device 100 includes an acquisition module 110, an identification module 120, a determination module 130, and a stitching module 140.

[0134] The acquisition module 110 is used to acquire video streams captured by multiple cameras located at multiple different locations in the target parking lot;

[0135] The recognition module 120 is used to perform vehicle recognition on each video stream and obtain the position sequence of each vehicle in each video stream.

[0136] The determination module 130 is used to determine the local trajectory of the position sequence of each vehicle in each video stream in the bird's-eye view based on the mapping relationship between the camera corresponding to each video stream and the bird's-eye view of the target parking lot.

[0137] The stitching module 140 is used to stitch together the local trajectories of the same vehicle in all video streams to obtain the global trajectory of the same vehicle in all video streams.

[0138] In an optional implementation, the position sequence of each vehicle in each video stream includes the vehicle image coordinates of each vehicle in each of the multiple video frames of the video stream; the determination module 130 is specifically used for:

[0139] Based on the mapping relationship, calculate the image coordinates of each vehicle in the position sequence of each vehicle in each video stream and map them to the world coordinates of the vehicle in the bird's-eye view.

[0140] By sequentially concatenating the vehicle world coordinates corresponding to the position sequence of each vehicle in each video stream, the local trajectory of the position sequence of each vehicle in each video stream in the bird's-eye view is obtained.

[0141] In an optional implementation, the splicing module 140 is specifically used for:

[0142] Based on the chronological order of the shooting times of all video streams, extract the appearance features of each vehicle in each video stream in sequence.

[0143] Based on the appearance features of each vehicle in each video stream, determine the target video stream in which the same vehicle appears in all video streams;

[0144] By stitching together the local trajectories of the target video streams of the same vehicle in chronological order of shooting time, the global trajectory of the same vehicle can be obtained.

[0145] In an optional implementation, the stitching module 140 is used to stitch together local trajectories of the target video stream of the same vehicle according to the chronological order of shooting time to obtain the global trajectory of the same vehicle. Specifically, it is used for:

[0146] For any vehicle, if the local trajectories in two adjacent target video streams are continuous, then the local trajectories in the two adjacent target video streams are directly spliced ​​together to obtain the merged trajectory of the two adjacent target video streams.

[0147] If the local trajectories in two adjacent target video streams are discontinuous, then the missing trajectory between the local trajectories in the two adjacent target video streams is determined.

[0148] Based on the missing trajectory, the local trajectories in two adjacent target video streams are completed to obtain the merged trajectory of the two adjacent target video streams;

[0149] The merged trajectories of adjacent target video streams are stitched together sequentially to obtain the global trajectory of the same vehicle.

[0150] In an optional implementation, the stitching module 140 is specifically used to: determine the missing trajectory between local trajectories in two adjacent target video streams.

[0151] Obtain the reachable path between the cameras corresponding to two adjacent target video streams;

[0152] Calculate the estimated distance based on the time difference between the capture times of two adjacent target video streams and the preset vehicle speed;

[0153] The missing trajectory is determined from the reachable paths whose lengths are within a preset range of estimated distances.

[0154] In an optional implementation, the determining module 130 is further configured to:

[0155] For any target camera among multiple cameras, select a target anchor point within the target camera's field of view;

[0156] Acquire a reference image captured by the target camera at the target anchor point;

[0157] Determine the target image coordinates of the target anchor point within the reference image;

[0158] Obtain the target world coordinates of the target anchor point in the bird's-eye view;

[0159] The mapping relationship between the target camera and the target image coordinates and the target world coordinates is determined.

[0160] In an optional implementation, the determining module 130 is further configured to:

[0161] Acquire verification images captured by the target camera targeting a preset target within the target parking lot;

[0162] Based on the image coordinates and mapping relationship of the reference point on the preset target in the verification image, determine the world coordinates of the reference point in the bird's-eye view;

[0163] Calculate the error between the world coordinates of the reference point in the bird's-eye view and the actual coordinates of the reference point in the bird's-eye view as measured beforehand;

[0164] If the error is greater than the calibration error, the target anchor point is reselected, and the mapping relationship of the target camera is re-determined based on the reselected target anchor point.

[0165] This invention provides a computer storage medium storing a computer program that, when executed by a controller, implements the vehicle trajectory reconstruction method as described in any one of these embodiments.

[0166] In summary, this invention provides a vehicle trajectory reconstruction method, apparatus, server, and computer storage medium. The method includes: acquiring video streams captured by multiple cameras at multiple different locations in a target parking lot; performing vehicle identification on each video stream to obtain the position sequence of each vehicle in each video stream; determining the local trajectory of the position sequence of each vehicle in each video stream in the bird's-eye view based on the mapping relationship between the camera corresponding to each video stream and the bird's-eye view of the target parking lot; and stitching together the local trajectories of the same vehicle in all video streams to obtain the global trajectory of the same vehicle in all video streams. Compared with the prior art, this embodiment has at least the following advantages: (1) It uses video streams and mapping relationships to obtain local trajectories, and then stitches the local trajectories together to obtain the global trajectory, thus achieving vehicle trajectory reconstruction without relying on GPS; (2) It converts these image coordinates frame by frame into world coordinate points in the bird's-eye view based on the mapping relationship of the cameras, and connects these points sequentially to form a trajectory line, which is the local trajectory of the vehicle within the field of view of the camera. This method of generating local trajectories ensures that there is a unified spatial reference benchmark between the local trajectories when stitching trajectories in the future, thereby improving the accuracy and consistency of trajectory reconstruction; (3) The trajectory association method based on appearance features effectively solves the identity matching problem in vehicle tracking under multi-camera coverage scenarios, and provides basic support for subsequent trajectory analysis and behavior understanding; (4) By judging the continuity status of local trajectories in adjacent video streams, a differentiated stitching strategy is adopted to construct a global vehicle trajectory that is continuous in both time and space dimensions; (5) By combining physical path accessibility and time-space constraints, the driving path of the vehicle that is not captured by the video stream between the two camera views is reasonably inferred, thereby providing a basis for trajectory stitching; (6) By comparing verifiable reference points with known real coordinates, the quality assessment of the mapping relationship has an objective basis, avoiding systematic deviations caused by improper initial anchor point selection or accumulated coordinate extraction errors.

[0167] The above descriptions are merely various embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for reconstructing vehicle trajectory, characterized in that, The method includes: Acquire video streams captured by multiple cameras located at different locations in the target parking lot; Vehicle identification is performed on each video stream to obtain the position sequence of each vehicle in each video stream; Based on the mapping relationship between the camera corresponding to each video stream and the bird's-eye view of the target parking lot, determine the local trajectory of the position sequence of each vehicle in each video stream in the bird's-eye view; By stitching together the local trajectories of the same vehicle in all the video streams, the global trajectory of the same vehicle in all the video streams is obtained. The step of stitching together the local trajectories of the same vehicle in all the video streams to obtain the global trajectory of the same vehicle in all the video streams includes: The local trajectories of the target video stream of the same vehicle are stitched together according to the chronological order of shooting time to obtain the global trajectory of the same vehicle. The target video stream is a video stream containing the same vehicle. The step of stitching together the local trajectories of the target video streams of the same vehicle according to the chronological order of shooting time to obtain the global trajectory of the same vehicle includes: For any given vehicle, if the local trajectories in two adjacent target video streams are discontinuous, then the missing trajectory between the local trajectories in the two adjacent target video streams is determined. Based on the missing trajectory, the local trajectories in two adjacent target video streams are completed to obtain the merged trajectory of the two adjacent target video streams; The merged trajectories of two adjacent target video streams are stitched together sequentially to obtain the global trajectory of the same vehicle; The step of determining the missing trajectory between two adjacent local trajectories in the target video streams includes: Obtain the reachable path between the cameras corresponding to two adjacent target video streams; The estimated distance is calculated based on the time difference between the shooting times of the two adjacent target video streams and the preset vehicle speed. The missing trajectory is determined from the length of the reachable path within a preset range of the estimated distance.

2. The method according to claim 1, characterized in that, The position sequence of each vehicle in each of the video streams includes the vehicle image coordinates of each vehicle in each of the multiple video frames of the video stream. The step of determining the local trajectory of the position sequence of each vehicle in each video stream in the bird's-eye view based on the mapping relationship between the camera corresponding to each video stream and the bird's-eye view of the target parking lot includes: Based on the mapping relationship, calculate the image coordinates of each vehicle in the position sequence of each vehicle in each video stream and map them to the vehicle world coordinates in the bird's-eye view; By sequentially connecting the vehicle world coordinates corresponding to the position sequence of each vehicle in each video stream, the local trajectory of the position sequence of each vehicle in each video stream in the bird's-eye view is obtained.

3. The method according to claim 1, characterized in that, The step of stitching together the local trajectories of the same vehicle in all the video streams to obtain the global trajectory of the same vehicle in all the video streams includes: According to the chronological order of the shooting time of all video streams, the appearance features of each vehicle in each video stream are extracted sequentially. Based on the appearance features of each vehicle in each of the video streams, a target video stream in which the same vehicle appears in all the video streams is determined; The local trajectories of the target video streams of the same vehicle are stitched together according to the chronological order of shooting time to obtain the global trajectory of the same vehicle.

4. The method according to claim 1, characterized in that, The method further includes: For any target camera among the plurality of cameras, a target anchor point is selected within the field of view of the target camera; Acquire a reference image captured by the target camera at the target anchor point; Determine the target image coordinates of the target anchor point in the reference image; Obtain the target world coordinates of the target anchor point in the bird's-eye view; The mapping relationship of the target camera is determined based on the target image coordinates and the target world coordinates.

5. The method according to claim 4, characterized in that, After the step of determining the mapping relationship of the target camera based on the target image coordinates and the target world coordinates, the following steps are included: Acquire verification images captured by the target camera targeting a preset target within the target parking lot; Based on the image coordinates of the reference point on the preset target in the verification image and the mapping relationship, determine the world coordinates of the reference point in the bird's-eye view; Calculate the error between the world coordinates of the reference point in the bird's-eye view and the pre-measured true coordinates of the reference point in the bird's-eye view; If the error is greater than the calibration error, a new target anchor point is selected, and the mapping relationship of the target camera is re-determined based on the newly selected target anchor point.

6. A vehicle trajectory reconstruction device, characterized in that, The device includes: The acquisition module is used to acquire video streams captured by multiple cameras located at different locations in the target parking lot; The identification module is used to identify vehicles in each of the video streams and obtain the position sequence of each vehicle in each of the video streams. The determining module is used to determine the local trajectory of the position sequence of each vehicle in each video stream in the bird's-eye view based on the mapping relationship between the camera corresponding to each video stream and the bird's-eye view of the target parking lot; The stitching module is used to stitch together the local trajectories of the same vehicle in all the video streams to obtain the global trajectory of the same vehicle in all the video streams. The stitching module is specifically used for: stitching together the local trajectories of the target video streams of the same vehicle according to the chronological order of shooting time to obtain the global trajectory of the same vehicle, wherein the target video streams are video streams containing the same vehicle; for any vehicle, if the local trajectories of two adjacent target video streams are discontinuous, then obtaining the reachable path between the cameras corresponding to the two adjacent target video streams; calculating the estimated distance based on the time difference between the shooting times of the two adjacent target video streams and a preset vehicle speed; determining the missing trajectory from the reachable paths within a preset range of the estimated distance based on the length of the reachable path; completing the local trajectories of the two adjacent target video streams based on the missing trajectory to obtain the merged trajectory of the two adjacent target video streams; and stitching together the merged trajectories of the two adjacent target video streams in sequence to obtain the global trajectory of the same vehicle.

7. A server, characterized in that, It includes a controller and a memory, the memory being used to store a program, and the controller being used to implement the vehicle trajectory reconstruction method as described in any one of claims 1-5 when executing the program.

8. A computer storage medium, characterized in that, It stores a computer program that, when executed by the controller, implements the vehicle trajectory reconstruction method as described in any one of claims 1-5.

Images