Target coordinate matching and analysis method and device, computer equipment and storage medium

Abstract

The invention belongs to the field of tracking and positioning, and relates to a target coordinate matching and analysis method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a video sequence from a plurality of cameras; detecting and tracking a target from the video sequence, and outputting pixel coordinates of the target; pixel coordinates of the target are converted into corresponding world coordinates, and track information of the target in a world coordinate system and time sequences of the target in different time periods are obtained; through coordinate matching and fusion, coordinate tracks belonging to the same target are identified and connected to form an integral target track, and the tracks belonging to the same target are fused to obtain a space-time track of the same target; extracting identity information of the target track; and according to the identity information, carrying out conjoint analysis on the target trajectory and the time sequence. Based on a multi-target multi-camera tracking and positioning mode, behavior analysis of a target in a three-dimensional space is realized, and a cross-camera mode enables a target track to be longer and behavior analysis to be more accurate.

Description

Technical Field

[0001] The present invention relates to the field of tracking and positioning, and in particular to a target coordinate matching and analysis method, device, computer equipment and storage medium. Background Art

[0002] Using video to track the movement trajectory of the target has been widely used in the field of public safety. However, the activity range of the target often spans multiple camera fields of view, and the target exhibits characteristics such as high freedom of movement and susceptibility to environmental influences in the video. The existing cross-camera target adaptive tracking method is difficult to meet the actual application requirements, so the accuracy of target coordinate matching and analysis is very low. The defects brought by this are: in the field of navigation and positioning, low accuracy may lead to serious deviations in the target position, which in turn affects the precise execution of tasks, such as drone delivery and military strikes, and increases the risk of mission failure; in geographic information system applications, inaccurate coordinate matching will directly affect the analysis and decision support of spatial data, and may lead to problems such as improper resource allocation and urban planning errors. For services that require high-precision positioning, such as self-driving cars and precision agriculture, low accuracy will seriously weaken the reliability and safety of the system and increase the probability of accidents. Therefore, improving the accuracy of target coordinate matching and analysis is crucial to ensure the normal operation of various applications and improve overall efficiency. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides a target coordinate matching and analysis method, which adopts the following technical solution, including:

[0004] Acquire a video sequence, wherein the video sequence comes from a plurality of cameras;

[0005] Detect and track the target P from the video sequence, and output the pixel coordinates P(x i ,y i ), x i Indicates the offset of the target P relative to the x-axis direction, y i Indicates the offset of the target P relative to the y-axis direction;

[0006] The pixel coordinates P(x i ,y i ) is converted into the corresponding world coordinate Q(X i ,Y i ,Z i ), obtain the trajectory information of the target P in the world coordinate system and the time series in different time periods;

[0007] Through coordinate matching and fusion, the coordinate tracks belonging to the same target are identified and connected into an overall target track G(X i ,Yi ,Z i ), and fuse the trajectories belonging to the same target to obtain the spatiotemporal trajectory of the same target;

[0008] Extract the target trajectory G(X i ,Y i ,Z i )’s identity information;

[0009] According to the identity information, the target trajectory G(X i ,Y i ,Z i ) and the time series are jointly analyzed.

[0010] Preferably, the step of acquiring a video sequence from multiple cameras specifically includes:

[0011] List all cameras and their positions that will be used to acquire the video sequence;

[0012] Start the video capture software and set the capture parameters;

[0013] The video sequences from multiple cameras are captured according to preset capture parameters.

[0014] Preferably, the detecting and tracking of the target P from the video sequence and outputting the pixel coordinates P(x i ,y i ), x i Indicates the offset of the target P relative to the x-axis direction, y i The steps of indicating the offset of the target P relative to the y-axis direction specifically include:

[0015] Using the TrtYOLO image recognition algorithm, the frame of the target P is detected from the video sequence;

[0016] Based on the framework, the DeepSort multi-target tracking algorithm is used to track the target P and output the pixel coordinates P(x i ,y i ) and the corresponding number, where x i Indicates the offset of the target P relative to the x-axis direction, y i Indicates the offset of the target P relative to the y-axis direction.

[0017] Preferably, the pixel coordinates P(x i ,y i ) is converted into the corresponding world coordinate Q(X i ,Y i ,Z i), the steps of obtaining the trajectory information of the target P in the world coordinate system and the time series in different time periods specifically include:

[0018] Given a point P'(x i ,y i ) and the corresponding world coordinate point Q'(X i ,Y i ,Z i ), there is a transformation matrix M for transforming between 3×4 pixel coordinates and world coordinates, there is a virtual-real transformation relationship P'=MQ', and the virtual-real transformation matrix M is established through at least six sets of virtual-real corresponding points, M=K[RT],

[0019] Through the virtual-real conversion matrix relationship P = MQ, according to the pixel coordinates P (x i ,y i ), calculate the world coordinates Q(X i ,Y i ,Z i );

[0020] Repeat the conversion process from pixel coordinates to world coordinates at multiple time points to obtain the set of world coordinates of the target P at different time points, that is, trajectory information;

[0021] Record the timestamp corresponding to each world coordinate, arrange the trajectory information in the order of the timestamps, and generate a time series;

[0022] Among them, w is the scaling factor, K is the 3×3 intrinsic parameter matrix of the camera, R is the 3×3 rotation matrix, T is the translation vector, f is the focal length of the camera, s is the camera offset, and x0 and y0 represent the main points of the video sequence.

[0023] Preferably, the coordinate tracks belonging to the same target are identified through coordinate matching and fusion, and connected into an overall target track G(X i ,Y i ,Z i ), and the steps of fusing the trajectories belonging to the same target and obtaining the spatiotemporal trajectory of the same target specifically include:

[0024] Obtain the overlapping area S in the video coverage areas of multiple cameras;

[0025] Traverse n target trajectories Qn and determine the trajectory segment Q in the same time period t and the same overlapping area S n (t)∈S;

[0026] Calculate the trajectory deviation value |Q m (t k )-Qv (t k )|≤d, where Q m (t k ) indicates that at time t k Track segment Q m The world coordinates, Q v (t k ) indicates that at time t k Track segment Q v The world coordinates of , d represents the preset distance threshold in the world coordinates;

[0027] Judge when Q m (t) and Q v (t) corresponds to time t k When the distances of m and Q v It’s the same goal;

[0028] Q m and Q v The trajectories at different time periods are connected in time series to obtain the overall target trajectory G(X i ,Y i ,Z i );

[0029] Break up the points of any two trajectories, reorder them according to the time component of the coordinates, and replace the original two points with the midpoint if the time component is the same. The reordered point sequence is the merged space-time trajectory.

[0030] Preferably, the extraction of the target trajectory G(X i ,Y i ,Z i The steps of obtaining the identity information of the user include:

[0031] According to the target feature of the target P, the target feature is compared with the identity database;

[0032] According to the comparison result, the target trajectory G(X i ,Y i ,Z i )’s identity information.

[0033] Preferably, the target trajectory G(X i ,Y i ,Z i ) and the step of performing joint analysis on the time series specifically comprises:

[0034] According to the target trajectory G(X i ,Y i ,Z i) and the time series, perform speed analysis and / or direction analysis and / or area analysis and / or track analysis.

[0035] In order to solve the above technical problems, the present invention also provides a target coordinate matching and analysis device, which adopts the following technical solution, including:

[0036] An acquisition module, used for acquiring a video sequence, wherein the video sequence comes from a plurality of cameras;

[0037] The detection and tracking module is used to detect and track the target P from the video sequence and output the pixel coordinates P (x i ,y i ), x i Indicates the offset of the target P relative to the x-axis direction, y i Indicates the offset of the target P relative to the y-axis direction;

[0038] The conversion module is used to convert the pixel coordinates P(x i ,y i ) is converted into the corresponding world coordinate Q(X i ,Y i ,Z i ), obtain the trajectory information of the target P in the world coordinate system and the time series in different time periods;

[0039] The fusion module is used for coordinate matching and fusion, identifying the coordinate tracks belonging to the same target and connecting them into an overall target track G(X i ,Y i ,Z i ), and fuse the trajectories belonging to the same target to obtain the spatiotemporal trajectory of the same target, G(X i ,Y i ,Z i ) represents the target P at time t i The world coordinates of

[0040] Extraction module, used to extract the target trajectory G(X i ,Y i ,Z i )’s identity information;

[0041] An analysis module is used to analyze the target trajectory G(X) according to the identity information. i ,Y i ,Z i ) and the time series are jointly analyzed.

[0042] In order to solve the above technical problems, the present invention also provides a computer device, which adopts the technical solution described below, including a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned target coordinate matching and analysis method when executing the computer-readable instructions.

[0043] In order to solve the above technical problems, the present invention also provides a computer-readable storage medium, which adopts the technical solution described below, and stores computer-readable instructions on the computer-readable storage medium. When the computer-readable instructions are executed by the processor, the steps of the above-mentioned target coordinate matching and analysis method are implemented.

[0044] Compared with the prior art, the present invention mainly has the following beneficial effects: by acquiring and tracking the target from the video sequences of multiple cameras, it is possible to achieve all-round monitoring of the target and effectively avoid monitoring blind spots; by converting pixel coordinates into world coordinates, not only the precise position of the target in space is obtained, but also its trajectory information and time series can be obtained, providing rich data support for subsequent trajectory analysis; through coordinate matching and fusion technology, the coordinate trajectories belonging to the same target can be accurately identified and connected to form a complete space-time trajectory, which is crucial for analyzing the movement mode and behavior characteristics of the target; by extracting the identity information of the target trajectory and combining it with the time series for joint analysis, the target's whereabouts can be deeply understood, providing a powerful decision-making basis for security monitoring, crowd flow analysis and other fields; based on the multi-target multi-camera tracking and positioning method, the behavior analysis of the target in three-dimensional space is realized, and the cross-camera method makes the target trajectory longer and the behavior analysis more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the scheme in the present invention, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 is a flow chart of an embodiment of the target coordinate matching and analysis method of the present invention;

[0047] Figure 2 It is a schematic diagram of an application scenario of the target coordinate matching and analysis method of the present invention;

[0048] Figure 3 is a target recognition flow chart used in the target coordinate matching and analysis method of the present invention;

[0049] Figure 4It is a flow chart of coordinate matching, fusion and identity confirmation adopted in the target coordinate matching and analysis method of the present invention;

[0050] Figure 5 is a schematic structural diagram of an embodiment of a target coordinate matching and analyzing device of the present invention;

[0051] Figure 6 is a schematic structural diagram of another embodiment of the target coordinate matching and analysis device of the present invention;

[0052] Figure 7 yes Figure 6 Summary flow chart of the target positioning module;

[0053] Figure 8 yes Figure 6 Summary flow chart of coordinate matching and fusion module and identity confirmation module;

[0054] Fig. 9 It is a basic structural block diagram of the computer device of this embodiment. DETAILED DESCRIPTION

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present invention; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention; the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present invention or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0056] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0057] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0058] It should be noted that the target coordinate matching and analysis method provided in the embodiment of the present invention is generally executed by a server / terminal device, and accordingly, the target coordinate matching and analysis device is generally disposed in the server / terminal device.

[0059] It should be understood that the number of terminal devices, networks and servers is only illustrative. Depending on the implementation requirements, there can be any number of terminal devices, networks and servers.

[0060] Embodiment 1

[0061] Please refer to Figure 1 , Figure 2 , respectively show a flow chart of an embodiment of the target coordinate matching and analysis method of the present invention and a schematic diagram of its application scenario. The target coordinate matching and analysis method comprises the following steps:

[0062] Step S1, obtaining a video sequence, where the video sequence comes from multiple cameras.

[0063] In this embodiment, the electronic device (e.g., server / terminal device) on which the target coordinate matching and analysis method is running can receive the target coordinate matching and analysis request through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAXX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0064] In this embodiment, step S1, obtaining a video sequence, where the video sequence is from multiple cameras, may further include the following steps:

[0065] S11, lists all cameras and their positions that will be used to acquire the video sequence.

[0066] In order to monitor the whereabouts of the target, multiple cameras are usually deployed at designated monitoring locations through the network according to actual needs. All cameras and their location information that will be used to obtain video sequences can be obtained through IP addresses. For example, three high-definition network cameras are selected and deployed at key locations in the target area: one is installed at the main entrance to capture the dynamics of people entering and leaving; one is set up in the central area to ensure full coverage and capture core activities; and another is deployed in a hidden corner to prevent missing any monitoring blind spots. These cameras all support remote control and real-time data transmission, and can be connected to the central monitoring room through a high-speed network.

[0067] S12, start the video capture software and set the capture parameters.

[0068] For example, start VidCap or other video capture tools. Create a capture file in the capture software to store the captured video sequence. Set the frame rate of the captured video as required (such as 24fps, 30fps, etc.). If continuous capture is required, set the start and end time of the capture. To save labor costs, set the automatic capture mode.

[0069] S13, capturing video sequences from multiple cameras according to preset capture parameters.

[0070] In the automatic capture mode, the capture software will automatically capture the video sequence according to the preset capture parameters and store the captured video sequence in a capture file.

[0071] Step S2: Detect and track the target P from the video sequence and output the pixel coordinates P(x i ,y i ), x i Indicates the offset of the target P relative to the x-axis direction, y i Indicates the offset of the target P relative to the y-axis direction.

[0072] In this embodiment, step S2 detects and tracks the target P from the video sequence and outputs the pixel coordinates P (x i ,y i ), x i Indicates the offset of the target P relative to the x-axis direction, y i The method of expressing the offset of the target P relative to the y-axis direction may further include the following steps:

[0073] S21, using the TrtYOLO image recognition algorithm, detects the frame of the target P from the video sequence.

[0074] The TrtYOLO image recognition algorithm, as a variant or optimized version of the YOLO series of algorithms, inherits the core ideas and technical advantages of the YOLO algorithm. In TrtYOLO, the algorithm divides the input image into grids and predicts one or more bounding boxes and corresponding confidence scores for each grid cell. These bounding boxes are used to locate the target object in the image, and the confidence score indicates the probability of the object existing in the bounding box. In this way, TrtYOLO can quickly and efficiently detect multiple objects in an image.

[0075] The original image is divided into S×S grids. If the center of an object falls into a grid, this grid is responsible for detecting the object.

[0076] Each grid needs to predict B boundary regions and C category probabilities Pr. C is the total number of network categories. In this embodiment, C=5, including the following categories: people, bicycles, buses, cars and motorcycles; each grid has only one C category, which is equivalent to ignoring B boundary regions and judging the category of each grid only once.

[0077] In addition to regressing its own position, each boundary region also needs to predict a confidence value. This confidence value represents two pieces of information: the confidence that the predicted box contains the object and how accurate the prediction of this boundary region is:

[0078]

[0079] If the center of an object falls within the grid, Pr(Object) = 1; otherwise Pr(Object) = 0. The second term is the intersection-over-union ratio between the predicted boundary area and the actual effective value. Therefore, each boundary area contains 5 prediction quantities: (x, y, w, h, confidence), where (x, y) represents the predicted box relative to the center of the grid, (w, h) is the ratio of the predicted box to the width and height of the image, and confidence is the above confidence. Among them, x, y, w and h are all normalized.

[0080] Since the input image is divided into S×S grids, each grid includes 5 prediction quantities: (x, y, w, h, confidence) and a C class, the network output size is S×S×(5×B+C).

[0081] When detecting an object, the class conditional probability predicted by each grid is multiplied by the confidence information predicted by the boundary area to obtain a class-specific confidence score for each boundary area:

[0082]

[0083] The category-specific confidence score includes both the probability that the boundary area ultimately belongs to a certain category and the accuracy of the boundary area location. Finally, a threshold is set to compare with the category-specific confidence score, and boxes with scores below the threshold are filtered out. Then, boxes with scores above the threshold are non-maximum suppressed to obtain the final detection box.

[0084] S22, based on the frame, uses the DeepSort multi-target tracking algorithm to track the target P and output the pixel coordinates P(x i ,y i ) and the corresponding number, where x i Indicates the offset of the target P relative to the x-axis direction, y i Indicates the offset of the target P relative to the y-axis direction.

[0085] The DeepSort multi-target tracking algorithm is used to track the detected target P. The DeepSort algorithm combines the deep learning model and the Kalman filter to predict the position of the target P in the subsequent frames, and calculates the matching degree between the targets in the previous and next frames through the Hungarian algorithm and cascade matching, and assigns a unique ID to each tracked target.

[0086] During the tracking process, for each successfully matched target P in each frame, the pixel coordinates of its frame are extracted. These coordinates represent the specific position of the target P in the current frame.

[0087] The pixel coordinates of the target P and its corresponding number (i.e., the assigned ID) are outputted. In this way, the target P can be continuously tracked in the entire video sequence and its accurate position information in each frame can be obtained.

[0088] Through the DeepSort multi-target tracking algorithm, the target features of the target P can also be obtained. The target features include but are not limited to: the color histogram of the target area as a feature, the contour features of the target, the texture features of the target, etc.

[0089] The multi-target tracking algorithm can also adopt mean shift algorithm, particle filter algorithm, target modeling algorithm and the like.

[0090] Step S3: The pixel coordinates P(x i ,y i ) is converted into the corresponding world coordinate Q(X i ,Y i ,Z i ), obtain the trajectory information of the target P in the world coordinate system and the time series in different time periods.

[0091] In this embodiment, step S3, the pixel coordinates P(x i ,y i ) is converted into the corresponding world coordinate Q(X i ,Y i ,Z i ), obtaining the trajectory information of the target P in the world coordinate system and the time series in different time periods may also include the following steps:

[0092] S31, given a point P'(x i ,y i ) and the corresponding world coordinate point Q'(X i ,Y i ,Z i ), there is a transformation matrix M for transforming between 3×4 pixel coordinates and world coordinates, there is a virtual-real transformation relationship P'=MQ', and the virtual-real transformation matrix M is established through at least six sets of virtual-real corresponding points, M=K[RT], Among them, w is the scaling factor, K is the 3×3 intrinsic parameter matrix of the camera, R is the 3×3 rotation matrix, T is the translation vector, f is the focal length of the camera, s is the camera offset, and x0 and y0 represent the main points of the video sequence.

[0093] Step S31 is to calibrate the camera, where x0 and y0 are generally the center points of the image. The more corresponding points calibrated, the more conducive it is to reducing errors and obtaining a more accurate matrix M and scaling factor w.

[0094] The world coordinate system is selected as WGS84 coordinates.

[0095] S32, through the virtual-real conversion matrix relationship P=MQ, according to the pixel coordinates P(x i ,y i ), calculate the world coordinates Q(X i ,Y i ,Z i );

[0096] Among them, w is the scaling factor, K is the 3×3 intrinsic parameter matrix of the camera, R is the 3×3 rotation matrix, T is the translation vector, f is the focal length of the camera, s is the camera offset, and x0 and y0 represent the main points of the video sequence.

[0097] The camera's internal and external parameters can be obtained through a calibration process or by reading a pre-stored parameter file.

[0098] S33, repeating the conversion process from pixel coordinates to world coordinates at multiple time points to obtain a set of world coordinates of the target P at different time points, that is, trajectory information;

[0099] The trajectory information refers to the set of world coordinates of the target P at different time points. In order to obtain the trajectory information, it is necessary to repeat the above-mentioned conversion process from pixel coordinates to world coordinates at multiple time points.

[0100] If the target P moves in the video frame, it is necessary to process the video frame by frame, obtain the pixel coordinates of the target P in each frame, and then convert them into world coordinates.

[0101] S34, recording the timestamp corresponding to each world coordinate, arranging the trajectory information in the order of the timestamps, and generating a time series.

[0102] In order to generate a time series, it is necessary to record the timestamp corresponding to each world coordinate. This can be achieved through the index or timestamp information of the video frame.

[0103] Assuming timestamps are in seconds and frame rate is fps:

[0104]

[0105] The timestamp can be calculated by the following code:

[0106] timestamp = frame_index / fps

[0107] timestamps.append(timestamp)

[0108] A time series is a data set that arranges trajectory information in chronological order. It can be obtained by combining world coordinates and corresponding timestamps. The trajectory information and timestamps are combined into a time series data set. The time series data can be stored in a file for subsequent analysis and processing. Step S4, through coordinate matching and fusion, the coordinate trajectories belonging to the same target are identified and connected into an overall target trajectory G(X i ,Y i ,Z i ), and fuse the trajectories belonging to the same target to obtain the spatiotemporal trajectory of the same target.

[0109] In this embodiment, step S4, through coordinate matching and fusion, the coordinate tracks belonging to the same target are identified and connected into an overall target track G(X i ,Y i ,Z i ), and merge the trajectories belonging to the same target to obtain the spatiotemporal trajectory of the same target, which may specifically include the following steps:

[0110] S41, obtaining an overlapping area S in the video coverage areas of multiple cameras.

[0111] The video frames from each camera are pre-processed, including grayscale, noise reduction, etc., to improve the accuracy of subsequent processing.

[0112] Image stitching algorithms such as SIFT (Scale Invariant Feature Transform) or SURF (Speeded Up Robust Features) are used to extract feature points in video frames, and the correspondence between adjacent frames or video frames from different cameras is determined through feature matching algorithms.

[0113] According to these correspondences, the video frames are spatially transformed and registered so that they are in the same coordinate system.

[0114] The overlapping area can be found by using image overlay or mask operation. Specifically, each video frame can be regarded as a layer, which is overlaid according to its spatial position, and the overlapping area S is determined by calculating the intersection of pixel values.

[0115] In order to accurately obtain the boundaries of the overlapping areas, edge detection and contour extraction algorithms can be further applied to refine the overlapping areas.

[0116] S42, traverse n target trajectories Qn, and determine the trajectory segment Q in the same time period t and the same overlapping area S n (t)∈S.

[0117] For each target trajectory Q n Perform preprocessing, including extracting key feature points, timestamps and other information for subsequent comparison and analysis.

[0118] According to the time period t, the video data of all cameras in the time period are filtered out, and the video data are analyzed to extract the area information that the target trajectory may pass through.

[0119] Using the spatial matching algorithm, the extracted trajectory features are compared with the regional information in the video data to find the overlapping area S in the video coverage area of ​​multiple cameras in the same time period t.

[0120] It is also possible to find the trajectory segment Q in the overlap area S n (t)∈S are further analyzed and verified to ensure their accuracy and reliability.

[0121] S43, calculate trajectory deviation |Q m (t k )-Q v (t k )|, where Q m (t k ) indicates that at time t k Track segment Q m The world coordinates, Q v (t k ) indicates that at time t k Track segment Q v The world coordinates of .

[0122] Take out time t k Track segment Q m The world coordinates and time t k Track segment Q v The world coordinates of the two are used to calculate the distance difference between them.

[0123] S44, judge when Q m (t) and Q v (t) corresponds to time t k The distances are all lower than the preset distance threshold d in the world coordinates, i.e. |Q m (t k )-Q v (t k)|≤d, it is considered that Q m and Q v It’s the same goal.

[0124] If the distance difference is within the preset distance threshold d, that is, |Q m (t k )-Q v (t k )|≤d holds, then Q m and Q v The same goal. In specific implementation, d can be determined according to actual needs, and no limitation is made here. In this embodiment, d is set to 10 cm.

[0125] S45, Q m and Q v The trajectories at different time periods are connected in time series to obtain the overall target trajectory G(X i ,Y i ,Z i ).

[0126] The location information of the target at different time points has been obtained, and then the time series analysis algorithm is used to sort and calibrate these discrete data points to ensure the accuracy and continuity of the timestamp. On this basis, the trajectory interpolation method can be used to reasonably infer and supplement the missing or large error data segments, so as to build a more complete and accurate trajectory model. Then, the processed trajectory data can be connected and displayed in time series through visualization tools to form the overall target trajectory. This process not only improves the accuracy and reliability of trajectory data, but also realizes the seamless connection of target trajectories in different time periods, and also provides a solid foundation for subsequent trajectory analysis and application.

[0127] S46, break up the points of any two trajectories, reorder them according to the time component of the coordinates, and take the midpoint to replace the original two points with the same time component. The reordered point sequence is the merged space-time trajectory.

[0128] The points in the two original trajectories are extracted separately to form two independent point sets. These points contain longitude, latitude and timestamp information. Then all the points in the two point sets are merged into a large point list while retaining the trajectory identifiers to which they belong.

[0129] Then sort all the points according to the timestamp. This step ensures that the time component becomes the main basis for reorganizing the point list. It should be noted that in the sorting process, the consistency of the time format and the time zone difference need to be handled to ensure the accuracy of the sorting.

[0130] For points with exactly the same time components, that is, the intersection of two trajectories at the same time point, the midpoint method is used to replace the original two points. The midpoint calculation can be based on the average value of the geographic coordinates, that is, the arithmetic mean of the longitude and latitude of the two points is taken as the position of the new point. At the same time, the timestamp remains unchanged.

[0131] Finally, after sorting and taking the midpoint, a new point sequence is obtained, which is the merged space-time trajectory. This trajectory not only contains all the information of the original two trajectories, but also integrates them in an orderly manner in the time dimension, which facilitates subsequent analysis and processing. The whole process requires precise time processing and geographic coordinate calculation to ensure the accuracy and continuity of the merged trajectory in time and space.

[0132] Step S5, extract the target trajectory G(X i ,Y i ,Z i )’s identity information.

[0133] In this embodiment, step S5, extracting the target trajectory G(X i ,Y i ,Z i )'s identity information may specifically include the steps of:

[0134] S51, according to the target feature of the target P, the target feature is compared with the identity database.

[0135] The identity database is a unique attribute library that can also be manually encoded uniquely, including but not limited to: face library, body library, gait library, iris, license plate, unique identification code, etc.

[0136] When the target is a person, a face database is used as a unique code.

[0137] When the target is a car, the license plate database is used as the unique code.

[0138] S52, according to the comparison result, obtain the target trajectory G(X i ,Y i ,Z i )’s identity information.

[0139] When the detected target is a person, the target image is captured and uploaded to the face database. If the confidence of the returned result is greater than 95%, the target identity information is assigned according to the returned result. Otherwise, it is uploaded to the human body database. If the confidence is greater than 95%, the target identity information is assigned according to the returned result.

[0140] The identity acquisition process starts from the moment the target appears on the screen and continues until valid information about the target is acquired or the target disappears. Identity information includes but is not limited to ID number, source (such as permanent population, drug abuse, theft), etc.

[0141] When the detected target is a car, the target image is captured and uploaded to the vehicle database. If the confidence of the returned result is greater than 95%, the target license plate information is assigned according to the returned result.

[0142] The license plate acquisition process starts from the moment the target appears in the picture and continues until valid target information is acquired or the target disappears. The license plate information includes the license plate number and whether it is a key vehicle.

[0143] Multiple cameras compare the identity of the target P. Since the positions, angles, and distances of the cameras are different, the comparison results of the cameras vary. However, as long as one camera successfully confirms the identity of the target P, the overall target trajectory G(X i ,Y i ,Z i )’s identity.

[0144] Step S6: According to the identity information, the target trajectory G(X i ,Y i ,Z i ) and time series for joint analysis.

[0145] In this embodiment, step S6, according to the identity information, the target trajectory G(X i ,Y i ,Z i ) and time series joint analysis can also include the following steps:

[0146] S61, according to the identity information, the target trajectory G(X i ,Y i ,Z i ) and time series, and conduct speed analysis and / or direction analysis and / or area analysis and / or track analysis.

[0147] For example, when the identity information is a person or a car, the trajectory and time series of the person and car are jointly analyzed to obtain valuable information, including but not limited to speed, direction of travel, area, characteristics of the track, etc., which can be further analyzed and judged for abnormalities and violations of the target.

[0148] G(X i ,Y i ,Z i ) indicates that at time t i The world coordinates of G(X j ,Y j ,Z j) indicates that at time t j world coordinates, without loss of generality, assuming that t i <t j ; O is the origin selected in the world coordinate system, U is the area of ​​interest, and l is a vector line.

[0149] Calculation of velocity v:

[0150] Based on the speed calculation results, it can be determined whether the target vehicle is speeding. For speeding vehicles, the police will be notified in time and video will be recorded for evidence.

[0151] Calculation of the direction of travel:

[0152] Based on the calculation results of the driving, it can be determined whether the target vehicle is going in the wrong direction. If a vehicle is going in the wrong direction, the police will be notified in time and video will be recorded for evidence.

[0153] Regional Analysis: G(X i ,Y i ,Z i )∈U means that the target is in the key prevention and control area; Indicates that the target is not in the key prevention and control area. This can be used to determine whether the target vehicle illegally occupies the emergency lane or fails to drive in the designated lane.

[0154] Track characteristics: According to G(X i ,Y i ,Z i ) and l can determine whether the target is on the "left" or "right" of the line; according to G(X i ,Y i ,Z i The change of the relative position of ) and l over time, such as left and right, can be used to determine whether the vehicle is changing lanes frequently or whether it is changing lanes in a solid line.

[0155] Figure 3 This is a target recognition flow chart used in the target coordinate matching and analysis method of the present invention. Figure 3 As shown, the target recognition process includes the following steps:

[0156] S101, create a dictionary:

[0157] The video frame queue dictionary (video_container) of the device, the queue dictionary (detected_info_queue_dict) of the monitoring information, and the dictionary (detobj listdict) of the device detection target list are stored.

[0158] S102, read the initialization device list: getDevlnfoFromini().

[0159] S103, instantiate the device tree: obtain the attribute parameters of the corresponding device init DeviceTree().

[0160] S104, initialize T-hot YOLO and deepsort models.

[0161] S105, create a video window: CreateQlabel list().

[0162] After step S105, the process may directly proceed to step S106 or step S108.

[0163] S106, acquiring a video image of each device.

[0164] S107, initialize the target detection thread: detectTRTFrameThread.

[0165] After step S107, go directly to step S110.

[0166] S108, creating queues: a queue for storing mot-processed video images (mot_frame_queue) and a queue for storing video images with people (personframe queue).

[0167] S109, initialization: initialization of the video interruption restart thread: videoInterruptRestrartThresd, initialization of the personnel information sending thread: sendPersonidInfoThread, initialization of the personnel information obtaining thread: getPersonIDTread, initialization of the tracking thread: motthread.

[0168] S110, start the previewAll function: start the target detection thread (detectTRTFrameThread); start the video interruption restart thread (videoInterruptRestartThread); start the send personnel information thread (sendPersonidInfoThread); start the get personnel information thread (getPersonIDTread); start the tracking thread (mot thread).

[0169] Figure 4 It is a flow chart of coordinate matching, fusion and identity confirmation used in the target coordinate matching and analysis method of the present invention.

[0170] like Figure 4 As shown, the coordinate matching and fusion and identity confirmation process includes the following steps:

[0171] The trajectory matching traversal begins and enters the trajectory matching process:

[0172] S201, select target track A.

[0173] S202, select a field of view to overlap other tracks B.

[0174] S203, calculating the average difference between the trajectories of A and B.

[0175] S204, judging whether the average difference of the trajectories is higher than a threshold value, if yes, proceeding to step S205, otherwise, the trajectory matching process ends.

[0176] S205, the matching is successful, and the track pair serial numbers A & B are recorded. The matching is completed and the process proceeds to step S206.

[0177] S206 , generating a trajectory chain according to the trajectory pair set, such as A&B&C . . . and so on.

[0178] After step S206, the original database (storing the target trajectory, etc.) can be returned to enter the data reading process, and then the process can be entered into step S207. After step S206, the process can also be directly entered into the trajectory merging process, and then the process can be entered into step S209.

[0179] S207, eliminating short trajectories.

[0180] S208, extracting data within 10 minutes.

[0181] After step S208, the trajectory matching process starts again and enters the trajectory matching process.

[0182] S209, traverse the trajectory chain.

[0183] S210, determine whether the identity information is consistent, if yes, go to step S212, otherwise go to step S211.

[0184] S211, merge the trajectory without identity into the trajectory with identity. Go to step S213.

[0185] S212, merge trajectory information, and proceed to step S213.

[0186] S213, merge other information.

[0187] S214, the merging process ends, and then the data secondary merging and storage process is entered, and the process proceeds to step S215.

[0188] S215, identity information matching.

[0189] S216, generate a trajectory chain, such as A&B&C...etc.

[0190] S217, perform a secondary merge, and then generate new data, which is stored in a new database.

[0191] The implementation of this embodiment has the following beneficial effects: by acquiring and tracking the target from the video sequences of multiple cameras, it is possible to achieve all-round monitoring of the target and effectively avoid monitoring blind spots; by converting pixel coordinates into world coordinates, not only the precise position of the target in space is obtained, but also its trajectory information and time series can be obtained, providing rich data support for subsequent trajectory analysis; through coordinate matching and fusion technology, the coordinate trajectories belonging to the same target can be accurately identified and connected to form a complete space-time trajectory, which is crucial for analyzing the movement pattern and behavior characteristics of the target; by extracting the identity information of the target trajectory and combining it with the time series for joint analysis, the target's whereabouts can be deeply understood, providing a strong decision-making basis for security monitoring, crowd flow analysis and other fields; based on the multi-target multi-camera tracking and positioning method, the behavior analysis of the target in three-dimensional space is realized, and the cross-camera method makes the target trajectory longer and the behavior analysis more accurate.

[0192] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0193] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0194] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0195] Embodiment 2

[0196] Further references Figure 5 , as a response to the above Figure 1 The present invention provides an embodiment of a target coordinate matching and analysis device, and the device embodiment is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0197] like Figure 5 As shown, the target coordinate matching and analysis device 70 of this embodiment includes: an acquisition module 71, a detection and tracking module 72, a conversion module 73, a fusion module 74, an extraction module 75 and an analysis module 76. Among them:

[0198] An acquisition module 71 is used to acquire a video sequence, where the video sequence comes from multiple cameras;

[0199] The detection and tracking module 72 is used to detect and track the target P from the video sequence and output the pixel coordinates P (x i ,y i ), x i Indicates the offset of the target P relative to the x-axis direction, y i Indicates the offset of the target P relative to the y-axis direction;

[0200] The conversion module 73 is used to convert the pixel coordinates P(x i ,y i ) is converted into the corresponding world coordinate Q(X i ,Y i ,Z i ), obtain the trajectory information of the target P in the world coordinate system and the time series in different time periods;

[0201] Fusion module 74 is used for coordinate matching and fusion, identifying the coordinate tracks belonging to the same target and connecting them into an overall target track G(X i ,Y i,Z i ), and fuse the trajectories belonging to the same target to obtain the spatiotemporal trajectory of the same target, G(X i ,Y i ,Z i ) represents the target P at time t i The world coordinates of

[0202] Extraction module 75, used to extract the target trajectory G(X i ,Y i ,Z i )’s identity information;

[0203] The analysis module 76 is used to analyze the target trajectory G(X i ,Y i ,Z i ) and time series for joint analysis.

[0204] The implementation of this embodiment has the following beneficial effects: by acquiring and tracking the target from the video sequences of multiple cameras, it is possible to achieve all-round monitoring of the target and effectively avoid monitoring blind spots; by converting pixel coordinates into world coordinates, not only the precise position of the target in space is obtained, but also its trajectory information and time series can be obtained, providing rich data support for subsequent trajectory analysis; through coordinate matching and fusion technology, the coordinate trajectories belonging to the same target can be accurately identified and connected to form a complete space-time trajectory, which is crucial for analyzing the movement pattern and behavior characteristics of the target; by extracting the identity information of the target trajectory and combining it with the time series for joint analysis, the target's whereabouts can be deeply understood, providing a strong decision-making basis for security monitoring, crowd flow analysis and other fields; based on the multi-target multi-camera tracking and positioning method, the behavior analysis of the target in three-dimensional space is realized, and the cross-camera method makes the target trajectory longer and the behavior analysis more accurate.

[0205] Embodiment 3

[0206] Figure 6 FIG. 1 is a schematic diagram of the structure of another embodiment of the target coordinate matching and analysis device of the present invention. Figure 6 As shown, the target coordinate matching and analysis device of the present invention includes a camera target recognition module, a camera target positioning module, a cross-camera coordinate matching and fusion module, an identity confirmation module, and a behavior analysis module, which ultimately obtains valuable information such as the target's speed, direction of travel, area, and track characteristics, and can further analyze and determine the target's abnormalities, illegal behaviors, etc., providing strong technical support for fields such as public security prevention and control.

[0207] Figure 7 yes Figure 6 The target positioning module overview flow chart is shown in Figure 2. Figure 7As shown in the figure, the target recognition module is used to detect and track the target P in the video sequence output by the camera, and output the pixel coordinates P (x i ,y i ). The target recognition module executes the target recognition and tracking program, and further includes a video image acquisition module, a target detection module, a target tracking module, a personnel information acquisition module, a personnel information sending module, a video disconnection processing module, etc. The target detection module adopts the TrtYOLO image recognition algorithm, and the target tracking module adopts the deepsort multi-target tracking algorithm.

[0208] The target positioning module is designed to accurately track and locate the target object. Its outline process is as follows:

[0209] First, high-precision measurement equipment is used to obtain control point data (1), which is the basis for subsequent coordinate conversion. Then, using the known control point data, the collected coordinates are converted into the globally accepted WGS84 coordinate system (2) to ensure the consistency and accuracy of the positioning results.

[0210] Camera calibration (3) is a key step to ensure that image information is accurately converted into spatial coordinates. It involves adjusting the camera's internal parameters. On this basis, the camera's internal and external parameters (4) are determined, which are crucial for subsequent target positioning.

[0211] The system then identifies and records the target number and its corresponding pixel coordinates (5), which are the direct representation of the target in the image. Using this information, the positioning algorithm module (6) starts working and converts the pixel coordinates into actual space coordinates through a complex calculation model.

[0212] In order to track the changes in the target's location, the system generates a time series of the target's location (7), recording the location information of the target at different time points. Finally, this information is stored in a database (8) to facilitate subsequent data analysis and mining.

[0213] The entire process is closely linked, and each link supports each other, jointly realizing the efficient and precise operation of the target positioning module.

[0214] Figure 8 yes Figure 6 The coordinate matching and fusion module and the identity confirmation module are shown in the figure. Figure 8 As shown in the figure, by eliminating short trajectories (1), short and unstable trajectory data caused by noise or false detection are filtered out, providing a reliable basis for subsequent processing. Then, according to the spatiotemporal uniqueness matching step (2), the temporal continuity and spatial consistency of the trajectory are used to establish associations between multiple camera perspectives to achieve cross-camera trajectory matching. Finally, through the trajectory fusion step (3), the successfully matched trajectories are merged to form a continuous and complete motion trajectory.

[0215] The identity confirmation module takes this a step further. First, the identity database comparison step (1) is performed to compare the fused trajectory data with the information in the identity database to find potential identity matches. Then, in the identity information different identity information discrimination step (2), the comparison results are carefully analyzed to distinguish between consistent and inconsistent identity information, and the inconsistent ones are further processed or marked.

[0216] During the entire process, the merged trajectory data, database and original trajectory data are interrelated and supportive of each other, together forming the core data processing framework of the coordinate matching and fusion module and the identity confirmation module.

[0217] The implementation of this embodiment has the following beneficial effects: by acquiring and tracking the target from the video sequences of multiple cameras, it is possible to achieve all-round monitoring of the target and effectively avoid monitoring blind spots; by converting pixel coordinates into world coordinates, not only the precise position of the target in space is obtained, but also its trajectory information and time series can be obtained, providing rich data support for subsequent trajectory analysis; through coordinate matching and fusion technology, the coordinate trajectories belonging to the same target can be accurately identified and connected to form a complete space-time trajectory, which is crucial for analyzing the movement pattern and behavior characteristics of the target; by extracting the identity information of the target trajectory and combining it with the time series for joint analysis, the target's whereabouts can be deeply understood, providing a strong decision-making basis for security monitoring, crowd flow analysis and other fields; based on the multi-target multi-camera tracking and positioning method, the behavior analysis of the target in three-dimensional space is realized, and the cross-camera method makes the target trajectory longer and the behavior analysis more accurate.

[0218] Embodiment 4

[0219] To solve the above technical problems, the embodiment of the present invention also provides a computer device. Fig. 9 , Fig. 9 This is a basic structural block diagram of the computer device in this embodiment.

[0220] The above-mentioned computer device 8 includes a memory 81, a processor 82, and a network interface 83 which are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 8 having components of a memory 81, a processor 82, and a network interface 83, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0221] The computer device may be a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with the user through a keyboard, a mouse, a remote control, a touch pad, or a voice control device.

[0222] The memory 81 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 81 can be an internal storage unit of the computer device 8, such as a hard disk or memory of the computer device 8. In other embodiments, the memory 81 can also be an external storage device of the computer device 8, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 8. Of course, the memory 81 can also include both the internal storage unit of the computer device 8 and its external storage device. In this embodiment, the memory 81 is usually used to store the operating system and various application software installed on the computer device 8, such as computer-readable instructions of the target coordinate matching and analysis method, etc. In addition, the memory 81 can also be used to temporarily store various types of data that have been output or are to be output.

[0223] The processor 82 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 82 is generally used to control the overall operation of the computer device 8. In this embodiment, the processor 82 is used to run the computer-readable instructions stored in the memory 81 or process data, such as running the computer-readable instructions of the target coordinate matching and analysis method.

[0224] The network interface 83 may include a wireless network interface or a wired network interface. The network interface 83 is generally used to establish a communication connection between the computer device 8 and other electronic devices.

[0225] The implementation of this embodiment has the following beneficial effects: by acquiring and tracking the target from the video sequences of multiple cameras, it is possible to achieve all-round monitoring of the target and effectively avoid monitoring blind spots; by converting pixel coordinates into world coordinates, not only the precise position of the target in space is obtained, but also its trajectory information and time series can be obtained, providing rich data support for subsequent trajectory analysis; through coordinate matching and fusion technology, the coordinate trajectories belonging to the same target can be accurately identified and connected to form a complete space-time trajectory, which is crucial for analyzing the movement pattern and behavior characteristics of the target; by extracting the identity information of the target trajectory and combining it with the time series for joint analysis, the target's whereabouts can be deeply understood, providing a strong decision-making basis for security monitoring, crowd flow analysis and other fields; based on the multi-target multi-camera tracking and positioning method, the behavior analysis of the target in three-dimensional space is realized, and the cross-camera method makes the target trajectory longer and the behavior analysis more accurate.

[0226] Embodiment 5

[0227] The present invention also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions. The computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the target coordinate matching and analysis method as described above.

[0228] The implementation of this embodiment has the following beneficial effects: by acquiring and tracking the target from the video sequences of multiple cameras, it is possible to achieve all-round monitoring of the target and effectively avoid monitoring blind spots; by converting pixel coordinates into world coordinates, not only the precise position of the target in space is obtained, but also its trajectory information and time series can be obtained, providing rich data support for subsequent trajectory analysis; through coordinate matching and fusion technology, the coordinate trajectories belonging to the same target can be accurately identified and connected to form a complete space-time trajectory, which is crucial for analyzing the movement pattern and behavior characteristics of the target; by extracting the identity information of the target trajectory and combining it with the time series for joint analysis, the target's whereabouts can be deeply understood, providing a strong decision-making basis for security monitoring, crowd flow analysis and other fields; based on the multi-target multi-camera tracking and positioning method, the behavior analysis of the target in three-dimensional space is realized, and the cross-camera method makes the target trajectory longer and the behavior analysis more accurate.

[0229] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the various embodiment methods of the present invention.

[0230] Obviously, the embodiments described above are only some embodiments of the present invention, rather than all embodiments. The preferred embodiments of the present invention are given in the accompanying drawings, but they do not limit the patent scope of the present invention. The present invention can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive. Although the present invention has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific embodiments, or to perform equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of the present invention, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present invention.

Claims

1. A target coordinate matching and analysis method, characterized in that: The steps include: Acquire a video sequence, wherein the video sequence comes from a plurality of cameras; Detect and track the target P from the video sequence, and output the pixel coordinates P(x i ,y i ), x i Indicates the offset of the target P relative to the x-axis direction, y i Indicates the offset of the target P relative to the y-axis direction; The pixel coordinates P(x i ,y i ) is converted into the corresponding world coordinates Q(X i ,Y i ,Z i ), obtain the trajectory information of the target P in the world coordinate system and the time series in different time periods; Through coordinate matching and fusion, the coordinate tracks belonging to the same target are identified and connected into an overall target track G(X i ,Y i ,Z i ), and fuse the trajectories belonging to the same target to obtain the spatiotemporal trajectory of the same target, G(X i ,Y i ,Z i ) represents the target P at time t i The world coordinates of Extract the target trajectory G(X i ,Y i ,Z i )’s identity information; According to the identity information, the target trajectory G(X i ,Y i ,Z i ) and the time series are jointly analyzed.

2. The target coordinate matching and analysis method according to claim 1, characterized in that: The step of obtaining a video sequence, wherein the video sequence comes from multiple cameras, specifically comprises: List all cameras and their positions that will be used to acquire the video sequence; Start the video capture software and set the capture parameters; The video sequences from multiple cameras are captured according to preset capture parameters.

3. The target coordinate matching and analysis method according to claim 1, characterized in that: The target P is detected and tracked from the video sequence, and the pixel coordinates P (x i ,y i ), x i Indicates the offset of the target P relative to the x-axis direction, y i The steps of indicating the offset of the target P relative to the y-axis direction specifically include: Using the TrtYOLO image recognition algorithm, the frame of the target P is detected from the video sequence; Based on the framework, the DeepSort multi-target tracking algorithm is used to track the target P and output the pixel coordinates P(x i ,y i ) and the corresponding number, where x i Indicates the offset of the target P relative to the x-axis direction, y i Indicates the offset of the target P relative to the y-axis direction.

4. The target coordinate matching and analysis method according to claim 1, characterized in that: The pixel coordinates P(x i ,y i ) is converted into the corresponding world coordinates Q(X i ,Y i ,Z i ), the steps of obtaining the trajectory information of the target P in the world coordinate system and the time series in different time periods specifically include: Given a point P'(x i ,y i ) and the corresponding world coordinate point Q'(X i ,Y i ,Z i ), there is a transformation matrix M for transforming between 3×4 pixel coordinates and world coordinates, there is a virtual-real transformation relationship P'=MQ', and the virtual-real transformation matrix M is established through at least six sets of virtual-real corresponding points, Through the virtual-real conversion matrix relationship P = MQ, according to the pixel coordinates P (x i ,y i ), calculate the world coordinates Q(X i ,Y i ,Z i ); Repeat the conversion process from pixel coordinates to world coordinates at multiple time points to obtain the set of world coordinates of the target P at different time points, that is, trajectory information; Record the timestamp corresponding to each world coordinate, arrange the trajectory information in the order of the timestamps, and generate a time series; Among them, w is the scaling factor, K is the 3×3 intrinsic parameter matrix of the camera, R is the 3×3 rotation matrix, T is the translation vector, f is the focal length of the camera, s is the camera offset, and x0 and y0 represent the main points of the video sequence.

5. The target coordinate matching and analysis method according to claim 1, characterized in that: Through coordinate matching and fusion, the coordinate tracks belonging to the same target are identified and connected into an overall target track G(X i ,Y i ,Z i ), and fuse the trajectories belonging to the same target to obtain the spatiotemporal trajectory of the same target, G(X i ,Y i ,Z i ) represents the target P at time t i The steps of obtaining the world coordinates include: Obtain the overlapping area S in the video coverage areas of multiple cameras; Traverse n target trajectories Qn and determine the trajectory segment Q in the same time period t and the same overlapping area S n (t)∈S; Calculate trajectory deviation |Q m (t k )-Q v (t k )|, where Q m (t k ) indicates that at time t k Track segment Q m The world coordinates, Q v (t k ) indicates that at time t k Track segment Q v The world coordinates of Judge when Q m (t) and Q v (t) The distance corresponding to the time tk is lower than the preset distance threshold d in the world coordinates, that is, |Q m (t k )-Q v (t k )|≤d, Qm and Qv are considered to be the same target; Q m and Q v The trajectories at different time periods are connected in time series to obtain the overall target trajectory G(X i ,Y i ,Z i ); Break up the points of any two trajectories, reorder them according to the time component of the coordinates, and replace the original two points with the midpoint if the time component is the same. The reordered point sequence is the merged space-time trajectory.

6. The target coordinate matching and analysis method according to claim 1, characterized in that: The extraction of the target trajectory G(X i ,Y i ,Z i The steps of obtaining the identity information of the user include: According to the target feature of the target P, the target feature is compared with the identity database; According to the comparison results, the target trajectory G(X i ,Y i ,Z i )’s identity information.

7. The target coordinate matching and analysis method according to any one of claims 1 to 6, characterized in that: The target trajectory G(X) is i ,Y i ,Z i ) and the step of performing joint analysis on the time series specifically comprises: According to the identity information, the target trajectory G(X i ,Y i ,Z i ) and the time series, perform speed analysis and / or direction analysis and / or area analysis and / or track analysis.

8. A target coordinate matching and analysis device, characterized in that: include: An acquisition module, used for acquiring a video sequence, wherein the video sequence comes from a plurality of cameras; The detection and tracking module is used to detect and track the target P from the video sequence and output the pixel coordinates P (x i ,y i ), x i Indicates the offset of the target P relative to the x-axis direction, y i Indicates the offset of the target P relative to the y-axis direction; The conversion module is used to convert the pixel coordinates P(x i ,y i ) is converted into the corresponding world coordinates Q(X i ,Y i ,Z i ), obtain the trajectory information of the target P in the world coordinate system and the time series in different time periods; The fusion module is used for coordinate matching and fusion, identifying the coordinate tracks belonging to the same target and connecting them into an overall target track G(X i ,Y i ,Z i ), and fuse the trajectories belonging to the same target to obtain the spatiotemporal trajectory of the same target, G(X i ,Y i ,Z i ) represents the target P at time t i The world coordinates of Extraction module, used to extract the target trajectory G(X i ,Y i ,Z i )’s identity information; An analysis module is used to analyze the target trajectory G(X) according to the identity information. i ,Y i ,Z i ) and the time series are jointly analyzed.

9. A computer device, comprising a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the target coordinate matching and analysis method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the target coordinate matching and analysis method according to any one of claims 1 to 7 are implemented.

Images