A mobile target positioning method and system considering camera pitch angle

Abstract

The present invention discloses a method and system for positioning a mobile target taking into account the pitch angle of a camera, and relates to the technical field of vehicle positioning. The method comprises the steps of: obtaining the vehicle trajectory of a mobile target in a video to be detected and performing coordinate conversion; performing positioning correction on the vehicle trajectory using an error correction method to obtain the final vehicle positioning, wherein the positioning correction process comprises: obtaining the camera height and the camera pitch angle, constructing the relationship between the camera height, the camera pitch angle and the coordinate conversion error, calculating the coordinate conversion error, and calculating the actual position of the vehicle target based on the coordinate conversion error to obtain the coordinates of the vehicle positioning. The present invention utilizes the imaging principle of the camera optical lens and the projection geometric relationship to derive the theoretical relationship between the positioning error and the camera installation height and angle, thereby performing accurate error correction, and further realizing high-precision positioning of the mobile target.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle positioning, and in particular to a method and system for positioning a mobile target taking into account a camera pitch angle. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Real-time collection of vehicle driving data on the road is of great significance for intelligent road monitoring, surveillance, operation and management. The rapid development of information collection technology has led to the installation of more and more surveillance cameras on the road, and the development of computer vision technology has further provided reliable technical support for the extraction of real-time vehicle trajectory data based on video. However, for mobile targets such as vehicles, due to the driving needs of the driver and the operating status of road traffic flow, the movement trajectory of the vehicle is not regular, but is always in a dynamic process. Therefore, the precise positioning of the movement trajectory of mobile targets based on video has gradually become a key and difficult point in current research.

[0004] At present, it is difficult to calibrate traffic surveillance cameras installed at fixed positions on the road, and the heights and angles of surveillance cameras on different roads are also different, which hinders the accurate positioning of moving vehicles. In addition, previous studies mainly considered the detection and tracking of moving targets, ignoring the error accumulation during the motion trajectory tracking process. Previous studies also did not consider the relationship between the camera installation position height, pitch angle and the positioning accuracy of moving targets. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method and system for positioning a mobile target taking into account the pitch angle of the camera, and to utilize the imaging principle of the camera optical lens and the projection geometry relationship to derive the theoretical relationship between the positioning error and the camera installation height and angle, so as to perform accurate error correction and further achieve high-precision positioning of the mobile target.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0007] A first aspect of the present invention provides a method for positioning a mobile target taking into account a camera pitch angle, comprising the following steps:

[0008] Obtain the video to be detected, perform target detection and target tracking on the moving target in the video, and obtain the vehicle trajectory;

[0009] The coordinate transformation method is used to transform the vehicle trajectory;

[0010] The vehicle trajectory is positioned and corrected using an error correction method to obtain the final vehicle positioning, wherein the positioning correction process includes: obtaining the camera height and the camera pitch angle, constructing the relationship between the camera height, the camera pitch angle and the coordinate conversion error, calculating the coordinate conversion error according to the relationship between the camera height, the camera pitch angle and the coordinate conversion error, calculating the actual position of the vehicle target based on the coordinate conversion error, and obtaining the coordinates of the vehicle positioning.

[0011] Furthermore, YOLOv5 is used to detect the moving target in the video to be detected to obtain the moving target, and DeepSORT is used to track the moving target to obtain the vehicle trajectory.

[0012] Furthermore, the specific steps of using YOLOv5 to detect moving targets in the video to be detected are:

[0013] Divide the video to be detected into grids;

[0014] Make a prediction for each grid and get the bounding box, and the probability that each bounding box belongs to a different category;

[0015] The non-maximum suppression algorithm is used to remove bounding boxes whose overlap exceeds a set threshold and select the final detection result.

[0016] Furthermore, the specific steps of using DeepSORT to track a moving target are as follows:

[0017] Use the recursive Kalman filter algorithm to predict the location and state of the target in the next frame, perform IOU matching between the predicted box and the bounding box, and select the one with higher confidence as the prediction result;

[0018] Data association is performed using the linear weighted function of the Mahalanobis distance based on motion information and the cosine similarity of appearance features, and tracking trajectory matching is performed through cascade allocation to obtain the tracking result;

[0019] Update the tracker parameters synchronously and loop the algorithm process.

[0020] Furthermore, the specific steps of using the coordinate transformation method to transform the vehicle trajectory are as follows:

[0021] Establish geodetic coordinate system based on projection principle;

[0022] Get camera parameters and establish pixel coordinate system;

[0023] Calculate the conversion relationship between the earth coordinate system and the pixel coordinate system according to the vehicle trajectory;

[0024] Based on the conversion relationship, the pixel coordinates of the moving target photographed by the camera are converted into geodetic coordinates.

[0025] Furthermore, the specific steps for constructing the relationship between camera height, camera pitch angle and coordinate conversion error are:

[0026] Calculate the centroid coordinates of the target vehicle bounding box in the pixel coordinate system;

[0027] Calculate the centroid coordinates of the target vehicle bounding box in the geodetic coordinate system after coordinate transformation;

[0028] The relationship between the camera height, camera pitch angle and coordinate conversion error is determined based on the difference between the centroid coordinates of the pixel coordinate system and the geodetic coordinate system.

[0029] Furthermore, the relationship between camera height, camera pitch angle and coordinate conversion error is expressed as:

[0030] ,

[0031] ,

[0032] in, Represents the horizontal coordinate error after coordinate transformation, Represents the ordinate error after coordinate transformation, , , , are the four vertex coordinates of the vehicle bounding box in pixel-level coordinates, , , , and They correspond to P0, P1, P2, P3 and P4 respectively. , is half of the projection length of any point at its horizontal projection position. The frame width and frame height of the video image are and , the focal length of the camera is , is the height of the camera from the ground, is the vertical field of view of the camera, is the vertical near angle of the camera.

[0033] A second aspect of the present invention provides a mobile target positioning system considering the camera pitch angle, comprising:

[0034] The target detection and tracking module is configured to obtain a video to be detected, perform target detection and target tracking on a moving target in the video, and obtain a vehicle trajectory;

[0035] A coordinate conversion module is configured to perform coordinate conversion on the vehicle trajectory using a coordinate conversion method;

[0036] The error correction module is configured to use the error correction method to perform positioning correction on the vehicle trajectory to obtain the final vehicle positioning, wherein the positioning correction process includes: obtaining the camera height and the camera pitch angle, constructing the relationship between the camera height, the camera pitch angle and the coordinate conversion error, calculating the coordinate conversion error according to the relationship between the camera height, the camera pitch angle and the coordinate conversion error, calculating the actual position of the vehicle target based on the coordinate conversion error, and obtaining the coordinates of the vehicle positioning.

[0037] A third aspect of the present invention provides a medium having a program stored thereon, which, when executed by a processor, implements the steps of the method for positioning a mobile target taking into account the pitch angle of the camera as described in the first aspect of the present invention.

[0038] The fourth aspect of the present invention provides a device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for positioning a mobile target taking into account the pitch angle of the camera as described in the first aspect of the present invention are implemented.

[0039] One or more of the above technical solutions have the following beneficial effects:

[0040] The present invention discloses a method and system for positioning a mobile target taking into account the pitch angle of a camera. Aiming at the problem of error accumulation in the current mobile vehicle target tracking process, in the process of converting the vehicle position from pixel coordinates to geodetic coordinates, the imaging principle of the camera optical lens and the projection geometry relationship are used to deduce the theoretical relationship between the positioning error and the camera installation height and angle, so as to perform accurate error correction and further realize high-precision positioning of the mobile target.

[0041] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0043] Figure 1 This is a flow chart of a method for positioning a mobile target taking into account the pitch angle of a camera in Embodiment 1 of the present invention;

[0044] Figure 2 A schematic diagram showing the change of the center of the vehicle under different camera states in the first embodiment of the present invention;

[0045] Figure 3 Schematic diagram of a pixel coordinate system in Embodiment 1 of the present invention;

[0046] Figure 4 This is a schematic diagram of the ground projection principle in the first embodiment of the present invention;

[0047] Figure 5 A schematic diagram of a bounding box in a pixel coordinate system in Embodiment 1 of the present invention;

[0048] Figure 6 It is a schematic diagram of projection in the geodetic coordinate system in the first embodiment of the present invention. DETAILED DESCRIPTION

[0049] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0050] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "include" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or their combinations;

[0051] Embodiment 1:

[0052] Embodiment 1 of the present invention provides a method for positioning a mobile target taking into account the pitch angle of the camera. First, yolov5+deepsort is used to detect and track the target and output the pixel-level trajectory of the vehicle. Then, a coordinate conversion method is used to convert the pixel-level trajectory into real-world coordinates. Finally, an error correction method is used to correct the positioning.

[0053] like Figure 1 As shown, the specific steps include:

[0054] A method for positioning a moving target considering a camera pitch angle comprises the following steps:

[0055] Step 1: Obtain the video to be detected, perform target detection and target tracking on the moving target in the video, and obtain the vehicle trajectory.

[0056] Step 1.1: Get the video to be detected.

[0057] The development of computer vision technology makes it possible to detect and track target objects according to their spatial and temporal features throughout the video frame. Technically speaking, tracking is to obtain an initial detection set, assign a unique ID to the target object, and then track them throughout the frame of the video source. Target tracking can generally be divided into two steps: Target detection: This step is responsible for detecting and locating objects in the video screen using some detectors (such as YOLOv5, CenterNet, etc.). Motion trajectory predictor: This step is responsible for predicting the future movement of the object using its past information. Considering that vehicle target detection and tracking based on video streams needs to track multiple targets at the same time, and it is easy to have frequent occlusion of targets, this embodiment considers combining YOLOv5 with the multi-target tracking algorithm DeepSORT to detect vehicle moving targets in real time and output the pixel-level trajectory of the target.

[0058] Step 1.2: Use YOLOv5 to detect the moving target in the video to be detected and obtain the moving target.

[0059] Compared with traditional target detection algorithms, YOLOv5 has faster detection speed and higher accuracy. It predicts bounding boxes and category information directly from input images by converting the target detection task into a regression problem. YOLOv5 uses a series of convolutional layers and feature pyramid networks to extract image features, and can automatically adapt to targets of different sizes and shapes. It has been widely used in target detection in transportation, mapping, and agriculture. It combines convolutional neural networks and Kalman filters to achieve accurate target tracking.

[0060] In this embodiment, the specific steps of using YOLOv5 to detect the moving target in the video to be detected are:

[0061] Step 1.2.1: Divide the video to be detected into grids.

[0062] Step 1.2.2: For each grid, make a prediction and get the bounding box, and the probability that each bounding box belongs to different categories.

[0063] Step 1.2.3: Use the non-maximum suppression algorithm to remove bounding boxes whose overlap exceeds the set threshold and select the final detection result.

[0064] Step 1.3: Use DeepSORT to track the moving target and obtain the vehicle trajectory.

[0065] DeepSORT (Deep Learning-based SORT) is a multi-target tracking algorithm based on deep learning, combining the advantages of deep learning and SORT (Simple Online and Realtime Tracking) algorithms. DeepSORT reduces identity switching by adding appearance descriptors, thereby improving tracking efficiency. It is mainly used for multi-target tracking in video surveillance, autonomous driving and other fields.

[0066] In this embodiment, the specific steps of using DeepSORT to track a moving target are:

[0067] Step 1.3.1: Use the recursive Kalman filter algorithm to predict the location and state of the target in the next frame, perform IOU matching between the predicted box and the bounding box, and select the one with higher confidence as the prediction result.

[0068] Step 1.3.2: Use the linear weighted function of the Mahalanobis distance based on motion information and the cosine similarity of appearance features to perform data association, and perform tracking trajectory matching through cascade allocation to obtain the tracking result.

[0069] Step 1.3.3: Synchronously update the tracker parameters and loop the algorithm flow.

[0070] However, there are inevitably some errors in the vehicle trajectories extracted from traffic videos, which may come from any stage of vehicle target detection and tracking. Figure 2 As shown, the vehicle detection algorithm usually outputs a rectangular box in each frame to identify and confirm the existence of the vehicle. Figure 2 The center of the three-dimensional shape of the vehicle (the point in Figure 2) is regarded as the vehicle position, and the continuous vehicle positions constitute the vehicle trajectory. The size and shape of the rectangular box used for vehicle recognition are affected by factors such as the camera shooting height and pitch angle. If there is no error, its actual center position should be the center of the vehicle's three-dimensional shape (the circle in Figure 2). However, in most cases, there is a difference between the determined position and the actual position, resulting in positioning error. Therefore, it is necessary to derive the relationship between the positioning error and its influencing factors (camera posture) in order to perform error correction and improve the accuracy of video-based recognition and positioning.

[0071] Considering that the road surface actually photographed can be approximately regarded as a 2D plane, and the height difference of the road surface is relatively small relative to the height of the traffic monitoring camera position, this embodiment considers using the imaging principle of the camera optical lens and the projection geometry relationship to derive the vehicle coordinate position and positioning error relationship. Since the video-based mobile target detection and tracking algorithm outputs the pixel-level trajectory information of the mobile target, the camera position height and pitch angle are first used to derive the conversion relationship between the earth and the screen coordinate system before further discussing the vehicle positioning error.

[0072] Step 2: Use the coordinate transformation method to transform the vehicle trajectory.

[0073] Step 2.1: Establish the geodetic coordinate system according to the projection principle.

[0074] like Figure 3 As shown, the frame width and frame height of the video image are and (pixels), the focal length of the camera is (pixels). Figure 4 shows the projection principle of the camera. Considering that the height of the vehicle is negligible compared to the height of the camera from the ground, the height of the vehicle is omitted in this embodiment. Point O is the shooting position of the camera in the real world. Point is the projection point of the camera on the ground, that is, there is a line segment Perpendicular to the ground, point K is the intersection of the camera’s sight line OK and the ground, and the coordinates of point K are (X c , Y c ), The size is equal to the pitch angle of the camera , is the vertical field of view of the camera , is the horizontal field of view of the camera , the sight line OK is perpendicular to the plane BCHG, the quadrilateral BCHG is a rectangle, the trapezoidal BCED area is the real-world ground area captured by the camera, where B and C are the two vertices of the bottom edge of the image, D and E are the two vertices of the top edge of the image, F and I are the lowest and highest points that the camera can capture in the vertical field of view, G and H are the closest and farthest points that the camera can capture in the horizontal field of view, line segment JM is the horizontal projection length of the camera sight line OK, and line segment LN is the vertical distance equal to Y s The corresponding horizontal projection length is is the height of the camera from the ground, is the actual distance between the bottom edge of the image and the camera, b min is the horizontal distance from the bottom edge of the image to the camera, b is half of the length of the projection of any point at its horizontal projection position. cis half of the projection length of the camera’s line of sight in the horizontal direction, S is any point within the shooting range, and the coordinate of S in the real world is (X s , Y s ), the coordinates output by the target detection algorithm are (x s ,y s ).by is the origin, The straight line is the y-axis, The straight line is the z-axis, Point perpendicular to face The straight line is taken as the x-axis to establish a rectangular coordinate system, namely, the earth coordinate system. In this embodiment, the trajectory in the camera is converted into an actual trajectory in the earth coordinate system by using the projection principle.

[0075] Step 2.2: Get the camera parameters and establish the pixel coordinate system.

[0076] The pixel coordinate system is a two-dimensional rectangular coordinate system that represents the position of pixels in an image. Its origin is located in the upper left corner of the image, and the horizontal axis (u axis) and vertical axis (v axis) are parallel to the columns and rows of the image respectively. The pixel coordinate system does not use physical units (such as millimeters), but is based on pixels, so it is very suitable for use in digital image processing.

[0077] Step 2.3: Calculate the transformation relationship between the earth coordinate system and the pixel coordinate system based on the vehicle trajectory.

[0078] like Figure 4 As shown, let the vertical near angle of the camera be , it is known that: ,in, is the pitch angle of the camera, is the vertical field of view of the camera, Figure 4 In the expression Any point S (x s ,y s ) The horizontal distance from the camera in real world space is X s and vertical distance Y s The calculation process is as follows.

[0079] First, the actual distance between the bottom edge and the top edge of the image and the camera , (m) are:

[0080] (1)

[0081] (2).

[0082] in, is the height of the camera from the ground (m).

[0083] The angular position of any point S in the image in the vertical field of view (°) is:

[0084] (3).

[0085] Among them, y s It is the ordinate value (pixel level) of any point S of the vehicle moving target trajectory. In this embodiment, the vehicle moving target trajectory is the output of the target detection and tracking algorithm based on YOLOv5+DeepSORT.

[0086] After the above solution process, the vertical distance Y of the actual position of the vehicle moving target in the geodetic coordinate system can be obtained: s for:

[0087] (4).

[0088] Then, solve the horizontal distance X from the camera in the real world space s :

[0089] The horizontal distance from the bottom edge of the image to the camera (m) is:

[0090] (5).

[0091] Since the vertical distance X of the actual position of the vehicle moving target in the earth coordinate system s and The length of To calculate the length, we need to use half of the horizontal projection length of the camera line of sight in the same direction. c The horizontal distance b from the bottom edge of the image to the camera min , so the calculation steps are as follows:

[0092] Half of the projection length of the camera line of sight OK in the horizontal direction is line segment KJ or line segment KM. The length b of line segment KJ or line segment KM c (m) is:

[0093] (6).

[0094] The distance between camera point K and the camera's projection point on the ground, point O', is a line segment. , line segment Length (m) is:

[0095] (7).

[0096] The corresponding projection length at the horizontal projection position is line segment LN, half the length of line segment LN (m) is:

[0097] (8).

[0098] After the above solution process, the vertical distance X of the actual position of the vehicle moving target in the earth coordinate system can be obtained: s for:

[0099] (9).

[0100] Step 2.4: Based on the conversion relationship, the pixel coordinates of the moving target photographed by the camera are converted into geodetic coordinates.

[0101] By deriving the above formula, we can get the conversion relationship between the earth and pixel coordinate systems, and obtain the position coordinates of any point S in the real world. Figure 2 The possible positioning errors are shown, and this embodiment further derives the positioning error relationship of the vehicle.

[0102] Step 3: Use the error correction method to correct the vehicle trajectory and obtain the final vehicle positioning. The positioning correction process includes:

[0103] Step 3.1: Get the camera height and camera pitch angle.

[0104] Step 3.2: Construct the relationship between camera height, camera pitch angle and coordinate conversion error.

[0105] Step 3.2.1: Calculate the centroid coordinates of the target vehicle bounding box in the pixel coordinate system.

[0106] like Figure 5 As shown, assuming that the four vertices of the vehicle rectangular bounding box in the pixel coordinate system are , , , , the output centroid coordinates are .

[0107] Step 3.2.2: Calculate the centroid coordinates of the target vehicle bounding box in the earth coordinate system after coordinate transformation.

[0108] Since the conversion between different coordinate systems is complex and nonlinear, transforming the four vertices of the rectangular bounding box separately will result in a new trapezoidal area, which can be simplified to an isosceles trapezoid, such as Figure 6 As shown. For an isosceles trapezoid, given the position coordinates of the four vertices of the isosceles trapezoid, its centroid coordinates are:

[0109] = .

[0110] in, , , , , the output centroid coordinates are , at this time, the centroid of the rectangle is not equal to the centroid of this new isosceles trapezoid .

[0111] Step 3.3.3: Determine the relationship between the camera height, camera pitch angle and coordinate conversion error based on the difference between the centroid coordinates of the pixel coordinate system and the geodetic coordinate system.

[0112] According to the above coordinate transformation formula, we can deduce the difference between the real center of mass of the vehicle in the real world and the output center of mass after the coordinate transformation, and get:

[0113] (10)

[0114] (11).

[0115] in, Represents the horizontal coordinate error after coordinate transformation, Represents the ordinate error after coordinate transformation, , , , are the four vertex coordinates of the vehicle bounding box in pixel-level coordinates, , , , and They correspond to P0, P1, P2, P3 and P4 respectively. , is half of the projection length of any point at its horizontal projection position. The frame width and frame height of the video image are and , the focal length of the camera is , is the height of the camera from the ground, is the vertical field of view of the camera, is the vertical near angle of the camera.

[0116] Any point corresponding to The calculation formula is , b c is half of the horizontal projection length of the camera’s line of sight, b min is the horizontal distance from the bottom edge of the image to the camera. b1, b2...bn The calculation method of The same, respectively represents half of the projection length of each point at its horizontal position projection, and n is the total number of point numbers.

[0117] From formula (11), we can know that any point corresponds to and about, that is to say also with and Furthermore, through formulas (10) and (11), we can know that the error of coordinate transformation is , It will be affected by the height and pitch angle of the camera, and the influence relationship between them is shown in formula (10) and (11). Therefore, the corrected vehicle position is obtained in this embodiment, as shown in formula (12) and (13). In addition, in subsequent research, the height and angle of the camera should be taken as important factors to eliminate errors.

[0118] Step 3.3: Calculate the target position of the vehicle based on the relationship between the camera height, the camera pitch angle and the coordinate conversion error, and obtain any point S (X s , Y s )The corrected vehicle positioning S'(X es , Y es ). The specific formula is:

[0119] (12)

[0120] (13).

[0121] Among them, X es , Y es are the position coordinates of the vehicle target after error correction.

[0122] In the field test, vehicles traveling along the road section were equipped with inertial navigation devices installed on them, and their measurements were used as the ground truth values ​​of positioning accuracy. At the same time, the position height of the camera installed on the drone changed from 30m to 50m (every 5m), and the pitch angle changed from 30° to 50° (every 5°), and the trajectory movement of the vehicle target was recorded under different camera postures. A total of 25 experiments were conducted. The vehicle trajectory was extracted using the proposed coordinate transformation method and the error-corrected coordinate calculation method, respectively. These trajectories were then compared with the ground truth measurements of the inertial navigation device, and the average absolute distance error was calculated to evaluate the positioning accuracy. The calculation results show that the values ​​of the proposed coordinate transformation method and the error-corrected method are 2.67 meters and 2.12 meters, respectively, which means that the proposed error positioning correction method effectively improves the positioning accuracy under different camera postures.

[0123] Embodiment 2:

[0124] Embodiment 2 of the present invention provides a mobile target positioning system taking into account the pitch angle of a camera, including:

[0125] The target detection and tracking module is configured to obtain a video to be detected, perform target detection and target tracking on a moving target in the video, and obtain a vehicle trajectory;

[0126] A coordinate conversion module is configured to perform coordinate conversion on the vehicle trajectory using a coordinate conversion method;

[0127] The error correction module is configured to use the error correction method to perform positioning correction on the vehicle trajectory to obtain the final vehicle positioning, wherein the positioning correction process includes: obtaining the camera height and the camera pitch angle, constructing the relationship between the camera height, the camera pitch angle and the coordinate conversion error, calculating the coordinate conversion error according to the relationship between the camera height, the camera pitch angle and the coordinate conversion error, calculating the actual position of the vehicle target based on the coordinate conversion error, and obtaining the coordinates of the vehicle positioning.

[0128] Embodiment three:

[0129] Embodiment 3 of the present invention provides a medium on which a program is stored. When the program is executed by a processor, the steps of the method for positioning a moving target considering the pitch angle of the camera as described in Embodiment 1 of the present invention are implemented.

[0130] Embodiment 4:

[0131] Embodiment 4 of the present invention provides a device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for positioning a mobile target taking into account the pitch angle of the camera as described in Embodiment 1 of the present invention are implemented.

[0132] The steps involved in the above embodiments 2, 3 and 4 correspond to the method embodiment 1. For the specific implementation methods, please refer to the relevant description part of embodiment 1.

[0133] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0134] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for positioning a moving target taking into account the pitch angle of a camera, characterized in that: The following steps are involved: Obtain the video to be detected, perform target detection and target tracking on the moving target in the video, and obtain the vehicle trajectory; The coordinate transformation method is used to transform the vehicle trajectory; The vehicle trajectory is positioned and corrected by using an error correction method to obtain the final vehicle positioning, wherein the positioning correction process includes: obtaining the camera height and the camera pitch angle, constructing the relationship between the camera height, the camera pitch angle and the coordinate conversion error, calculating the coordinate conversion error according to the relationship between the camera height, the camera pitch angle and the coordinate conversion error, calculating the actual position of the vehicle target based on the coordinate conversion error, and obtaining the coordinates of the vehicle positioning; The specific steps to construct the relationship between camera height, camera pitch angle and coordinate transformation error are: Calculate the centroid coordinates of the target vehicle bounding box in the pixel coordinate system; Calculate the centroid coordinates of the target vehicle bounding box in the geodetic coordinate system after coordinate transformation; Determine the relationship between the camera height, camera pitch angle and coordinate conversion error based on the difference between the centroid coordinates of the pixel coordinate system and the geodetic coordinate system; The relationship between camera height, camera pitch angle and coordinate conversion error is expressed as: Among them, ΔX represents the horizontal coordinate error after coordinate transformation, ΔY represents the vertical coordinate error after coordinate transformation, P1(x1,y1), P2(x2,y1), P3(x2,y2), P4(x1,y2) are the four vertex coordinates corresponding to the vehicle bounding box in pixel-level coordinates, b0, b1, b2, b3 and b4 are the b values ​​corresponding to P0, P1, P2, P3 and P4 respectively. s , b s is half of the projection length of any point at its horizontal projection position, the frame width and frame height of the video image are w and h0 respectively, the focal length of the camera is f, h is the height of the camera from the ground, θ is the vertical field of view of the camera, φ is the vertical near angle of the camera, and α is the pitch angle of the camera, and P0 is the centroid coordinate of the vehicle's rectangular bounding box in the pixel coordinate system.

2. The method for positioning a mobile target taking into account the camera pitch angle as claimed in claim 1, characterized in that: YOLOv5 is used to detect the moving target in the video to be detected to obtain the moving target, and DeepSORT is used to track the moving target to obtain the vehicle trajectory.

3. The method for positioning a mobile target taking into account the camera pitch angle as claimed in claim 1, characterized in that: The specific steps of using YOLOv5 to detect moving targets in the video to be detected are: Divide the video to be detected into grids; Make a prediction for each grid and get the bounding box, and the probability that each bounding box belongs to a different category; The non-maximum suppression algorithm is used to remove bounding boxes whose overlap exceeds a set threshold and select the final detection result.

4. The method for positioning a mobile target taking into account the camera pitch angle as claimed in claim 3, characterized in that: The specific steps of using DeepSORT to track a moving target are: Use the recursive Kalman filter algorithm to predict the location and state of the target in the next frame, perform IOU matching between the predicted box and the bounding box, and select the one with higher confidence as the prediction result; Data association is performed using the linear weighted function of the Mahalanobis distance based on motion information and the cosine similarity of appearance features, and tracking trajectory matching is performed through cascade allocation to obtain the tracking result; Update the tracker parameters synchronously and loop the algorithm process.

5. The method for positioning a mobile target taking into account the camera pitch angle as claimed in claim 4, characterized in that: The specific steps of using the coordinate transformation method to transform the vehicle trajectory are as follows: Establish geodetic coordinate system based on projection principle; Get camera parameters and establish pixel coordinate system; Calculate the conversion relationship between the earth coordinate system and the pixel coordinate system according to the vehicle trajectory; Based on the conversion relationship, the pixel coordinates of the moving target photographed by the camera are converted into geodetic coordinates.

6. A mobile target positioning system considering the camera pitch angle, characterized in that: include: The target detection and tracking module is configured to obtain a video to be detected, perform target detection and target tracking on a moving target in the video, and obtain a vehicle trajectory; A coordinate conversion module is configured to perform coordinate conversion on the vehicle trajectory using a coordinate conversion method; The error correction module is configured to use the error correction method to perform positioning correction on the vehicle trajectory to obtain the final vehicle positioning, wherein the positioning correction process includes: obtaining the camera height and the camera pitch angle, constructing the relationship between the camera height, the camera pitch angle and the coordinate conversion error, calculating the coordinate conversion error according to the relationship between the camera height, the camera pitch angle and the coordinate conversion error, calculating the actual position of the vehicle target based on the coordinate conversion error, and obtaining the coordinates of the vehicle positioning; The specific steps to construct the relationship between camera height, camera pitch angle and coordinate transformation error are: Calculate the centroid coordinates of the target vehicle bounding box in the pixel coordinate system; Calculate the centroid coordinates of the target vehicle bounding box in the geodetic coordinate system after coordinate transformation; Determine the relationship between the camera height, camera pitch angle and coordinate conversion error based on the difference between the centroid coordinates of the pixel coordinate system and the geodetic coordinate system; The relationship between camera height, camera pitch angle and coordinate conversion error is expressed as: Among them, ΔX represents the horizontal coordinate error after coordinate transformation, ΔY represents the vertical coordinate error after coordinate transformation, P1(x1,y1), P2(x2,y1), P3(x2,y2), P4(x1,y2) are the four vertex coordinates corresponding to the vehicle bounding box in pixel-level coordinates, b0, b1, b2, b3 and b4 are the b values ​​corresponding to P0, P1, P2, P3 and P4 respectively. s , b s is half of the projection length of any point at its horizontal projection position, the frame width and frame height of the video image are w and h0 respectively, the focal length of the camera is f, h is the height of the camera from the ground, θ is the vertical field of view of the camera, φ is the vertical near angle of the camera, and α is the pitch angle of the camera, and P0 is the centroid coordinate of the vehicle's rectangular bounding box in the pixel coordinate system.

7. A computer-readable storage medium, characterized in that: A plurality of instructions are stored therein, and the instructions are suitable for being loaded by a processor of a terminal device and executed by the method for positioning a mobile target taking into account the pitch angle of a camera as described in any one of claims 1-5.

8. A terminal device, characterized in that: The method comprises a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; and the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the method for positioning a mobile target considering the pitch angle of the camera according to any one of claims 1 to 5.

Images