Coordinate conversion method, device and storage medium

By establishing the camera and virtual world coordinate system and solving the coordinate conversion relationship, the coordinate conversion of the gimbal camera is realized, and the accuracy and applicability of the coordinate conversion method in the existing technology is solved, the application scenarios are expanded and errors are reduced.

CN118587290BActive Publication Date: 2024-11-29ZHEJIANG DAHUA TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202411073059.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-29
Estimated Expiration
2044-08-06

AI Technical Summary

Technical Problem

In the prior art, the coordinate conversion method is mainly applicable to fixed-focus cameras, and requires accurate camera installation angle and all calibration targets to be on the same altitude plane, resulting in limited application scenarios and easy introduction of errors, affecting the accuracy of coordinate conversion.

Method used

By using the camera light center of the gimbal camera as the origin, we can obtain the deployment location to establish the virtual world coordinate system, determine the camera and virtual world coordinate expressions of the preset calibration points, and solve the coordinates between the camera and the virtual world coordinate system. Convert relationships, using this relationship to convert known world coordinates into camera coordinates or camera coordinates into world coordinates, reducing the number of preset calibration points and reducing errors introduced by inaccurate installation positions.

Benefits of technology

On the premise of ensuring the accuracy of coordinate conversion, the number of preset calibration points is reduced, the errors introduced due to inaccurate camera installation position and inaccurate north-oriented angle are reduced, and the applicable scenarios are expanded.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118587290B_ABST
    Figure CN118587290B_ABST
Patent Text Reader

Abstract

The present application discloses a coordinate conversion method, device and storage medium, the coordinate conversion method comprising: establishing a camera coordinate system with the camera optical center of the gimbal camera as the origin, and establishing a virtual world coordinate system with the deployment position of the gimbal camera as the origin; determining the camera coordinate expression of a preset calibration point in the camera coordinate system, and determining the virtual world coordinate expression of the preset calibration point in the virtual world coordinate system; solving the coordinate conversion relationship between the camera coordinate system and the virtual world coordinate system based on the camera coordinate expression and the virtual world coordinate expression of the preset calibration point; using the coordinate conversion relationship, converting the known world coordinates into the camera coordinates in the camera coordinate system, or converting the known camera coordinates into the world coordinates in the virtual world coordinate system, and performing coordinate mapping by the automatically constructed camera coordinate system and the virtual world coordinate system, which can reduce errors introduced by inaccurate camera installation position, inaccurate north angle, etc.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of target positioning technology, and in particular to a coordinate conversion method, device and storage medium. Background Art

[0002] At present, tracking and observing targets through gimbal cameras or GPS (Global Positioning System) has been widely used in many industries, such as target tracking in urban roads or shipping ports, and forest fire prevention, which generally requires mutual conversion between camera coordinates and GPS coordinates.

[0003] In the prior art, the conversion relationship between the camera coordinates and the GPS coordinates is calculated based on the GPS coordinates of the calibration target and the position of the calibration target in the camera image.

[0004] However, the above method is only applicable to fixed-focus cameras with fixed shooting angles. The camera installation angle also needs to be accurate enough, and all calibration targets need to be on the same altitude plane, which results in fewer applicable scenarios and is prone to errors, affecting the accuracy of coordinate conversion. Summary of the invention

[0005] In order to solve the above technical problems, the present application at least provides a coordinate conversion method, device and storage medium.

[0006] In a first aspect, the present application provides a coordinate conversion method, which includes: establishing a camera coordinate system with the camera optical center of a gimbal camera as the origin, and obtaining the deployment position of the gimbal camera, and establishing a virtual world coordinate system with the deployment position as the origin; determining the screen position of a preset calibration point in a screen corresponding to the gimbal camera, and obtaining the camera coordinate expression of the preset calibration point in the camera coordinate system based on the screen position; and obtaining the virtual world coordinate expression of the preset calibration point in the virtual world coordinate system based on the calibration position of the preset calibration point; solving the coordinate conversion relationship between the camera coordinate system and the virtual world coordinate system based on the camera coordinate expression and the virtual world coordinate expression of the preset calibration point; and using the coordinate conversion relationship, converting the known world coordinates into the camera coordinates in the camera coordinate system, or converting the known camera coordinates into the world coordinates in the virtual world coordinate system.

[0007] In one embodiment, the image position of a preset calibration point in an image corresponding to a gimbal camera is determined, and a camera coordinate expression of the preset calibration point in a camera coordinate system is obtained based on the image position, including: obtaining the pitch angle and yaw angle of the gimbal camera when the preset calibration point is at the center of the image of the gimbal camera; and constructing a camera coordinate expression of the preset calibration point in the camera coordinate system based on the pitch angle and yaw angle of the gimbal camera.

[0008] In one embodiment, based on the calibration position of the preset calibration point, a virtual world coordinate expression of the preset calibration point in the virtual world coordinate system is obtained, including: calculating the straight-line distance and vertical altitude distance between the calibration position and the deployment position, and calculating the angle between the calibration position and the virtual world coordinate system; based on the straight-line distance, vertical altitude distance and angle, constructing the virtual world coordinate expression of the preset calibration point in the virtual world coordinate system.

[0009] In one embodiment, based on the camera coordinate expression and the virtual world coordinate expression of preset calibration points, the coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system is solved, including: based on the camera coordinate expression and the virtual world coordinate expression of preset calibration points, the rotation matrix and the translation vector between the camera coordinate expression and the virtual world coordinate are solved; based on the rotation matrix and the translation vector, the coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system is obtained.

[0010] In one embodiment, the known world coordinates are converted into camera coordinates in a camera coordinate system, including: mapping the known world coordinates into a virtual world coordinate system to obtain virtual world coordinates; based on a coordinate conversion relationship between the camera coordinate system and the virtual world coordinate system, converting the virtual world coordinates into the camera coordinate system to obtain camera coordinates corresponding to the known world coordinates.

[0011] In one embodiment, the known camera coordinates are coordinate expressions corresponding to the posture of the gimbal camera when the target corresponding to the known camera coordinates is at the center of the screen of the gimbal camera; the known camera coordinates are converted into world coordinates in a virtual world coordinate system, including: determining whether the altitude parameters corresponding to the target are known; if the altitude parameters of the target are known, then based on the altitude parameters of the target and the coordinate conversion relationship, the known camera coordinates are converted into the virtual world coordinate system to obtain the world coordinates corresponding to the known camera coordinates; if the altitude parameters of the target are unknown, then the world coordinate modulus parameters are set step by step, and the target predicted altitude corresponding to the world coordinate modulus parameters is calculated based on the coordinate conversion relationship and the elevation map, and the world coordinates corresponding to the known camera coordinates are obtained based on the target predicted altitude that meets the preset conditions.

[0012] In one embodiment, a target predicted altitude corresponding to a world coordinate modulus parameter is calculated based on a coordinate conversion relationship and an elevation map, and a world coordinate corresponding to a known camera coordinate is obtained based on the target predicted altitude that satisfies a preset condition, including: obtaining a first altitude based on a target predicted altitude corresponding to the world coordinate corresponding to the world coordinate modulus parameter in the elevation map; and obtaining a second altitude by calculating the target predicted altitude based on the world coordinate modulus parameter, a pitch angle of a virtual world coordinate corresponding to the world coordinate modulus parameter, and an altitude corresponding to a deployment position; calculating a difference between the second altitude and the first altitude, and determining whether the difference satisfies a preset condition; if the difference satisfies the preset condition, using the world coordinate corresponding to the target predicted altitude that satisfies the preset condition as the world coordinate corresponding to the known camera coordinate.

[0013] In one embodiment, if the origins of the camera coordinate system and the virtual world coordinate system coincide, the number of preset calibration points is at least two; if the origins of the camera coordinate system and the virtual world coordinate system do not coincide, the number of preset calibration points is at least three.

[0014] According to a second aspect of the present application, there is provided a coordinate conversion device, which includes: a coordinate system construction module, which is used to establish a camera coordinate system with the camera optical center of the gimbal camera as the origin, and to obtain the deployment position of the gimbal camera and to establish a virtual world coordinate system with the deployment position as the origin; a coordinate expression acquisition module, which is used to determine the screen position of a preset calibration point in the screen corresponding to the gimbal camera, and obtain the camera coordinate expression of the preset calibration point in the camera coordinate system based on the screen position; and, based on the calibration position of the preset calibration point, obtain the virtual world coordinate expression of the preset calibration point in the virtual world coordinate system; a conversion relationship solving module, which is used to solve the coordinate conversion relationship between the camera coordinate system and the virtual world coordinate system based on the camera coordinate expression and the virtual world coordinate expression of the preset calibration point; and a coordinate conversion module, which is used to convert the known world coordinates into the camera coordinates in the camera coordinate system, or to convert the known camera coordinates into the world coordinates in the virtual world coordinate system, by using the coordinate conversion relationship.

[0015] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned coordinate conversion method.

[0016] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the above-mentioned coordinate conversion method when the program instructions are executed by a processor.

[0017] The above scheme establishes a camera coordinate system with the camera optical center of the gimbal camera as the origin, and establishes a virtual world coordinate system with the deployment position of the gimbal camera as the origin; determines the camera coordinate expression of the preset calibration point in the camera coordinate system, and determines the virtual world coordinate expression of the preset calibration point in the virtual world coordinate system; based on the camera coordinate expression and the virtual world coordinate expression of the preset calibration point, solves the coordinate conversion relationship between the camera coordinate system and the virtual world coordinate system; uses the coordinate conversion relationship to convert the known world coordinates into the camera coordinates in the camera coordinate system, or converts the known camera coordinates into the world coordinates in the virtual world coordinate system, and uses the virtual world coordinate system as a bridge for coordinate conversion. Under the premise of ensuring the accuracy of coordinate conversion, the number of required preset calibration points is reduced. At the same time, by performing coordinate mapping between the automatically constructed camera coordinate system and the virtual world coordinate system, the error introduced by the inaccurate camera installation position can be reduced, and because there is no need to obtain the north angle of the gimbal camera, the error introduced by the inaccurate north angle can be avoided.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0020] Figure 1 It is a schematic diagram of a solution implementation environment provided by an embodiment of the present application;

[0021] Figure 2 is a flow chart of a coordinate conversion method shown in an exemplary embodiment of the present application;

[0022] Figure 3 is a schematic diagram of solving a coordinate transformation relationship shown in an exemplary embodiment of the present application;

[0023] Figure 4 is a block diagram of a coordinate conversion device shown in an exemplary embodiment of the present application;

[0024] Figure 5 is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present application;

[0025] Figure 6 It is a schematic diagram of the structure of a computer-readable storage medium shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0026] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0027] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0028] The term "and / or" in this article is only an association information describing the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0029] The coordinate conversion method provided in the embodiment of the present application is described below.

[0030] Please refer to Figure 1 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment may include a pan-tilt camera 110 and a server 120, and the pan-tilt camera 110 and the server 120 are connected to each other in communication.

[0031] The pan-tilt camera 110 is used to capture scene images. The pan-tilt camera 110 can be any image capture device with a pan-tilt, which can capture images from multiple angles, such as a spherical camera.

[0032] Server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0033] In one example, the server 120 can perform target positioning for the scene image acquired from the gimbal camera 110, obtain the camera coordinates of the target, and convert the camera coordinates into world coordinates; or, the server 120 can obtain the world coordinates of the target, convert the world coordinates of the target into camera coordinates. Of course, the server 120 can store the converted coordinates locally, transmit them back to the gimbal camera 110, or transmit them to other terminals.

[0034] In the coordinate conversion method provided in the embodiment of the present application, the execution subject of each step can be the gimbal camera 110, or the server 120, or the gimbal camera 110 and the server 120 can be executed in interaction with each other, that is, part of the steps of the method are executed by the gimbal camera 110 and the other steps are executed by the server 120.

[0035] See also Figure 2 , Figure 2 is a flowchart of a coordinate conversion method shown in an exemplary embodiment of the present application. The coordinate conversion method can be applied to Figure 1 It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments, and this embodiment does not limit the implementation environment to which the method is applicable.

[0036] like Figure 2 As shown, the coordinate conversion method at least includes steps S210 to S240, which are described in detail as follows:

[0037] Step S210: establishing a camera coordinate system with the camera optical center of the gimbal camera as the origin, and obtaining the deployment position of the gimbal camera, and establishing a virtual world coordinate system with the deployment position as the origin.

[0038] The steps of establishing the camera coordinate system may include: taking the optical center of the gimbal camera as the origin, the x-axis and y-axis are parallel and in the same direction as the x-axis and y-axis of the captured image, and then determining the z-axis according to the right-hand rule to obtain the camera coordinate system.

[0039] Of course, other methods may also be used to determine the x-axis, y-axis, and z-axis of the camera coordinate system, which is not limited in this application.

[0040] The steps of establishing the virtual world coordinate system may include: obtaining the deployment position of the gimbal camera, recording the deployment position as point A, taking A as the origin, and obtaining any position B different from position A, taking position A as the origin, the line connecting AB as the z-axis, the direction from A to B, the height direction as the y-axis, the direction vertically pointing to the ground, and then determining the x-axis according to the right-hand rule, thereby obtaining the virtual world coordinate system. Among them, the deployment position of the gimbal camera includes the GPS coordinates of the gimbal camera, and may further include the altitude of the gimbal camera.

[0041] Of course, other methods may also be used to determine the x-axis, y-axis, and z-axis of the virtual world coordinate system, which is not limited in this application.

[0042] Step S220: Determine the screen position of the preset calibration point in the screen corresponding to the gimbal camera, and obtain the camera coordinate expression of the preset calibration point in the camera coordinate system based on the screen position; and, based on the calibration position of the preset calibration point, obtain the virtual world coordinate expression of the preset calibration point in the virtual world coordinate system.

[0043] During the coordinate calibration process, preset calibration points are deployed simultaneously in the deployment scene of the gimbal camera. The calibration position of the preset calibration point is known. The calibration position includes the GPS coordinates of the preset calibration point and may further include the altitude of the preset calibration point.

[0044] It should be noted that, since the constructed virtual world coordinate system is used as a bridge for coordinate conversion, when deploying preset calibration points, the number of deployed preset calibration points can be reduced while ensuring the accuracy of coordinate conversion.

[0045] Specifically, if the origins of the camera coordinate system and the virtual world coordinate system coincide, the number of preset calibration points is at least two; if the origins of the camera coordinate system and the virtual world coordinate system do not coincide, the number of preset calibration points is at least three.

[0046] The coordinate expression of each preset calibration point in the camera coordinate system and the virtual world coordinate system is obtained respectively.

[0047] The step of obtaining the camera coordinate expression includes: determining the screen position of the preset calibration point in the screen corresponding to the gimbal camera, and obtaining the camera coordinate expression of the preset calibration point in the camera coordinate system based on the screen position.

[0048] Exemplarily, if the posture of the gimbal camera is fixed, the screen position corresponding to the preset calibration point is directly mapped to the camera coordinate system to obtain the camera coordinate expression. If the posture of the gimbal camera is not fixed, the screen position corresponding to the preset calibration point and the posture of the gimbal camera when collecting the above picture can be combined to map the screen position corresponding to the preset calibration point to the camera coordinate system to obtain the camera coordinate expression; the image acquisition angle of the gimbal camera can also be adjusted to make the preset calibration point at the center of the gimbal camera's picture, and the posture of the gimbal camera when the preset calibration point is at the center of the gimbal camera's picture is obtained, and the camera coordinate expression is obtained according to the posture of the gimbal camera.

[0049] Of course, in addition to the above embodiments, other methods may be used to obtain the camera coordinate expression of the preset calibration point in the camera coordinate system, which is not limited in the present application.

[0050] The step of acquiring the virtual world coordinate expression includes: obtaining the virtual world coordinate expression of the preset calibration point in the virtual world coordinate system based on the calibration position of the preset calibration point.

[0051] Exemplarily, since the position of the origin of the virtual world coordinate system (i.e., the deployment position) is known, the calibration position of the preset calibration point can be mapped to the virtual world coordinate system according to the relative spatial relationship between the calibration position of the preset calibration point and the deployment position to obtain a virtual world coordinate expression.

[0052] Of course, in addition to the above embodiments, other methods may be used to obtain the virtual world coordinate expression of the preset calibration point in the virtual world coordinate system, and this application does not limit this.

[0053] Step S230: Based on the camera coordinate expression and the virtual world coordinate expression of the preset calibration point, the coordinate conversion relationship between the camera coordinate system and the virtual world coordinate system is solved.

[0054] After obtaining the camera coordinate expression and the virtual world coordinate expression of the preset calibration point, since the camera coordinate expression and the virtual world coordinate expression should represent the same position, the coordinate transformation relationship between the camera coordinate expression and the virtual world coordinate expression can be solved and used as the coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system.

[0055] Exemplarily, a transformation relationship between the camera coordinate system and the virtual world coordinate system is constructed, and the transformation relationship contains the parameters to be solved. The camera coordinate expression and the virtual world coordinate expression of each preset calibration point are substituted into the above transformation relationship to derive the value of the parameter to be solved, thereby obtaining the coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system.

[0056] Step S240: using a coordinate conversion relationship, converting the known world coordinates into camera coordinates in the camera coordinate system, or converting the known camera coordinates into world coordinates in the virtual world coordinate system.

[0057] If the world coordinates of the target are known, the known world coordinates can be converted into camera coordinates in the camera coordinate system according to the coordinate transformation relationship. If the camera coordinates of the target are known, the known camera coordinates can be converted into world coordinates in the virtual world coordinate system according to the coordinate transformation relationship. The world coordinates in the virtual world coordinate system can also be further mapped to GPS coordinates.

[0058] Next, some embodiments of the present application are described in detail.

[0059] In some embodiments, the posture of the gimbal camera includes a pitch angle and a yaw angle. Obtaining the camera coordinate expression of the preset calibration point in the camera coordinate system in step S220 includes: obtaining the pitch angle and yaw angle of the gimbal camera when the preset calibration point is at the center of the picture of the gimbal camera; based on the pitch angle and yaw angle of the gimbal camera, constructing the camera coordinate expression of the preset calibration point in the camera coordinate system.

[0060] For example: Assuming that the preset calibration point is at the center of the gimbal camera's image, the gimbal camera's pitch angle is T, and the yaw angle is P, then the camera coordinate expression corresponding to the preset calibration point is It can be expressed as the following formula 1:

[0061] (Formula 1)

[0062] In some embodiments, obtaining the virtual world coordinate expression of the preset calibration point in the virtual world coordinate system in step S220 includes: calculating the straight-line distance and vertical altitude distance between the calibration position and the deployment position, and calculating the angle between the calibration position and the virtual world coordinate system; based on the straight-line distance, vertical altitude distance and angle, constructing the virtual world coordinate expression of the preset calibration point in the virtual world coordinate system.

[0063] For example, the deployment position and calibration position both contain GPS coordinates and altitude. The camera coordinates corresponding to the preset calibration point are expressed as It can be expressed as the following formula 2:

[0064] (Formula 2)

[0065] Where theta is the angle between the calibration position of the preset calibration point and the z-axis in the clockwise direction in the virtual world coordinate system, d is the straight-line distance between the deployment position and the calibration position, is the altitude corresponding to the deployment location of the gimbal camera, The altitude corresponding to the calibration position of the preset calibration point.

[0066] Furthermore, according to the camera coordinate expression and the virtual world coordinate expression of the preset calibration point, the coordinate conversion relationship between the camera coordinate system and the virtual world coordinate system is solved.

[0067] For example, see Figure 3 , Figure 3 A schematic diagram of solving the coordinate transformation relationship is shown in an exemplary embodiment of the present application, such as Figure 3 As shown, the number of preset calibration points is 3, and the camera coordinate expression and the virtual world coordinate expression of each preset calibration point are obtained. The camera coordinate expression and the virtual world coordinate expression of each preset calibration point are combined to solve the rotation matrix and the translation vector between the camera coordinate system and the virtual world coordinate system, and the rotation matrix and the translation vector are used as calibration parameters of the coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system.

[0068] Exemplarily, if the origins of the camera coordinate system and the virtual world coordinate system are different, the step of solving the coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system may include: based on the camera coordinate expression and the virtual world coordinate expression of preset calibration points, solving the rotation matrix and translation vector between the camera coordinate expression and the virtual world coordinates; based on the rotation matrix and the translation vector, obtaining the coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system.

[0069] For example, the conversion relationship between the camera coordinate expression and the virtual world coordinate expression can be expressed as the following formula 3:

[0070] (Formula 3)

[0071] in, is the camera coordinate expression; is the virtual world coordinate expression; R is the rotation matrix, which can be a 3*3 unitary matrix, indicating the rotation relationship between coordinate systems; t is the translation vector, whose vector size can be 3*1; is the scale factor, which changes with the preset calibration point.

[0072] Exemplarily, if the origins of the camera coordinate system and the virtual world coordinate system are the same, the step of solving the coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system may include: solving the rotation matrix between the camera coordinate expression and the virtual world coordinate based on the camera coordinate expression and the virtual world coordinate expression of preset calibration points; and obtaining the coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system based on the rotation matrix.

[0073] For example, the conversion relationship between the camera coordinate expression and the virtual world coordinate expression can be expressed as the following formula 4:

[0074] (Formula 4)

[0075] That is, relative to Formula 3, there is no translation vector.

[0076] The rotation matrix and translation vector are obtained by solving the above formula 3 or formula 4. The rotation matrix and translation vector represent the coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system.

[0077] By using the coordinate transformation relationship, the known world coordinates are transformed into the camera coordinates in the camera coordinate system, or the known camera coordinates are transformed into the world coordinates in the virtual world coordinate system.

[0078] In some embodiments, converting the known world coordinates into camera coordinates in a camera coordinate system includes: mapping the known world coordinates into a virtual world coordinate system to obtain virtual world coordinates; and based on a coordinate conversion relationship between the camera coordinate system and the virtual world coordinate system, converting the virtual world coordinates into the camera coordinate system to obtain camera coordinates corresponding to the known world coordinates.

[0079] Specifically, the known world coordinates of the target are first converted into the target's expression in the virtual world coordinate system, and then the target's expression in the virtual world coordinate system is mapped to the expression in the camera coordinate system according to the coordinate conversion relationship to obtain the camera coordinates corresponding to the known world coordinates.

[0080] Furthermore, the trigonometric function relationship can be used to calculate the posture of the gimbal camera when the target is located at the center of the gimbal camera's screen to obtain the expected posture. According to the current actual posture of the gimbal camera and the above-mentioned expected posture, the shooting angle adjustment value of the gimbal camera is determined, thereby achieving target tracking.

[0081] In some implementations, the known camera coordinates are transformed into world coordinates in a virtual world coordinate system.

[0082] Exemplarily, the known camera coordinates are the coordinate expression corresponding to the posture of the gimbal camera when the target corresponding to the known camera coordinates is at the center of the screen of the gimbal camera; the known camera coordinates are converted into world coordinates in a virtual world coordinate system, including: determining whether the altitude parameters corresponding to the target are known; if the altitude parameters of the target are known, then based on the altitude parameters of the target and the coordinate conversion relationship, the known camera coordinates are converted into the virtual world coordinate system to obtain the world coordinates corresponding to the known camera coordinates; if the altitude parameters of the target are unknown, then the world coordinate modulus parameters are set step by step, and the predicted altitude of the target corresponding to the world coordinate modulus parameters is calculated based on the coordinate conversion relationship and the elevation map, and the world coordinates corresponding to the known camera coordinates are obtained based on the predicted altitude of the target that meets the preset conditions.

[0083] That is, when converting known camera coordinates into world coordinates in the virtual world coordinate system, there are two cases:

[0084] Case 1: The altitude parameters corresponding to the target are known.

[0085] For example, in target positioning scenarios on urban roads or shipping ports, the target's altitude parameters do not fluctuate much, so the altitude of the gimbal camera can be used as the target's altitude; or, the target's altitude can be obtained from feedback from other terminals.

[0086] When the target's altitude parameters are known, based on the target's altitude parameters and the coordinate conversion relationship, the known camera coordinates are converted to the virtual world coordinate system to obtain the world coordinates corresponding to the known camera coordinates. Specifically, please refer to the following formula 5:

[0087] (Formula 5)

[0088] in, is the known camera coordinate. Since the target’s altitude parameter is known, refer to Formula 2 and we can get , so that according to the known , R, t, Calculate the scale factor , to transform the known camera coordinates into the virtual world coordinate system and obtain the virtual world coordinates .

[0089] Furthermore, the world coordinates of the target in the virtual world coordinate system can be Convert to polar coordinates and finally calculate the GPS coordinates of the target.

[0090] Case 2: The altitude parameters corresponding to the target are unknown.

[0091] For example, in target positioning scenarios in forests, mountains and other environments, the target's altitude parameters fluctuate greatly, and the altitude of the gimbal camera cannot be used as the target's altitude.

[0092] When the altitude parameters of the target are unknown, the world coordinate modulus parameters are set step by step, and the target predicted altitude corresponding to the world coordinate modulus parameters is solved based on the coordinate conversion relationship and the elevation map, and the world coordinates corresponding to the known camera coordinates are obtained based on the target predicted altitude that meets the preset conditions.

[0093] It should be noted that when the altitude parameters of the target are unknown, in order to ensure the accuracy of coordinate conversion, when constructing the camera coordinate system and the virtual world coordinate system, it is necessary to ensure that the origins of the camera coordinate system and the virtual world coordinate system coincide with each other. For example, when collecting the GPS coordinates of the deployment location of the gimbal camera, reduce the error between the collected GPS coordinates and the actual gimbal camera coordinates.

[0094] Specifically, when the origin of the camera coordinate system and the virtual world coordinate system coincide, the calibration parameters in the calculated coordinate transformation relationship only contain the rotation matrix R, and there is no translation vector t. Then the coordinate transformation relationship is transformed to obtain Formula 6:

[0095] (Formula 6)

[0096] That is, we get the virtual world coordinates with a modulus of 1 , it can be seen that the GPS coordinates of the target are Therefore, the world coordinate modulus length parameter di is set step by step to determine the optimal target altitude parameter.

[0097] Specifically, a first altitude is obtained by calculating a target predicted altitude corresponding to a world coordinate of a world coordinate modulus parameter in an elevation map; and a second altitude is obtained by calculating a target predicted altitude based on the world coordinate modulus parameter, the pitch angle of the world coordinate corresponding to the world coordinate modulus parameter, and the altitude corresponding to the deployment position; a difference between the second altitude and the first altitude is calculated to determine whether the difference meets a preset condition; if the difference meets the preset condition, the world coordinate corresponding to the target predicted altitude that meets the preset condition is used as the world coordinate corresponding to the known camera coordinates.

[0098] For the world coordinate modulus parameter di, the target predicted altitude corresponding to the GPS coordinate under di in the elevation map is calculated to obtain the first altitude, which is recorded as h1.

[0099] In addition, based on the world coordinate modulus parameter, the pitch angle of the virtual world coordinate corresponding to the world coordinate modulus parameter, and the altitude corresponding to the deployment position, the target predicted altitude is calculated to obtain the second altitude h2. The relevant formula may be the following formula 7:

[0100] (Formula 7)

[0101] in, is the altitude of the gimbal camera, For di The pitch angle is combined with the trigonometric function relationship to calculate the second altitude h2.

[0102] The difference between the second altitude and the first altitude is calculated. The difference calculation formula may be the following formula 8:

[0103] (Formula 8)

[0104] Wherein, delta_h is the calculated difference.

[0105] The world coordinates corresponding to the target predicted altitude that meets the preset conditions are used as the world coordinates corresponding to the known camera coordinates. It should be noted that the world coordinates corresponding to the known camera coordinates can be coordinates in a virtual world coordinate system or GPS coordinates.

[0106] For example, if the signs of delta_h of two adjacent steps are different, it is considered that the preset condition is met, and stepping is stopped. At this time, the GPS coordinates corresponding to d=di are output; or if the absolute value of delta_h is less than a preset threshold, it is considered that the preset condition is met, and stepping is stopped. At this time, the GPS coordinates corresponding to d=di are output.

[0107] The coordinate conversion method disclosed in the present application establishes a camera coordinate system with the camera optical center of the gimbal camera as the origin, and establishes a virtual world coordinate system with the deployment position of the gimbal camera as the origin; determines the camera coordinate expression of the preset calibration point in the camera coordinate system, and determines the virtual world coordinate expression of the preset calibration point in the virtual world coordinate system; based on the camera coordinate expression and the virtual world coordinate expression of the preset calibration point, solves the coordinate conversion relationship between the camera coordinate system and the virtual world coordinate system; utilizes the coordinate conversion relationship to convert the known world coordinates into the camera coordinates in the camera coordinate system, or converts the known camera coordinates into the world coordinates in the virtual world coordinate system, and uses the virtual world coordinate system as a bridge for coordinate conversion, thereby reducing the number of preset calibration points required while ensuring the accuracy of coordinate conversion, and at the same time, by performing coordinate mapping between the automatically constructed camera coordinate system and the virtual world coordinate system, it is possible to reduce errors introduced by inaccurate camera installation positions, and because there is no need to obtain the north angle of the gimbal camera, errors introduced by inaccurate north angles are avoided.

[0108] Figure 4 FIG. 1 is a block diagram of a coordinate conversion device shown in an exemplary embodiment of the present application. Figure 4 As shown, the exemplary coordinate conversion device 400 includes: a coordinate system construction module 410, a coordinate expression acquisition module 420, a conversion relationship solving module 430 and a coordinate conversion module 440. Specifically:

[0109] A coordinate system construction module 410 is used to establish a camera coordinate system with the camera optical center of the gimbal camera as the origin, and to obtain the deployment position of the gimbal camera and establish a virtual world coordinate system with the deployment position as the origin;

[0110] The coordinate expression acquisition module 420 is used to determine the screen position of the preset calibration point in the screen corresponding to the gimbal camera, and obtain the camera coordinate expression of the preset calibration point in the camera coordinate system based on the screen position; and obtain the virtual world coordinate expression of the preset calibration point in the virtual world coordinate system based on the calibration position of the preset calibration point;

[0111] The conversion relationship solving module 430 is used to solve the coordinate conversion relationship between the camera coordinate system and the virtual world coordinate system based on the camera coordinate expression and the virtual world coordinate expression of the preset calibration point;

[0112] The coordinate conversion module 440 is used to convert the known world coordinates into the camera coordinates in the camera coordinate system, or convert the known camera coordinates into the world coordinates in the virtual world coordinate system, by using the coordinate conversion relationship.

[0113] It should be noted that the coordinate conversion device provided in the above embodiment and the coordinate conversion method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the coordinate conversion device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0114] See also Figure 5 , Figure 5 1 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 500 includes a memory 501 and a processor 502, and the processor 502 is used to execute program instructions stored in the memory 501 to implement the steps in any of the above coordinate conversion method embodiments. In a specific implementation scenario, the electronic device 500 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 500 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0115] Specifically, the processor 502 is used to control itself and the memory 501 to implement the steps in any of the above coordinate conversion method embodiments. The processor 502 can also be called a central processing unit (CPU). The processor 502 may be an integrated circuit chip with signal processing capabilities. The processor 502 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 502 can be implemented by an integrated circuit chip.

[0116] See also Figure 6 , Figure 6 The computer-readable storage medium 600 stores program instructions 610 that can be executed by a processor, and the program instructions 610 are used to implement the steps in any of the above coordinate conversion method embodiments.

[0117] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0118] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0119] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0120] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

Claims

1. A coordinate conversion method, characterized in that: include: Establishing a camera coordinate system with the camera optical center of the gimbal camera as the origin, and obtaining the deployment position of the gimbal camera, and establishing a virtual world coordinate system with the deployment position as the origin; Determine the image position of the preset calibration point in the image corresponding to the gimbal camera, and obtain the camera coordinate expression of the preset calibration point in the camera coordinate system based on the image position; and, based on the calibration position of the preset calibration point, obtaining a virtual world coordinate expression of the preset calibration point in the virtual world coordinate system; Based on the camera coordinate expression and the virtual world coordinate expression of the preset calibration point, solving the coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system; Acquire known camera coordinates corresponding to a target, and if an altitude parameter of the target is unknown, set a world coordinate modulus parameter in a step-by-step manner, transform the coordinate conversion relationship to obtain a world coordinate corresponding to the world coordinate modulus parameter, and obtain a first altitude based on a target predicted altitude corresponding to the world coordinate corresponding to the world coordinate modulus parameter in an elevation map; and calculate a target predicted altitude based on the world coordinate modulus parameter, a pitch angle of a virtual world coordinate corresponding to the world coordinate modulus parameter, and an altitude corresponding to the deployment position, to obtain a second altitude; Calculating the difference between the second altitude and the first altitude, and determining whether the difference meets a preset condition; If the difference satisfies a preset condition, the world coordinates corresponding to the target predicted altitude that satisfies the preset condition are used as the world coordinates corresponding to the known camera coordinates.

2. The method according to claim 1, characterized in that The step of determining a position of a preset calibration point in a picture corresponding to the gimbal camera, and obtaining a camera coordinate expression of the preset calibration point in the camera coordinate system based on the picture position, includes: Acquire the pitch angle and yaw angle of the gimbal camera when the preset calibration point is at the center of the image of the gimbal camera; Based on the pitch angle and yaw angle of the gimbal camera, a camera coordinate expression of the preset calibration point in the camera coordinate system is constructed.

3. The method according to claim 1, characterized in that The obtaining, based on the calibration position of the preset calibration point, a virtual world coordinate expression of the preset calibration point in the virtual world coordinate system comprises: Calculating the straight-line distance and vertical altitude distance between the calibrated position and the deployed position, and calculating the angle between the calibrated position and the virtual world coordinate system; Based on the straight-line distance, the vertical altitude distance and the angle, a virtual world coordinate expression of the preset calibration point in the virtual world coordinate system is constructed.

4. The method according to claim 1, characterized in that: The step of calculating the coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system based on the camera coordinate expression and the virtual world coordinate expression of the preset calibration point includes: Based on the camera coordinate expression and the virtual world coordinate expression of the preset calibration point, solving the rotation matrix and the translation vector between the camera coordinate expression and the virtual world coordinate; Based on the rotation matrix and the translation vector, a coordinate transformation relationship between the camera coordinate system and the virtual world coordinate system is obtained.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Mapping the known world coordinates into the virtual world coordinate system to obtain virtual world coordinates; Based on the coordinate conversion relationship between the camera coordinate system and the virtual world coordinate system, the virtual world coordinates are converted into the camera coordinate system to obtain the camera coordinates corresponding to the known world coordinates.

6. The method according to any one of claims 1 to 4, characterized in that: The known camera coordinates are coordinate expressions corresponding to the posture of the gimbal camera when the target corresponding to the known camera coordinates is at the center of the screen of the gimbal camera; the method further includes: If the altitude parameters of the target are known, then based on the altitude parameters of the target and the coordinate conversion relationship, the known camera coordinates are converted into the virtual world coordinate system to obtain the world coordinates corresponding to the known camera coordinates.

7. The method according to claim 1, characterized in that If the origins of the camera coordinate system and the virtual world coordinate system coincide with each other, the number of the preset calibration points is at least two; if the origins of the camera coordinate system and the virtual world coordinate system do not coincide with each other, the number of the preset calibration points is at least three.

8. An electronic device, characterized in that: The electronic device comprises a memory and a processor, and the processor is used to execute program instructions stored in the memory to implement the steps in the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and the program instructions can be executed by a processor to implement the steps in the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Method for acquiring coordinate transformation matrix and positioning target in monitoring picture

    CN111461994A