A method, device and system for positioning based on a monocular fixed-focus gimbal camera

Abstract

The application provides a method, device and system for positioning based on a monocular fixed-focus gimbal camera. The method comprises: based on the calibrated monocular fixed-focus gimbal camera, determining a first coordinate and a first pose, a second coordinate and a second pose of a camera center in a world coordinate system at a first position and a second position respectively, and acquiring a first image and a second image including a static target respectively; based on an image coordinate system, determining the coordinates of the static target in the first image and the second image respectively; according to the calibration parameters of the camera, the first coordinate and the first pose, the second coordinate and the second pose, and the coordinates of the static target in the image, determining a first deflection vector and a second deflection vector from the camera center to the static target in the world coordinate system respectively; and according to the first coordinate, the second coordinate, the first deflection vector and the second deflection vector, determining the coordinate information of the static target in the world coordinate system. The method is simple and efficient to implement, and is particularly suitable for indoor close-range application scenarios.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a positioning technology based on a monocular fixed-focus gimbal camera. Background Technology

[0002] In existing technologies, stereo cameras, zoom cameras, etc., can usually be used to locate spatial points (or static targets), but there is no existing method that can locate static targets based on a monocular fixed-focus gimbal camera. Summary of the Invention

[0003] The purpose of this invention is to provide a method, apparatus, and system for positioning based on a monocular fixed-focus gimbal camera, so as to solve the technical problem that there is no existing technology that can achieve static target positioning based on a monocular fixed-focus gimbal camera.

[0004] According to one aspect of the present invention, a method for positioning based on a monocular fixed-focus gimbal camera is provided, characterized in that the method includes:

[0005] Based on the calibrated monocular fixed-focus gimbal camera, determine the first coordinate information and first attitude information of the camera center in the world coordinate system when it is in the first position, and acquire the first image including the static target at its first position.

[0006] Adjust the camera pose, determine the second coordinate information and second attitude information in the world coordinate system corresponding to the second position of the camera center, and acquire a second image including the static target at the second position.

[0007] Based on the image coordinate system, the coordinate information of the static target in the first image and the second image are determined respectively;

[0008] Based on the calibration parameters of the monocular fixed-focus gimbal camera, the first coordinate information and the first attitude information, the second coordinate information and the second attitude information, the coordinate information of the static target in the first image and the coordinate information of the static target in the second image, the first deflection amount and the second deflection amount from the camera center to the static target in the world coordinate system are determined respectively.

[0009] Based on the first coordinate information, the second coordinate information, the first deflection amount, and the second deflection amount, the coordinate information of the static target in the world coordinate system is determined.

[0010] Optionally, determining the deflection of the camera center to the static target in the world coordinate system includes:

[0011] Based on the calibration parameters of the monocular fixed-focus gimbal camera and the coordinate information of the static target in the image, an imaging model is used to determine the relative deflection of the camera center to the static target in the camera coordinate system.

[0012] Based on the coordinate and attitude information of the camera center in the world coordinate system, as well as the relative deflection, the deflection from the camera center to the static target in the world coordinate system is determined.

[0013] Optionally, the imaging model includes a pinhole imaging model, wherein,

[0014] The relative deflection of the camera center to the static target in the camera coordinate system is determined using the following formula (1).

[0015]

[0016] Where K is the projection matrix in the pinhole imaging model, (U,V,1) T f is the relative deflection of the camera center to the static target in the camera coordinate system. x f y c x c y Here are the camera calibration parameters, (u,v) represents the coordinate information of the static target in the image coordinate system, and (u,v,1) represents the coordinates. T Let (u,v) be the homogeneous coordinate representation;

[0017] The deflection of the camera center to the static target in the world coordinate system is determined using the following formula (2).

[0018]

[0019] Where P is the static target in the world coordinate system, C is the coordinate information of the camera center in the world coordinate system, and R is the rotation matrix of the camera coordinate system relative to the world coordinate system.

[0020] Optionally, the method further includes:

[0021] Identify several static targets corresponding to a stationary object within the field of view of the monocular fixed-focus gimbal camera;

[0022] Traverse each static target, determine the coordinate information of each static target in the world coordinate system, and determine the size of the stationary object based on the coordinate information of each static target in the world coordinate system.

[0023] Optionally, the method further includes:

[0024] Traverse each static target within the field of view of the monocular fixed-focus gimbal camera and determine the coordinate information of each static target in the world coordinate system;

[0025] Based on the coordinate information of all static targets in the world coordinate system, a three-dimensional map of the field of view of the monocular fixed-focus gimbal camera is constructed.

[0026] Optionally, the method further includes:

[0027] Based on the coordinate information of the static target in the world coordinate system, the distance of the static target relative to the rotation center of the monocular fixed-focus gimbal camera is determined.

[0028] According to another aspect of the present invention, a device for positioning based on a monocular fixed-focus gimbal camera is provided, wherein the device comprises:

[0029] The first module is used to determine the first coordinate information and first attitude information of the camera center in the world coordinate system when the camera center is in the first position, based on the calibrated monocular fixed-focus gimbal camera, and to acquire the first image including the static target at the first position.

[0030] The second module is used to adjust the camera pose, determine the second coordinate information and second attitude information in the world coordinate system corresponding to the camera center when it is in the second position, and acquire a second image including the static target at its second position.

[0031] The third module is used to determine the coordinate information of the static target in the first image and the second image based on the image coordinate system;

[0032] The fourth module is used to determine the first and second offsets from the camera center to the static target in the world coordinate system based on the calibration parameters of the monocular fixed-focus gimbal camera, the first coordinate information and the first attitude information, the second coordinate information and the second attitude information, the coordinate information of the static target in the first image and the coordinate information of the static target in the second image.

[0033] The fifth module is used to determine the coordinate information of the static target in the world coordinate system based on the first coordinate information, the second coordinate information, the first deflection amount, and the second deflection amount.

[0034] Optionally, the device further includes:

[0035] The sixth module is used to determine several static targets corresponding to a stationary object within the field of view of the monocular fixed-focus gimbal camera; and to traverse each static target, determine the coordinate information of each static target in the world coordinate system, and determine the size of the stationary object based on the coordinate information of each static target in the world coordinate system.

[0036] Optionally, the device further includes:

[0037] The seventh module is used to traverse each static target within the field of view of the monocular fixed-focus gimbal camera, determine the coordinate information of each static target in the world coordinate system, and construct a three-dimensional map of the field of view of the monocular fixed-focus gimbal camera based on the coordinate information of all static targets in the world coordinate system.

[0038] According to another aspect of the present invention, a system for positioning based on a monocular fixed-focus gimbal camera is provided, characterized in that the system comprises:

[0039] The calibrated monocular fixed-focus gimbal camera is used to photograph static targets within the visible range;

[0040] The host computer is used to acquire the first coordinate information and first attitude information of the camera center of the monocular fixed-focus gimbal camera in the world coordinate system at the first position, and the second coordinate information and second attitude information in the world coordinate system at the second position, and to acquire the first image and the second image including the static target captured at the first position and the second position, respectively; to determine the coordinate information of the static target in the first image and the second image based on the image coordinate system; to determine the first deflection amount and the second deflection amount from the camera center to the static target in the world coordinate system according to the calibration parameters of the monocular fixed-focus gimbal camera, the first coordinate information and the first attitude information, the second coordinate information and the second attitude information, the coordinate information of the static target in the first image and the coordinate information of the static target in the second image, respectively; and to determine the coordinate information of the static target in the world coordinate system according to the first coordinate information, the second coordinate information, the first deflection amount and the second deflection amount.

[0041] Compared with existing technologies, the present invention provides a method, apparatus, and system for positioning based on a monocular fixed-focus gimbal camera. The method includes: determining, based on a calibrated monocular fixed-focus gimbal camera, first coordinate information and first attitude information in the world coordinate system corresponding to the camera center at a first position, and acquiring a first image including a static target at that first position; adjusting the camera pose to determine, second coordinate information and second attitude information in the world coordinate system corresponding to the camera center at a second position, and acquiring a second image including the static target at that second position; determining, based on the image coordinate system, the coordinate information of the static target in the first and second images respectively; determining, based on the calibration parameters of the monocular fixed-focus gimbal camera, the first coordinate information and first attitude information, the second coordinate information and second attitude information, the coordinate information of the static target in the first image, and the coordinate information of the static target in the second image, a first deflection amount and a second deflection amount from the camera center to the static target in the world coordinate system respectively; and determining, based on the first coordinate information, the second coordinate information, the first deflection amount, and the second deflection amount, the coordinate information of the static target in the world coordinate system.

[0042] The technical effects that this invention can bring are:

[0043] The method of this invention can locate static targets by processing images captured by a monocular fixed-focus gimbal camera at different poses on the same static target. This method is simple and efficient to implement, and is particularly suitable for static target localization in indoor close-range applications. Attached Figure Description

[0044] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0045] Figure 1 A schematic diagram of a positioning method based on a monocular fixed-focus gimbal camera according to one aspect of the present invention is shown.

[0046] Figure 2 A schematic diagram of the pose of the camera center in the world coordinate system according to an embodiment of one aspect of the present invention;

[0047] Figure 3 A schematic diagram showing the relative position of the camera center and a static target according to an embodiment of one aspect of the present invention;

[0048] Figure 4 A schematic diagram illustrating the deflection of the camera center from different positions to a static target in the world coordinate system according to an embodiment of one aspect of the present invention;

[0049] Figure 5A schematic diagram of a device for positioning based on a monocular fixed-focus gimbal camera according to another aspect of the present invention is shown.

[0050] Figure 6 A schematic diagram of a positioning system based on a monocular fixed-focus gimbal camera according to another aspect of the present invention is shown.

[0051] The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0052] The present invention will now be described in further detail with reference to the accompanying drawings.

[0053] In a typical configuration of various embodiments of the present invention, the method execution entity, each trusted party of the system and / or each module of the device may include one or more processors (CPU), input / output interfaces, network interfaces and memory.

[0054] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0055] Computer-readable media can include permanent and non-permanent, removable and non-removable media, and can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Computer storage media can include, but is not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0056] This invention can locate static targets (or static spatial points) within the visible spatial range of the camera and measure the distance between the camera and the static targets within the visible spatial range using a host computer 100 and a monocular fixed-focus gimbal camera 200. The host computer 100 is electrically connected to the monocular fixed-focus gimbal camera, and the electrical connection can be a wired electrical connection or a wireless electrical connection, which is not limited here.

[0057] To further illustrate the technical means adopted and the effects achieved by the present invention, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and various embodiments.

[0058] Figure 1 The diagram illustrates a method for positioning based on a monocular fixed-focus gimbal camera according to one aspect of the present invention, wherein one embodiment of the method includes:

[0059] Based on the calibrated monocular fixed-focus gimbal camera, S101 determines the first coordinate information and first attitude information of the camera center in the world coordinate system when it is in the first position, and acquires the first image including the static target at its first position.

[0060] S102 adjusts the camera pose, determines the second coordinate information and second attitude information in the world coordinate system corresponding to the second position of the camera center, and acquires a second image including the static target at the second position.

[0061] S103 determines the coordinate information of the static target in the first image and the second image based on the image coordinate system;

[0062] S104 determines the first and second offset turning amounts from the camera center to the static target in the world coordinate system based on the calibration parameters of the monocular fixed-focus gimbal camera, the first coordinate information and the first attitude information, the second coordinate information and the second attitude information, the coordinate information of the static target in the first image and the coordinate information of the static target in the second image.

[0063] S105 determines the coordinate information of the static target in the world coordinate system based on the first coordinate information, the second coordinate information, the first deflection amount, and the second deflection amount.

[0064] The method embodiments of this application are implemented through a host computer and a calibrated monocular fixed-focus camera. The host computer can be a computer device. This computer device includes, but is not limited to, personal computers, laptops, industrial computers, servers, network hosts, and single network servers. Here, the computer device is merely an example; other existing or future devices and / or resources that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0065] The camera center refers to the focal point of the camera, a key point in the camera imaging process, and the origin of the camera coordinate system. In this application, after the monocular fixed-focus gimbal camera 200 is calibrated, the host computer 100 can obtain the calibration parameters of the calibrated monocular fixed-focus gimbal camera 200. Under different poses of the calibrated monocular fixed-focus gimbal camera 200 (i.e., the gimbal rotation center remains stationary, but the camera's pose changes relative to the gimbal rotation center), the coordinate information of the camera center in the world coordinate system is different.

[0066] In this embodiment, in step S101, based on the calibrated monocular fixed-focus gimbal camera, the first coordinate information and the first attitude information in the world coordinate system corresponding to the camera center at the first position can be determined, and a first image including a static target can be acquired at its first position.

[0067] In this scenario, when the calibrated monocular fixed-focus gimbal camera is in a certain pose, the static target is within its spatial visibility range. The position of the camera center can be taken as the first position. The host computer obtains the first coordinate information and first pose information of the camera center in the world coordinate system corresponding to the first position, and acquires the first image including the static target at the first position.

[0068] Continuing in this embodiment, in step S102, the camera pose can be adjusted to determine the second coordinate information and second attitude information in the world coordinate system corresponding to the second position of the camera center, and a second image including the static target can be acquired at the second position.

[0069] After acquiring the first image, the pose of the calibrated monocular fixed-focus gimbal camera can be adjusted, and the position of its camera center can be used as the second position. The host computer can acquire the second coordinate information and the second attitude information in the world coordinate system corresponding to the camera center at the second position, and acquire the second image including the static target at the second position.

[0070] Continuing in this embodiment, in step S103, the coordinate information of the static target in the first image and the second image can be determined based on the image coordinate system.

[0071] The host computer determines the coordinate information of the static target in the first image and the coordinate information in the second image based on the image coordinate system.

[0072] Continuing in this embodiment, in step S104, the first and second deflection angles from the camera center to the static target in the world coordinate system can be determined based on the calibration parameters of the monocular fixed-focus gimbal camera, the first coordinate information and the first attitude information, the second coordinate information and the second attitude information, the coordinate information of the static target in the first image and the coordinate information of the static target in the second image.

[0073] The host computer can determine the first and second deflection amounts from the camera center to the static target in the world coordinate system based on the calibration parameters of the calibrated monocular fixed-focus gimbal camera, the first coordinate information and the first attitude information obtained in step S101, the second coordinate information and the second attitude information obtained in step S102, and the coordinate information of the static target in the first image and the second image based on the image coordinate system determined in step S103.

[0074] Optionally, in step S104, determining the deflection of the camera center to the static target in the world coordinate system includes:

[0075] Based on the calibration parameters of the monocular fixed-focus gimbal camera and the coordinate information of the static target in the image, an imaging model is used to determine the relative deflection of the camera center to the static target in the camera coordinate system.

[0076] Based on the coordinate and attitude information of the camera center in the world coordinate system, as well as the relative deflection, the deflection from the camera center to the static target in the world coordinate system is determined.

[0077] The host computer can determine the relative offset between the camera center and the static target in the camera coordinate system based on the calibration parameters of the calibrated monocular fixed-focus gimbal camera and the coordinate information of the static target in the image obtained in step S103, using an imaging model; and then determine the deflection of the camera center to the static target in the world coordinate system based on the coordinate information and attitude information of the camera center in the world coordinate system obtained in the aforementioned steps, as well as the relative offset.

[0078] The imaging model used can be an orthogonal projection model, a pinhole imaging model, etc.

[0079] Optionally, if the imaging model is a pinhole imaging model, then,

[0080] The relative deflection of the camera center to the static target in the camera coordinate system is determined using the following formula (1).

[0081]

[0082] Where K is the projection matrix in the pinhole imaging model, (U,V,1) T f is the relative deflection of the camera center to the static target in the camera coordinate system. x f y c x c y Here are the camera calibration parameters, (u,v) represents the coordinate information of the static target in the image coordinate system, and (u,v,1) represents the coordinates. TLet (u,v) be the homogeneous coordinate representation;

[0083] The deflection of the camera center to the static target in the world coordinate system is determined using the following formula (2).

[0084]

[0085] Where P is the static target in the world coordinate system, C is the coordinate information of the camera center in the world coordinate system, and R is the rotation matrix of the camera coordinate system relative to the world coordinate system.

[0086] In this optional embodiment, after calibration, the relevant calibration parameters f of the monocular fixed-focus gimbal camera are... x f y c x c y The distance r between the gimbal rotation center O and the camera center C can be determined. For example... Figure 2 , Figure 3 , Figure 4 We can construct a world coordinate system by taking the gimbal rotation center O of the monocular fixed-focus gimbal camera as the origin, construct a camera coordinate system by taking the camera center C as the origin, and construct an image coordinate system by taking the top left corner of the image as the origin. We can then determine the rotation matrix R of the camera coordinate system relative to the world coordinate system. The coordinate information of a static target P within the visible range of the monocular fixed-focus gimbal camera in the world coordinate system can then be represented as (x, y, z). The coordinate information of the camera center C of the monocular fixed-focus gimbal camera in a certain pose in the world coordinate system can be represented as (x, y, z). c ,y c ,z c The attitude information can be represented as The relationship between the coordinate information and attitude information of the camera center C can be expressed by the following formula (3).

[0087]

[0088] Where r is the distance between the gimbal rotation center O and the camera center C.

[0089] The coordinate information of a static target in an image in the image coordinate system can be represented as (u,v) and (u,v,1). T Expressed in homogeneous coordinates, the relative deflection of the camera center C to the static target P in the camera coordinate system can be represented as (U,V,1). T .

[0090] An example of using a pinhole imaging model, when a monocular fixed-focus gimbal camera is in its first pose, the first coordinate information of its camera center in the world coordinate system at the first position C1 can be represented as (x c1 ,y c1 ,zc1 The first attitude information can be represented as: Based on the first posture information The coordinate information (x) of C1 can be determined using formula (3). c1 ,y c1 ,z c1 It is also possible to obtain the first image of the static target P when the camera center is at the first position C1, and determine the coordinate information of the static target P in the image coordinate system, which can be expressed as (u1, v1). Then, without considering camera distortion factors, the first relative deflection of the camera center to the static target P in the camera coordinate system at the first position C1 can be determined according to formula (1), which can be expressed as (U1, V1, 1). T Then, according to formula (2), the first deflection of the camera center from the static target P at the first position C1 in the world coordinate system can be determined, which can be expressed as: After adjusting the camera pose, the camera center is at the second position C2, and its second coordinate information in the world coordinate system can be represented as (x... c2 ,y c2 ,z c2 The attitude information can be represented as Similarly, based on the second attitude information The coordinate information (x) of C2 can be determined using formula (3). c2 ,y c2 ,z c2 The second image that the camera center can acquire at the second position C2 includes the static target P. The coordinate information of the static target P in the image coordinate system can be represented as (u2, v2). Without considering camera distortion, the second relative deflection of the camera center at the second position C2 to the static target P in the camera coordinate system can also be determined according to formula (1), which can be represented as (U2, V2, 1). T Then, according to formula (2), the second deflection of the camera center from the static target P at the second position C2 in the world coordinate system can be determined, which can be expressed as:

[0091] Continuing in this embodiment, in step S105, the coordinate information of the static target in the world coordinate system is determined based on the first coordinate information, the second coordinate information, the first deflection amount, and the second deflection amount.

[0092] The host computer can determine the coordinate information of the static target in the world coordinate system based on the first coordinate information obtained in step S101, the second coordinate information obtained in step S102, and the first and second deflection values ​​obtained in step S104.

[0093] Continuing with the above example, the first deflection of the camera center in the determined world coordinate system from the first position C1 to the static target P is respectively... and the second deflection of the second position C2 to the static target P Then, based on the first coordinate information (x) of the camera center at the first position C1 in the world coordinate system, it can also be determined that... c1 ,y c1 ,z c1 ) and the second coordinate information (x) at the second position C2 c2 ,y c2 ,z c2 The vector is determined according to the following formula (4).

[0094]

[0095] Then, according to the following formula (5),

[0096]

[0097] Determine the coordinate information of the static target P in the world coordinate system.

[0098] Optionally, after locating a static target within the spatial field of view using a calibrated monocular fixed-focus gimbal camera, the method may further include, for static objects with regular shapes or whose size can be determined by a finite number of visible static targets within the spatial field of view:

[0099] S106 determines several static targets corresponding to a stationary object within the field of view of the monocular fixed-focus gimbal camera;

[0100] S107 Iterates through each static target, determines the coordinate information of each static target in the world coordinate system, and determines the size of the stationary object based on the coordinate information of each static target in the world coordinate system.

[0101] In this optional embodiment, in step S106, several necessary spatial points whose size can be determined can be identified within the visible range of a stationary object in the space where the monocular fixed-focus gimbal camera 200 is located. Each necessary spatial point is taken as a static target, such as at least 4 different vertices of a cube, at least 1 pair of points with the largest distance on the surface of a sphere, etc.

[0102] Continuing in this optional embodiment, in step S107, each static target determined in step S106 can be traversed. According to the aforementioned steps S101 to S105, the coordinate information of each static target of the stationary object in the world coordinate system can be determined. Then, combined with the shape of the stationary object, the size of the stationary object can be determined according to the coordinate information of each static target in the world coordinate system. For example, the length, width, and height of the cube can be determined according to the coordinate information of at least four vertices of the cube in the world coordinate system; the diameter of the sphere can be determined according to the coordinate information of a pair of points with the largest distance on the surface of the sphere.

[0103] Optionally, after locating static targets within the spatial field of view using a calibrated monocular fixed-focus gimbal camera, the method may further include, in conjunction with each locatable static target within the spatial field of view:

[0104] S108 traverses each static target within the field of view of the monocular fixed-focus gimbal camera and determines the coordinate information of each static target in the world coordinate system;

[0105] S109 constructs a three-dimensional map of the field of view of the monocular fixed-focus gimbal camera based on the coordinate information of all static targets in the world coordinate system.

[0106] In this optional embodiment, in step S108, each locatable spatial point within the visible range of the monocular fixed-focus gimbal camera 200 can be taken as a static target, and the coordinate information of each static target within the visible range in the world coordinate system can be determined according to the aforementioned steps S101 to S105.

[0107] In this optional embodiment, in step S109, a three-dimensional map of the visible range of the space where the monocular fixed-focus gimbal camera 200 is located can be constructed based on the coordinate information of all static targets in the world coordinate system. For example, a three-dimensional map can be constructed using three-dimensional modeling software (Pro / E, Blender, etc.), or a three-dimensional map can be drawn using Python software tools, and so on.

[0108] Optionally, after the calibrated monocular fixed-focus gimbal camera has achieved localization of static targets within the spatial field of view, the method may further include:

[0109] S110 determines the distance of the static target relative to the rotation center of the monocular fixed-focus gimbal camera based on the coordinate information of the static target in the world coordinate system.

[0110] In this optional embodiment, the origin of the world coordinate system is the gimbal rotation center O of the monocular fixed-focus gimbal camera 200, and its coordinate information in the world coordinate system is (0,0,0). In step S110, the host computer 100 can, based on the determined coordinate information (x,y,z) of the static target P in the world coordinate system, use the following formula (6) to...

[0111]

[0112] The distance of the static target P relative to the gimbal rotation center O of the monocular fixed-focus gimbal camera 200 can be determined.

[0113] Figure 5 An apparatus for positioning based on a monocular fixed-focus gimbal camera according to another aspect of the present invention is shown, wherein, in one embodiment, the apparatus includes:

[0114] The first module 510 is used to determine the first coordinate information and first attitude information of the camera center in the world coordinate system when the camera center is in the first position, based on the calibrated monocular fixed-focus gimbal camera, and to acquire a first image including static targets at its first position.

[0115] The second module 520 is used to adjust the camera pose, determine the second coordinate information and second attitude information in the world coordinate system corresponding to the camera center when it is in the second position, and acquire a second image including the static target at its second position.

[0116] The third module 530 is used to determine the coordinate information of the static target in the first image and the second image based on the image coordinate system;

[0117] The fourth module 540 is used to determine the first and second offsets from the camera center to the static target in the world coordinate system based on the calibration parameters of the monocular fixed-focus gimbal camera, the first coordinate information and the first attitude information, the second coordinate information and the second attitude information, the coordinate information of the static target in the first image and the coordinate information of the static target in the second image.

[0118] The fifth module 550 is used to determine the coordinate information of the static target in the world coordinate system based on the first coordinate information, the second coordinate information, the first deflection amount, and the second deflection amount.

[0119] In this embodiment, part or all of the device can be deployed in the host computer 100 that performs the aforementioned method embodiments and / or optional embodiments, and in conjunction with the calibrated monocular fixed-focus gimbal camera, the aforementioned method embodiments and / or optional embodiments can be implemented.

[0120] After the monocular fixed-focus gimbal camera 200 is calibrated, the host computer 100 can obtain the calibration parameters of the calibrated monocular fixed-focus gimbal camera 200. When the calibrated monocular fixed-focus gimbal camera 200 is in a certain pose, the static target is within its spatial visibility range. The position of the camera center of the calibrated monocular fixed-focus gimbal camera 200 can be taken as the first position. In this embodiment, the first module 510 of the device obtains the first coordinate information and the first pose information in the world coordinate system corresponding to the camera center at the first position, and obtains the first image including the static target at the first position.

[0121] Continuing in this embodiment, when the camera pose is adjusted so that the camera center is in the second position, the second module 520 of the device can obtain the second coordinate information and the second attitude information in the world coordinate system corresponding to the camera center in the second position, and obtain a second image including the static target at the second position.

[0122] Continuing in this embodiment, the third module 530 of the device can determine the coordinate information of the static target in the first image and the coordinate information in the second image based on the image coordinate system.

[0123] Continuing in this embodiment, through the fourth module 540 of the device, the relative offset between the camera center and the static target in the camera coordinate system can be determined by using the imaging model based on the calibration parameters of the calibrated monocular fixed-focus gimbal camera and the coordinate information of the static target in the image obtained in step S103; then, based on the coordinate information and attitude information of the camera center in the world coordinate system obtained in the aforementioned steps, and the relative offset, the offset of the camera center to the static target in the world coordinate system can be determined.

[0124] Continuing in this embodiment, the fifth module 550 of the device can determine the coordinate information of the static target in the world coordinate system based on the first coordinate information obtained by the first module 510, the second coordinate information obtained by the second module 520, and the first and second deflection values ​​obtained by the fourth module 540.

[0125] Optionally, the device further includes:

[0126] The sixth module 560 is used to determine several static targets corresponding to a stationary object within the field of view of the monocular fixed-focus gimbal camera; and to traverse each static target, determine the coordinate information of each static target in the world coordinate system, and determine the size of the stationary object based on the coordinate information of each static target in the world coordinate system.

[0127] In this optional embodiment, the sixth module 560 of the device can determine several necessary spatial points within the visible range of the monocular fixed-focus gimbal camera 200 to determine the size of a stationary object. Each necessary spatial point is taken as a static target, such as at least four different vertices of a cube, at least one pair of points with the greatest distance on the surface of a sphere, etc. Then, each static target is traversed, and the coordinate information of each static target in the world coordinate system is determined by the aforementioned modules 510-550 of the device. Combined with the shape of the stationary object, the size of the stationary object can be determined based on the coordinate information of each static target in the world coordinate system. For example, the length, width, and height of the cube can be determined based on the coordinate information of the at least four vertices of the cube in the world coordinate system; the diameter of the sphere can be determined based on the coordinate information of the pair of points with the greatest distance on the surface of the sphere in the world coordinate system.

[0128] Optionally, the device further includes:

[0129] The seventh module 570 is used to traverse each static target within the field of view of the monocular fixed-focus gimbal camera, determine the coordinate information of each static target in the world coordinate system, and construct a three-dimensional map of the field of view of the monocular fixed-focus gimbal camera based on the coordinate information of all static targets in the world coordinate system.

[0130] In this optional embodiment, through the seventh module 570 of the device, each locatable spatial point within the visible range of the space where the monocular fixed-focus gimbal camera 200 is located can be taken as a static target. Through the aforementioned modules 510 to 550 of the device, the coordinate information of each static target within the visible range in the world coordinate system is determined, and a three-dimensional map of the visible range of the space where the monocular fixed-focus gimbal camera 200 is located is constructed based on the determined coordinate information of all static targets in the world coordinate system.

[0131] Optionally, the device further includes:

[0132] The eighth module 580 is used to determine the distance of the static target relative to the rotation center of the monocular fixed-focus gimbal camera based on the coordinate information of the static target in the world coordinate system.

[0133] In this optional embodiment, the eighth module 580 of the device can determine the distance of the static target P relative to the gimbal rotation center O of the monocular fixed-focus gimbal camera 200 by formula (6) based on the coordinate information (x,y,z) of the static target P in the world coordinate system.

[0134] In the various embodiments and / or optional embodiments of the above-described apparatus, the parts of the method steps performed by each module that are not mentioned in the above-described related method embodiments and / or optional embodiments are the same as those described in the above-described related method embodiments and / or optional embodiments, and will not be repeated here.

[0135] Figure 6 A system for positioning based on a monocular fixed-focus gimbal camera according to another aspect of the present invention is shown, wherein, in one embodiment, the system includes:

[0136] The calibrated monocular fixed-focus gimbal camera 200 is used to photograph static targets within the visible range;

[0137] The host computer 100 is used to acquire the first coordinate information and first attitude information of the camera center of the monocular fixed-focus gimbal camera in the world coordinate system at the first position, and the second coordinate information and second attitude information in the world coordinate system at the second position, and to acquire the first image and the second image including the static target captured at the first position and the second position, respectively; to determine the coordinate information of the static target in the first image and the second image based on the image coordinate system; to determine the first deflection amount and the second deflection amount from the camera center to the static target in the world coordinate system according to the calibration parameters of the monocular fixed-focus gimbal camera, the first coordinate information and the first attitude information, the second coordinate information and the second attitude information, the coordinate information of the static target in the first image and the coordinate information of the static target in the second image, respectively; and to determine the coordinate information of the static target in the world coordinate system according to the first coordinate information, the second coordinate information, the first deflection amount and the second deflection amount.

[0138] In this embodiment, the system includes a calibrated monocular fixed-focus gimbal camera 200 and a host computer 100. After the monocular fixed-focus gimbal camera 200 is calibrated, the host computer 100 can acquire the calibration parameters of the calibrated monocular fixed-focus gimbal camera 200. The coordinate information of the camera center in the world coordinate system differs depending on the pose of the calibrated monocular fixed-focus gimbal camera 200. The calibrated monocular fixed-focus gimbal camera 200 can take pictures of static targets in different poses. The host computer 100 accordingly acquires the first coordinate information and first pose information of the camera center in the world coordinate system at the first position, and the second coordinate information and second pose information in the world coordinate system at the second position, and acquires a first image and a second image including the static target taken at the first and second positions, respectively. The host computer 100 also determines the coordinate information of the static target in the first and second images based on the image coordinate system. The host computer 100 then determines the first and second offset rotation amounts from the camera center to the static target in the world coordinate system based on the acquired calibration parameters of the calibrated monocular fixed-focus gimbal camera 100, the aforementioned first coordinate information and first attitude information, the aforementioned second coordinate information and second attitude information, the coordinate information of the static target in the first image, and the coordinate information of the static target in the second image. Then, the host computer 100 determines the coordinate information of the static target in the world coordinate system based on the acquired first and second coordinate information, and the determined first and second offset rotation amounts.

[0139] Through the modules of the various embodiments and / or optional embodiments of the above-described device, or the components of the above-described system embodiments, some or all of the steps of the method embodiments and / or optional embodiments can be executed. Based on a monocular fixed-focus gimbal camera, it is possible to locate and measure the distance of static targets (static spatial points) within its spatial field of view, as well as identify the size of stationary objects of specific shapes, and construct a three-dimensional map of the field of view. It does not rely on zoom and / or binocular cameras, the method is simple and efficient to implement, and is particularly suitable for indoor close-range application scenarios.

[0140] According to another aspect of the present invention, a computer-readable medium is also provided, the computer-readable medium storing computer-readable instructions that can be executed by a processor to implement some or all of the foregoing methods.

[0141] It should be noted that the various method embodiments of the present invention can be implemented partially or entirely in software and / or a combination of software and hardware. The software program involved in the present invention can be executed by a processor to implement some or all of the steps or functions of the above embodiments. Similarly, the software program of the present invention (including related data structures) can be stored in a computer-readable recording medium.

[0142] Furthermore, part or all of the present invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the present invention through the operation of the computer. The program instructions invoking the methods of the present invention may be stored in a fixed or removable recording medium, and / or transmitted via a data stream in a broadcast or other signal carrying medium, and / or stored in the working memory of a computer device operating according to the program instructions.

[0143] According to another aspect of the present invention, a device for positioning based on a monocular fixed-focus gimbal camera is also provided. The device includes: a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute part or all of the methods and / or technical solutions as described in the foregoing embodiments and / or optional embodiments.

[0144] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The steps in the method claims do not necessarily define the order of execution / implementation, and multiple units or devices stated in the apparatus claims may also be implemented by a single unit or device through software and / or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

Claims

1. A positioning method based on a monocular fixed-focus gimbal camera, characterized in that, The method includes: Based on the calibrated monocular fixed-focus gimbal camera, determine the first coordinate information and first attitude information of the camera center in the world coordinate system when it is in the first position, and acquire the first image including the static target at its first position. Adjust the camera pose, determine the second coordinate information and second attitude information in the world coordinate system corresponding to the second position of the camera center, and acquire a second image including the static target at the second position. Based on the image coordinate system, the coordinate information of the static target in the first image and the second image are determined respectively; Based on the calibration parameters of the monocular fixed-focus gimbal camera, the first coordinate information and the first attitude information, the second coordinate information and the second attitude information, the coordinate information of the static target in the first image and the coordinate information of the static target in the second image, the first deflection amount and the second deflection amount from the camera center to the static target in the world coordinate system are determined respectively. Based on the first coordinate information, the second coordinate information, the first deflection amount, and the second deflection amount, the coordinate information of the static target in the world coordinate system is determined.

2. The method according to claim 1, characterized in that, Determining the deflection of the camera center to the static target in the world coordinate system includes: Based on the calibration parameters of the monocular fixed-focus gimbal camera and the coordinate information of the static target in the image, an imaging model is used to determine the relative deflection of the camera center to the static target in the camera coordinate system. Based on the coordinate and attitude information of the camera center in the world coordinate system, as well as the relative deflection, the deflection from the camera center to the static target in the world coordinate system is determined.

3. The method according to claim 2, characterized in that, The imaging model includes a pinhole imaging model, wherein... The relative deflection of the camera center to the static target in the camera coordinate system is determined using the following formula (1). (1) Where K is the projection matrix in the pinhole imaging model, (U,V,1) T f is the relative deflection of the camera center to the static target in the camera coordinate system. x f y c x c y Here are the camera calibration parameters, (u,v) represents the coordinate information of the static target in the image coordinate system, and (u,v,1) represents the coordinates. T Let (u,v) be the homogeneous coordinate representation; The deflection of the camera center to the static target in the world coordinate system is determined using the following formula (2). (2) Where P is the static target in the world coordinate system, C is the coordinate information of the camera center in the world coordinate system, and R is the rotation matrix of the camera coordinate system relative to the world coordinate system.

4. The method according to claim 1, characterized in that, The method further includes: Identify several static targets corresponding to a stationary object within the field of view of the monocular fixed-focus gimbal camera; Traverse each static target, determine the coordinate information of each static target in the world coordinate system, and determine the size of the stationary object based on the coordinate information of each static target in the world coordinate system.

5. The method according to claim 1, characterized in that, The method further includes: Traverse each static target within the field of view of the monocular fixed-focus gimbal camera and determine the coordinate information of each static target in the world coordinate system; Based on the coordinate information of all static targets in the world coordinate system, a three-dimensional map of the field of view of the monocular fixed-focus gimbal camera is constructed.

6. The method according to claim 1, characterized in that, The method further includes: Based on the coordinate information of the static target in the world coordinate system, the distance of the static target relative to the rotation center of the monocular fixed-focus gimbal camera is determined.

7. A device for positioning based on a monocular fixed-focus gimbal camera, characterized in that, The device includes: The first module is used to determine the first coordinate information and first attitude information of the camera center in the world coordinate system when the camera center is in the first position, based on the calibrated monocular fixed-focus gimbal camera, and to acquire the first image including the static target at the first position. The second module is used to adjust the camera pose, determine the second coordinate information and second attitude information in the world coordinate system corresponding to the camera center when it is in the second position, and acquire a second image including the static target at its second position. The third module is used to determine the coordinate information of the static target in the first image and the second image based on the image coordinate system; The fourth module is used to determine the first and second offsets from the camera center to the static target in the world coordinate system based on the calibration parameters of the monocular fixed-focus gimbal camera, the first coordinate information and the first attitude information, the second coordinate information and the second attitude information, the coordinate information of the static target in the first image and the coordinate information of the static target in the second image. The fifth module is used to determine the coordinate information of the static target in the world coordinate system based on the first coordinate information, the second coordinate information, the first deflection amount, and the second deflection amount.

8. The apparatus according to claim 7, characterized in that, The device further includes: The sixth module is used to determine several static targets corresponding to a stationary object within the field of view of the monocular fixed-focus gimbal camera; and to traverse each static target, determine the coordinate information of each static target in the world coordinate system, and determine the size of the stationary object based on the coordinate information of each static target in the world coordinate system.

9. The apparatus according to claim 7, characterized in that, The device further includes: The seventh module is used to traverse each static target within the field of view of the monocular fixed-focus gimbal camera, determine the coordinate information of each static target in the world coordinate system, and construct a three-dimensional map of the field of view of the monocular fixed-focus gimbal camera based on the coordinate information of all static targets in the world coordinate system.

10. A positioning system based on a monocular fixed-focus gimbal camera, characterized in that, The system includes: The calibrated monocular fixed-focus gimbal camera is used to photograph static targets within the visible range; The host computer is used to acquire the first coordinate information and first attitude information of the camera center of the monocular fixed-focus gimbal camera in the world coordinate system at the first position, and the second coordinate information and second attitude information in the world coordinate system at the second position, and to acquire the first image and the second image including the static target captured at the first position and the second position, respectively; to determine the coordinate information of the static target in the first image and the second image based on the image coordinate system; to determine the first deflection amount and the second deflection amount from the camera center to the static target in the world coordinate system according to the calibration parameters of the monocular fixed-focus gimbal camera, the first coordinate information and the first attitude information, the second coordinate information and the second attitude information, the coordinate information of the static target in the first image and the coordinate information of the static target in the second image, respectively; and to determine the coordinate information of the static target in the world coordinate system according to the first coordinate information, the second coordinate information, the first deflection amount and the second deflection amount.

11. A computer-readable medium, characterized in that, It stores computer-readable instructions that are executed by a processor to implement the method as described in any one of claims 1 to 6.

12. A device for positioning based on a monocular fixed-focus gimbal camera, characterized in that, The device includes: One or more processors; and A memory storing computer-readable instructions, which, when executed, cause the processor to perform the operations of the method as described in any one of claims 1 to 6.

Images