A rapid calibration method, apparatus, device, and medium for dynamic camera coordinate mapping

Abstract

This application relates to a rapid calibration method, apparatus, device, and medium for dynamic camera coordinate mapping. The method calculates a coordinate transformation model between pixel coordinates and world coordinates using the PTZ values, pixel coordinates, and world coordinates of the original base calibration point and its corresponding derived calibration point set. Then, using the PT difference between the current base coordinate point and the original base coordinate point, and the derived calibration point set corresponding to the original base calibration point, it obtains the set of derived coordinate points corresponding to the current base coordinate point. Based on the current base coordinate point and its corresponding derived coordinate point set, it updates the extrinsic parameter matrix of the coordinate transformation model to obtain a new mapping relationship between pixel coordinates and world coordinates. This method achieves rapid calibration of dynamic camera coordinate mapping without being affected by camera offset amplitude or image quality.

Description

Technical Field

[0001] This invention relates to the field of data processing technology for camera calibration, and in particular to a rapid calibration method, apparatus, device, and medium for dynamic camera coordinate mapping. Background Technology

[0002] The core purpose of dynamic camera coordinate calibration is to establish and maintain in real time the precise mapping relationship between image pixel coordinates and real-world 3D coordinates, solve the coordinate drift caused by camera motion / zoom / attitude changes, and ensure the accuracy and robustness of tasks such as positioning, tracking, measurement, and multi-camera fusion.

[0003] Chinese patent CN116993830A describes an automatic calibration method for dynamic camera coordinate mapping. This method acquires image pairs before and after the offset, extracts and pairs feature points using the SIFT algorithm, optimizes camera parameters (azimuth, pitch, and roll) using a genetic algorithm, and then weights and processes multiple sets of results to achieve automatic calibration. However, this method fails to achieve automatic calibration when the camera offset is too large, resulting in no overlap between the two images, or when the image quality is poor, leading to feature point matching failures. Summary of the Invention

[0004] Therefore, the purpose of this invention is to provide a rapid calibration method for dynamic camera coordinate mapping. This method is not affected by the camera offset amplitude and image quality. Based on the original calibration data, the camera posture information corresponding to the calibration data is processed to achieve rapid calibration of camera coordinate mapping.

[0005] A fast calibration method for dynamic camera coordinate mapping includes the following steps:

[0006] S10: Control the pan-tilt-zoom (PTZ) rotation to center the reference object in the video frame, and control the zoom level relative to the original baseline calibration point. With the same scaling factor, the current base calibration point of the camera is obtained. of value; S20: Calculate the current base calibration point of Values ​​and original baseline calibration points of The difference in values ​​is used to obtain the azimuth difference ΔP and the elevation difference ΔT. S30: Use azimuth difference △P and elevation difference △T to calibrate the original foundation points. The corresponding set of derived calibration points of The value is corrected to obtain the current base calibration point. The corresponding current derived calibration point set of value; S40: Set the current base calibration point and the current set of derived calibration points The input is fed into the pixel coordinate-world coordinate mapping model constructed based on the intrinsic and extrinsic parameter matrices of the PTZ camera. The extrinsic parameter matrix is ​​then updated to obtain a new pixel coordinate-world coordinate mapping model.

[0007] Furthermore, the mapping model between pixel coordinates and world coordinates constructed based on the intrinsic and extrinsic parameter matrices of the PTZ camera in step S40 is as follows:

[0008] Where M is the intrinsic parameter matrix of the PTZ camera, [R, T] is the extrinsic parameter matrix of the PTZ camera, R is a 3×3 rotation matrix, and T is a 3×1 translation matrix; For pixel coordinates, Used as world coordinates.

[0009] The intrinsic and extrinsic parameter matrices are based on the original baseline calibration points. and its corresponding set of derived calibration points confirm.

[0010] Furthermore, the world coordinates are obtained as follows: Original base calibration points and its corresponding set of derived calibration points latitude and longitude coordinates Convert to planar coordinates :

[0011] In the formula: The length of the meridian arc; The radius of curvature of the meridian; ; , Longitude of the central meridian; ; , This is the second eccentricity of the ellipsoid; Then based on latitude and longitude coordinates Obtain latitude and longitude from the elevation file DEM The corresponding altitude h is used to obtain world coordinates. .

[0012] Furthermore, in step S30, the original basic calibration point The corresponding set of derived calibration points The methods to obtain it are as follows: Position the reference object in the exact center of the video frame, and use this calibration point as the original base calibration point. Obtain the original basic calibration point. The latitude and longitude coordinates on the map, along with the PTZ value and pixel coordinates of the corresponding camera area, yield the following: ; Set a search area centered on the center position in the video frame. Rotate the pan-tilt head to change the position of the reference object within the search area, creating several new calibration points as derived calibration points. Obtain the PTZ value and pixel coordinates of each derived calibration point within the corresponding camera area. The derived calibration point set is obtained. ; m represents the number of derived calibration points, where m > 1.

[0013] Furthermore, the current basic calibration point The corresponding current derived calibration point set satisfy: ; in, ,..., ; ,..., .

[0014] Furthermore, the azimuth difference ΔP satisfies: △P=P n -P0; Among them, P n P0 is the azimuth of the current base calibration point, and P1 is the azimuth of the original base calibration point. The pitch angle difference ΔT satisfies: △T=T n -T0; Among them, T n T0 is the pitch angle of the current base calibration point, and T0 is the pitch angle of the original base calibration point.

[0015] Compared with existing technologies, this application calculates the coordinate transformation model between pixel coordinates and world coordinates using the PTZ values, pixel coordinates, and world coordinates of the original base calibration point and its corresponding derived calibration point set. Then, using the PT difference between the current base coordinate point and the original base coordinate point, and the derived calibration point set corresponding to the original base calibration point, the derived coordinate point set corresponding to the current base coordinate point is obtained. Based on the current base coordinate point and its corresponding derived coordinate point set, the extrinsic parameter matrix of the coordinate transformation model is updated to obtain a new mapping relationship between pixel coordinates and world coordinates. This method achieves rapid calibration of dynamic camera coordinate mapping and is unaffected by camera offset amplitude and image quality.

[0016] Meanwhile, the present invention provides a rapid calibration device for dynamic camera coordinate mapping, including a current basic calibration point PT value acquisition unit, a PT value deviation calculation unit, a calibration point correction unit, and a coordinate mapping model update unit; The current base calibration point PT value acquisition unit is used to control the pan-tilt unit to rotate, so that the reference object is located in the center of the video frame, and to control the scaling factor relative to the original base calibration point. With the same scaling factor, the current base calibration point of the camera is obtained. of value; The PT value deviation calculation unit is used to calculate the current basic calibration point. of Values ​​and original baseline calibration points of The difference in values ​​is used to obtain the azimuth difference ΔP and the elevation difference ΔT. The calibration point correction unit is used to correct the original base calibration point using azimuth difference ΔP and elevation difference ΔT. The corresponding set of derived calibration points of The value is corrected to obtain the current base calibration point. The corresponding current derived calibration point set of value; The coordinate mapping model update unit is used to update the current base calibration point. and the current set of derived calibration points The input is fed into the pixel coordinate-world coordinate mapping model constructed based on the intrinsic and extrinsic parameter matrices of the PTZ camera. The extrinsic parameter matrix is ​​then updated to obtain a new pixel coordinate-world coordinate mapping model.

[0017] Compared with the prior art, the beneficial effects of the fast calibration device for dynamic camera coordinate mapping proposed in this invention are the same as those of the fast calibration method for dynamic camera coordinate mapping, and will not be repeated here. Attached Figure Description

[0018] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings.

[0019] Figure 1 This is a schematic diagram of the rapid calibration device for dynamic camera coordinate mapping according to the present invention; Figure 2 This is a flowchart of the fast calibration method for dynamic camera coordinate mapping according to the present invention. Detailed Implementation

[0020] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings of the embodiments of the present invention. The described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0021] When the camera shifts excessively, resulting in no overlap between consecutive images, or when image quality is poor, existing automatic camera calibration techniques based on feature point matching become inapplicable. Since camera shifts due to the natural environment only affect the camera's extrinsic parameter matrix, this application designs a method for updating the extrinsic parameter matrix to achieve rapid calibration of dynamic camera coordinate mapping.

[0022] This invention proposes a rapid calibration method for dynamic camera coordinate mapping. Based on the original calibration data, this method processes the camera posture information corresponding to the calibration data, which can achieve rapid calibration of camera coordinate mapping regardless of the camera offset amplitude and image quality.

[0023] The present invention discloses a rapid calibration method for dynamic camera coordinate mapping, which is implemented by a rapid calibration device for dynamic camera coordinate mapping.

[0024] Please see Figure 1 The rapid calibration device for dynamic camera coordinate mapping is communicatively connected to the PTZ camera.

[0025] The PTZ camera rotates and captures images according to the rapid calibration device of dynamic camera coordinate mapping, and transmits the PTZ value of the camera and the captured images to the rapid calibration device of dynamic camera coordinate mapping.

[0026] Please see Figure 2 The rapid calibration device for dynamic camera coordinate mapping is used to execute a rapid calibration method for dynamic camera coordinate mapping, including a current basic calibration point PT value acquisition unit, a PT value deviation calculation unit, a calibration point correction unit, and a coordinate mapping model update unit.

[0027] The current base calibration point PT value acquisition unit is used to execute step S10: control the pan-tilt unit to rotate so that the reference object is located in the center of the video frame, and control the scaling factor to match the original base calibration point. Maintaining a consistent scaling factor, we obtain the current base calibration point of the camera. of value.

[0028] In practice, the original basic calibration point is set as follows: Where P0 is the azimuth of the original base calibration point, T0 is the elevation of the original base calibration point, Z0 is the scaling factor of the original base calibration point, u0 is the x-coordinate of the pixel of the original base calibration point, v0 is the y-coordinate of the pixel of the original base calibration point, lon is the longitude of the original base calibration point, lat is the latitude of the original base calibration point, and h is the elevation of the original base calibration point. The pixel coordinates are in the center of the image, that is, the reference object is in the center of the image.

[0029] Due to natural factors and other reasons, the equipment shifted, causing the reference object corresponding to the latitude and longitude of the original base calibration point to be inconsistent with the reference object corresponding to the pixel coordinates in the image. By controlling the pan-tilt-zoom (PTZ) to rotate the camera, the reference object was positioned in the exact center of the video frame, and the scaling factor was made consistent with the scaling factor of the original base calibration point, thus obtaining the current base calibration point as follows: , where P n For the current base calibration point azimuth angle, T n The pitch angle of the current base calibration point.

[0030] The PT value deviation calculation unit is used to execute step S20: calculate the current basic calibration point. of Values ​​and original baseline calibration points of The difference in values ​​yields the azimuth difference ΔP and the elevation difference ΔT.

[0031] In specific implementation, the azimuth difference ΔP satisfies: △P=P n -P0.

[0032] The pitch angle difference ΔT satisfies: △T=T n -T0.

[0033] The calibration point correction unit is used to perform step S30: using azimuth difference △P and elevation difference △T to correct the original base calibration point. The corresponding set of derived calibration points of Value correction yields the current base calibration point. The corresponding current derived calibration point set of value.

[0034] In practice, the original basic calibration points are selected. The corresponding set of derived calibration points : ; m represents the number of derived calibration points, m > 1; For the derived calibration point set The P-values ​​and T-values ​​of each derived calibration point are corrected and calculated to obtain the current basic calibration point. The corresponding current derived calibration point set : ; in, ,..., ; ,..., ; The original basic calibration point The corresponding set of derived calibration points It is obtained through the following methods: Position the reference object in the exact center of the video frame, and use this calibration point as the original base calibration point. Obtain the original basic calibration point. The latitude and longitude coordinates on the map, along with the PTZ value and pixel coordinates of the corresponding camera area, yield the following: ; A search area centered on the center position is set in the video frame. The pan-tilt head is rotated to change the position of the reference object within the search area, creating several new calibration points as derived calibration points. The PTZ value and pixel coordinates of each derived calibration point in the corresponding camera area are then obtained to obtain the set of derived calibration points. .

[0035] The coordinate mapping model update unit is used to execute step S40: update the current base calibration point. and the current set of derived calibration points The input is fed into the pixel coordinate-world coordinate mapping model constructed based on the intrinsic and extrinsic parameter matrices of the PTZ camera. The extrinsic parameter matrix is ​​then updated to obtain a new pixel coordinate-world coordinate mapping model.

[0036] In practice, a projection mapping model between the pixel coordinates of the video frame and world coordinates is constructed, utilizing the original base calibration points. and its corresponding set of derived calibration points Determine the camera's intrinsic and extrinsic parameter matrices:

[0037] Where M is the intrinsic parameter matrix of the PTZ camera, [R, T] is the extrinsic parameter matrix of the PTZ camera, R is a 3×3 rotation matrix, and T is a 3×1 translation matrix; For pixel coordinates, Used as world coordinates.

[0038] Among them, the original basic calibration points and its corresponding set of derived calibration points latitude and longitude coordinates Need to be converted to planar coordinates The specific calculations are as follows:

[0039] In the formula: The length of the meridian arc; The radius of curvature of the meridian; ; , Longitude of the central meridian; ; , It is the second eccentricity of the ellipsoid.

[0040] Based on latitude and longitude coordinates Obtain latitude and longitude from the elevation file DEM The corresponding altitude h is used to obtain world coordinates. .

[0041] Current baseline calibration point and the current set of derived calibration points Input into the above pixel coordinate to world coordinate mapping model and update the extrinsic parameter matrix [R, T].

[0042] Since the camera shifts due to the natural environment, it only affects the camera's pose, meaning the extrinsic parameters change while the intrinsic parameters remain unchanged. Therefore, when fitting the mapping relationship between pixel coordinates and world coordinates, the intrinsic parameters are used as known quantities and the extrinsic parameters as unknown quantities. Based on the corrected calibration data, the extrinsic parameter matrix of the camera is calculated, thereby establishing a new mapping relationship between pixel coordinates and world coordinates, and realizing rapid calibration of dynamic camera coordinate mapping.

[0043] Meanwhile, the rapid calibration device for dynamic camera coordinate mapping is stored in an electronic device and is executed by the electronic device to implement the rapid calibration method for dynamic camera coordinate mapping.

[0044] The electronic devices include, but are not limited to, memory, processor, and network interface that can communicate with each other via a system bus.

[0045] The electronic device can be a rack server, blade server, tower server, or cabinet server, or other computing device. The electronic device can be a standalone server or a server cluster composed of multiple servers.

[0046] The memory includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. The memory can be an internal storage unit of the electronic device, such as the hard disk or RAM of the electronic device. The memory can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. The memory may also include both internal storage units and external storage devices of the electronic device.

[0047] The processor can be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor is typically used to control the overall operation of the electronic device, such as performing control and processing related to data interaction or communication with the electronic device. The processor is used to run program code stored in the memory or process data, such as running the rapid calibration method for dynamic camera coordinate mapping.

[0048] The network interface may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the electronic device and other electronic devices. For example, the network interface is used to connect the electronic device to an external data platform via a network, establishing a data transmission channel and communication connection between the electronic device and the external data platform. The network may be an intranet, the Internet, Global System for Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0049] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0050] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" and "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0051] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and the present invention also intends to include these modifications and variations.

Claims

1. A rapid calibration method for dynamic camera coordinate mapping, characterized in that, Includes the following steps: S10: Control the pan-tilt-zoom (PTZ) rotation to center the reference object in the video frame, and control the zoom level relative to the original baseline calibration point. With the same scaling factor, the current base calibration point of the camera is obtained. of value; S20: Calculate the current base calibration point of Values ​​and original baseline calibration points of The difference in values ​​is used to obtain the azimuth difference ΔP and the elevation difference ΔT. S30: Use azimuth difference △P and elevation difference △T to calibrate the original foundation points. The corresponding set of derived calibration points of The value is corrected to obtain the current base calibration point. The corresponding current derived calibration point set of value; S40: Set the current base calibration point and the current set of derived calibration points The input is fed into the pixel coordinate-world coordinate mapping model constructed based on the intrinsic and extrinsic parameter matrices of the PTZ camera. The extrinsic parameter matrix is ​​then updated to obtain a new pixel coordinate-world coordinate mapping model.

2. The rapid calibration method for dynamic camera coordinate mapping according to claim 1, characterized in that, The pixel coordinate to world coordinate mapping model constructed based on the intrinsic and extrinsic parameter matrices of the PTZ camera in step S40 is as follows: Where M is the intrinsic parameter matrix of the PTZ camera, [R, T] is the extrinsic parameter matrix of the PTZ camera, R is a 3×3 rotation matrix, and T is a 3×1 translation matrix; For pixel coordinates, Used as world coordinates. The intrinsic and extrinsic parameter matrices are based on the original baseline calibration points. and its corresponding set of derived calibration points confirm.

3. The rapid calibration method for dynamic camera coordinate mapping according to claim 2, characterized in that, The world coordinates are obtained as follows: Original base calibration points and its corresponding set of derived calibration points latitude and longitude coordinates Convert to planar coordinates : In the formula: The length of the meridian arc; The radius of curvature of the meridian; ; , Longitude of the central meridian; ; , This is the second eccentricity of the ellipsoid; Then based on latitude and longitude coordinates Obtain latitude and longitude from the elevation file DEM The corresponding altitude h is used to obtain world coordinates. .

4. The rapid calibration method for dynamic camera coordinate mapping according to claim 1, characterized in that, The original basic calibration point in step S30 The corresponding set of derived calibration points The methods to obtain it are as follows: Position the reference object in the exact center of the video frame, and use this calibration point as the original base calibration point. Obtain the original basic calibration point. The latitude and longitude coordinates on the map, along with the PTZ value and pixel coordinates of the corresponding camera area, yield the following: ; Set a search area centered on the center position in the video frame. Rotate the pan-tilt head to change the position of the reference object within the search area, creating several new calibration points as derived calibration points. Obtain the PTZ value and pixel coordinates of each derived calibration point within the corresponding camera area. The derived calibration point set is obtained. ; m represents the number of derived calibration points, where m > 1.

5. The rapid calibration method for dynamic camera coordinate mapping according to claim 4, characterized in that, The current base calibration point The corresponding current derived calibration point set satisfy: ； in, ,..., ; ，...， 。 6. The rapid calibration method for dynamic camera coordinate mapping according to claim 1, characterized in that, The azimuth difference ΔP satisfies: △P=P n -P0; Among them, P n P0 is the azimuth of the current base calibration point, and P1 is the azimuth of the original base calibration point. The pitch angle difference ΔT satisfies: △T=T n -T0; Among them, T n T0 is the pitch angle of the current base calibration point, and T0 is the pitch angle of the original base calibration point.

7. A rapid calibration device for dynamic camera coordinate mapping, characterized in that, This includes a unit for obtaining the PT value of the current basic calibration point, a unit for calculating the PT value deviation, a unit for correcting the calibration point, and a unit for updating the coordinate mapping model. The current base calibration point PT value acquisition unit is used to control the pan-tilt unit to rotate, so that the reference object is located in the center of the video frame, and to control the scaling factor relative to the original base calibration point. With the same scaling factor, the current base calibration point of the camera is obtained. of value; The PT value deviation calculation unit is used to calculate the current basic calibration point. of Values ​​and original baseline calibration points of The difference in values ​​is used to obtain the azimuth difference ΔP and the elevation difference ΔT. The calibration point correction unit is used to correct the original base calibration point using azimuth difference ΔP and elevation difference ΔT. The corresponding set of derived calibration points of The value is corrected to obtain the current base calibration point. The corresponding current derived calibration point set of value; The coordinate mapping model update unit is used to update the current base calibration point. and the current set of derived calibration points The input is fed into the pixel coordinate-world coordinate mapping model constructed based on the intrinsic and extrinsic parameter matrices of the PTZ camera. The extrinsic parameter matrix is ​​then updated to obtain a new pixel coordinate-world coordinate mapping model.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing programs or instructions that can run on the processor, the programs or instructions being executed by the processor to implement the steps of the rapid calibration method for dynamic camera coordinate mapping as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the rapid calibration method for dynamic camera coordinate mapping as described in any one of claims 1 to 6.

Images