A calibration method, apparatus, electronic device, and storage medium for a camera device.

Abstract

This application provides a calibration method, apparatus, electronic device, and storage medium for camera equipment, applicable to the field of water traffic management, and solves the problem that camera equipment cannot accurately obtain the position of a target object due to changes in water level. The method includes: acquiring the target vertical roll angle of the gimbal module and the target focus value of the camera equipment, wherein the target vertical roll angle and target focus value are the vertical roll angle and focus value when the image of the calibration object captured by the gimbal module meets a preset clarity level; the calibration object is the water surface or an object on the water surface; based on the focus-object distance mapping relationship and the target focus value, determining the target object distance value corresponding to the target focus value; determining the target height above the water for the camera equipment based on the target object distance value and the target vertical roll angle; and correcting the camera conversion relationship based on the target height above the water.

Description

Technical Field

[0001] This application relates to the field of water traffic management, and in particular to a calibration method, apparatus, electronic device and storage medium for a camera device. Background Technology

[0002] With the development of technology, integrated camera equipment that includes radar and pan-tilt units is also widely used to assist in water traffic management.

[0003] Compared to road transportation, water transportation has certain unique characteristics. Water surfaces are wider, and boats move more freely. Therefore, vessels are typically detected first by radar, and then a pan-tilt unit rotates the camera to capture images of the vessel. However, because radar and the pan-tilt unit use different coordinate systems, coordinate transformation between them is necessary. This transformation is affected by the equipment's mounting height; therefore, during dry seasons, flood seasons, or after rainfall, the coordinate transformation relationship between radar and pan-tilt unit coordinates changes due to water level fluctuations.

[0004] Therefore, determining the coordinate transformation relationship between the current radar coordinates and the gimbal coordinates based on the real-time water level has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a calibration method, apparatus, electronic device, and storage medium for camera equipment, used for calibrating real-time water levels.

[0006] Firstly, this application provides a calibration method for a camera device, applied to a camera device including a pan-tilt module and a radar module. The radar module is used to obtain radar position information of a target object. The pan-tilt module rotates the lens portion of the camera device based on offset parameters to capture an image of the target object. The offset parameters are obtained by converting the radar position information of the target object based on a thundercloud conversion relationship. The parameters of the thundercloud conversion relationship include the height of the camera device above the water; the target object is the water surface or an object on the water surface; the method includes: obtaining the target vertical roll angle of the pan-tilt module and the vertical roll angle of the camera device. The target focus value, target vertical roll angle, and target focus value are the vertical roll angle and focus value when the image of the calibrated object meets the preset clarity level; the calibrated object is the water surface or an object on the water surface; based on the focus-object distance mapping relationship and the target focus value, the target object distance value corresponding to the target focus value is determined; the focus-object distance mapping relationship represents the object distance value corresponding to the focus value of the gimbal module when the image meets the preset clarity level; based on the target object distance value and the target vertical roll angle, the target height above the water of the camera device is determined; based on the target height above the water, the thundercloud conversion relationship is corrected.

[0007] As can be seen from the above technical solution, the thundercloud conversion relationship in the camera equipment depends on the camera equipment's height above the water, and this height changes with the rise and fall of the water level. This method determines the target object distance value corresponding to the target focus value by mapping the focus distance to the target object distance and the target focus value when the camera equipment captures the target object (water surface or an object on the water surface). Finally, it determines the current target height above the water, thereby correcting the thundercloud conversion relationship to adapt it to the current water level (target height above the water).

[0008] In one possible implementation, before determining the target object distance value corresponding to the target focus value based on the focus distance mapping relationship and the target focus value, the camera calibration method further includes: acquiring radar position information of multiple calibration points on the water surface based on the radar module within the target time period, and capturing images of multiple calibration points; ensuring that the change in the camera's height above the water falls within a preset error range within the target time period; determining the calibration object distance between each calibration point and the camera based on the radar position information of each calibration point and the camera's height above the water within the target time period; acquiring the calibration focus value of each calibration point if the captured images of the calibration points meet a preset clarity level; and fitting the focus distance mapping relationship based on the calibration focus value and calibration object distance of each calibration point.

[0009] In one possible implementation, a focus-object distance mapping relationship is fitted based on the calibration focus value and calibration object distance of each calibration point, including: determining the calibration focus value and calibration object distance of multiple calibration points; determining the relationship between the focus value and object distance within a target focus interval for a first calibration point and a second calibration point; the target focus interval is the interval from the first calibration value of the first calibration point to the second calibration focus value of the second calibration point, where the first calibration point and the second calibration point are any two calibration points with the closest calibration object distances among the multiple calibration points; and fitting a focus-object distance mapping relationship based on the relationship between the focus value and object distance within the multiple target focus intervals.

[0010] In one possible implementation, the target height above the water of the camera device is determined based on the target distance value and the target vertical roll angle, including: determining the target height above the water of the camera device based on the target distance value, the target vertical roll angle, and the camera device's mounting pitch angle.

[0011] In one possible implementation, before determining the target height above the water for the camera based on the target distance value and the target vertical roll angle, the calibration method for the camera further includes: obtaining the calibration offset parameter of the gimbal module, wherein the calibration offset parameter is the calibration radar position information of the radar module corresponding to the image of the calibration point being captured, provided that the image meets a preset clarity level; and determining the mounting pitch angle of the camera based on the calibration offset parameter and the calibration radar position information.

[0012] In one possible implementation, obtaining the target vertical roll angle of the gimbal module and the target focus value of the camera device includes: if the image of the calibration object captured meets the preset clarity level, determining the vertical roll angle of the gimbal module as the target offset, and determining the focus value of the calibration object captured by the camera device as the target focus value.

[0013] In one possible implementation, the calibration method for the camera device further includes: when the radar module acquires the radar position information of the target object on the water surface, determining the target parameters corresponding to the gimbal module when shooting the target object based on the corrected radar cloud conversion relationship and radar position information, the target parameters include the vertical roll angle and the horizontal offset; and controlling the gimbal module of the camera device according to the target parameters to capture the image of the target object.

[0014] Secondly, this application provides a calibration device for a camera device, comprising: a gimbal data acquisition unit for acquiring the target vertical roll angle of the gimbal module and the target focus value of the camera device, wherein the target vertical roll angle and target focus value are the vertical roll angle and focus value when the image of the calibration object captured by the gimbal module meets a preset level of clarity; the calibration object is a water surface or an object on the water surface; an object distance determination unit for determining the target object distance value corresponding to the target focus value based on the focus object distance mapping relationship and the target focus value, wherein the focus object distance mapping relationship characterizes the object distance value corresponding to the focus value of the gimbal module when the image captured by the gimbal module meets the preset level of clarity; a water height determination unit for determining the target water height of the camera device based on the target object distance value and the target vertical roll angle; and a correction unit for correcting the thundercloud conversion relationship based on the target water height.

[0015] In one possible embodiment, the calibration device for the camera equipment further includes: a data determination unit, a radar data determination unit, and a fitting unit. The data determination unit acquires radar position information of multiple calibration points on the water surface based on the radar module within a target time period and captures images of the multiple calibration points; the change in the camera equipment's height above the water within the target time period falls within a preset error range. The radar data determination unit determines the calibration distance between each calibration point and the camera equipment based on the radar position information of each calibration point and the camera equipment's height above the water within the target time period. The gimbal data acquisition unit is further configured to acquire the calibration focus value of each calibration point, provided that the captured images of the calibration points meet a preset clarity level. The fitting unit is configured to fit a focus-distance mapping relationship based on the calibration focus value and calibration distance of each calibration point.

[0016] In one possible embodiment, the object distance determination unit is specifically configured to: determine the calibration focus value and calibration object distance of multiple calibration points; for the first calibration point and the second calibration point, determine the relationship between the focus value and the object distance within a target focus interval; the target focus interval is the interval from the first calibration value of the first calibration point to the second calibration focus value of the second calibration point, wherein the first calibration point and the second calibration point are any two of the multiple calibration points whose calibration object distances are closest; and fit a focus object distance mapping relationship based on the relationship between the focus value and the object distance within the multiple target focus intervals.

[0017] In one possible embodiment, the height above water determination unit is specifically used to: determine the height above water of the target of the camera equipment based on the target distance value, the target vertical roll angle, and the camera equipment mounting pitch angle.

[0018] In one possible embodiment, the calibration device of the camera equipment further includes a pitch angle determination unit, which is used to obtain the calibration offset parameter of the gimbal module. The calibration offset parameter is the calibration radar position information of the radar module corresponding to the image of the calibration point when the image meets the preset clarity. Based on the calibration offset parameter and the calibration radar position information, the mounting pitch angle of the camera equipment is determined.

[0019] In one possible embodiment, the gimbal data acquisition unit is specifically used to determine the vertical roll angle of the gimbal module as the target offset and the focus value of the camera device capturing the calibration object as the target focus value, provided that the image of the calibration object captured by the camera meets the preset clarity level.

[0020] In one possible embodiment, the calibration device for the camera equipment further includes: a target parameter determination unit, used to determine the target parameters corresponding to the gimbal module when shooting the target object, based on the corrected radar cloud conversion relationship and radar position information, when the radar module acquires the radar position information of the target object on the water surface; the target parameters include the vertical roll angle and the horizontal offset; and a target information acquisition unit, used to control the gimbal module of the camera equipment according to the target parameters to capture an image of the target object.

[0021] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code including computer instructions; wherein, when the processor executes the computer instructions, it causes the electronic device to perform a calibration method for a camera device as described in the first aspect and any possible design thereof.

[0022] Fourthly, this application provides a computer-readable storage medium comprising: computer software instructions; when the computer software instructions are executed in a calibration device of a camera device, the calibration device of the camera device causes the camera device to implement the method described in the first aspect.

[0023] Fifthly, this application provides a computer program product that, when run on a calibration device of a camera device, causes the calibration device of the camera device to perform the steps of the relevant method described in the first aspect above, so as to implement the method of the first aspect above.

[0024] The beneficial effects of the second to fifth aspects mentioned above can be referred to the corresponding description of the first aspect, and will not be repeated here. Attached Figure Description

[0025] Figure 1 A schematic diagram of the structure of the camera device 11 in the calibration method of the camera device provided in this application;

[0026] Figure 2 A flowchart illustrating a calibration method for a camera device provided in this application. Figure 1 ;

[0027] Figure 3 A schematic diagram illustrating the positional relationship between the camera device and the target object in a calibration method for a camera device provided in this application;

[0028] Figure 4 A schematic diagram illustrating the positional relationship between the camera device and the calibration object in a calibration method for a camera device provided in this application;

[0029] Figure 5 A flowchart illustrating a calibration method for a camera device provided in this application. Figure 2 ;

[0030] Figure 6 A flowchart illustrating a calibration method for a camera device provided in this application. Figure 3 ;

[0031] Figure 7 A schematic diagram of the camera equipment and calibration points for a calibration method of camera equipment provided in this application;

[0032] Figure 8 A schematic diagram of the focus distance mapping relationship of a calibration method for a camera device provided in this application;

[0033] Figure 9 A schematic diagram of the structure of a calibration device for a camera device provided in this application;

[0034] Figure 10 This is a schematic diagram of the hardware composition of a calibration device for a camera provided in this application. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] It should be noted that in the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0037] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0038] As the background technology indicates, water transportation has certain unique characteristics compared to road transportation. Water surfaces are wider, and vessels move more freely. Therefore, vessels are typically detected first by radar, and then image information is obtained by a pan-tilt-zoom (PTZ) camera. However, since radar and the PTZ camera use different coordinate systems, coordinate transformation between radar and PTZ coordinates is necessary. This coordinate transformation is affected by the equipment's mounting height; therefore, the coordinate transformation relationship between radar and PTZ coordinates will change due to fluctuations in water levels during dry seasons, flood seasons, or after rainfall.

[0039] Therefore, determining the coordinate transformation relationship between the current radar coordinates and the gimbal coordinates based on the real-time water level has become an urgent technical problem to be solved.

[0040] To address this issue, embodiments of this application provide a calibration method, apparatus, electronic device, and storage medium for a camera device.

[0041] In one possible implementation, such as Figure 1 The schematic diagram of the camera device 11 shown illustrates a calibration method for a camera device provided in this application embodiment, which can be applied to a camera device 11 including a pan-tilt module 101 and a radar module 102.

[0042] In one possible embodiment, the camera device 11 can acquire the target vertical roll angle of the gimbal module 101 and the target focus value of the camera device 11. The target vertical roll angle and target focus value are the vertical roll angle and focus value when the image of the calibrated object meets the preset clarity level. The calibrated object is the water surface or an object on the water surface. Based on the focus-object distance mapping relationship and the target focus value, the target object distance value corresponding to the target focus value is determined. The focus-object distance mapping relationship represents the object distance value corresponding to the focus value of the gimbal module 101 when the image meets the preset clarity level. According to the target object distance value and the target vertical roll angle, the target height above the water of the camera device is determined. Based on the target height above the water, the thundercloud conversion relationship is corrected.

[0043] In one possible embodiment, the camera device 11 can also acquire radar position information of multiple calibration points on the water surface based on the radar module 102 within a target time period, and capture images of multiple calibration points; the change in the camera device's height above the water within the target time period falls within a preset error range; the calibration distance between each calibration point and the camera device is determined based on the radar position information of each calibration point and the camera device's height above the water within the target time period; if the captured images of the calibration points meet a preset clarity level, the calibration focus value of each calibration point is acquired; and a focus-distance mapping relationship is fitted based on the calibration focus value and calibration distance of each calibration point.

[0044] In one possible embodiment, the camera device 11 can further determine the calibration focus value and calibration object distance of multiple calibration points; for the first calibration point and the second calibration point, determine the relationship between the focus value and the object distance within the target focus range; the target focus range is the range from the first calibration value of the first calibration point to the second calibration focus value of the second calibration point, and the first calibration point and the second calibration point are any two calibration points whose calibration object distances are closest among the multiple calibration points; and fit a focus object distance mapping relationship according to the relationship between the focus value and the object distance in the multiple target focus ranges.

[0045] In one possible embodiment, the camera device 11 can also determine the target height above the water based on the target distance value, the target vertical roll angle, and the camera device's mounting pitch angle.

[0046] In one possible embodiment, the camera device 11 can also acquire the calibration offset parameter of the gimbal module 101. The calibration offset parameter is the calibration radar position information of the radar module 102 corresponding to the image of the calibration point when the captured image meets the preset clarity. Based on the calibration offset parameter and the calibration radar position information, the mounting pitch angle of the camera device is determined.

[0047] In one possible embodiment, the camera device 11 can also determine the vertical roll angle of the gimbal module 101 as the target offset and the focus value of the camera device capturing the calibration object as the target focus value, provided that the image of the calibration object captured by the camera device meets the preset clarity level.

[0048] In one possible embodiment, the camera device 11 can also determine the target parameters corresponding to the gimbal module 101 when shooting the target object based on the corrected thundercloud conversion relationship and radar position information after the radar module 102 acquires the radar position information of the target object. The target parameters include the vertical roll angle and the horizontal offset. Based on the target parameters, the camera device 11 controls the gimbal module 101 to capture the image of the target object.

[0049] The camera device 11 can be a PTZ camera with a radar module, or a camera with both a PTZ module and a radar module.

[0050] In one possible embodiment, Figure 2 This is a flowchart illustrating a calibration method for a camera device provided in an embodiment of this application. The method is applied to camera devices, such as... Figure 2 As shown, the calibration method for the camera device provided in this application embodiment may specifically include the following steps S201 to S204.

[0051] The camera equipment includes a pan-tilt module and a radar module. The radar module is used to obtain the radar position information of the target object. The pan-tilt module rotates the lens of the camera equipment based on offset parameters to capture images of the target object. The offset parameters are obtained by converting the radar position information of the target object based on the thundercloud conversion relationship, which includes the camera equipment's height above the water. The target object is the water surface or an object on the water surface.

[0052] It should be understood that, such as Figure 3 As shown, using the water surface as a planar coordinate system, the radar module can obtain the X and Y coordinates of objects on the water surface in this planar coordinate system. A gimbal module typically includes at least three parameters: P (Pan) and T (Tilt), namely the horizontal offset parameter Pan, the vertical roll angle Tilt, and the zoom parameter Zoom. For camera devices with adjustable fields of view, the camera also has a zoom parameter Zoom. The vertical roll angle refers to the rotation angle of the camera device with a gimbal around its own front-to-back axis.

[0053] First, the coordinates in the radar coordinate system (centered on the projection of the installation position on the water surface) are mapped to the Cartesian coordinates of the gimbal module, as shown in Equation 1 below.

[0054] Where x and y represent the coordinates of the gimbal module in Cartesian coordinates, x0 and y0 represent the coordinates in the radar coordinate system, and h represents the height above the water, i.e., the height of the camera device above the water surface. The matrix formed by h11-h33 is the homography matrix of the gimbal module.

[0055]

[0056] The positional relationship between the target object and the camera equipment can be as follows: Figure 3 As shown, the plane represents the horizontal plane, and the XOY coordinates are the radar coordinate system of the radar module. The distance h between the camera and the plane represents the height above the water, the angle pan represents the horizontal offset parameter of the gimbal module when shooting the target object, and the angle tilt represents the vertical offset parameter of the gimbal module when shooting the target object. R represents the object distance between the camera and the target object.

[0057] Given that the horizontal offset parameters tilt and pan, as well as the height h above the water, are known in the radar coordinate system when the gimbal module is photographing the target object, we can obtain Equations 2 and 3 to characterize the Cartesian coordinates of the target object under the gimbal module.

[0058]

[0059]

[0060] Combining Equations 1, 2, and 3 above, we can obtain the conversion relationship between the coordinates of the target object determined by the radar module and the parameters of the gimbal module, as shown in Equation 4 below. Equation 4 can characterize the above-mentioned radar-cloud conversion relationship.

[0061]

[0062] As shown in Equation 4, the thundercloud conversion relationship is related to the distance between the camera and the water surface, i.e., the height h above the water. During dry seasons, flood seasons, and under the influence of rainfall, the height of the camera above the water changes due to water level fluctuations. Therefore, performing the thundercloud conversion based on the previously known height above the water, as shown in Equation 4, will result in a significant error, causing the pan-tilt module to fail to accurately capture the target object. Therefore, it is necessary to obtain the real-time height above the water to correct the thundercloud conversion relationship.

[0063] S201, obtain the target vertical roll angle of the gimbal module and the target focus value of the camera device.

[0064] The target vertical roll angle and target focus value are the vertical roll angle and focus value when the image of the calibration object meets the preset sharpness level. The calibration object is the water surface or an object on the water surface.

[0065] S202, based on the focus-object distance mapping relationship and the target focus value, determine the target object distance value corresponding to the target focus value.

[0066] Among them, the focus-object distance mapping relationship represents the object distance value corresponding to the focus value of the gimbal module when the captured image meets the preset clarity level.

[0067] In one possible implementation, obtaining the target vertical roll angle of the gimbal module and the target focus value of the camera device includes: if the image of the calibration object captured by the gimbal module meets the preset clarity level, determining the vertical roll angle of the gimbal module as the target offset and the focus value of the gimbal module as the target focus value.

[0068] Zooming changes the focal length of a lens, thereby altering the field of view in an image. The zoom mentioned above corresponds to the scaling parameter (Zoom) of a camera device.

[0069] Focusing (or adjusting focus) is achieved by moving the relative distance between the CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide-Semiconductor) and the lens to obtain the sharpest image. For camera lens imaging, the image sharpness is only high on a specific plane; other areas will be significantly blurry. This sharp plane is called the imaging plane. This sharply focused imaging plane is also related to the distance between the object being photographed and the lens. When a convex lens is used as a camera lens, the sharpest image generally does not fall exactly on the lens's focal point; the specific distance depends on the distance between the object being photographed and the lens (object distance).

[0070] When a camera captures a subject, the desired image is a sharp image, not a blurry one. This means that the camera's focus must be adjusted during shooting to achieve a preset level of sharpness. The focus value required to achieve this preset level of sharpness is related to the object distance between the camera and the subject.

[0071] Therefore, this method first obtains the target focus value when the image of the calibration object captured by the camera device meets the preset clarity level, and then, based on the focus-object distance mapping relationship, determines the target object distance value corresponding to the target focus value.

[0072] S203, determine the target height above the water for the camera equipment based on the target distance value and the target vertical roll angle.

[0073] When the camera equipment photographs the calibration object, its geometric relationship can be simplified as follows: Figure 4As shown, since the calibration object is the water surface or an object on the water surface, given that the target distance between the camera equipment and the calibration object is known, and the target's vertical roll angle is also known, the target's height above the water can be calculated using Equation 5 below. In Equation 5, H represents the target's height above the water, D represents the target distance, and tilt represents the target's vertical roll angle.

[0074]

[0075] S204, based on the target's height above the water, corrects the thundercloud conversion relationship.

[0076] It should be understood that the target's height above the water is the same as the current height of the camera equipment above the water. After calculation, the target's height above the water, H, can be substituted into the equation 4 above as h. This corrects the thundercloud conversion relationship. The corrected thundercloud conversion relationship is determined based on the currently calibrated target height above the water, and therefore adapts to changing water levels.

[0077] As can be seen from the above technical solution, the thundercloud conversion relationship in the camera equipment depends on the camera equipment's height above the water, and this height changes with the rise and fall of the water level. This method determines the target object distance value corresponding to the target focus value by mapping the focus distance to the target object distance and the target focus value when the camera equipment captures the target object (water surface or an object on the water surface). Finally, it determines the current target height above the water, thereby correcting the thundercloud conversion relationship to adapt it to the current water level (target height above the water).

[0078] In one possible implementation, the target height above the water of the camera device is determined based on the target distance value and the target vertical roll angle, including: determining the target height above the water of the camera device based on the target distance value, the target vertical roll angle, and the camera device's mounting pitch angle.

[0079] It should be understood that, since the camera equipment may not be perfectly perpendicular to the ground during installation, there will be a certain error between the vertical roll angle determined by the camera equipment and the actual vertical roll angle. This error is the installation pitch angle. Therefore, as shown in Equation 6, the installation pitch angle can be used to correct the target's vertical roll angle during calculation, thereby making the calculated height above the water more accurate.

[0080]

[0081] In one possible implementation, before determining the target height above the water for the camera based on the target distance and the target vertical roll angle, the camera calibration method further includes: obtaining the calibration offset parameters of the gimbal module, where the calibration offset parameters are the calibration radar position information corresponding to the radar module, provided that the image at the calibration point meets a preset clarity level. Based on the calibration offset parameters and the calibration radar position information, the camera's mounting pitch angle is determined.

[0082] As shown in Equation 4, if the calibration offset coefficients (tilt and pan in Equation 4), the calibration radar position information (x0 and y0), and the height above the water h are all known, the calculation results on both sides of Equation 4 should be equal. However, since the mounting pole may not be completely perpendicular to the horizontal plane during the installation of the camera equipment (or the initial state of the camera equipment may not be completely parallel to the horizontal plane), i.e., there is a mounting pitch angle, the results on both sides of Equation 4 are not equal. Therefore, we can assume that there is a mounting pitch angle θ, correct tilt and pan in Equation 4 with θ, and then deduce the size of the mounting pitch angle θ through the calculation in Equation 4.

[0083] In one possible implementation, the camera calibration method further includes: after the radar module acquires the radar position information of the target object on the water surface, determining the target parameters corresponding to the gimbal module when shooting the target object based on the corrected radar cloud conversion relationship and radar position information. The target parameters include the vertical roll angle and the horizontal offset. Based on the target parameters, the gimbal module of the camera is controlled to capture an image of the target object.

[0084] For example, in Equation 4, after correcting the camera conversion relationship based on the calibrated target height above the water, the radar position information (such as x0 and y0 in Equation 4) can be determined based on the radar module, and the target parameters of the pan-tilt module (such as tilt and pan in Equation 4) can be determined. Therefore, based on the target parameters, the pan-tilt module of the camera device is controlled to rotate the camera device to the corresponding position to capture an image of the target object.

[0085] In one possible implementation, such as Figure 5 As shown, the calibration method for the camera equipment also includes the following S301 to S304, which are used to determine the focus distance mapping relationship.

[0086] S301, within the target time period, acquires radar position information of multiple calibration points on the water surface based on the radar module, and captures images of multiple calibration points.

[0087] Among them, the change in the height of the camera above the water within the target time period falls within the preset error range.

[0088] It should be understood that the radar position information of the radar module and the images captured by the gimbal module for the calibration point are both acquired within the target time period. Therefore, both the radar position information of the calibration point and the information acquired when the camera captures images are obtained at the same or within the preset error range at the same height above the water.

[0089] S302, based on the radar position information of each calibration point and the height of the camera equipment above the water during the target time period, determine the calibration distance between each calibration point and the camera equipment.

[0090] It should be understood that at this time, the radar module can obtain radar position information of multiple calibration points on the water surface, in order to (x i y i Let ) represent the position information of the i-th calibration point among multiple calibration points, let height represent the height of the camera equipment above the water during the target time period, and let δ represent the near-far ratio coefficient of the radar module. Then, the calibration object distance dis between the i-th calibration point and the camera equipment at this time is... i It can be calculated as shown in Equation 7.

[0091]

[0092] S303 acquires the calibration focus value of each calibration point if the image captured at the calibration point meets the preset clarity level.

[0093] S304, based on the calibration focus value and calibration object distance at each calibration point, fits the focus-object distance mapping relationship.

[0094] When the calibration focus value and calibration object distance are known at multiple calibration points, the values ​​at multiple calibration points can be fitted to obtain the focus-object distance mapping relationship. Specifically, the fitting method can be either direct fitting based on the focus value and object distance corresponding to multiple calibration points, or constructing a piecewise function based on the focus value and object distance corresponding to multiple calibration points, and then fitting the piecewise function. This application does not impose any specific limitations on this approach.

[0095] As can be seen from the above technical solution, since there is a correlation between the focus value and the object distance, this method can fit the focus-object distance mapping relationship of the camera device by collecting the real focus value and object distance.

[0096] In one possible implementation, based on the calibration focus value and calibration object distance at each calibration point, a focus-object distance mapping relationship is fitted, including the following steps S401 to S403, such as... Figure 6 As shown.

[0097] S401 determines the calibration focus value and calibration object distance for multiple calibration points.

[0098] It should be understood that the calibration focus value here is obtained when the image of the calibration point meets the preset sharpness level.

[0099] S402, for the first calibration point and the second calibration point, determine the relationship between the focus value and the object distance within the target focus range.

[0100] The target focusing range is the interval between the first calibration value of the first calibration point and the second calibration focusing value of the second calibration point. The first calibration point and the second calibration point are any two calibration points whose calibration object distances are closest among multiple calibration points.

[0101] For example, such as Figure 7 As shown, among calibration points A (10m), B (20m), C (30m), D (40m), and E (50m), the two calibration points with the closest calibration distances can be any pair from A and B, B and C, C and D, or D and E. Therefore, based on these four pairs of calibration points with the closest calibration distances, and using the data from each pair as the two endpoints of a linear function, four linear functions can be determined. As shown in Equation 8, the calibration points are ordered by their suffixes ABCDE, and D... i The calibration object distance representing the i-th calibration point, a i x i +b i The expression representing the function of the i-th straight line segment, a i and b i It is a constant.

[0102]

[0103] S403 fits the focus-object distance mapping relationship based on the relationship between focus value and object distance in multiple target focus intervals.

[0104] Following the above embodiments, further, f(x) i If ) represents the i-th piecewise function in the piecewise function f(x), then the linear fitting function P(x) i It can be shown in Equation 9 below.

[0105] ε i =p(x i )-f(x i (Equation 9)

[0106] Where, vector e = [ε1, ε2, ..., ε n ].

[0107] Then, the least squares method is used to calculate the following equation 10, which minimizes the polynomial squared difference. The final p(xi) obtained is the best fitted curve.

[0108]

[0109] Furthermore, for the five calibration points collected, within the piecewise linear region, they can be viewed as the difference of squared polynomials and the coefficient a. i b i The multivariate function is then used to find the extrema of the multivariate function fitting the polynomial. Let the fitting curve be p(x) = m0x. 2 The extremum principle of the multivariate function +m1x+m2 satisfies the minimum value formula, and the following equation can be obtained: 11.

[0110]

[0111] Based on Equation 11, Equation 12 can be obtained, where n is the number of calibration points, and then the binary equations m0 and m1, m2 are obtained, as shown in Equation 12.

[0112]

[0113] Eliminating variables from Equation 12, we obtain the fitted curve by taking the coefficients m0, m1, and m2.

[0114] For example, the focus distance mapping relationship obtained by fitting can be as follows: Figure 8 As shown in the diagram, the object distance is represented by the x-axis, and the focus value is represented by the y-axis. As the object distance increases, the focus value gradually decreases. Of course, this diagram is merely an example; other devices may have different focus-object distance mapping relationships, and this application does not impose any specific limitations on this.

[0115] In one possible implementation, the camera calibration method further includes: determining the target field of view of the object to be captured by the camera based on the length of the target object, the pixel ratio of the target object in the image frame, and the object distance between the target object and the camera. Based on the target scaling parameters corresponding to the target field of view, the camera is then controlled to capture images of the target object.

[0116] For example, as in Equation 13, the appropriate target field of view fov is determined when the camera takes pictures of the target object, using the length L of the target object, the pixel ratio of the target object in the image screenRatio, and the target object distance value dis between the target object and the camera device.

[0117]

[0118] Furthermore, by establishing the correspondence between the camera's field of view and the scaling parameters, the appropriate scaling parameters for capturing the target object can be determined. Therefore, based on the target scaling parameters, the camera can be controlled to capture the target object.

[0119] In one possible implementation, the target object is a vessel, and the calibration method for the camera equipment further includes: identifying the vessel's license plate based on the captured image of the target object, and determining the vessel identifier based on the identification result of the vessel's license plate. Under preset conditions, the identified vessel identifier of the target object is determined as a pre-selected identifier for the target object.

[0120] It should be understood that after controlling the camera device based on the target scaling parameters corresponding to the target field of view, the camera device can clearly capture the target object with a suitable field of view, and can identify the target object's signboard based on the captured image.

[0121] In one possible implementation, the preset condition is that within a preset time period, the movement direction of the target object's boat sign is similar to the movement direction of the target object.

[0122] Since the boat tags are located on the hull and move with the boat, if a tag belongs to one boat, both tags will move in the same or similar directions. Conversely, if the tags and boats move in significantly different directions, then the tags and boats...

[0123] In another possible implementation, the preset condition is a pre-selected identifier of the target object to be identified, which is different from the ship identifier determined in a preset historical period.

[0124] Since multiple vessels may exist simultaneously within the field of view of the camera equipment, the vessel identifiers determined in the preset historical time period are those vessels that have already been captured and identified by the camera equipment. Vessel identifiers that have already been identified do not need to be identified again.

[0125] In one possible implementation, the calibration method for the camera device further includes: comparing a pre-selected identifier of the target object with identifiers in a ship number database, the ship number database storing ship number identifiers. The pre-selected identifier is then matched with a similar identifier in the ship number identifier database, which is used as the target identifier for the target object. The target similar identifier is the identifier in the ship number identifier database that has the highest similarity to the target identifier and whose similarity is greater than a preset similarity.

[0126] Because some vessel registration plates may have stains or lack reflective markings, making them difficult to identify, the identified vessel number may differ from the actual number. Therefore, after determining a pre-selected identifier for the target object based on the captured image, the identified vessel number can be compared with vessel number identifiers in a database. The vessel number identifier that best matches the pre-selected identifier in the database, or the vessel number identifier in the database that matches the pre-selected identifier, is ultimately determined as the target identifier for the target object (vessel).

[0127] For example, if the pre-selected identifier is "Cargo Transportation 6129" and the ship number database stores "A Cargo Transportation 6129", then "Cargo Transportation 6129" can be confirmed as the target identifier of the target object.

[0128] For example, if the preselected identifier is "goods transportation 6129" and the ship number database stores the identifier "cargo transportation 6129" that is most similar to the preselected identifier, since the two are very similar and the preselected identifier "goods transportation 6129" is contained within "cargo transportation 6129", the preselected identifier "goods transportation 6129" is regarded as a substring of "cargo transportation 6129", and thus "cargo transportation 6129" can be identified as the target identifier of the target object.

[0129] For example, if the preselected identifier is "Cargo Transportation 8129" and the ship number database stores the identifier "Cargo Transportation 6129" which is most similar to the preselected identifier, then since the two are very similar and the process of identifying the preselected identifier may be misjudged, "Cargo Transportation 6129" can be identified as the target identifier of the target object.

[0130] In one possible embodiment, this application provides a calibration device for a camera device, such as... Figure 9 As shown, the calibration device for the camera equipment includes:

[0131] The gimbal data acquisition unit 501 is used to acquire the target vertical roll angle of the gimbal module and the target focus value of the camera device. The target vertical roll angle and target focus value are the vertical roll angle and focus value when the image of the calibration object captured by the gimbal module meets the preset clarity level. The calibration object is the water surface or an object on the water surface. The object distance determination unit 502 is used to determine the target object distance value corresponding to the target focus value based on the focus object distance mapping relationship and the target focus value. The focus object distance mapping relationship represents the object distance value corresponding to the focus value of the gimbal module when the image captured by the gimbal module meets the preset clarity level. The water height determination unit 503 is used to determine the target water height of the camera device according to the target object distance value and the target vertical roll angle. The correction unit 504 is used to correct the thundercloud conversion relationship based on the target water height.

[0132] In one possible embodiment, the calibration device for the camera equipment further includes: a data determination unit, a radar data determination unit, and a fitting unit. The data determination unit acquires radar position information of multiple calibration points on the water surface based on the radar module within a target time period and captures images of the multiple calibration points; the change in the camera equipment's height above the water within the target time period falls within a preset error range. The radar data determination unit determines the calibration distance between each calibration point and the camera equipment based on the radar position information of each calibration point and the camera equipment's height above the water within the target time period. The gimbal data acquisition unit 501 is further configured to acquire the calibration focus value of each calibration point, provided that the captured images of the calibration points meet a preset clarity level. The fitting unit is configured to fit a focus-distance mapping relationship based on the calibration focus value and calibration distance of each calibration point.

[0133] In one possible embodiment, the object distance determination unit 502 is specifically used for: determining the calibration focus value and calibration object distance of multiple calibration points; determining the relationship between the focus value and object distance within a target focus interval for a first calibration point and a second calibration point; the target focus interval is the interval from the first calibration value of the first calibration point to the second calibration focus value of the second calibration point, wherein the first calibration point and the second calibration point are any two calibration points whose calibration object distances are closest among the multiple calibration points; and fitting a focus object distance mapping relationship based on the relationship between the focus value and object distance within the multiple target focus intervals.

[0134] In one possible embodiment, the height above water determination unit 503 is specifically used to: determine the height above water of the target of the camera equipment based on the target distance value, the target vertical roll angle and the camera equipment mounting pitch angle.

[0135] In one possible embodiment, the calibration device of the camera equipment further includes a pitch angle determination unit, which is used to obtain the calibration offset parameter of the gimbal module. The calibration offset parameter is the calibration radar position information of the radar module corresponding to the image of the calibration point when the image meets the preset clarity. Based on the calibration offset parameter and the calibration radar position information, the mounting pitch angle of the camera equipment is determined.

[0136] In one possible embodiment, the gimbal data acquisition unit 501 is specifically used to determine the vertical roll angle of the gimbal module as the target offset and the focus value of the camera device capturing the calibration object as the target focus value, provided that the image of the calibration object captured by the camera meets the preset clarity level.

[0137] In one possible embodiment, the calibration device for the camera equipment further includes: a target parameter determination unit, used to determine the target parameters corresponding to the gimbal module when shooting the target object, based on the corrected radar cloud conversion relationship and radar position information, when the radar module acquires the radar position information of the target object on the water surface; the target parameters include the vertical roll angle and the horizontal offset; and a target information acquisition unit, used to control the gimbal module of the camera equipment according to the target parameters to capture an image of the target object.

[0138] In the case of implementing the functions of the integrated units described above in hardware, this application embodiment provides a schematic diagram of the hardware composition of a calibration device for a camera equipment, as shown below. Figure 10 As shown, the calibration device for the camera equipment may include: a processor 602, a communication interface 603, and a bus 604. Optionally, the calibration device for the camera equipment may also include a memory 601.

[0139] Processor 602 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 602 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 602 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0140] Communication interface 603 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0141] The memory 601 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0142] As one possible implementation, the memory 601 can exist independently of the processor 602. The memory 601 can be connected to the processor 602 via a bus 604 and is used to store instructions or program code. When the processor 602 calls and executes the instructions or program code stored in the memory 601, it can implement the calibration method of the camera device provided in this application embodiment.

[0143] In another possible implementation, the memory 601 can also be integrated with the processor 602.

[0144] Bus 604 can be an extended industry standard architecture (EISA) bus, etc. Bus 604 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0145] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the calibration device of the camera equipment can be divided into different functional modules to complete all or part of the functions described above.

[0146] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by computer instructions instructing related hardware. The program can be stored in the computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be any of the foregoing embodiments or memory. The computer-readable storage medium can also be an external storage device of the calibration apparatus of the camera device, such as a pluggable hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the calibration apparatus of the camera device. Further, the computer-readable storage medium can include both internal storage units of the calibration apparatus of the camera device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the calibration apparatus of the camera device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0147] This application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to execute any of the camera device calibration methods provided in the above embodiments.

[0148] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple components. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0149] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

[0150] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A calibration method for a camera device, characterized in that, The method is applied to a camera device, which includes a pan-tilt module and a radar module. The radar module is used to obtain the radar position information of the target object. The pan-tilt module rotates the lens part of the camera device based on an offset parameter to capture an image of the target object. The offset parameter is obtained by converting the radar position information of the target object based on a thundercloud conversion relationship. The parameters of the thundercloud conversion relationship include the height of the camera device above the water. The target object is a water surface or an object on a water surface; the method includes: The target vertical roll angle of the gimbal module and the target focus value of the camera device are obtained. The target vertical roll angle and the target focus value are the vertical roll angle and focus value when the image of the calibration object meets the preset clarity. The calibration object is the water surface or an object on the water surface. Based on the focus distance mapping relationship and the target focus value, the target object distance value corresponding to the target focus value is determined. The focus distance mapping relationship represents the object distance value corresponding to the focus value of the gimbal module when the captured image meets the preset clarity level. The target height above the water is determined by the target distance value and the target vertical roll angle. The thundercloud conversion relationship is corrected based on the target's height above the water.

2. The calibration method for a camera device according to claim 1, characterized in that, Before determining the target object distance value corresponding to the target focus value based on the focus distance mapping relationship and the target focus value, the method further includes: During the target time period, the radar module acquires radar position information of multiple calibration points on the water surface and captures images of the multiple calibration points; the change in the height of the camera above the water during the target time period falls within a preset error range. Based on the radar position information of each calibration point and the distance of the camera device from the water during the target time period, the calibration object distance between each calibration point and the camera device is determined; If the images captured at the calibration points meet the preset clarity level, obtain the calibration focus value for each calibration point; Based on the calibration focus value and the calibration object distance at each calibration point, the focus object distance mapping relationship is fitted.

3. The calibration method for a camera device according to claim 2, characterized in that, The process of fitting the focus-object distance mapping relationship based on the calibration focus value and the calibration object distance at each calibration point includes: Determine the calibration focus value and calibration object distance at multiple calibration points; For the first calibration point and the second calibration point, the relationship between the focus value and the object distance within the target focus range is determined; the target focus range is the range from the first calibration value of the first calibration point to the second calibration focus value of the second calibration point, and the first calibration point and the second calibration point are any two of the plurality of calibration points whose calibration object distances are closest; Based on the relationship between the focus value and the object distance in multiple target focus intervals, the focus object distance mapping relationship is fitted.

4. The calibration method for a camera device according to claim 2, characterized in that, Determining the target height above the water for the camera device based on the target distance value and the target vertical roll angle includes: The target height above the water is determined based on the target distance value, the target vertical roll angle, and the camera equipment's mounting pitch angle.

5. The calibration method for a camera device according to claim 4, characterized in that, Before determining the target height above the water for the camera device based on the target distance value and the target vertical roll angle, the method further includes: Obtain the calibration offset parameter of the gimbal module. The calibration offset parameter is the calibration radar position information of the radar module when the image of the captured calibration point meets the preset clarity. Based on the calibration offset parameters and calibration radar position information, the elevation angle of the camera equipment is determined.

6. The calibration method for a camera device according to claim 1, characterized in that, The step of obtaining the target vertical roll angle of the gimbal module and the target focus value of the camera device includes: If the image of the calibration object captured meets the preset clarity level, the vertical roll angle of the gimbal module is determined as the target offset, and the focus value of the calibration object captured by the camera device is determined as the target focus value.

7. The calibration method for a camera device according to claim 1, characterized in that, The method further includes: When the radar module acquires the radar position information of the target object on the water surface, the target parameters corresponding to the gimbal module when shooting the target object are determined according to the corrected thundercloud conversion relationship and the radar position information. The target parameters include the vertical roll angle and the horizontal offset. Based on the target parameters, the pan-tilt module of the camera device is controlled to capture an image of the target object.

8. A calibration device for a camera, characterized in that, The device includes: The gimbal data acquisition unit is used to acquire the target vertical roll angle of the gimbal module and the target focus value of the camera device. The target vertical roll angle and the target focus value are the vertical roll angle and focus value when the image of the calibration object captured by the gimbal module meets the preset clarity level. The calibration object is the water surface or an object on the water surface. The object distance determination unit is used to determine the target object distance value corresponding to the target focus value based on the focus object distance mapping relationship and the target focus value. The focus object distance mapping relationship represents the object distance value corresponding to the focus value of the gimbal module when the image captured by the gimbal module meets the preset clarity level. The water height determination unit is used to determine the target water height of the camera device based on the target distance value and the target vertical roll angle; The correction unit is used to correct the thundercloud conversion relationship based on the target's height above the water.

9. An electronic device, characterized in that, The device includes a processor and a memory, the memory being used to store computer instructions, and the processor being used to retrieve and execute the computer instructions from the memory to implement the calibration method of the camera device according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes: computer software instructions; when the computer software instructions are executed in the calibration device of the camera device, the calibration device of the camera device implements the calibration method of the camera device as described in any one of claims 1-7.

Images