Camera parameter calibration method and electronic equipment

By performing object detection and iterative optimization of the scene images acquired by the camera, the camera parameters are automatically calibrated, which solves the problems of low efficiency and large error in the existing technology, and achieves efficient and accurate camera parameter calibration.

CN118608628BActive Publication Date: 2024-11-08ZHEJIANG DAHUA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411073058.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-08
Estimated Expiration
2044-08-06

AI Technical Summary

Technical Problem

The existing camera parameter calibration methods mainly rely on manual work, are inefficient and easy to introduce errors, making it difficult to accurately obtain the internal and external parameters of the camera.

Method used

By obtaining the scene images collected by the camera, the target object's position parameters and prior dimensions are obtained, the actual size representation is constructed, and the camera parameters are iteratively optimized, and the camera's internal and external parameters are automatically calibrated with the dimension error function as the target.

Benefits of technology

Improve the efficiency and accuracy of camera parameter calibration, reduce manual errors, and realize automated calibration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118608628B_ABST
    Figure CN118608628B_ABST
Patent Text Reader

Abstract

The present application discloses a camera parameter calibration method and electronic device, the camera parameter calibration method comprising: performing target detection on a scene image, obtaining the position parameters of the target object in the scene image, and obtaining the prior size corresponding to the target object; constructing the actual size representation of the target object in the scene image based on the mapping relationship between the actual size and position parameters of the target object and the camera parameters to be calibrated; performing difference calculation between the actual size representation and the prior size to obtain a size error function; taking the value of the camera parameter to be calibrated approaching zero as the goal, iteratively optimizing the value of the camera parameter to be calibrated, and using the value of the camera parameter to be calibrated that meets the preset conditions as the camera parameter calibration result corresponding to the camera. The present application combines the prior size information and the actual size representation of the target to construct a size error function, obtains the internal and external parameters of the camera by iteratively optimizing the size error function, and does not require manual dotting, thereby improving the calibration efficiency and accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a camera parameter calibration method and electronic equipment. Background Art

[0002] The camera is one of the important perception sensors, but only the two-dimensional pixel coordinates of the scene elements can be obtained through the camera image. In practical applications, it is often necessary to know the three-dimensional world coordinates of the scene elements. At this time, it is necessary to analyze the mapping relationship between the camera's two-dimensional pixel coordinates and the three-dimensional world coordinates.

[0003] When using a camera, you need to unify the camera coordinate system with the world coordinate system, that is, calibrate the camera coordinate system to the world coordinate system. After completing the unification of the camera coordinate system to the world coordinate system, you need to calculate the world coordinates corresponding to the pixel coordinates of each scene element through the camera's extrinsic parameters and camera intrinsic parameters.

[0004] Therefore, the above-mentioned camera extrinsic parameters and camera intrinsic parameters are camera parameters that need to be calibrated in advance. The current implementation method is generally to obtain them by manual marking, which is inefficient and has low accuracy, and is prone to introducing manual errors. Summary of the invention

[0005] In order to solve the above technical problems, the present application at least provides a camera parameter calibration method and an electronic device.

[0006] In a first aspect, the present application provides a camera parameter calibration method, which includes: acquiring a scene image captured by a camera, performing target detection on the scene image, obtaining position parameters of a target object in the scene image, and obtaining a priori size corresponding to the target object; constructing an actual size representation of the target object in the scene image based on a mapping relationship between the actual size and position parameters of the target object and camera parameters to be calibrated; performing difference calculation on the actual size representation and the priori size to obtain a size error function; iteratively optimizing the value of the camera parameter to be calibrated with the goal of approaching zero for the value of the size error function, and using the value of the camera parameter to be calibrated that meets preset conditions as the camera parameter calibration result corresponding to the camera.

[0007] In one embodiment, obtaining a priori size corresponding to the target object includes: determining the object type to which the target object belongs; querying an object size that matches the object type, and using the matched object size as the priori size corresponding to the target object.

[0008] In one embodiment, the position parameters include the bottom point coordinates and the top point coordinates of the target detection frame corresponding to the target object, and the camera parameters to be calibrated include the camera principal point, the camera focal length, the camera pitch angle and the camera installation height.

[0009] In one embodiment, based on the mapping relationship between the actual size of the target object and the position parameters and the camera parameters to be calibrated, the actual size representation of the target object in the scene image is constructed, including: constructing a coordinate system based on the optical center point of the camera, and determining the optical axis direction corresponding to the optical center point; combining the bottom point coordinates, the vertex coordinates, the optical center point, the camera principal point, the optical axis direction, the camera focal length, the camera pitch angle and the camera installation height to construct the actual size representation of the target object in the scene image.

[0010] In one embodiment, with the goal of making the value of the size error function approach zero, the value of the camera parameter to be calibrated is iteratively optimized, including: determining the initial value corresponding to the camera parameter to be calibrated; setting the numerical constraint range corresponding to the camera parameter to be calibrated based on the initial value; detecting the relationship between the value of the camera parameter to be calibrated corresponding to the current iteration and the numerical constraint range, and obtaining a target optimization strategy that matches the detection result; and optimizing the value of the camera parameter to be calibrated using the target optimization strategy.

[0011] In one embodiment, the relationship between the value of the camera parameter to be calibrated corresponding to the current iteration and the numerical constraint range is detected, and a target optimization strategy matching the detection result is obtained, including: if the detection result shows that the value of the camera parameter to be calibrated corresponding to the current iteration is within the numerical constraint range, then the unconstrained optimization strategy is used as the target optimization strategy; if the detection result shows that the value of the camera parameter to be calibrated corresponding to the current iteration is not within the numerical constraint range, then the constrained optimization strategy is used as the target optimization strategy; wherein the constrained optimization strategy optimizes the value of the camera parameter to be calibrated by increasing the value of the size error function as a constraint condition.

[0012] In one embodiment, target detection is performed on a scene image to obtain position parameters of a target object in the scene image, including: performing target detection on the scene image, taking the detected target as a candidate target, and obtaining a candidate target set; filtering difference targets in the candidate target set, and taking the candidate targets in the filtered candidate target set as selected target objects; wherein the difference target refers to a candidate target whose actual size and a priori size have a difference greater than a preset threshold; and obtaining position parameters corresponding to the selected target object.

[0013] In one embodiment, before filtering the difference targets in the candidate target set, the method also includes: calculating the camera parameters of the camera using the position parameters corresponding to each candidate target to obtain the initial camera parameters corresponding to the camera; predicting the actual size corresponding to the candidate target based on the initial camera parameters and the position parameters corresponding to the candidate target; if the difference between the predicted actual size and the prior size corresponding to the candidate target is greater than a preset threshold, the candidate target is used as a difference target.

[0014] In one embodiment, the method further includes: calculating the distance between the target object and the camera, the size of the target object, and the moving speed of the target object in the scene image based on the camera parameter calibration result corresponding to the camera.

[0015] According to a second aspect of the present application, there is provided a camera parameter calibration device, which includes: a parameter acquisition module, which is used to acquire a scene image captured by a camera, perform target detection on the scene image, obtain position parameters of the target object in the scene image, and obtain a priori size corresponding to the target object; a size expression construction module, which is used to construct an actual size representation of the target object in the scene image based on a mapping relationship between the actual size and position parameters of the target object and the camera parameters to be calibrated; an error function construction module, which is used to perform difference calculation between the actual size representation and the priori size to obtain a size error function; and an iterative optimization module, which is used to iteratively optimize the values ​​of the camera parameters to be calibrated with the goal of making the value of the size error function approach zero, and use the values ​​of the camera parameters to be calibrated that meet preset conditions as the camera parameter calibration results corresponding to the camera.

[0016] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned camera parameter calibration method.

[0017] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the above-mentioned camera parameter calibration method when the program instructions are executed by a processor.

[0018] The above scheme obtains the scene image collected by the camera, performs target detection on the scene image, obtains the position parameters of the target object in the scene image, and obtains the prior size corresponding to the target object; based on the mapping relationship between the actual size of the target object and the position parameters and the camera parameters to be calibrated, constructs the actual size representation of the target object in the scene image; calculates the difference between the actual size representation and the prior size to obtain a size error function; with the value of the size error function approaching zero as the goal, iteratively optimizes the value of the camera parameters to be calibrated, and uses the value of the camera parameters to be calibrated that meets the preset conditions as the camera parameter calibration result corresponding to the camera; the size error function can be constructed by combining the prior size information and the actual size representation of the target, so as to iteratively optimize the size error function to obtain the internal and external parameters of the camera without manual dotting, thereby improving the calibration efficiency and accuracy.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0021] Figure 1 It is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;

[0022] Figure 2 is a flowchart of a camera parameter calibration method shown in an exemplary embodiment of the present application;

[0023] Figure 3 is a schematic diagram of camera parameter calibration shown in an exemplary embodiment of the present application;

[0024] Figure 4 is a schematic diagram of camera imaging shown in an exemplary embodiment of the present application;

[0025] Figure 5 is a schematic diagram of calculating moving speed according to an exemplary embodiment of the present application;

[0026] Figure 6 is a block diagram of a camera parameter calibration device shown in an exemplary embodiment of the present application;

[0027] Figure 7 is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present application;

[0028] Figure 8 It is a schematic diagram of the structure of a computer-readable storage medium shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0029] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0030] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0031] The term "and / or" in this article is only an association information describing the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0032] The camera parameter calibration method provided in the embodiment of the present application is described below.

[0033] Please refer to Figure 1 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment may include a camera 110 and a server 120, and the camera 110 and the server 120 are connected to each other for communication.

[0034] Server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0035] In one example, server 120 may obtain a scene image captured by camera 110, and obtain calibrated camera parameters corresponding to camera 110 by analyzing the scene image. Of course, server 120 may store the calibrated camera parameters locally, transmit them back to camera 110, or transmit them to other terminals.

[0036] In one example, the camera 110 may directly analyze the captured scene image to obtain calibrated camera parameters.

[0037] In the camera parameter calibration method provided in the embodiment of the present application, the execution subject of each step can be the camera 110, or the server 120, or the camera 110 and the server 120 can cooperate with each other to execute the method, that is, part of the steps of the method are executed by the camera 110 and the other steps are executed by the server 120.

[0038] See also Figure 2 , Figure 2 is a flowchart of a camera parameter calibration method shown in an exemplary embodiment of the present application. The camera parameter calibration method can be applied to Figure 1 It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments, and this embodiment does not limit the implementation environment to which the method is applicable.

[0039] like Figure 2 As shown, the camera parameter calibration method at least includes steps S210 to S240, which are described in detail as follows:

[0040] Step S210: Acquire a scene image captured by a camera, perform target detection on the scene image, obtain position parameters of the target object in the scene image, and obtain a priori size corresponding to the target object.

[0041] The camera performs an image acquisition operation to obtain a scene image.

[0042] Perform target detection on the scene image to determine the target objects contained in the scene image, including but not limited to target objects with small size changes such as cars, trucks, motorcycles, animals, etc., and obtain the position parameters of the target objects obtained by target detection.

[0043] For example, the position parameter may be the coordinates of the detection frame of the target object; the position parameter may also be the coordinates of the two end points corresponding to the height mark obtained after the target object is height-marked according to the detection frame of the target object; the position parameter may also be the coordinates of the two end points corresponding to the width mark obtained after the target object is width-marked according to the detection frame of the target object, and this application does not limit this.

[0044] Furthermore, a priori size corresponding to the target object is obtained, wherein the priori size may be at least one size information of the height, width, length, etc. of the target object.

[0045] In some embodiments, for ease of calculation, the acquired position parameters and prior dimensions of the target object are matched with each other. For example, the position parameters may be the coordinates of the two end points corresponding to the height mark, and the prior dimension may be the prior height of the target object.

[0046] Step S220: constructing a representation of the actual size of the target object in the scene image based on a mapping relationship between the actual size of the target object and the position parameters and the camera parameters to be calibrated.

[0047] The camera parameters to be calibrated include at least one of camera intrinsic parameters and camera extrinsic parameters.

[0048] It should be noted that, under the premise that both the camera intrinsic parameters and the camera extrinsic parameters are known, the actual size of the target object can be calculated according to the position parameters of the target object, the camera intrinsic parameters and the camera extrinsic parameters. Conversely, the actual size representation of the target object in the scene image can be constructed according to the mapping relationship between the actual size of the target object and the position parameters and the camera parameters to be calibrated.

[0049] Of course, the mapping relationship between the actual size of the target object and the position parameters and the camera parameters to be calibrated can be represented by a variety of mathematical formulas. Different mathematical formulas are selected, and the actual size representation obtained is also different. This application does not limit this.

[0050] Step S230: Calculate the difference between the actual size representation and the priori size to obtain a size error function.

[0051] The difference between the actual size representation and the prior size is calculated to obtain a size error function, the value of which indicates the difference between the actual size of the target object obtained by the solution and the prior size for the current value of the camera parameter to be calibrated.

[0052] Step S240: with the goal of making the value of the size error function approach zero, iteratively optimize the value of the camera parameter to be calibrated, and use the value of the camera parameter to be calibrated that meets the preset conditions as the camera parameter calibration result corresponding to the camera.

[0053] It should be noted that, the closer the difference between the actual size of the target object calculated by the current value of the camera parameter to be calibrated and the prior size is to zero, the more accurate the value of the current camera parameter to be calibrated is; conversely, the less the difference between the actual size of the target object calculated by the current value of the camera parameter to be calibrated and the prior size is to zero, the less accurate the value of the current camera parameter to be calibrated is.

[0054] Therefore, with the goal of making the value of the size error function approach zero, the values ​​of the camera parameters to be calibrated are iteratively optimized.

[0055] In some embodiments, during each iteration, the adjustment range of the camera parameters to be calibrated corresponding to the current iteration can be determined according to the value of the size error function corresponding to the previous iteration. For example, the larger the value of the size error function corresponding to the previous iteration, the larger the adjustment range of the camera parameters to be calibrated corresponding to the current iteration. Conversely, the smaller the value of the size error function corresponding to the previous iteration, the smaller the adjustment range of the camera parameters to be calibrated corresponding to the current iteration.

[0056] In some implementations, a Gauss-Newton iteration method, a Newton's method, or the like may be used to iteratively optimize the values ​​of the camera parameters to be calibrated.

[0057] The values ​​of the camera parameters to be calibrated are iteratively optimized, and after each iteration, it is determined whether the values ​​of the camera parameters to be calibrated corresponding to the current iteration meet the preset conditions. If the preset conditions are met, the iteration is terminated and the values ​​of the camera parameters to be calibrated corresponding to the current iteration are used as the camera parameter calibration results corresponding to the camera.

[0058] The camera parameter calibration result includes at least one of the parameters such as camera focal length, camera principal point, camera pitch angle and camera installation height.

[0059] The preset condition may be that the absolute value of the size error function is less than a preset error threshold; the preset condition may also be that the number of iterations reaches a preset number and the absolute value of the size error function is less than a preset error threshold.

[0060] In some embodiments, see Figure 3 , Figure 3 A schematic diagram of camera parameter calibration is shown in an exemplary embodiment of the present application. Figure 3 As shown, the scene image contains q target objects, and the values ​​of the camera parameters to be calibrated are iteratively optimized by combining the position parameters and prior size of each target object.

[0061] For example, the size error function corresponding to each target object is constructed respectively, and then, for the value of the camera parameter to be calibrated corresponding to the current iteration, the value of the size error function corresponding to each target object is calculated respectively according to the size error function corresponding to each target object. According to the value of the size error function corresponding to each target object, the value of the camera parameter to be calibrated is iteratively optimized to obtain a camera parameter calibration result that meets the preset conditions.

[0062] For example: the mean value of the size error function corresponding to each target object may be calculated, and the value of the camera parameter to be calibrated may be optimized according to the mean calculation result to obtain the value of the camera parameter to be calibrated corresponding to the next iteration; the maximum value of the size error function corresponding to each target object may be selected, and the value of the camera parameter to be calibrated may be optimized according to the maximum value to obtain the value of the camera parameter to be calibrated corresponding to the next iteration; the weighted parameter corresponding to the target object may be determined according to the object type to which the target object belongs, and the value of the size error function corresponding to each target object may be weightedly calculated according to the weighted parameter of each target object, and the value of the camera parameter to be calibrated may be optimized according to the weighted calculation result to obtain the value of the camera parameter to be calibrated corresponding to the next iteration.

[0063] Optionally, the iterative optimization strategy can be determined according to the current number of iterative optimizations. For example, if the number of iterative optimizations is less than a preset threshold, the value of the camera parameter to be calibrated is optimized according to the above mean value calculation result or weighted calculation result to obtain the value of the camera parameter to be calibrated corresponding to the next iteration; if the number of iterative optimizations is not less than the preset threshold, the value of the camera parameter to be calibrated is optimized according to the above maximum value to obtain the value of the camera parameter to be calibrated corresponding to the next iteration.

[0064] Optionally, the iterative optimization strategy may be determined according to the object type to which the target object in the scene image belongs. For example, if there is a target object with a large size change in the scene image, the value of the camera parameter to be calibrated is optimized according to the above mean calculation result or weighted calculation result to obtain the value of the camera parameter to be calibrated corresponding to the next iteration; if there is no target object with a large size change in the scene image, the value of the camera parameter to be calibrated is optimized according to the above maximum value to obtain the value of the camera parameter to be calibrated corresponding to the next iteration.

[0065] Of course, the iterative optimization strategy may also be determined in combination with the current number of iterative optimization times, the object type to which the target object in the scene image belongs, and the like.

[0066] By comprehensively considering multiple target objects to perform iterative optimization methods, the flexibility and efficiency of iterative optimization can be improved, and the robustness of camera parameter calibration can be improved, avoiding the situation where camera distortion exists in some image contents and leads to inaccurate camera parameter calibration.

[0067] In addition, when the scene image contains multiple target objects, the step of judging whether the value of the camera parameter to be calibrated corresponding to the current iteration meets the preset conditions may be: the absolute value of the size error function corresponding to each target object is less than the preset error threshold; the number of target objects whose absolute value of the size error function is less than the preset error threshold is greater than the preset number threshold; the number of iterations reaches the preset number and the absolute value of the size error function corresponding to each target object is less than the preset error threshold; the number of iterations reaches the preset number and the absolute value of the size error function is less than the preset error threshold is greater than the preset number threshold. Wherein, the preset number threshold may be determined according to the total number of target objects in the scene image.

[0068] Next, some embodiments of the present application are described in detail.

[0069] In some implementations, performing target detection on the scene image in step S210 to obtain position parameters of the target object in the scene image includes:

[0070] Step S2111: perform target detection on the scene image, use the detected targets as candidate targets, and obtain a candidate target set.

[0071] Step S2112: filtering the difference targets in the candidate target set, and taking the candidate targets in the filtered candidate target set as the selected target objects; wherein the difference targets refer to the candidate targets whose actual size and prior size have a difference greater than a preset threshold.

[0072] Since the difference between the actual size and the prior size of some detected candidate targets is too large, if the position parameters of these candidate targets are used to calculate the camera parameters, the accuracy of the calculated camera parameters will be affected.

[0073] Therefore, the difference targets in the candidate target set are identified, and the difference targets are deleted and filtered, and the candidate targets in the filtered candidate target set are used as the selected target objects.

[0074] Exemplarily, the step of identifying and filtering difference targets in a set of candidate targets may include: calculating the camera parameters of a camera using the position parameters corresponding to each candidate target to obtain initial camera parameters corresponding to the camera; predicting the actual size corresponding to the candidate target based on the initial camera parameters and the position parameters corresponding to the candidate target; if the difference between the predicted actual size and the prior size corresponding to the candidate target is greater than a preset threshold, treating the candidate target as a difference target.

[0075] The camera parameter calibration method exemplified in the above steps S210 to S240 can be used to calculate the initial camera parameters corresponding to the camera, or other camera parameter calculation methods can be used to calculate the initial camera parameters corresponding to the camera, such as using a PnP (Perspective-n-Point) algorithm to calculate the initial camera parameters corresponding to the camera, which is not limited in this application.

[0076] The initial camera parameters include, but are not limited to, camera focal length, camera principal point, camera pitch angle, and camera installation height.

[0077] Then, based on the initial camera parameters and the position parameters corresponding to the candidate targets, the actual size corresponding to the candidate targets is predicted. For example, based on the actual size representation constructed in step S220, the actual size corresponding to the candidate targets is predicted. Of course, other size calculation algorithms can also be used to predict the actual size corresponding to the candidate targets, and this application does not limit this.

[0078] The difference between the actual size predicted and the prior size corresponding to the candidate target is calculated. The larger the difference, the less favorable the candidate target is for solving the camera parameters. The smaller the difference, the more favorable the candidate target is for solving the camera parameters.

[0079] Specifically, if the difference between the predicted actual size and the prior size corresponding to the candidate target is greater than a preset threshold, the candidate target is taken as a difference target. The preset threshold may be pre-set or determined according to the object type to which the target object belongs, and this application does not limit this.

[0080] According to the difference target recognition method of the above embodiment, difference target recognition and filtering are performed on each candidate target in the scene image, and the candidate targets in the filtered candidate target set are taken as selected target objects.

[0081] Step S2113: Obtain the position parameters corresponding to the selected target object.

[0082] In some implementations, obtaining a priori size corresponding to the target object in step S210 includes:

[0083] Step S2121: Determine the object type to which the target object belongs.

[0084] For example, object types may include vehicle types, animal types, etc.

[0085] Step S2122: querying the object size that matches the object type, and using the matching object size as the prior size corresponding to the target object.

[0086] For example, if the target object is a vehicle, the prior size includes the prior height, and the prior height of the vehicle obtained by querying may be 1.5 meters.

[0087] According to the mapping relationship between the actual size of the target object and the position parameters and the camera parameters to be calibrated obtained above, the actual size representation of the target object in the scene image is constructed.

[0088] In some embodiments, the prior size includes a prior height, the position parameters include the bottom point coordinates and the vertex coordinates of the target detection box corresponding to the target object, and in addition, the camera parameters to be calibrated include the camera principal point, the camera focal length, the camera pitch angle and the camera installation height.

[0089] On the basis of the above embodiment, in step S220, based on the mapping relationship between the actual size of the target object and the position parameter and the camera parameter to be calibrated, constructing the actual size representation of the target object in the scene image includes:

[0090] Step S221: construct a coordinate system based on the optical center point of the camera, and determine the direction of the optical axis corresponding to the optical center point.

[0091] Step S222: The actual size representation of the target object in the scene image is constructed by combining the bottom point coordinates, the vertex coordinates, the optical center point, the camera principal point, the optical axis direction, the camera focal length, the camera pitch angle and the camera installation height.

[0092] For an example, see Figure 4 , Figure 4 A schematic diagram of camera imaging is shown as an exemplary embodiment of the present application. Figure 4As shown in the figure, let the optical center of the camera be point P; point A is the intersection of point P along the optical axis and the ground; point B is the intersection of the target object and the ground, that is, the bottom point of the target object; point C is the vertex of the target object; point D is the intersection of the extension line of the line connecting the vertex of the target and the optical center and the ground; OP represents the camera installation height, and BC represents the actual height of the target. According to the pinhole imaging principle, the positions of point C and point D in the picture are the same, so point C can be equivalent to point D. Take point O as the origin of the coordinate system and construct the camera coordinate system.

[0093] If the coordinates of the bottom point B of the target object are known to be (x1, y1), and the coordinates of the vertex C are (x1, y2); and assuming that the focal length of the camera is f, the camera installation height is h, the coordinates of the camera principal point are (cx, cy), and the camera pitch angle is pitch, then the actual height BC of the target object in the scene image can be constructed by combining the bottom point coordinates, vertex coordinates, optical center point, camera principal point, optical axis direction, camera focal length, camera pitch angle and camera installation height. The actual height representation can be seen in the following formula 1:

[0094] BC = BD / tan (OPA + APD) (Formula 1)

[0095] Among them, angle OPA = pitch; angle APD = -atan((y2-cy) / f); BD = OD-OB, , , angle APB=-atan((y1-cy) / f).

[0096] Furthermore, the size error function is constructed according to Formula 1, and the obtained size error function can be expressed as the following Formula 2:

[0097] error = BC - object_height (Formula 2)

[0098] Among them, object_height is the prior size, that is, the prior height of the target object.

[0099] It should be noted that the above Formulas 1 and 2 are schematic illustrations taking height as an example. In actual application scenarios, you can also select the width or length of the target object, or select multiple heights, widths, and lengths of the target object to construct a size error function. This application does not limit this.

[0100] In some implementations, in step S240, the value of the camera parameter to be calibrated is iteratively optimized with the goal of making the value of the size error function approach zero, including:

[0101] Step S241: Determine the initial values ​​corresponding to the camera parameters to be calibrated.

[0102] Exemplarily, the initial value corresponding to each camera parameter to be calibrated can be pre-estimated, for example, the initial value corresponding to each camera parameter to be calibrated can be pre-estimated through a PnP algorithm; or scene recognition can be performed on the scene image to obtain the scene category of the camera deployment scene, and the preset initial value matching the scene category is queried according to the scene category. This application does not limit the method for obtaining the initial value.

[0103] Step S242: setting the numerical constraint range corresponding to the camera parameter to be calibrated based on the initial value.

[0104] For example, if the initial camera focal length is f0, the numerical constraint range of the camera focal length can be set to , a1 and a2 are range coefficients, and the range coefficients can be preset values, such as a1 is 0.7 and a2 is 1.5.

[0105] Optionally, different types of camera parameters to be calibrated may correspond to different range coefficients, and the numerical constraint range corresponding to the camera parameter to be calibrated may be calculated according to the initial value of the camera parameter to be calibrated and the range coefficient of the camera parameter to be calibrated.

[0106] Step S243: Detect the relationship between the value of the camera parameter to be calibrated corresponding to the current iteration and the numerical constraint range, and obtain a target optimization strategy that matches the detection result.

[0107] Exemplarily, if the detection result shows that the value of the camera parameter to be calibrated corresponding to the current iteration is within the numerical constraint range, the unconstrained optimization strategy is used as the target optimization strategy; if the detection result shows that the value of the camera parameter to be calibrated corresponding to the current iteration is not within the numerical constraint range, the constrained optimization strategy is used as the target optimization strategy; wherein the constrained optimization strategy optimizes the value of the camera parameter to be calibrated by increasing the value of the size error function as a constraint condition.

[0108] That is, if the detection result shows that the value of the camera parameter to be calibrated corresponding to the current iteration is not within the numerical constraint range, it is necessary to add constraints, which are used to increase the value of the size error function. For example, increasing the penalty factor so that the value of the camera parameter to be calibrated is not within the numerical constraint range will cause the value of the size error function to be forced to fall back into the numerical constraint range in subsequent iterations, which can prevent the value of the camera parameter to be calibrated from falling into an abnormal solution and improve the accuracy of the camera parameter calibration result obtained by the final iterative optimization.

[0109] Step S244: Optimize the values ​​of the camera parameters to be calibrated using a target optimization strategy.

[0110] Through the above-mentioned embodiment, the values ​​of the camera parameters to be calibrated are iteratively optimized, and the values ​​of the camera parameters to be calibrated that meet the preset conditions are used as the camera parameter calibration results corresponding to the camera.

[0111] In some embodiments, the method further includes: calculating the distance between the target object and the camera, the size of the target object, and the moving speed of the target object in the scene image based on the camera parameter calibration result corresponding to the camera.

[0112] Among them, the distance between the target object and the camera is calculated (such as Figure 4 OB in), the size of the target object (such as Figure 4 The method of BC) in FIG. 1 can refer to the above embodiment and will not be described in detail here.

[0113] For an example, see Figure 5 , Figure 5 FIG. 1 is a schematic diagram showing a method for calculating a moving speed according to an exemplary embodiment of the present application. Figure 5 As shown, assuming that the target moves from point M to point N, determine the length of MN and the time it takes for the target to move from point M to point N. Calculate the length of MN divided by the time to get the moving speed of the target.

[0114] Specifically, the coordinates of point M and point N are decomposed onto the x-axis and y-axis. Note the image coordinates of point M (xm, ym), decompose point M into x-axis coordinates (xm, 0), and decompose into y-axis coordinates (0, ym); note the image coordinates of point N (xn, yn), decompose point N into x-axis coordinates (xn, 0), and decompose into y-axis coordinates (0, yn). According to the relevant formulas for calculating the distance between the target object and the camera introduced in the above embodiment, the x-axis and y-axis components of points M and N can be calculated, so that the MN length can be obtained according to the Pythagorean theorem. Then, by calculating the acquisition time of the image corresponding to point M and the acquisition time of the image corresponding to point N, the time required for the target to move from point M to point N is obtained. Thus, the moving speed of the target is obtained by combining the above-obtained MN length and dividing it by time.

[0115] The camera parameter calibration method provided by the present application obtains a scene image captured by a camera, performs target detection on the scene image, obtains position parameters of the target object in the scene image, and obtains a priori size corresponding to the target object; constructs an actual size representation of the target object in the scene image based on a mapping relationship between the actual size and position parameters of the target object and the camera parameters to be calibrated; performs difference calculation between the actual size representation and the priori size to obtain a size error function; iteratively optimizes the values ​​of the camera parameters to be calibrated with the goal of the value of the size error function approaching zero, and uses the values ​​of the camera parameters to be calibrated that meet preset conditions as the camera parameter calibration results corresponding to the camera; the size error function can be constructed in combination with the priori size information and the actual size representation of the target, thereby iteratively optimizing the size error function to obtain the internal and external parameters of the camera without manual dotting, thereby improving the calibration efficiency and accuracy.

[0116] Figure 6 FIG. 1 is a block diagram of a camera parameter calibration device shown in an exemplary embodiment of the present application. Figure 6 As shown, the exemplary camera parameter calibration device 600 includes: a parameter acquisition module 610, a size expression construction module 620, an error function construction module 630 and an iterative optimization module 640. Specifically:

[0117] The parameter acquisition module 610 is used to acquire the scene image captured by the camera, perform target detection on the scene image, obtain the position parameters of the target object in the scene image, and obtain the prior size corresponding to the target object;

[0118] A size expression construction module 620 is used to construct an actual size representation of the target object in the scene image based on a mapping relationship between the actual size of the target object and the position parameters and the camera parameters to be calibrated;

[0119] The error function construction module 630 is used to calculate the difference between the actual size representation and the prior size to obtain a size error function;

[0120] The iterative optimization module 640 is used to iteratively optimize the values ​​of the camera parameters to be calibrated with the goal of making the value of the size error function approach zero, and use the values ​​of the camera parameters to be calibrated that meet preset conditions as the camera parameter calibration results corresponding to the camera.

[0121] It should be noted that the camera parameter calibration device provided in the above embodiment and the camera parameter calibration method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the camera parameter calibration device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0122] See also Figure 7 , Figure 7 7 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 700 includes a memory 701 and a processor 702, and the processor 702 is used to execute the program instructions stored in the memory 701 to implement the steps in any of the above-mentioned camera parameter calibration method embodiments. In a specific implementation scenario, the electronic device 700 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 700 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0123] Specifically, the processor 702 is used to control itself and the memory 701 to implement the steps in any of the above-mentioned camera parameter calibration method embodiments. The processor 702 can also be called a central processing unit (CPU). The processor 702 may be an integrated circuit chip with signal processing capabilities. The processor 702 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 702 can be implemented by an integrated circuit chip.

[0124] See also Figure 8 , Figure 8 The computer-readable storage medium 800 stores program instructions 810 that can be executed by a processor, and the program instructions 810 are used to implement the steps in any of the above-mentioned camera parameter calibration method embodiments.

[0125] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0126] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0127] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0128] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

Claims

1. A camera parameter calibration method, characterized in that: include: Acquire a scene image captured by the camera, perform target detection on the scene image, obtain position parameters of a target object in the scene image, and determine the object type to which the target object belongs; Querying the size of an object matching the object type, and using the matching object size as the prior size corresponding to the target object; wherein the position parameter includes the coordinates of the target detection frame of the target object, or the coordinates of the two end points corresponding to the height or width mark obtained after the target object is marked with the height or width according to the target detection frame; Based on the mapping relationship between the actual size of the target object and the position parameter and the camera parameters to be calibrated, construct an actual size representation of the target object in the coordinate system corresponding to the scene image; wherein the camera parameters to be calibrated include at least one of the camera focal length, the camera principal point, the camera pitch angle and the camera installation height parameter, and the coordinate system corresponding to the scene image is constructed based on the optical center point of the camera; Calculating the difference between the actual size representation and the priori size to obtain a size error function; With the goal of making the value of the size error function approach zero, the value of the camera parameter to be calibrated is iteratively optimized, and the value of the camera parameter to be calibrated that meets the preset conditions is used as the camera parameter calibration result corresponding to the camera.

2. The method according to claim 1, characterized in that The position parameters include the bottom point coordinates and the vertex coordinates of the target detection frame corresponding to the target object, and the camera parameters to be calibrated include the camera principal point, camera focal length, camera pitch angle and camera installation height.

3. The method according to claim 2, characterized in that The constructing, based on the mapping relationship between the actual size of the target object and the position parameter and the camera parameter to be calibrated, an actual size representation of the target object in the coordinate system corresponding to the scene image comprises: Constructing a coordinate system based on the optical center point of the camera, and determining the direction of the optical axis corresponding to the optical center point; The actual size representation of the target object in the scene image is constructed by combining the bottom point coordinates, the vertex coordinates, the optical center point, the camera principal point, the optical axis direction, the camera focal length, the camera pitch angle and the camera installation height.

4. The method according to claim 1, characterized in that: The iterative optimization of the value of the camera parameter to be calibrated with the goal of the value of the size error function approaching zero includes: Determine the initial value corresponding to the camera parameter to be calibrated; Setting a numerical constraint range corresponding to the camera parameter to be calibrated based on the initial value; Detecting the relationship between the value of the camera parameter to be calibrated corresponding to the current iteration and the numerical constraint range, and acquiring a target optimization strategy that matches the detection result; The target optimization strategy is adopted to optimize the values ​​of the camera parameters to be calibrated.

5. The method according to claim 4, characterized in that The detecting the relationship between the value of the camera parameter to be calibrated corresponding to the current iteration and the numerical constraint range, and obtaining a target optimization strategy matching the detection result, includes: If the detection result indicates that the value of the to-be-calibrated camera parameter corresponding to the current iteration is within the numerical constraint range, the unconstrained optimization strategy is used as the target optimization strategy; If the detection result indicates that the value of the camera parameter to be calibrated corresponding to the current iteration is not within the numerical constraint range, the constrained optimization strategy is used as the target optimization strategy; wherein the constrained optimization strategy optimizes the value of the camera parameter to be calibrated by increasing the value of the size error function as a constraint condition.

6. The method according to claim 1, characterized in that The performing target detection on the scene image to obtain position parameters of the target object in the scene image includes: Performing target detection on the scene image, taking the detected targets as candidate targets, and obtaining a candidate target set; Filtering the difference targets in the candidate target set, and taking the candidate targets in the filtered candidate target set as the selected target objects; wherein the difference target refers to a candidate target whose difference between the actual size and the prior size is greater than a preset threshold; Get the position parameters corresponding to the selected target object.

7. The method according to claim 6, characterized in that Before filtering the difference targets in the candidate target set, the method further includes: Calculating the camera parameters of the camera using the position parameters corresponding to each of the candidate targets to obtain the initial camera parameters corresponding to the camera; Based on the initial camera parameters and the position parameters corresponding to the candidate objects, predicting the actual size corresponding to the candidate objects; If the difference between the predicted actual size and the prior size corresponding to the candidate target is greater than a preset threshold, the candidate target is taken as a difference target.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Based on the camera parameter calibration result corresponding to the camera, the distance between the target object in the scene image and the camera, the size of the target object, and the moving speed of the target object are calculated.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the steps in the method according to any one of claims 1 to 8.