Calibration method, device and electronic equipment

Abstract

The present disclosure relates to a calibration method, device and electronic device for calibrating an image acquisition device in a virtual shooting scene, wherein the method comprises: calibrating an internal parameter corresponding to a first focal length; acquiring a captured image of each focal length among a plurality of focal lengths, wherein the captured image is an image obtained by the image acquisition device capturing a screen image at the same posture and each focal length among the plurality of focal lengths; and calibrating the internal parameter of each focal length among the plurality of focal lengths except the first focal length based on the calibrated internal parameter corresponding to the first focal length and the captured image of each focal length among the plurality of focal lengths. Through the present disclosure, a mode of main focal length (i.e., first focal length) + multiple sub-focal lengths (i.e., each focal length among the plurality of focal lengths except the first focal length) is adopted to realize the calibration of the zoom lens of the image acquisition device, thereby saving the calibration time and effectively improving the calibration efficiency of the zoom lens.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a calibration method, device, and electronic equipment. Background Art

[0002] Virtual shooting means: the scene image rendered by the virtual engine is projected onto the screen for display, and then the actors use the screen as a background to perform. The image acquisition device simultaneously shoots the actors and the screen. Afterwards, the captured image is synthesized with the original scene image, thereby placing the real actors in the virtual scene, achieving the effect of shooting exterior scenes or science fiction backgrounds in the studio.

[0003] Before performing virtual shooting, it is usually necessary to calibrate the lens of the image acquisition device. Existing calibration technologies are aimed at fixed-focus lenses, but lack calibration for zoom lenses. Summary of the Invention

[0004] In view of this, the present disclosure proposes a calibration method, device, electronic device, storage medium and computer program product.

[0005] According to one aspect of the present disclosure, a calibration method is provided for calibrating an image acquisition device in a virtual shooting scene, the method comprising:

[0006] calibrating an internal parameter corresponding to a first focal length, where the first focal length is one of multiple focal lengths of the image acquisition device;

[0007] Acquire a captured image at each of the multiple focal lengths, where the captured image is an image obtained by the image capture device capturing a screen image at the same posture and at each of the multiple focal lengths;

[0008] The internal parameter of each focal length among the multiple focal lengths except the first focal length is calibrated according to the calibrated internal parameter corresponding to the first focal length and the captured image of each focal length among the multiple focal lengths.

[0009] In a possible implementation, the captured image at each of the multiple focal lengths includes an image obtained by shooting the image on the screen once at each of the multiple focal lengths.

[0010] In a possible implementation, calibrating the intrinsic parameters of each focal length among the multiple focal lengths except the first focal length according to the calibrated intrinsic parameters corresponding to the first focal length and the captured images of each focal length among the multiple focal lengths includes:

[0011] Determining an initial value of the internal parameter corresponding to the second focal length based on an internal parameter corresponding to at least one focal length adjacent to the second focal length among the multiple focal lengths; wherein the second focal length is any focal length among the multiple focal lengths except the first focal length;

[0012] Determine a calibration value of the intrinsic parameter corresponding to the second focal length according to the initial value of the intrinsic parameter corresponding to the second focal length and the captured image of the second focal length,

[0013] When the second focal length is adjacent to the first focal length, at least one focal length adjacent to the second focal length among the multiple focal lengths includes the first focal length.

[0014] In a possible implementation, the captured image at each of the multiple focal lengths includes multiple feature points;

[0015] The determining, based on the initial value of the intrinsic parameter corresponding to the second focal length and the captured image of the second focal length, a calibration value of the intrinsic parameter corresponding to the second focal length includes:

[0016] Determining the two-dimensional coordinates of each feature point in the captured image of the second focal length, and the three-dimensional coordinates of each feature point on the screen;

[0017] The calibration value of the intrinsic parameter corresponding to the second focal length is determined according to the initial value of the intrinsic parameter corresponding to the second focal length, the two-dimensional coordinates of each feature point in the captured image of the second focal length, and the three-dimensional coordinates of each feature point on the screen.

[0018] In one possible implementation, determining the calibration value of the intrinsic parameter corresponding to the second focal length according to the initial value of the intrinsic parameter corresponding to the second focal length, the two-dimensional coordinates of each feature point in the captured image of the second focal length, and the three-dimensional coordinates of each feature point on the screen includes:

[0019] The calibration value of the internal parameter corresponding to the second focal length is determined according to the posture of the image acquisition device, the two-dimensional coordinates of each feature point in the captured image of the second focal length and the three-dimensional coordinates of each feature point on the screen, and the initial value of the internal parameter corresponding to the second focal length.

[0020] In one possible implementation, determining the calibration value of the intrinsic parameter corresponding to the second focal length based on the posture of the image acquisition device, the two-dimensional coordinates of each feature point in the captured image of the second focal length and the three-dimensional coordinates of each feature point on the screen, and the initial value of the intrinsic parameter corresponding to the second focal length includes:

[0021] According to the posture of the image acquisition device and the value of the internal parameter corresponding to the second focal length, the two-dimensional reference coordinates corresponding to the three-dimensional coordinates of each feature point in the captured image of the second focal length on the screen are determined; and based on the two-dimensional reference coordinates and the two-dimensional coordinates of each feature point in the captured image of the second focal length, the value of the internal parameter corresponding to the second focal length is iteratively optimized, and the value of the internal parameter corresponding to the second focal length when the preset conditions are met is used as the calibration value; wherein, the initial value of the internal parameter corresponding to the second focal length is used as the initial value of the internal parameter corresponding to the second focal length during the iterative optimization process.

[0022] In a possible implementation, the method further includes:

[0023] Determining a third focal length, wherein the third focal length is any focal length within a zoom range of the image acquisition device;

[0024] determining one or more focal lengths adjacent to the third focal length among the plurality of focal lengths;

[0025] An internal parameter corresponding to the third focal length is determined based on the internal parameters corresponding to the one or more focal lengths.

[0026] In a possible implementation, calibrating the intrinsic parameter corresponding to the first focal length includes:

[0027] The internal parameter corresponding to the first focal length is calibrated by using a fixed focus calibration method.

[0028] In a possible implementation, the first focal length is the minimum focal length among the multiple focal lengths.

[0029] According to another aspect of the present disclosure, a calibration device is provided for calibrating an image acquisition device in a virtual shooting scene, the device comprising:

[0030] a calibration module, configured to calibrate an internal parameter corresponding to a first focal length, where the first focal length is one of the plurality of focal lengths of the image acquisition device;

[0031] an acquisition module, configured to acquire an acquired image at each of the multiple focal lengths, wherein the acquired image is an image obtained by the image acquisition device capturing a screen image at the same posture and at each of the multiple focal lengths;

[0032] The calibration module is further configured to calibrate the internal parameters of each focal length among the multiple focal lengths except the first focal length based on the internal parameters corresponding to the calibrated first focal length and the captured images of each focal length among the multiple focal lengths.

[0033] In a possible implementation, the captured image at each of the multiple focal lengths includes an image obtained by shooting the image on the screen once at each of the multiple focal lengths.

[0034] In one possible implementation, the calibration module is further used to: determine an initial value of the internal parameter corresponding to the second focal length based on the internal parameter corresponding to at least one focal length adjacent to the second focal length among the multiple focal lengths; wherein, the second focal length is any focal length among the multiple focal lengths except the first focal length; determine the calibration value of the internal parameter corresponding to the second focal length based on the initial value of the internal parameter corresponding to the second focal length and the captured image of the second focal length, wherein, when the second focal length is adjacent to the first focal length, at least one focal length adjacent to the second focal length among the multiple focal lengths includes the first focal length.

[0035] In one possible implementation, the captured image of each of the multiple focal lengths includes multiple feature points; the calibration module is further used to: determine the two-dimensional coordinates of each feature point in the captured image of the second focal length, and the three-dimensional coordinates of each feature point on the screen; determine the calibration value of the intrinsic parameter corresponding to the second focal length based on the initial value of the intrinsic parameter corresponding to the second focal length, the two-dimensional coordinates of each feature point in the captured image of the second focal length, and the three-dimensional coordinates of each feature point on the screen.

[0036] In one possible implementation, the calibration module is further used to determine the calibration value of the internal parameter corresponding to the second focal length based on the posture of the image acquisition device, the two-dimensional coordinates of each feature point in the captured image of the second focal length and the three-dimensional coordinates of each feature point on the screen, and the initial value of the internal parameter corresponding to the second focal length.

[0037] In one possible implementation, the calibration module is further used to: determine the two-dimensional reference coordinates corresponding to the three-dimensional coordinates of each feature point in the captured image of the second focal length on the screen according to the posture of the image acquisition device and the value of the internal parameter corresponding to the second focal length; and iteratively optimize the value of the internal parameter corresponding to the second focal length based on the two-dimensional reference coordinates and the two-dimensional coordinates of each feature point in the captured image of the second focal length, and use the value of the internal parameter corresponding to the second focal length when the preset conditions are met as the calibration value; wherein the initial value of the internal parameter corresponding to the second focal length is used as the initial value of the internal parameter corresponding to the second focal length during the iterative optimization process.

[0038] In one possible implementation, the calibration module is further used to: determine a third focal length, wherein the third focal length is any focal length within the zoom range of the image acquisition device; determine one or more focal lengths adjacent to the third focal length among the multiple focal lengths; and determine the internal parameters corresponding to the third focal length based on the internal parameters corresponding to the one or more focal lengths.

[0039] In a possible implementation, the calibration module is further configured to calibrate an intrinsic parameter corresponding to the first focal length using a fixed-focus calibration method.

[0040] In a possible implementation, the first focal length is the minimum focal length among the multiple focal lengths.

[0041] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0042] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0043] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0044] Through the embodiments of the present disclosure, a mode of one main focal length (first focal length) + multiple sub-focal lengths (each focal length among the multiple focal lengths except the first focal length) is adopted; first, the internal parameters corresponding to the first focal length are calibrated, and then based on the internal parameters corresponding to the first focal length and the captured images obtained by shooting the screen at the same posture and each focal length, the internal parameters corresponding to each focal length among the multiple focal lengths except the first focal length are calibrated to realize the calibration of the zoom lens of the image acquisition device; as an example, only one image needs to be captured for each focal length to complete the calibration, thereby greatly reducing the amount of data collection and data processing, saving calibration time, and effectively improving the calibration efficiency of the zoom lens.

[0045] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0047] Figure 1 A schematic diagram of a virtual shooting scene according to an embodiment of the present disclosure is shown.

[0048] Figure 2 A flowchart of a calibration method according to an embodiment of the present disclosure is shown.

[0049] Figure 3 A schematic diagram of feature points in a captured image according to an embodiment of the present disclosure is shown.

[0050] Figure 4 A schematic diagram of feature points and positioning identifiers in a captured image according to an embodiment of the present disclosure is shown.

[0051] Figure 5 A flowchart of another calibration method according to an embodiment of the present disclosure is shown.

[0052] Figure 6 A flowchart of another calibration method according to an embodiment of the present disclosure is shown.

[0053] Figure 7 A structural diagram of a calibration device according to an embodiment of the present disclosure is shown.

[0054] Figure 8 A schematic structural diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0055] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0056] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present disclosure. Thus, phrases such as "exemplary," "in one embodiment," "in some other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0057] In the present disclosure, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0058] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0059] The following first provides an illustrative description of the applicable application scenarios of the embodiments of the present disclosure.

[0060] Figure 1 A schematic diagram of a virtual shooting scene according to an embodiment of the present disclosure is shown. Figure 1 As shown, the virtual shooting scene may include a display device 10 and an image acquisition device 20; wherein the display device 10 is used to display the rendered scene, and the image acquisition device 20 is used to capture the picture displayed by the display device 10. For example, an actor can stand in an appropriate position in front of the display device 10 and perform with the display device 10 as the background. The image acquisition device 20 can simultaneously capture the actor and the picture displayed by the display device 10, thereby completing the shooting of the virtual scene.

[0061] Exemplarily, the display device 10 may be a LED (Light-Emitting Diode) screen or a screen made of other materials. The shape of the screen may be a flat screen, a curved screen, etc. The display device 10 may also be a projection screen, etc., which is not limited in the embodiments of the present disclosure.

[0062] Exemplarily, the image acquisition device 20 may be a device with a shooting function such as a camera or a video camera. The image acquisition device 20 is equipped with a zoom lens, that is, the focal length of the lens of the image acquisition device 20 can be changed to meet different shooting requirements.

[0063] For a zoom lens, when the focal length changes, its imaging parameters generally change. Therefore, during the process of virtual shooting using the image acquisition device 20, it is crucial to correctly track the changes in the focal length of the zoom lens and the corresponding changes in the imaging parameters. In order to accurately track the imaging parameters of the zoom lens of the image acquisition device 20 at different focal lengths during the virtual shooting process, it is necessary to calibrate the image acquisition device 20 before using the image acquisition device 20 for virtual shooting, that is, to solve the imaging parameters of the zoom lens of the image acquisition device 20 corresponding to different focal lengths. Exemplarily, the imaging parameters of the image acquisition device 20 may include intrinsic parameters and / or extrinsic parameters, wherein the extrinsic parameters may include the pose, that is, the position of the zoom lens in space and the orientation of the zoom lens, which can be represented by the rotation matrix R and the translation matrix T; the intrinsic parameters may include the resolution, field of view (FOV), principal point position, distortion parameters, etc. of the zoom lens.

[0064] For example, one method for calibrating a zoom lens is as follows: multiple focal lengths are selected within the zoom range of the zoom lens, and a fixed-focus lens is calibrated for each focal length to obtain the lens' internal parameters at each focal length, forming an internal parameter table. During actual use, the internal parameter table is searched based on the real-time focal length information, and the real-time lens internal parameters are obtained through interpolation. However, this method requires collecting a large number of images (generally around 10) for each focal length that needs to be calibrated, which results in a long calibration time. This is especially true when the zoom range of the zoom lens is large, as the amount of data collected is even greater, making the calibration very time-consuming and inefficient.

[0065] In order to solve the above technical problems, the present disclosure proposes a calibration method for a zoom lens (detailed description see below), which can be used to calibrate the above Figure 1 In the illustrated virtual shooting scene, the image acquisition device 20 is calibrated using a single primary focal length and multiple sub-focal lengths. First, the intrinsic parameters corresponding to the primary focal length are calibrated. Then, based on images captured by the image acquisition device at the same position and focal lengths, the intrinsic parameters corresponding to each sub-focal length are calibrated to achieve zoom lens calibration. In some examples, calibration can be completed by capturing only one image at each focal length, significantly reducing the amount of data collected, saving calibration time, and improving zoom lens calibration efficiency.

[0066] It should be noted that the above-mentioned application scenarios described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. A person skilled in the art will appreciate that the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems in response to the emergence of other similar or new scenarios, such as virtual studio scenarios.

[0067] The calibration method provided by the embodiment of the present disclosure is described in detail below.

[0068] Figure 2 The flowchart of a calibration method according to an embodiment of the present disclosure is shown. The method can be executed by an electronic device with data processing function such as a processor or a server. Figure 2 As shown, the method may include the following steps:

[0069] Step 201: calibrate an internal parameter corresponding to a first focal length, where the first focal length is one of multiple focal lengths of the image acquisition device.

[0070] The focal length can be obtained using a set of encoders fixed on the zoom lens of the image acquisition device. The zoom value of the encoder indicates the current position in the entire zoom range, that is, the current focal length. The number of focal lengths included in the multiple focal lengths and the intervals between adjacent focal lengths can be set according to actual needs. For example, since the image acquisition device is equipped with a zoom lens, that is, there is more than one focal length within the zoom range of the image acquisition device, multiple focal lengths can be randomly selected from the zoom range of the image acquisition device. For example, multiple focal lengths can be selected at equal intervals within the zoom range of the image acquisition device so that the calibrated focal lengths are evenly distributed within the zoom range. The first focal length can be arbitrarily selected from the multiple focal lengths. As an example, the first focal length can be the minimum focal length among the multiple focal lengths, that is, the wide-angle end of the zoom lens of the image acquisition device can be used as the first focal length. For example, if the zoom range of the image acquisition device is 10mm-110mm, 10 focal lengths can be selected at intervals of 10mm, that is, the focal lengths are 10mm, 20mm, 30mm, 40mm, 50mm, 60mm, 70mm, 80mm, 90mm, and 100mm; and the 10mm focal length can be used as the first focal length.

[0071] In one possible implementation, calibrating the intrinsic parameter corresponding to the first focal length may include calibrating the intrinsic parameter corresponding to the first focal length using a fixed-focus calibration method. For example, the intrinsic parameter corresponding to the first focal length may be calibrated using an existing fixed-focus lens calibration method; for example, a calibration interface of an image acquisition device (such as an OpenCV calibration interface) may be invoked to obtain the intrinsic parameter corresponding to the first focal length.

[0072] Step 202: Acquire a captured image at each of the multiple focal lengths, where the captured image is an image obtained by the image capture device capturing a screen image at the same posture and at each of the multiple focal lengths.

[0073] The image on the screen refers to the image displayed on the screen, which can also be called the upper screen image; as an example, the image acquisition device can be the above-mentioned Figure 1 The image acquisition device 20 may have a screen as described above. Figure 1 Display device 10. In the embodiment of the present disclosure, when the image acquisition device captures the image on the screen in a fixed posture, that is, the positions of the image acquisition device and the screen are fixed, and the direction in which the image acquisition device captures the screen remains unchanged. For example, in the process of capturing an image by the image acquisition device, the posture of the image acquisition device can be fixed first, and then the focal length of the image acquisition device can be adjusted, and then the focal length can be switched to each of the multiple focal lengths selected above in turn, and the image on the screen can be captured at each focal length, thereby obtaining a captured image at each focal length; for example, the shooting field of view at each focal length can cover the image on the screen.

[0074] In one possible implementation, the captured images at each of the multiple focal lengths may include images obtained by capturing the screen image once at each of the multiple focal lengths. That is, for each focal length to be calibrated, only one image of the screen image captured by the image capture device at that focal length needs to be captured. This significantly reduces image capture time, the amount of data required for processing, and improved calibration efficiency.

[0075] As an example, the captured image of each of the multiple focal lengths may include multiple feature points. The feature point represents a point where the grayscale value in the captured image changes dramatically or a point with a larger curvature on the edge of the captured image. The point can reflect the essential characteristics of the image and can identify information such as objects or positions in the captured image. In the embodiment of the present disclosure, the picture on the screen may include multiple preset feature points, so that the image capture device captures the picture on the screen at each focal length, and the generated captured image of each focal length contains the multiple feature points. Exemplarily, the picture on the screen may be a checkerboard pattern, a dot dot matrix pattern, or a combination of a checkerboard pattern and a dot dot matrix pattern, for example, Figure 3 FIG. 1 shows a schematic diagram of feature points in a captured image according to an embodiment of the present disclosure. Figure 3 As shown, the left picture (a) is a checkerboard picture, which is composed of black and white squares. The point formed by the junction of two adjacent squares of the same color is called a corner point, and the corner point is the feature point; the right picture (b) is a dot matrix picture, which is composed of white dots, and each white dot is a feature point.

[0076] As another example, the image capture device captures a picture on the screen including the above-mentioned multiple feature points and a positioning identifier with known preset position information; in this way, after the image capture device captures the picture on the screen at various focal lengths, the captured image obtained also includes the positioning identifier and the multiple feature points. Exemplarily, the positioning identifier can be an identifier generated based on an Aruco code (a two-dimensional code), or a combination of the two; wherein the Aruco code is a synthetic square mark consisting of a wide black border and an internal binary matrix that can determine its identifier. In the embodiment of the present disclosure, a positioning identifier can be generated based on the Aruco code, for example: the Aruco code can be used as a positioning identifier alone; other identification information can be added to the Aruco code as a positioning identifier; the Aruco code can also be modified in certain shapes, and the improved graphic can be used as a positioning identifier, and so on. Figure 4 A schematic diagram of feature points and positioning identifiers in a captured image according to an embodiment of the present disclosure is shown. Figure 4 As shown in the figure, the left picture (a) is a dot matrix picture, the white dots are feature points, and the four circles in the middle are positioning identifiers; the right picture (b) is a picture combining the Aruco code with the chessboard, the corner points in the picture are feature points, and a single Aruco code is embedded in a white square grid on the chessboard. The white square grid embedded with the ArUco code is the positioning identifier.

[0077] Step 203: Calibrate the intrinsic parameters of each focal length among the multiple focal lengths except the first focal length according to the calibrated intrinsic parameters corresponding to the first focal length and the captured images of each focal length among the multiple focal lengths.

[0078] In one possible implementation, a step-by-step intrinsic parameter initial value estimation strategy can be used to calibrate the internal parameters corresponding to each focal length among multiple focal lengths except the first focal length. For example, the initial value of the internal parameter corresponding to the second focal length can be determined based on the internal parameter corresponding to at least one focal length adjacent to the second focal length among the multiple focal lengths, i.e., the internal parameter corresponding to the second focal length can be initialized. The second focal length is any focal length among the multiple focal lengths except the first focal length. When the second focal length is adjacent to the first focal length, at least one focal length adjacent to the second focal length among the multiple focal lengths includes the first focal length. Furthermore, based on the initial value of the internal parameter corresponding to the second focal length and a captured image of the second focal length, a calibration value of the internal parameter corresponding to the second focal length is determined. For example, if there is a focal length among multiple focal lengths that is adjacent to the second focal length and has its internal parameters calibrated, the calibrated internal parameter corresponding to this focal length is used as the initial value of the internal parameter corresponding to the second focal length; if there are two focal lengths among multiple focal lengths that are adjacent to the second focal length and have their internal parameters calibrated, the calibrated internal parameter corresponding to any one of the two focal lengths can be used as the initial value of the internal parameter corresponding to the second focal length, or the average of the calibrated internal parameters corresponding to the two focal lengths can be used as the initial value of the internal parameter corresponding to the second focal length.

[0079] As an example, the focal lengths that need to be calibrated can be selected in sequence from multiple focal lengths in ascending order of distance from the first focal length; for example, multiple focal lengths are sorted in ascending order according to the focal length values, namely focal lengths A, B, C, D, and E; the first focal length is focal length A, and the focal length B closest to focal length A can be selected from focal lengths B, C, D, and E. Since the internal parameter a of focal length A has been calibrated, the calibrated internal parameter of focal length A can be used as the initial value of the internal parameter b corresponding to focal length B, and then the internal parameter b can be calibrated according to the initial value of the internal parameter b and the captured image of focal length B, so as to determine the calibration value of the internal parameter b corresponding to focal length B. Next, the internal parameter c corresponding to focal length C can be calibrated. Since focal length B is a focal length adjacent to focal length C and has already been calibrated, the calibration value of the internal parameter b corresponding to focal length B can be used as the initial value of the internal parameter c corresponding to focal length C. Then, based on the initial value of the internal parameter c and the captured image of focal length C, the internal parameter c can be calibrated to determine the calibration value of the internal parameter c corresponding to focal length C. Similarly, the parameter d corresponding to focal length D and the parameter e corresponding to focal length E can be calibrated in sequence. In this way, a step-by-step internal parameter initial value estimation strategy is adopted. When optimizing the internal parameter corresponding to the new focal length, the calibrated internal parameter corresponding to the adjacent focal length is used as the initial value. Since the two focal lengths are adjacent, the corresponding internal parameters are not much different. By making the initial value close to the true value of the internal parameter corresponding to the focal length, the correct internal parameter value (i.e., the calibration value) corresponding to the focal length can be obtained more easily and quickly, simplifying the estimation of the corresponding internal parameter of the new focal length.

[0080] In the disclosed embodiment, a mode of one main focal length (first focal length) + multiple sub-focal lengths (each focal length among the multiple focal lengths except the first focal length) is adopted; first, the internal parameters corresponding to the first focal length are calibrated, and then based on the internal parameters corresponding to the first focal length and the captured images obtained by respectively shooting the pictures on the screen at the same posture and each focal length, the internal parameters corresponding to each focal length among the multiple focal lengths except the first focal length are calibrated to realize the calibration of the zoom lens of the image acquisition device; as an example, only one image needs to be captured for each focal length to complete the calibration, thereby greatly reducing the amount of data collection and data processing, saving calibration time, and effectively improving the calibration efficiency of the zoom lens.

[0081] The specific implementation process of calibrating the internal parameters corresponding to each focal length except the first focal length among multiple focal lengths using the above-mentioned step-by-step internal parameter initial value estimation strategy is exemplified below.

[0082] Figure 5 FIG. 1 shows a flow chart of another calibration method according to an embodiment of the present disclosure. Figure 5 As shown, the method may include the following steps:

[0083] Step 501: Calibrate the internal parameters corresponding to the first focal length.

[0084] This step 501 is similar to the above Figure 2 Step 201 is the same as above and will not be described again here.

[0085] Step 502: Acquire a captured image of each focal length of the multiple focal lengths.

[0086] This step 502 is similar to the above Figure 2 Step 202 is the same as above and will not be described again here.

[0087] Step 503: perform feature point detection on the captured image of each focal length among the multiple focal lengths.

[0088] In this step, each captured image at each focal length contains feature points. Through this step, each feature point in the captured image at the first focal length and each feature point in the captured image at any focal length other than the first focal length can be detected.

[0089] For example, feature point detection can be implemented using a variety of different algorithms, such as a corner detection algorithm or a spot detection algorithm, to perform feature point detection to determine multiple feature points in the captured image at each of the multiple focal lengths. Figure 3 The chessboard image shown can be used to perform chessboard corner point detection to obtain the corner points in the image, that is, to detect the feature points.

[0090] Exemplarily, for a captured image containing a positioning identifier, feature point detection and positioning identifier recognition may be performed.

[0091] Step 504: Determine the two-dimensional coordinates of each feature point in the captured image at each focal length and the three-dimensional coordinates of each feature point on the screen.

[0092] Through this step, the two-dimensional coordinates of each feature point in the captured image of the first focal length and the three-dimensional coordinates of each feature point on the screen can be determined. The two-dimensional coordinates of each feature point in the captured image of any focal length other than the first focal length (i.e., the second focal length) among multiple focal lengths and the three-dimensional coordinates of each feature point on the screen can also be determined.

[0093] Exemplarily, for a certain feature point on the captured image, the two-dimensional coordinates of the feature point in the corresponding captured image can be expressed by the coordinate value of the feature point in the image coordinate system; wherein, the image coordinate system is a two-dimensional rectangular coordinate system, which can take the center of the captured image as the coordinate origin, or a point in the lower left corner of the captured image, or a point in the lower right corner of the captured image as the coordinate origin, etc.; the X-axis and Y-axis of the image coordinate system are respectively parallel to the X-axis and Y-axis of the image acquisition device coordinate system; the image acquisition device coordinate system can take the focusing center of the image acquisition device as the origin, and the optical axis of the image acquisition device as the Z axis; exemplarily, the intersection of the optical axis of the image acquisition device and the plane where the captured image is located can be used as the coordinate origin of the image coordinate system. In this way, based on the established image coordinate system, after the feature points of the captured image are detected as described above, the coordinate value of each feature point in the image coordinate system (i.e., the two-dimensional coordinate in the captured image) can be determined; for example, for the above Figure 3 As shown in the chessboard image, after detecting each corner point in the image, the coordinate value of each corner point in the image coordinate system can be determined.

[0094] For example, for a certain feature point on the captured image, the three-dimensional coordinates of the feature point on the screen can be represented by the coordinate value of the feature point on the screen captured by the image acquisition device in the screen coordinate system; wherein, the screen coordinate system is a three-dimensional coordinate system set when performing screen modeling, and the definition method of the screen coordinate system can be set according to actual conditions, for example: the center of the screen can be used as the coordinate origin, or a point in the lower left corner of the screen, or a point in the lower right corner of the screen can be used as the coordinate origin, etc.

[0095] Exemplarily, the three-dimensional coordinates of each detected feature point on the screen may be determined in the following manner.

[0096] Method 1: When generating a picture containing feature points (i.e., the picture on the screen), the position of the reference feature point on the picture can be obtained in advance. The reference feature point is any one or more feature points in the picture, and the two-dimensional display area of the picture in the established screen model can be determined in advance. Based on the position of the reference feature point on the picture and the position of the above-mentioned display area, the two-dimensional coordinates of the reference feature point in the screen model can be determined: texture mapping coordinates (UV coordinates); in addition, when modeling the screen, for each 3D point in the screen model, the correspondence between its three-dimensional coordinates in the screen coordinate system and the texture mapping coordinates can be established in advance. In this way, after determining the two-dimensional coordinates of each feature point in the corresponding captured image, the position of each feature point on the picture can be determined based on the relative position relationship between each feature point and the position of the above-mentioned reference feature point on the picture, and then the UV coordinates of each feature point in the screen model can be determined. Then, combined with the above-mentioned correspondence, the three-dimensional coordinate value of each feature point in the screen coordinate system, that is, the three-dimensional coordinates of each feature point on the screen, can be obtained.

[0097] For example, taking the collected image as the above Figure 3 Taking the chessboard image shown as an example, the feature points are all the corner points in the chessboard image, and the acquired image contains all the corner points in the chessboard image. When performing corner point detection, the corner points in the acquired image can be detected; reference corner points are pre-selected, and the positions of the reference corner points on the screen are obtained; after detecting the two-dimensional coordinates of all the corner points in the chessboard image in the acquired image, the position of each corner point in the screen can be obtained based on the relative position relationship between the corner points and the position of the above-mentioned reference corner points on the screen; and then based on the position of the screen in the display area of the screen model and the position of each corner point in the screen, the UV coordinates of each corner point in the screen model can be obtained; in addition, when establishing the screen model, for each 3D point in the screen model, a correspondence between its three-dimensional coordinates and UV coordinates in the screen coordinate system is pre-established. In this way, after obtaining the UV coordinates of each corner point in the screen model, the three-dimensional coordinate value of each corner point in the screen coordinate system, that is, the three-dimensional coordinates of each corner point on the screen, can be obtained based on the correspondence.

[0098] Method 2: The image acquisition device captures the image on the screen and the captured image includes a plurality of feature points and a positioning identifier with known preset position information, and the position information can be the position on the screen; after determining the two-dimensional coordinates of each feature point detected in the corresponding captured image, the positioning identifier can be used to determine the three-dimensional coordinates of each detected feature point on the screen. For example, based on the plurality of feature points and the positioning identifier detected in the captured image, the relative position relationship between each feature point and the positioning identifier can be obtained, and then based on the relative position relationship and the position information of the positioning identifier, the position of each feature point in the image can be determined; then, based on the position of each feature point in the image and the two-dimensional display area of the image in the established screen model, the UV coordinates of each feature point in the screen model are determined; finally, based on the correspondence between the three-dimensional coordinates and the UV coordinates in the screen coordinate system, the three-dimensional coordinate value of each feature point in the screen coordinate system, that is, the three-dimensional coordinates of each feature point on the screen, is obtained.

[0099] For example, taking the collected image as the above Figure 4 Taking the picture shown in which the Aruco code is combined with the chessboard as an example, in the picture, a single Aruco code is embedded in a white square of the chessboard to generate a positioning identifier, and the corner points of the chessboard are used as feature points. After detecting the corner points of the chessboard and any Aruco code, any detected Aruco code is decoded and identified to obtain the identification information of any Aruco code, and based on the identification information, the position of any Aruco code on the screen is determined; then, based on the relative position relationship between each corner point and any Aruco code, the position of each corner point on the screen is obtained, and then the UV coordinates of each corner point in the screen model are determined; finally, based on the correspondence between the three-dimensional coordinates in the screen coordinate system and the UV coordinates, the three-dimensional coordinate value of each corner point in the screen coordinate system, that is, the three-dimensional coordinates of each corner point on the screen, is obtained.

[0100] Through this step, for any feature point detected above, the two-dimensional coordinates of the feature point in the captured image and the three-dimensional coordinates of the feature point on the screen are determined, thereby constructing a 2D-3D point pair.

[0101] Step 505: Determine the position and posture of the image acquisition device according to the calibrated internal parameters corresponding to the first focal length, the two-dimensional coordinates of each feature point in the captured image of the first focal length, and the three-dimensional coordinates of each feature point on the screen.

[0102] For example, an existing algorithm such as solvepnp can be used to process the internal parameters corresponding to the calibrated first focal length, the two-dimensional coordinates of each feature point in the captured image of the first focal length, and the three-dimensional coordinates of each feature point on the screen to obtain the pose corresponding to the first focal length; this pose can represent the coordinate transformation relationship between the image acquisition device coordinate system and the screen coordinate system. The solvepnp algorithm can be a P3P camera pose estimation algorithm, the Direct Least-Squares Method (DLS), the Efficient Perspective-n-Point (EPnP) camera pose estimation algorithm, or an iterative method, etc.

[0103] It can be understood that since the captured images at each focal length are obtained by the image capture device capturing the screen image at the same position, that is, the pose corresponding to each focal length is the same; therefore, the pose corresponding to the first focal length determined is the pose of the image capture device. In this way, the image capture device captures the screen image without changing its position to obtain the captured images at each focal length, fixing the extrinsic parameters of the image capture device. After obtaining the pose corresponding to one focal length, there is no need to calibrate the poses corresponding to other focal lengths, thereby greatly reducing calibration complexity and improving calibration efficiency.

[0104] This step 505 is an optional step. The position of the image acquisition device can also be determined in other ways, or the image acquisition device can be placed in a state with a known position when the image acquisition device is used to capture the image on the screen. This embodiment of the present application is not limited to this.

[0105] Step 506: Determine a calibration value of the intrinsic parameter corresponding to the second focal length according to the initial value of the intrinsic parameter corresponding to the second focal length and the captured image of the second focal length.

[0106] The initial value of the internal parameter corresponding to the second focal length can be determined by the method in step 203 above.

[0107] For example, the calibration value of the internal parameter corresponding to the second focal length can be determined based on the initial value of the internal parameter corresponding to the second focal length, the two-dimensional coordinates of each feature point in the captured image at the second focal length, and the three-dimensional coordinates of each feature point on the screen. For any feature point, the conversion between the two-dimensional coordinates of the feature point in the captured image and the three-dimensional coordinates of the feature point on the screen is related to the internal parameter corresponding to the second focal length. Thus, based on the initial value of the internal parameter corresponding to the second focal length, the value of the internal parameter corresponding to the second focal length can be continuously optimized using a nonlinear optimization algorithm or other method. Through the optimized internal parameter value, the two-dimensional coordinates of the feature point in the captured image can be transformed into the three-dimensional coordinates of the feature point on the screen.

[0108] In one possible implementation, the calibration value of the internal parameter corresponding to the second focal length can be determined based on the posture of the image acquisition device, the two-dimensional coordinates of each feature point in the captured image of the second focal length and the three-dimensional coordinates of each feature point on the screen, and the initial value of the internal parameter corresponding to the second focal length.

[0109] Exemplarily, determining the calibration value of the internal parameter corresponding to the second focal length based on the posture of the image acquisition device, the two-dimensional coordinates of each feature point in the captured image of the second focal length and the three-dimensional coordinates of each feature point on the screen, and the initial value of the internal parameter corresponding to the second focal length may include: determining the two-dimensional reference coordinates corresponding to the three-dimensional coordinates of each feature point in the captured image of the second focal length on the screen based on the posture of the image acquisition device and the value of the internal parameter corresponding to the second focal length; and iteratively optimizing the value of the internal parameter corresponding to the second focal length based on the two-dimensional reference coordinates and the two-dimensional coordinates of each feature point in the captured image of the second focal length, and using the value of the internal parameter corresponding to the second focal length when the preset conditions are met as the calibration value; wherein the initial value of the internal parameter corresponding to the second focal length is used as the initial value of the internal parameter corresponding to the second focal length during the iterative optimization process. Exemplarily, the preset conditions may include: reaching a preset number of iterations, reaching a preset iteration time, the deviation of the results obtained from two consecutive iterations being less than a preset value, the difference between the two-dimensional reference coordinates corresponding to each feature point and the two-dimensional coordinates of each feature point being minimized, etc.; in this way, the intrinsic parameter value corresponding to the second focal length is continuously optimized through a nonlinear optimization algorithm, which simplifies the estimation process of the intrinsic parameter of the zoom lens at the focal length, thereby making it easier and faster to converge to the accurate intrinsic parameter value.

[0110] For example, for each feature point of a certain second focal length, the three-dimensional coordinates P_i of each feature point in the captured image corresponding to the second focal length (i.e., the three-dimensional coordinates P_i in the screen coordinate system) can be calculated based on the posture of the image acquisition device and the initial value of the internal parameter corresponding to the second focal length, combined with the projection equation, and projected onto the two-dimensional reference coordinates P_t in the image coordinate system; then the internal parameter values corresponding to the second focal length can be continuously optimized through the above-mentioned nonlinear optimization algorithm, and a new two-dimensional reference coordinate P_t can be calculated based on the optimized internal parameter values corresponding to the second focal length, until the Euclidean distance between the two-dimensional reference coordinate P_t and the two-dimensional coordinates P'_t of each feature point in the captured image reaches the minimum, and the value of the internal parameter corresponding to this time is used as the calibration value of the internal parameter corresponding to the second focal length.

[0111] In the disclosed embodiment, a step-by-step strategy for estimating the initial value of the intrinsic parameter is adopted, and a nonlinear optimization algorithm is used to continuously optimize the intrinsic parameter values corresponding to each focal length. On the basis of ensuring the accuracy of the intrinsic parameter calibration value, the efficiency of the intrinsic parameter calibration is effectively improved; in addition, the captured images of each focal length are images obtained by the image acquisition device shooting the screen at the same posture, which fixes the external parameters of the image acquisition device, greatly reduces the parameters that need to be estimated, and further improves the accuracy of the intrinsic parameter calibration.

[0112] Furthermore, after completing the calibration of the internal parameters corresponding to each of the multiple focal lengths using the above method, the internal parameters corresponding to other focal lengths within the zoom range of the image acquisition device can be calibrated based on the calibrated internal parameters corresponding to each focal length; or based on the calibrated internal parameters corresponding to each focal length, the internal parameters corresponding to the focal length adjusted in real time can be determined during the virtual shooting process.

[0113] Figure 6 FIG. 1 shows a flow chart of another calibration method according to an embodiment of the present disclosure. Figure 6 As shown, the method may include the following steps:

[0114] Step 601: Calibrate an internal parameter corresponding to a first focal length, where the first focal length is one of multiple focal lengths of the image acquisition device.

[0115] Step 602: Acquire a captured image at each of the multiple focal lengths, where the captured image is an image obtained by the image capture device capturing a screen image at the same posture and at each of the multiple focal lengths.

[0116] Step 603: Calibrate the intrinsic parameters of each focal length among the multiple focal lengths except the first focal length according to the calibrated intrinsic parameters corresponding to the first focal length and the captured images of each focal length among the multiple focal lengths.

[0117] Among them, steps 601-603 are the same as above Figure 2 Steps 201-203 are the same and will not be repeated here.

[0118] Step 604: Determine a third focal length, wherein the third focal length is any focal length within the zoom range of the image acquisition device.

[0119] Exemplarily, the third focal length is any focal length within the zoom range of the image acquisition device except the multiple focal lengths calibrated above (ie, the first focal length and the second focal length).

[0120] For example, when the third focal length is used to capture the image on the screen, the posture of the image acquisition device remains consistent with the posture of the image on the screen captured at multiple focal lengths during the above calibration process; that is, the external parameters corresponding to the third focal length are the same as the external parameters corresponding to the first focal length and the external parameters corresponding to the second focal length.

[0121] Step 605: Determine one or more focal lengths adjacent to the third focal length from the multiple focal lengths.

[0122] As an example, for any third focal length, two focal lengths adjacent to the third focal length can be determined from the plurality of focal lengths, where one of the two focal lengths is greater than the third focal length and the other is less than the third focal length. For example, if the plurality of focal lengths are 10mm, 20mm, 30mm, 40mm, 50mm, 60mm, 70mm, 80mm, 90mm, and 100mm, and the third focal length is 25mm, then 20mm and 30mm are selected as the two focal lengths adjacent to 35mm.

[0123] Step 606: Determine an internal parameter corresponding to the third focal length based on the internal parameters corresponding to the one or more focal lengths.

[0124] For example, the internal parameter corresponding to the third focal length can be determined by interpolation. For example, if the third focal length is 25 mm, and among the multiple calibrated focal lengths, the focal lengths adjacent to 35 mm are 20 mm and 30 mm, if the calibrated value of the internal parameter corresponding to the focal length of 20 mm is d1 and the calibrated value of the internal parameter corresponding to the focal length of 30 mm is d2, then the internal parameter d3 corresponding to the focal length of 25 mm is (25-20)*(d2-d1) / (30-20)=(d2-d1) / 2.

[0125] In the embodiment of the present disclosure, after calibrating the internal parameters corresponding to multiple focal lengths (i.e., the first focal length and the second focal length), the internal parameters corresponding to any focal length within the zoom range of the image acquisition device can be determined based on the internal parameters, so that the internal parameters corresponding to any focal length can be calibrated before virtual shooting; or, during the actual virtual shooting process, the internal parameters corresponding to the current focal length can be calculated in real time to meet the needs of virtual shooting.

[0126] Based on the same inventive concept of the above method embodiment, the embodiment of the present disclosure further provides a calibration device, which can be used to execute the technical solution described in the above method embodiment. Figure 2 、 Figure 5 or Figure 6 The steps of the calibration method are shown in .

[0127] Figure 7 FIG. 1 shows a structural diagram of a calibration device according to an embodiment of the present disclosure, wherein the device is used to calibrate an image acquisition device in a virtual shooting scene. Figure 7 As shown, the device may include:

[0128] A calibration module 701 is configured to calibrate an intrinsic parameter corresponding to a first focal length, where the first focal length is one of multiple focal lengths of the image acquisition device;

[0129] An acquisition module 702 is configured to acquire an acquired image at each of the multiple focal lengths, where the acquired image is an image obtained by the image acquisition device capturing a screen image at the same posture and at each of the multiple focal lengths.

[0130] The calibration module 701 is further configured to calibrate the internal parameters of each focal length among the multiple focal lengths except the first focal length according to the calibrated internal parameters corresponding to the first focal length and the captured images of each focal length among the multiple focal lengths.

[0131] In the disclosed embodiment, a mode of one main focal length (first focal length) + multiple sub-focal lengths (each focal length among the multiple focal lengths except the first focal length) is adopted; first, the internal parameters corresponding to the first focal length are calibrated, and then based on the internal parameters corresponding to the first focal length and the captured images obtained by respectively shooting the pictures on the screen at the same posture and each focal length, the internal parameters corresponding to each focal length among the multiple focal lengths except the first focal length are calibrated to realize the calibration of the zoom lens of the image acquisition device; as an example, only one image needs to be captured for each focal length to complete the calibration, thereby greatly reducing the amount of data collection and data processing, saving calibration time, and effectively improving the calibration efficiency of the zoom lens.

[0132] In a possible implementation, the captured image at each of the multiple focal lengths includes an image obtained by shooting the image on the screen once at each of the multiple focal lengths.

[0133] In one possible implementation, the calibration module 701 is further used to: determine an initial value of the internal parameter corresponding to the second focal length based on the internal parameter corresponding to at least one focal length adjacent to the second focal length among the multiple focal lengths; wherein the second focal length is any focal length among the multiple focal lengths except the first focal length; determine a calibration value of the internal parameter corresponding to the second focal length based on the initial value of the internal parameter corresponding to the second focal length and a captured image of the second focal length, wherein, when the second focal length is adjacent to the first focal length, at least one focal length adjacent to the second focal length among the multiple focal lengths includes the first focal length.

[0134] In one possible implementation, the captured image of each of the multiple focal lengths includes multiple feature points; the calibration module 701 is further used to: determine the two-dimensional coordinates of each feature point in the captured image of the second focal length, and the three-dimensional coordinates of each feature point on the screen; determine the calibration value of the intrinsic parameter corresponding to the second focal length based on the initial value of the intrinsic parameter corresponding to the second focal length, the two-dimensional coordinates of each feature point in the captured image of the second focal length, and the three-dimensional coordinates of each feature point on the screen.

[0135] In one possible implementation, the calibration module 701 is further used to: determine the two-dimensional coordinates of each feature point in the captured image of the first focal length, and the three-dimensional coordinates of each feature point on the screen; determine the posture of the image acquisition device based on the calibrated internal parameters corresponding to the first focal length, the two-dimensional coordinates of each feature point in the captured image of the first focal length, and the three-dimensional coordinates of each feature point on the screen.

[0136] In one possible implementation, the calibration module 701 is further used to determine the calibration value of the internal parameter corresponding to the second focal length based on the posture of the image acquisition device, the two-dimensional coordinates of each feature point in the captured image of the second focal length and the three-dimensional coordinates of each feature point on the screen, and the initial value of the internal parameter corresponding to the second focal length.

[0137] In one possible implementation, the calibration module 701 is further used to: determine the two-dimensional reference coordinates corresponding to the three-dimensional coordinates of each feature point in the captured image of the second focal length on the screen according to the posture of the image acquisition device and the value of the internal parameter corresponding to the second focal length; and iteratively optimize the value of the internal parameter corresponding to the second focal length based on the two-dimensional reference coordinates and the two-dimensional coordinates of each feature point in the captured image of the second focal length, and use the value of the internal parameter corresponding to the second focal length when the preset conditions are met as the calibration value; wherein the initial value of the internal parameter corresponding to the second focal length is used as the initial value of the internal parameter corresponding to the second focal length during the iterative optimization process.

[0138] In one possible implementation, the calibration module 701 is further used to: determine a third focal length, wherein the third focal length is any focal length within the zoom range of the image acquisition device; determine one or more focal lengths adjacent to the third focal length among the multiple focal lengths; and determine the internal parameters corresponding to the third focal length based on the internal parameters corresponding to the one or more focal lengths.

[0139] In a possible implementation, the calibration module 701 is further configured to calibrate the intrinsic parameter corresponding to the first focal length by adopting a fixed-focus calibration method.

[0140] In a possible implementation, the first focal length is the minimum focal length among the multiple focal lengths.

[0141] above Figure 7 The technical effects and specific descriptions of the calibration device shown and its various possible implementation methods can be found in the above-mentioned calibration method, which will not be repeated here.

[0142] It should be understood that the division of the modules in the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into a physical entity, or they can be physically separated. In addition, the modules in the device can be implemented in the form of a processor calling software; for example, the device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of the modules of the device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the modules in the device can be implemented in the form of hardware circuits, and the functions of some or all modules can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC), which realizes the functions of some or all of the above modules by designing the logical relationship of the components in the circuit. For another example, in another implementation, the hardware circuit can be implemented by a programmable logic device (PLD). Taking a field programmable gate array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by configuring the configuration file, thereby realizing the functions of some or all of the above modules. All modules of the above devices can be implemented in the form of software called by the processor, or in the form of hardware circuits, or in part by the form of software called by the processor, and the rest by hardware circuits.

[0143] In the embodiments of the present disclosure, a processor is a circuit capable of processing signals. In one implementation, the processor may be a circuit capable of reading and executing instructions, such as a CPU, a microprocessor, a graphics processing unit (GPU), a digital signal processor (DSP), a neural-network processing unit (NPU), a tensor processing unit (TPU), etc. In another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit may be fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above modules.

[0144] It can be seen that each module in the above apparatus can be one or more processors (or processing circuits) configured to implement the above embodiment methods, such as: CPU, GPU, NPU, TPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms. In addition, each module in the above apparatus can be fully or partially integrated together, or can be implemented independently, without limitation.

[0145] The present disclosure also provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the method of the above embodiment when executing the instructions. Figure 2 、 Figure 5 or Figure 6 The steps of the calibration method are shown in .

[0146] Figure 8 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Figure 8 As shown, the electronic device may include: at least one processor 801 , a communication line 802 , a memory 803 and at least one communication interface 804 .

[0147] The processor 801 can be a general-purpose central processing unit, a microprocessor, a specific application integrated circuit, or one or more integrated circuits for controlling the execution of the program of the disclosed solution; the processor 801 can also include a heterogeneous computing architecture of multiple general-purpose processors, for example, it can be a combination of at least two of a CPU, a GPU, a microprocessor, a DSP, an ASIC, and an FPGA; as an example, the processor 801 can be a CPU+GPU or a CPU+ASIC or a CPU+FPGA.

[0148] Communication link 802 may include a pathway for transmitting information between the aforementioned components.

[0149] The communication interface 804 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (WLAN), etc.

[0150] The memory 803 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory can be independent and connected to the processor via a communication line 802. The memory can also be integrated with the processor. The memory provided in the embodiment of the present disclosure can generally have non-volatility. Among them, the memory 803 is used to store computer-executable instructions for executing the disclosed solution, and is controlled by the processor 801 for execution. The processor 801 is used to execute the computer-executable instructions stored in the memory 803, thereby implementing the method provided in the above embodiment of the present disclosure; illustratively, the above Figure 2 、 Figure 5 or Figure 6 The steps of the calibration method are shown in .

[0151] Optionally, the computer-executable instructions in the embodiments of the present disclosure may also be referred to as application code, which is not specifically limited in the embodiments of the present disclosure.

[0152] Exemplarily, the processor 801 may include one or more CPUs, for example, Figure 8 The processor 801 may also include a CPU, and any one of a GPU, an ASIC, and an FPGA, for example, Figure 8 CPU0+GPU0 or CPU 0+ASIC0 or CPU0+FPGA0 in.

[0153] For example, an electronic device may include multiple processors, such as Figure 8 801 and processor 807 in FIG. Each of these processors can be a single-core (single-CPU) processor, a multi-core (multi-CPU) processor, or a heterogeneous computing architecture including multiple general-purpose processors. A processor here can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0154] In a specific implementation, as an embodiment, the electronic device may further include an output device 805 and an input device 806. The output device 805 communicates with the processor 801 and can display information in a variety of ways. For example, the output device 805 can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. For example, it can be a display device such as a vehicle-mounted HUD, AR-HUD, or a display. The input device 806 communicates with the processor 801 and can receive user input in a variety of ways. For example, the input device 806 can be a mouse, a keyboard, a touch screen device, or a sensing device, etc.

[0155] The embodiments of the present disclosure provide a computer-readable storage medium having computer program instructions stored thereon, which implement the method in the above embodiments when the computer program instructions are executed by a processor. Figure 2 、 Figure 5 or Figure 6 The steps of the calibration method are shown in .

[0156] The embodiments of the present disclosure provide a computer program product, which may include computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code; when the computer program product is run on a computer, the computer is caused to execute the method in the above embodiment. Figure 2 、 Figure 5 or Figure 6 The steps of the calibration method are shown in .

[0157] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0158] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0159] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0160] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0161] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0162] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0163] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0164] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0165] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A calibration method, characterized in that: For calibrating an image acquisition device in a virtual shooting scene, the method includes: calibrating an internal parameter corresponding to a first focal length, where the first focal length is one of multiple focal lengths of the image acquisition device; Acquire a captured image at each of the multiple focal lengths, where the captured image is an image obtained by the image capture device capturing a screen image at the same posture and at each of the multiple focal lengths; calibrating the intrinsic parameters of each focal length among the multiple focal lengths except the first focal length according to the calibrated intrinsic parameters corresponding to the first focal length and the captured images of each focal length among the multiple focal lengths; The step of calibrating the internal parameters of each focal length among the multiple focal lengths except the first focal length according to the calibrated internal parameters corresponding to the first focal length and the captured images of each focal length among the multiple focal lengths includes: Determining an initial value of the internal parameter corresponding to the second focal length based on a calibrated internal parameter corresponding to at least one focal length adjacent to the second focal length among the multiple focal lengths; wherein the second focal length is any focal length among the multiple focal lengths except the first focal length; Determine a calibration value of the intrinsic parameter corresponding to the second focal length according to the initial value of the intrinsic parameter corresponding to the second focal length and the captured image of the second focal length, When the second focal length is adjacent to the first focal length, at least one focal length adjacent to the second focal length among the multiple focal lengths includes the first focal length.

2. The method according to claim 1, characterized in that The captured image at each of the multiple focal lengths includes an image obtained by shooting the image on the screen once at each of the multiple focal lengths.

3. The method according to claim 1, characterized in that The captured image of each focal length of the multiple focal lengths includes multiple feature points; The method further comprises: Determining the two-dimensional coordinates of each feature point in the captured image of the second focal length, and the three-dimensional coordinates of each feature point on the screen; The determining, based on the initial value of the intrinsic parameter corresponding to the second focal length and the captured image of the second focal length, a calibration value of the intrinsic parameter corresponding to the second focal length includes: The calibration value of the intrinsic parameter corresponding to the second focal length is determined according to the initial value of the intrinsic parameter corresponding to the second focal length, the two-dimensional coordinates of each feature point in the captured image of the second focal length, and the three-dimensional coordinates of each feature point on the screen.

4. The method according to claim 3, characterized in that The step of determining a calibration value of the intrinsic parameter corresponding to the second focal length according to the initial value of the intrinsic parameter corresponding to the second focal length, the two-dimensional coordinates of each feature point in the captured image of the second focal length, and the three-dimensional coordinates of each feature point on the screen includes: The calibration value of the internal parameter corresponding to the second focal length is determined according to the posture of the image acquisition device, the two-dimensional coordinates of each feature point in the captured image of the second focal length and the three-dimensional coordinates of each feature point on the screen, and the initial value of the internal parameter corresponding to the second focal length.

5. The method according to claim 4, characterized in that The method of determining a calibration value of the intrinsic parameter corresponding to the second focal length according to the posture of the image acquisition device, the two-dimensional coordinates of each feature point in the captured image of the second focal length and the three-dimensional coordinates of each feature point on the screen, and the initial value of the intrinsic parameter corresponding to the second focal length includes: According to the posture of the image acquisition device and the value of the internal parameter corresponding to the second focal length, the two-dimensional reference coordinates corresponding to the three-dimensional coordinates of each feature point in the captured image of the second focal length on the screen are determined; and based on the two-dimensional reference coordinates and the two-dimensional coordinates of each feature point in the captured image of the second focal length, the value of the internal parameter corresponding to the second focal length is iteratively optimized, and the value of the internal parameter corresponding to the second focal length when the preset conditions are met is used as the calibration value; wherein, the initial value of the internal parameter corresponding to the second focal length is used as the initial value of the internal parameter corresponding to the second focal length during the iterative optimization process.

6. The method according to claim 1, characterized in that The method further comprises: Determining a third focal length, wherein the third focal length is any focal length within a zoom range of the image acquisition device; determining one or more focal lengths adjacent to the third focal length among the plurality of focal lengths; An internal parameter corresponding to the third focal length is determined based on the internal parameters corresponding to the one or more focal lengths.

7. The method according to claim 1, characterized in that The calibrating the internal parameter corresponding to the first focal length includes: The internal parameter corresponding to the first focal length is calibrated by using a fixed focus calibration method.

8. The method according to any one of claims 1 to 7, characterized in that The first focal length is the smallest focal length among the multiple focal lengths.

9. A calibration device, characterized in that: Used to calibrate an image acquisition device in a virtual shooting scene, the device comprising: a calibration module, configured to calibrate an internal parameter corresponding to a first focal length, where the first focal length is one of the plurality of focal lengths of the image acquisition device; an acquisition module, configured to acquire an acquired image at each of the multiple focal lengths, wherein the acquired image is an image obtained by the image acquisition device capturing a screen image at the same posture and at each of the multiple focal lengths; The calibration module is further configured to calibrate the intrinsic parameters of each focal length among the multiple focal lengths except the first focal length based on the calibrated intrinsic parameters corresponding to the first focal length and the captured images of each focal length among the multiple focal lengths; The calibration module is further used to determine the initial value of the internal parameter corresponding to the second focal length based on the calibrated internal parameter corresponding to at least one focal length adjacent to the second focal length among the multiple focal lengths; wherein the second focal length is any focal length among the multiple focal lengths except the first focal length; and determine the calibration value of the internal parameter corresponding to the second focal length based on the initial value of the internal parameter corresponding to the second focal length and the captured image of the second focal length, wherein, when the second focal length is adjacent to the first focal length, at least one focal length adjacent to the second focal length among the multiple focal lengths includes the first focal length.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 8 when executing the instructions stored in the memory.

11. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images