Calibration method and device of binocular camera, computer readable medium and equipment

Abstract

Embodiments of the present application provide a method, device, computer readable medium and equipment for calibrating a binocular camera. The method comprises: obtaining a first captured image and a second captured image captured by each camera for the same captured content; determining feature point information corresponding to each calibration image in the first captured image and the second captured image to determine target calibration information corresponding to the binocular camera, and determining a first conversion matrix and a second conversion matrix of each captured image to a coplanar line alignment plane; performing depth recognition on a face image to be detected included in each captured image to determine corresponding depth information; and comparing the depth information with an actual distance to determine whether a calibration result is valid. The technical solution of the embodiments of the present application can improve the calibration efficiency of the binocular camera and ensure the accuracy of the calibration result.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a calibration method, apparatus, computer-readable medium, and device for a binocular camera. Background Technology

[0002] In the field of facial recognition, binocular cameras can construct 3D depth information of faces, effectively preventing various liveness detection attacks such as those using images and videos, thus gaining widespread application. Current technical solutions require the binocular camera to take photos of a calibration board from different positions for calibration, or multiple calibration boards can be combined into a single unit and positioned at different angles, with the binocular camera taking a series of photos for calibration. However, these methods require the binocular camera to take multiple photos, and the distance between the calibration board and the camera is fixed, resulting in insufficient depth information and a certain loss of scaling accuracy. Therefore, improving the calibration efficiency of binocular cameras while ensuring the accuracy of the calibration results has become an urgent technical problem to be solved. Summary of the Invention

[0003] The embodiments of this application provide a calibration method, apparatus, computer-readable medium, and device for a binocular camera, which can at least to some extent improve the calibration efficiency of the binocular camera and ensure the accuracy of the calibration results.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to one aspect of the embodiments of this application, a calibration method for a binocular camera is provided, the method comprising:

[0006] Acquire a first image and a second image captured by the first and second cameras of a binocular camera for the same subject matter. The subject matter includes multiple calibration images with different distance settings and a face image to be verified. The calibration images contain several feature points that are equidistant in both horizontal and vertical directions, and the face image to be verified does not obstruct the calibration images or any pair of calibration images.

[0007] Image recognition is performed on the first captured image and the second captured image to determine the feature point information corresponding to each of the calibration images in the first captured image and the second captured image, respectively.

[0008] Based on the feature point information corresponding to each calibration image in the first captured image and the second captured image, the target calibration information corresponding to the binocular camera is determined. The target calibration information includes single-target calibration information and binocular stereo calibration information.

[0009] Based on the target calibration information, and using the Bouguet algorithm, the first transformation matrix and the second transformation matrix are determined respectively from the first captured image and the second captured image to the coplanar row alignment plane;

[0010] Based on the first transformation matrix and the second transformation matrix, depth recognition is performed on the face image to be detected contained in the first captured image and the second captured image to determine the depth information corresponding to the face image to be detected;

[0011] The depth information is compared with the actual distance between the face image to be detected and the binocular camera. If the comparison result meets the predetermined rules, the calibration result is determined to be valid, and the target calibration information, the first transformation matrix, and the second transformation matrix are stored together.

[0012] In one aspect of this application, a calibration device for a binocular camera is provided, the device comprising:

[0013] The acquisition module is used to acquire a first image and a second image captured by the first and second cameras of the binocular camera for the same shooting content. The shooting content includes face images to be verified at different distances and multiple calibration images. The calibration images contain several feature points that are equidistant in the horizontal and vertical directions, and the face images to be verified and the calibration images do not obstruct each other or between any two of the calibration images.

[0014] The first determining module is used to perform image recognition on the first captured image and the second captured image, and respectively determine the feature point information corresponding to each of the calibration images in the first captured image and the second captured image;

[0015] The second determining module is used to determine the target calibration information corresponding to the binocular camera based on the feature point information corresponding to each calibration image in the first captured image and the second captured image. The target calibration information includes single-target calibration information and binocular stereo calibration information.

[0016] The third determining module is used to determine, based on the target calibration information and the Bouguet algorithm, the first transformation matrix and the second transformation matrix from the first captured image and the second captured image to the coplanar row alignment plane, respectively.

[0017] The fourth determining module is used to perform depth recognition on the face image to be detected contained in the first captured image and the second captured image based on the first transformation matrix and the second transformation matrix, and determine the depth information corresponding to the face image to be detected.

[0018] The processing module is used to compare the depth information with the actual distance between the face image to be detected and the binocular camera. If the comparison result meets the predetermined rules, the calibration result is determined to be valid, and the target calibration information, the first transformation matrix and the second transformation matrix are associated and stored.

[0019] According to one aspect of the embodiments of this application, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the binocular camera calibration method as described in the above embodiments.

[0020] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the binocular camera calibration method as described in the above embodiments.

[0021] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the binocular camera calibration method provided in the above embodiments.

[0022] In some embodiments of this application, the technical solutions involve acquiring first and second images captured by a first and second camera in a binocular camera, respectively, of the same subject matter. The subject matter includes multiple calibration images at different distances and a face image to be verified. The calibration images contain several equidistant feature points, and the face image to be verified and the calibration images do not obstruct each other, nor do the calibration images obstruct each other. Image recognition is performed on the first and second images to determine the feature point information corresponding to each calibration image in the first and second images. Then, the target calibration information corresponding to the binocular camera is determined. Based on this calibration information and the Bouguet algorithm, a first transformation matrix and a second transformation matrix are determined from the first and second images to the coplanar alignment plane. According to these first and second transformation matrices, depth recognition is performed on the face image to be detected in the first and second images to determine the depth information corresponding to the face image to be detected. This depth information is then compared with the actual distance between the face image to be detected and the binocular camera to determine whether the calibration result is valid. Therefore, by using multiple calibration images set at different distances, the depth information required for calibration can be obtained. Furthermore, the binocular camera only needs to take one photo to achieve calibration and verification, which improves calibration efficiency while ensuring the accuracy of calibration results.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0025] Figure 1 A flowchart illustrating a calibration method for a binocular camera according to an embodiment of this application is shown;

[0026] Figure 2 A schematic diagram of a first captured image and a second captured image according to an embodiment of this application is shown;

[0027] Figure 3 A block diagram of a calibration device for a binocular camera according to one embodiment of this application is shown;

[0028] Figure 4A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0029] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0030] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0031] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0032] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0033] Figure 1 A flowchart illustrating a binocular camera calibration method according to an embodiment of this application is shown. This method can be applied to terminal devices, including but not limited to one or more of smartphones, tablets, laptops, and desktop computers; it can also be applied to servers, such as physical servers or cloud servers, etc., and this application does not impose any special limitations thereon.

[0034] Reference Figure 1 As shown, the calibration method for this binocular camera includes at least steps S110 to S160, which are described in detail below:

[0035] In step S110, a first image and a second image captured by the first and second cameras of the binocular camera for the same subject are obtained. The subject includes multiple calibration images with different distance settings and a face image to be verified. The calibration images contain several feature points that are equidistant in the horizontal and vertical directions, and the face image to be verified and the calibration images do not obstruct each other.

[0036] The calibration image can be a checkerboard image or a dot matrix image with several feature points, wherein the horizontal and vertical distances between the feature points are equal.

[0037] In one embodiment, this application also provides a calibration device, which includes a module placement platform, multiple calibration images of different sizes, a face image to be detected, and a support. During calibration, the face image to be detected and the multiple calibration images can be placed at different distances on the module placement platform. It should be understood that the different distances mentioned in this application are different from the distances between the two-lens cameras and the plane they occupy. In one example, the number of calibration images can be four. Since the focal length of a face recognition module is generally around 60cm, and the effective recognition distance is 30cm to 150cm, the placement distances of the four calibration images can be 30cm, 70cm, 110cm, and 150cm, respectively, and the face image to be detected can be placed at 90cm. Furthermore, the face image to be detected and the calibration images do not obstruct each other, nor do the calibration images obstruct each other. For example, the four calibration images can be arranged in a four-square grid, and the face image to be detected can be placed in the middle of the four calibration images, etc.

[0038] The first and second cameras in the binocular camera system can capture images of the face to be detected and the calibration images, respectively, to obtain the corresponding first and second images. It should be understood that the first and second images should each contain the face to be detected and multiple calibration images.

[0039] In step S120, image recognition is performed on the first captured image and the second captured image to determine the feature point information corresponding to each of the calibration images in the first captured image and the second captured image, respectively.

[0040] In this embodiment, image recognition can be performed on the first captured image and the second captured image to determine the feature point information corresponding to each calibration image in the first captured image and the second captured image.

[0041] In one embodiment of this application, the calibration image can be a checkerboard image, and the corner points of the checkerboard image are the feature points.

[0042] Then, image recognition is performed on the first captured image and the second captured image to determine the feature point information corresponding to each of the calibration images in the first captured image and the second captured image, including:

[0043] Image recognition is performed on the regions where each calibration image is located in the first captured image and the second captured image to determine the number of corner points and the coordinates of each corner point corresponding to each calibration image in the first captured image and the second captured image, which are used as feature point information.

[0044] Taking multiple calibration images arranged in a four-grid layout as an example (e.g.) Figure 2 As shown, the captured image can be divided into four grids according to the center point, and the four calibration images are displayed completely in different grids. The face image to be detected is placed in the middle so as not to obscure the corners of the calibration images.

[0045] At this point, image recognition is performed on each grid cell to identify the number and coordinates of corner points within each cell (i.e., each calibrated image). The corner points can be numbered and sorted according to a matrix to obtain the corresponding corner point sequence. Thus, a single captured image can yield four corner point sequences. Both the first and second captured images can be processed in the manner described above.

[0046] In one example, after performing image recognition on the regions where each calibration image is located in the first captured image and the second captured image, and determining the number of corner points and their coordinates corresponding to each calibration image in the first captured image and the second captured image as feature point information, the method further includes:

[0047] The number of corner points corresponding to the same calibration image in the first captured image and the second captured image are compared. If the number of corner points corresponding to the same calibration image is different, the images are captured again by the first camera and the second camera.

[0048] In this embodiment, to ensure the accuracy of subsequent calibration results, if the number of corner points corresponding to the same calibration image is different in two captured images, the image can be recaptured after adjustment. For example, whether the image of the face to be detected occludes the calibration image, or whether the camera has been properly focused.

[0049] In step S130, target calibration information corresponding to the binocular camera is determined based on the feature point information corresponding to each calibration image in the first captured image and the second captured image. The target calibration information includes single-target calibration information and binocular stereo calibration information.

[0050] In one embodiment, target calibration information can be obtained by calibrating the binocular camera based on the feature point information corresponding to each calibration image in the first captured image and the second captured image. The target calibration information may include single-target calibration information and binocular stereo calibration information.

[0051] Specifically, based on the feature point information corresponding to each calibration image in the first and second captured images, and using the Zhang Zhengyou calibration method, single-target calibration can be performed on the first and second cameras to determine the corresponding intrinsic parameter matrix, extrinsic parameter matrix, and radial distortion parameter of the first and second cameras, so as to obtain single-target calibration information.

[0052] Based on the intrinsic and extrinsic parameter matrices and radial distortion parameters of each camera, the first and second cameras are then subjected to binocular stereo calibration to determine the relative relationship between the coordinate systems of the first and second cameras. This relative relationship includes the rotation and translation matrices between the two cameras, thereby obtaining the binocular stereo calibration information.

[0053] Specifically, the relative relationship between the coordinate systems of the two cameras (the first camera and the second camera) can be described using the rotation matrix R and the translation matrix T, as follows:

[0054] Establish a world coordinate system using the left camera. Assume there is a point P in space, whose coordinates in the world coordinate system are P_t. w Its coordinates in the left and right camera coordinate systems can be expressed as follows:

[0055] P l =R l P w +T l (1)

[0056] P r =R r P w +T r (2)

[0057] Based on equation (1), we can deduce... Substituting into equation (2), we get:

[0058]

[0059] According to equation (3):

[0060]

[0061]

[0062] Among them, R l T lLet R be the rotation matrix and translation vector relative to the calibration object obtained by single-target calibration using the left camera. l T l Let R be the rotation matrix and translation vector relative to the calibration object obtained by single-target calibration using the right camera. By performing single-target calibration on both the left and right cameras respectively, R can be obtained. l T l R l T l Substituting these values ​​into the above equation, we can calculate the rotation matrix R and translation T between the left and right cameras.

[0063] In step S140, based on the target calibration information and the Bouguet algorithm, the first transformation matrix and the second transformation matrix from the first captured image and the second captured image to the coplanar row alignment plane are determined respectively.

[0064] In this embodiment, based on the obtained target calibration information, the first camera and the second camera are subjected to binocular stereo correction using the Bouguet algorithm, and the first transformation matrix and the second transformation matrix from the first captured image and the second captured image to the coplanar alignment plane are calculated.

[0065] Bouguet's method involves decomposing the rotation matrix R and translation matrix T into rotation and translation matrices R that rotate the left and right cameras by half. l T l R l T l The principle of decomposition is to minimize the distortion caused by the reprojection of the left and right images and maximize the common area of ​​the left and right views.

[0066] Specifically, the rotation matrix of the right image plane relative to the left image plane is decomposed into two matrices R. l and R r This is called the composite rotation matrix of the left and right cameras.

[0067]

[0068]

[0069] in, yes The inverse matrix.

[0070] Rotate each of the left and right cameras by half until their optical axes are parallel. At this point, the imaging planes of the left and right cameras are parallel, but the baselines are not parallel to the imaging plane.

[0071] Construct the transformation moment matrix R rect This ensures the baseline is parallel to the imaging plane. This is achieved using the offset matrix T of the right camera relative to the left camera.

[0072] Constructing e1, the transformation matrix transforms the pole of the left view to infinity, making the epipolar line horizontal. Therefore, the translation vector between the projection centers of the left and right cameras is the direction of the left pole.

[0073]

[0074] The direction e2 is orthogonal to the principal optical axis, along the image direction, and perpendicular to e1. ​​Therefore, the direction e2 can be obtained by taking the cross product of e1 and the principal optical axis and normalizing the result.

[0075]

[0076] Having obtained e1 and e2, e3 is orthogonal to both e1 and e2, so e3 is naturally the cross product of the two:

[0077] e3 = e1 × e2

[0078] This allows the pole of the left camera to be transformed to matrix R at infinity. rect ,as follows:

[0079]

[0080] The overall rotation matrix of the left and right cameras is obtained by multiplying the synthesis rotation matrix and the transformation matrix. Multiplying the coordinate systems of the left and right cameras by their respective overall rotation matrices ensures that the principal optical axes of the left and right cameras are parallel, and that the image plane is parallel to the baseline. Using these two overall rotation matrices, an ideally parallel binocular stereo image can be obtained.

[0081] E l =R rect R l ;

[0082] E r =R rect R r .

[0083] In step S150, depth recognition is performed on the face image to be detected contained in the first captured image and the second captured image according to the first transformation matrix and the second transformation matrix to determine the depth information corresponding to the face image to be detected.

[0084] In this embodiment, based on the first transformation matrix and the second transformation matrix, face detection and depth recognition can be performed on the face images to be detected contained in the first and second captured images to determine the depth information corresponding to the face images to be detected.

[0085] Specifically, face detection can be performed on the first and second captured images to identify the first coordinate information of the face contour markers corresponding to each captured image. For example, each face can be identified with 68 face contour markers, and the coordinate information of each face contour marker can be determined as the first coordinate information.

[0086] The first coordinate information of the face contour markers corresponding to the first and second captured images is then transformed to a coplanar alignment plane through the first and second transformation matrices, respectively, to determine the second coordinate information of the face contour markers corresponding to the first and second captured images.

[0087] Furthermore, based on the coordinate difference in the X-direction between the second coordinate information of the same face contour marker point, the depth information corresponding to each face contour marker point is determined. Specifically, the depth information Z = Bf / d, where B is the center distance between the first and second cameras, also called the baseline distance, which is known; f is the camera focal length, which can be calculated from the camera's intrinsic parameter matrix; and d = X... l -X r This refers to the difference in the X-values ​​of the second coordinate information of the same facial contour marker point. Similarly, the depth information corresponding to each facial contour marker point can be calculated.

[0088] Based on the depth information of each facial contour marker point, the target depth information of the face to be detected can be determined. For example, the mean of all depth information can be calculated as the target depth information of the face image to be detected, or the median of all depth information can be selected as the target depth information of the face image to be detected, or a depth information can be randomly selected as the target depth information corresponding to the face image to be detected, and so on. Those skilled in the art can choose the corresponding determination method according to the actual implementation needs, and this application does not make any special limitations in this regard.

[0089] Please continue to refer to this. Figure 1 In step S160, the depth information is compared with the actual distance between the face image to be detected and the binocular camera. If the comparison result meets the predetermined rules, the calibration result is determined to be valid, and the target calibration information, the first transformation matrix and the second transformation matrix are associated and stored.

[0090] In this embodiment, after determining the target depth information corresponding to the face image to be detected, the target depth information can be compared with the actual distance between the face image to be detected and the binocular camera. Those skilled in the art can pre-set predetermined rules based on prior experience to determine whether the comparison result meets actual usage requirements. For example, the predetermined rules could be that the target depth information is equal to the actual distance, or that there can be a certain error between the target depth information and the actual distance.

[0091] Therefore, if the comparison result meets the predetermined rules, it means that the calibration result is accurate. So the target calibration information, the first transformation matrix, and the second transformation matrix can be associated and stored for later use. If the comparison result does not meet the predetermined rules, it means that the calibration result is poor and can be recalibrated after adjustment.

[0092] Therefore, based on Figure 1 The illustrated embodiment acquires first and second images captured by the first and second cameras of a binocular camera system, respectively, of the same subject matter. The subject matter includes face images to be verified at different distances and multiple calibration images. Each calibration image contains several equidistant feature points, and the face images to be verified and the calibration images do not obstruct each other. Image recognition is performed on the first and second images to determine the feature point information corresponding to each calibration image in the first and second images. Then, the target calibration information corresponding to the binocular camera is determined. Based on this calibration information and the Bouguet algorithm, a first transformation matrix and a second transformation matrix are determined from the first and second images to the coplanar alignment plane. According to these transformation matrices, depth recognition is performed on the face images to be detected in the first and second images to determine the depth information corresponding to the face images to be detected. This depth information is then compared with the actual distance between the face images to be detected and the binocular camera system to determine whether the calibration result is valid. Therefore, by using multiple calibration images set at different distances, the depth information required for calibration can be obtained. Furthermore, the binocular camera only needs to take one photo to achieve calibration and verification, which improves calibration efficiency while ensuring the accuracy of calibration results.

[0093] The binocular camera calibration method provided in this application only requires one shot to complete the camera parameter calibration and calibration verification. The calibration image and the face image to be detected are fixed, the camera does not need to move, and no motion error is introduced. Furthermore, the distance between the multiple calibration images and the camera is different, resulting in higher calibration accuracy. The depth information of the face at different distances can be well reconstructed, ensuring the accuracy of the calibration effect.

[0094] The following describes an embodiment of the apparatus described in this application, which can be used to execute the binocular camera calibration method described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the binocular camera calibration method described above in this application.

[0095] Figure 3 A block diagram of a calibration device for a binocular camera according to one embodiment of this application is shown.

[0096] Reference Figure 3 As shown, a binocular camera calibration device according to an embodiment of this application includes:

[0097] The acquisition module 310 is used to acquire a first image and a second image captured by the first camera and the second camera in the binocular camera for the same shooting content. The shooting content includes multiple calibration images with different distance settings and a face image to be verified. The calibration image contains several feature points that are equidistant in the horizontal and vertical directions, and the face image to be verified and the calibration image do not obstruct each other or between any two of the calibration images.

[0098] The first determining module 320 is used to perform image recognition on the first captured image and the second captured image, and respectively determine the feature point information corresponding to each of the calibration images in the first captured image and the second captured image;

[0099] The second determining module 330 is used to determine the target calibration information corresponding to the binocular camera based on the feature point information corresponding to each calibration image in the first captured image and the second captured image. The target calibration information includes single-target calibration information and binocular stereo calibration information.

[0100] The third determining module 340 is used to determine, based on the target calibration information and the Bouguet algorithm, the first transformation matrix and the second transformation matrix from the first captured image and the second captured image to the coplanar row alignment plane, respectively.

[0101] The fourth determining module 350 is used to perform depth recognition on the face image to be detected contained in the first captured image and the second captured image according to the first transformation matrix and the second transformation matrix, and determine the depth information corresponding to the face image to be detected.

[0102] The processing module 360 ​​is used to compare the depth information with the actual distance between the face image to be detected and the binocular camera. If the comparison result meets the predetermined rules, the calibration result is determined to be valid, and the target calibration information, the first transformation matrix and the second transformation matrix are associated and stored.

[0103] In one embodiment of this application, the second determining module 330 is configured to: perform single-target calibration on the first camera and the second camera based on the feature point information corresponding to each calibration image in the first captured image and the second captured image, respectively, and determine the intrinsic parameter matrix, extrinsic parameter matrix and radial distortion parameter corresponding to each of the first camera and the second camera; perform binocular stereo calibration on the first camera and the second camera based on the intrinsic parameter matrix, extrinsic parameter matrix and radial distortion parameter corresponding to each of the first camera and the second camera, and determine the relative relationship between the coordinate systems of the first camera and the second camera, wherein the relative relationship includes the rotation matrix and translation matrix between the two.

[0104] In one embodiment of this application, the fourth determining module 350 is configured to: perform face detection on the first captured image and the second captured image respectively to obtain the first coordinate information of the face contour marker points corresponding to the first captured image and the second captured image respectively; transform the first coordinate information of the face contour marker points corresponding to the first captured image and the second captured image respectively to a coplanar row alignment plane through the first transformation matrix and the second transformation matrix respectively to determine the second coordinate information of the face contour marker points corresponding to the first captured image and the second captured image respectively; determine the depth information corresponding to each face contour marker point according to the coordinate difference in the X direction between the second coordinate information of the same face contour marker point; and determine the target depth information of the face image to be detected according to the depth information corresponding to each face contour marker point.

[0105] In one embodiment of this application, the fourth determining module 350 is used to: determine the mean of all depth information based on the depth information corresponding to each of the face contour marker points as the target depth information of the face image to be detected.

[0106] In one embodiment of this application, the calibration image is a checkerboard image, and the corner points of the checkerboard image are the feature points; the first determining module 320 is used to: perform image recognition on the regions where each calibration image is located in the first captured image and the second captured image, and determine the number of corner points and the coordinates of the corner points corresponding to each calibration image in the first captured image and the second captured image as feature point information.

[0107] In one embodiment of this application, after performing image recognition on the regions where each calibration image is located in the first captured image and the second captured image to determine the number of corner points and the corner point coordinates corresponding to each calibration image in the first captured image and the second captured image as feature point information, the acquisition module 310 is further configured to: compare the number of corner points corresponding to the same calibration image in the first captured image and the second captured image; if there are different numbers of corner points corresponding to the same calibration image, then recapture the image through the first camera and the second camera.

[0108] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0109] It should be noted that, Figure 4 The computer system of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0110] like Figure 4 As shown, the computer system includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 402 or programs loaded from storage portion 408 into Random Access Memory (RAM) 403, such as performing the methods described in the above embodiments. The RAM 403 also stores various programs and data required for system operation. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An Input / Output (I / O) interface 405 is also connected to the bus 404.

[0111] The following components are connected to I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to I / O interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 410 as needed so that computer programs read from it can be installed into storage section 408 as needed.

[0112] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs various functions defined in the system of this application.

[0113] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0115] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0116] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0117] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0118] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0119] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0120] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A calibration method for a binocular camera, characterized in that, The method includes: Acquire a first image and a second image captured by the first and second cameras of a binocular camera for the same subject matter. The subject matter includes multiple calibration images with different distance settings and a face image to be verified. The calibration images contain several feature points that are equidistant in both horizontal and vertical directions, and the face image to be verified does not obstruct the calibration images or any pair of calibration images. Image recognition is performed on the first captured image and the second captured image to determine the feature point information corresponding to each of the calibration images in the first captured image and the second captured image, respectively. Based on the feature point information corresponding to each calibration image in the first captured image and the second captured image, the target calibration information corresponding to the binocular camera is determined. The target calibration information includes single-target calibration information and binocular stereo calibration information. Based on the target calibration information, and using the Bouguet algorithm, the first transformation matrix and the second transformation matrix are determined respectively from the first captured image and the second captured image to the coplanar row alignment plane; Based on the first transformation matrix and the second transformation matrix, depth recognition is performed on the face image to be detected contained in the first captured image and the second captured image to determine the depth information corresponding to the face image to be detected; The depth information is compared with the actual distance between the face image to be detected and the binocular camera. If the comparison result meets the predetermined rules, the calibration result is determined to be valid, and the target calibration information, the first transformation matrix and the second transformation matrix are stored together. Specifically, based on the first transformation matrix and the second transformation matrix, depth recognition is performed on the face image to be detected contained in the first captured image and the second captured image to determine the depth information corresponding to the face image to be detected, including: Face detection is performed on the first captured image and the second captured image respectively to obtain the first coordinate information of the face contour marker points corresponding to the first captured image and the second captured image respectively; The first coordinate information of the face contour marker points corresponding to the first captured image and the second captured image are transformed to the coplanar row alignment plane through the first transformation matrix and the second transformation matrix, respectively, to determine the second coordinate information of the face contour marker points corresponding to the first captured image and the second captured image. Based on the coordinate difference in the X direction between the second coordinate information of the same face contour marker point, the depth information corresponding to each face contour marker point is determined; The target depth information of the face image to be detected is determined based on the depth information corresponding to each of the face contour marker points.

2. The method according to claim 1, characterized in that, Based on the feature point information corresponding to each calibration image in the first captured image and the second captured image, the target calibration information corresponding to the binocular camera is determined, including: Based on the feature point information corresponding to each calibration image in the first captured image and the second captured image, single-target calibration is performed on the first camera and the second camera to determine the intrinsic parameter matrix, extrinsic parameter matrix and radial distortion parameter corresponding to the first camera and the second camera, respectively. Based on the intrinsic parameter matrix, extrinsic parameter matrix, and radial distortion parameter of the first camera and the second camera respectively, a binocular stereo calibration is performed on the first camera and the second camera to determine the relative relationship between the coordinate systems of the first camera and the second camera. The relative relationship includes the rotation matrix and translation matrix between the two.

3. The method according to claim 2, characterized in that, Based on the depth information corresponding to each of the aforementioned facial contour marker points, the target depth information of the face image to be detected is determined, including: Based on the depth information corresponding to each of the facial contour marker points, the mean of all depth information is determined as the target depth information of the face image to be detected.

4. The method according to claim 1, characterized in that, The calibration image is a checkerboard image, and the corner points of the checkerboard image are the feature points; Then, image recognition is performed on the first captured image and the second captured image to determine the feature point information corresponding to each of the calibration images in the first captured image and the second captured image, including: Image recognition is performed on the regions where each calibration image is located in the first captured image and the second captured image to determine the number of corner points and the coordinates of each corner point corresponding to each calibration image in the first captured image and the second captured image, which are used as feature point information.

5. The method according to claim 4, characterized in that, After performing image recognition on the regions where each calibration image is located in the first captured image and the second captured image, and determining the number of corner points and their coordinates corresponding to each calibration image in the first captured image and the second captured image as feature point information, the method further includes: The number of corner points corresponding to the same calibration image in the first captured image and the second captured image are compared. If the number of corner points corresponding to the same calibration image is different, the images are captured again by the first camera and the second camera.

6. A calibration device for a binocular camera, characterized in that, The device includes: The acquisition module is used to acquire a first image and a second image captured by the first and second cameras of the binocular camera for the same shooting content. The shooting content includes multiple calibration images with different distance settings and a face image to be verified. The calibration image contains several feature points that are equidistant in the horizontal and vertical directions, and the face image to be verified and the calibration image do not obstruct each other or between any two of the calibration images. The first determining module is used to perform image recognition on the first captured image and the second captured image, and respectively determine the feature point information corresponding to each of the calibration images in the first captured image and the second captured image; The second determining module is used to determine the target calibration information corresponding to the binocular camera based on the feature point information corresponding to each calibration image in the first captured image and the second captured image. The target calibration information includes single-target calibration information and binocular stereo calibration information. The third determining module is used to determine, based on the target calibration information and the Bouguet algorithm, the first transformation matrix and the second transformation matrix from the first captured image and the second captured image to the coplanar row alignment plane, respectively. The fourth determining module is used to perform depth recognition on the face image to be detected contained in the first captured image and the second captured image based on the first transformation matrix and the second transformation matrix, and determine the depth information corresponding to the face image to be detected. The processing module is used to compare the depth information with the actual distance between the face image to be detected and the binocular camera. If the comparison result meets the predetermined rules, the calibration result is determined to be valid, and the target calibration information, the first transformation matrix and the second transformation matrix are associated and stored. The fourth determining module is also used for, Face detection is performed on the first captured image and the second captured image respectively to obtain the first coordinate information of the face contour marker points corresponding to the first captured image and the second captured image respectively; The first coordinate information of the face contour marker points corresponding to the first captured image and the second captured image are transformed to the coplanar row alignment plane through the first transformation matrix and the second transformation matrix, respectively, to determine the second coordinate information of the face contour marker points corresponding to the first captured image and the second captured image. Based on the coordinate difference in the X direction between the second coordinate information of the same face contour marker point, the depth information corresponding to each face contour marker point is determined; The target depth information of the face image to be detected is determined based on the depth information corresponding to each of the face contour marker points.

7. The apparatus according to claim 6, characterized in that, The second determining module is used for: Based on the feature point information corresponding to each calibration image in the first captured image and the second captured image, single-target calibration is performed on the first camera and the second camera to determine the intrinsic parameter matrix, extrinsic parameter matrix and radial distortion parameter corresponding to the first camera and the second camera respectively. Based on the intrinsic parameter matrix, extrinsic parameter matrix, and radial distortion parameter of the first camera and the second camera respectively, a binocular stereo calibration is performed on the first camera and the second camera to determine the relative relationship between the coordinate systems of the first camera and the second camera. The relative relationship includes the rotation matrix and translation matrix between the two.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the calibration method for the binocular camera as described in any one of claims 1 to 5.

9. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the calibration method for a binocular camera as described in any one of claims 1 to 5.

Images