A method and device for calibration based on graphic card photographing

Abstract

The objective of this application is to provide a method and device for calibration based on graphic card shooting. This application obtains a picture taken by using the camera of a shooting device to align with a graphic card, and sets at least one group of identification and positioning points in the picture; determines the three-dimensional coordinate information of the at least one group of identification and positioning points in the graphic card coordinate system and the planar coordinate information in the picture pixels; determines the internal parameters of the camera based on the distance between the camera and the graphic card, the three-dimensional coordinate information of the at least one group of identification and positioning points in the graphic card coordinate system, and the planar coordinate information in the picture pixels; determines the rotation angle and movement coordinates of the camera based on the internal parameters of the camera, and determines the shooting distance between the camera and the graphic card during the next shooting according to the rotation angle and movement coordinates. It can keep the camera always aligned with the shooting plane, can control the alignment accuracy during shooting, and improve the calibration accuracy of shooting.

Description

Technical Field

[0001] This application relates to the field of computers, and in particular, to a method and device for camera shooting calibration based on a pattern card. Background Art

[0002] Currently, the common method for aligning a pattern card during shooting is mainly through manual adjustment. By visually observing, a relatively aligned position is found, and after recording the position, the photo is taken at a fixed position. Additionally, there is also a method of identifying specific color areas on the pattern card and calculating the relative position information between the camera and the pattern card using the centroid coordinates of the contour. Usually, technical parameters such as the focal length or field of view angle need to be set for estimation. Manual adjustment of the alignment position has insufficient accuracy, and it is difficult to adjust the pattern card to the centered and aligned position in the field of view. Through color recognition, relevant professional parameters need to be set, and it is not easy to obtain some device parameters. Moreover, color recognition is easily affected by objects of the same color in the field of view, making it inconvenient to operate and the calculation accuracy is not high enough. Summary of the Invention

[0003] An object of this application is to provide a method and device for camera shooting calibration based on a pattern card, which solves the problems of insufficient accuracy during manual adjustment and inconvenient operation and insufficient calculation accuracy during color recognition in the prior art.

[0004] According to one aspect of this application, a method for camera shooting calibration based on a pattern card is provided. The method includes:

[0005] Obtain a picture taken by aligning a camera of a shooting device with a pattern card, and set at least one set of recognition and positioning points in the picture;

[0006] Determine the three-dimensional coordinate information of the at least one set of recognition and positioning points in the pattern card coordinate system and the planar coordinate information in the picture pixels;

[0007] Determine the internal parameters of the camera according to the distance between the camera and the pattern card, the three-dimensional coordinate information of the at least one set of recognition and positioning points in the pattern card coordinate system, and the planar coordinate information in the picture pixels;

[0008] Based on the internal parameters of the camera, determine the rotation angle and movement coordinates of the camera, and determine the shooting distance between the camera and the pattern card for the next shooting according to the rotation angle and movement coordinates.

[0009] Optionally, the pattern card includes a regular dot pattern card or a checkerboard pattern card, and the method includes:

[0010] Establish a three-dimensional coordinate system on the pattern card according to the point distance on the regular dot pattern card or the side length of the checkerboard on the checkerboard pattern card.

[0011] Optionally, determining the camera internal parameters of the camera based on the distance between the camera and the image card, the three-dimensional coordinate information of the at least one set of identification and positioning points in the image card coordinate system, and the planar coordinate information in the picture pixels, includes:

[0012] Obtaining the horizontal direction coordinate information and vertical direction coordinate information of each identification and positioning point in the image card from the three-dimensional coordinate information of the at least one set of identification and positioning points in the image card coordinate system;

[0013] Obtaining the picture pixel width, picture pixel height, horizontal direction pixel coordinate information, and vertical direction pixel coordinate information of each identification and positioning point in the picture from the planar coordinate information of the at least one set of identification and positioning points in the picture pixels;

[0014] Determining the horizontal pixel information of the camera focal length based on the horizontal direction pixel coordinate information, horizontal direction coordinate information, and the distance between the camera and the image card;

[0015] Determining the vertical pixel information of the camera focal length based on the vertical direction pixel coordinate information, vertical direction coordinate information, and the distance between the camera and the image card;

[0016] Determining the horizontal coordinate of the principal point of the camera in pixels according to the picture pixel width of each identification and positioning point, and determining the vertical coordinate of the principal point of the camera in pixels according to the picture pixel height of each identification and positioning point.

[0017] Optionally, determining the rotation angle and movement coordinates of the camera based on the camera internal parameters of the camera, includes:

[0018] Establishing a camera internal parameter matrix based on the camera internal parameters of the camera, and determining the pose matrix diagram of the camera according to the camera internal parameter matrix;

[0019] Determining the three-dimensional coordinate information of the angular rotation of the camera based on the pose matrix diagram of the camera;

[0020] Determining the rotation angle and movement coordinates of the camera according to the three-dimensional coordinate information of the angular rotation of the camera, the three-dimensional coordinate information of the at least one set of identification and positioning points in the image card coordinate system, and the planar coordinate information in the picture pixels.

[0021] Optionally, establishing a camera internal parameter matrix based on the camera internal parameters of the camera, includes:

[0022] Establishing a camera internal parameter matrix according to the horizontal pixel information of the camera focal length, the vertical pixel information of the camera focal length, the horizontal coordinate of the principal point of the camera in pixels, and the vertical coordinate of the principal point of the camera in pixels, wherein the camera internal parameter matrix satisfies the following form:

[0023]

[0024] Among them, fx represents the horizontal pixel information of the camera focal length, fy represents the vertical pixel information of the camera focal length, cx represents the abscissa of the principal point pixel of the camera, and cy represents the ordinate of the principal point pixel of the camera.

[0025] Optionally, the photographing device is fixed on a robotic arm with an interface, and the method includes:

[0026] Input the rotation angle, movement coordinates of the camera, and the shooting distance between the camera and the figure card during the next shooting into the interface, so that the robotic arm moves the photographing device to the corresponding position.

[0027] Optionally, at least one set of recognition positioning points is set in the picture, including:

[0028] Training all the recognition points in the picture using a training model;

[0029] Selecting at least one set of recognition positioning points from all the recognition points.

[0030] Optionally, training all the recognition points in the picture using a training model includes:

[0031] Determining positive samples and negative samples, where the positive samples include the recognition point marks to be recognized, and the negative samples are obtained from the environment where the positive samples are located;

[0032] Normalizing the positive samples and placing them in the positive sample folder under the root file, and directly placing the negative samples in the negative sample folder;

[0033] Creating a positive sample editing document in the positive sample folder and a negative sample editing document in the negative sample folder;

[0034] Creating a classifier folder under the root file, calling a training execution program to train the positive sample editing document and the negative sample editing document to obtain a training model, and storing the training model under the classifier folder;

[0035] Inputting the picture into the training model under the classifier folder, and outputting all the recognition points in the picture.

[0036] According to another aspect of the present application, a device based on figure card shooting correction is provided. The device includes:

[0037] A setting device, configured to obtain a picture taken by aligning the camera of the photographing device with a figure card, and set at least one set of recognition positioning points in the picture;

[0038] A determination device for determining the three-dimensional coordinate information of the at least one set of identification and positioning points in the coordinate system of the chart card and the planar coordinate information in the picture pixels;

[0039] A calculation device for determining the internal parameters of the camera according to the distance between the camera and the chart card, the three-dimensional coordinate information of the at least one set of identification and positioning points in the coordinate system of the chart card, and the planar coordinate information in the picture pixels;

[0040] A calibration device for determining the rotation angle and movement coordinates of the camera based on the internal parameters of the camera, and determining the shooting distance between the camera and the chart card during the next shooting according to the rotation angle and movement coordinates.

[0041] According to another aspect of the present application, there is also provided a device based on chart card shooting calibration, and the device includes:

[0042] One or more processors; and

[0043] A memory storing computer-readable instructions, and when the computer-readable instructions are executed, the processor executes the operations of the method as described above.

[0044] According to still another aspect of the present application, there is also provided a computer-readable medium, on which computer-readable instructions are stored, and the computer-readable instructions can be executed by a processor to implement the method as described above.

[0045] Compared with the prior art, the present application obtains a picture of using a camera of a shooting device to align with a chart card for shooting, sets at least one set of identification and positioning points in the picture; determines the three-dimensional coordinate information of the at least one set of identification and positioning points in the coordinate system of the chart card and the planar coordinate information in the picture pixels; determines the internal parameters of the camera according to the distance between the camera and the chart card, the three-dimensional coordinate information of the at least one set of identification and positioning points in the coordinate system of the chart card, and the planar coordinate information in the picture pixels; determines the rotation angle and movement coordinates of the camera based on the internal parameters of the camera, and determines the shooting distance between the camera and the chart card during the next shooting according to the rotation angle and movement coordinates. It can make the camera always remain aligned with the shooting plane during the image test process, can control the alignment accuracy during shooting, correct the position during the next shooting according to the parameters calculated after each current shooting, thereby improving the calibration accuracy of shooting and the accuracy of test results. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present application will become more obvious:

[0047] Figure 1Schematic diagram of a method for calibration based on figure card shooting provided according to an aspect of the present application;

[0048] Figure 2 Schematic diagram of the installation of a shooting device and a figure card in an embodiment of the present application;

[0049] Figure 3 Schematic diagram of the structure of a device for calibration based on figure card shooting provided according to another aspect of the present application.

[0050] Identical or similar reference numerals in the drawings represent identical or similar components. Detailed Description of the Embodiments

[0051] The present application will be further described in detail below with reference to the drawings.

[0052] In a typical configuration of the present application, the terminal, the device of the service network, and the trusted party all include one or more processors (such as a Central Processing Unit (CPU)), an input / output interface, a network interface, and a memory.

[0053] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of, for example, read only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0054] A computer-readable medium includes permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, Phase-Change RAM (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash memory or other memory technologies, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disk (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer-readable medium does not include transitory media such as modulated data signals and carrier waves.

[0055] Figure 1 A schematic flowchart of a method for camera shooting calibration according to an aspect of the present application is shown. The method includes: steps S11 to S14, where step S11 is to obtain a picture of shooting a graphic card with a camera of a shooting device and set at least one set of recognition and positioning points in the picture; step S12 is to determine the three-dimensional coordinate information of the at least one set of recognition and positioning points in the graphic card coordinate system and the planar coordinate information in the picture pixels; step S13 is to determine the camera internal parameters of the camera according to the distance between the camera and the graphic card, the three-dimensional coordinate information of the at least one set of recognition and positioning points in the graphic card coordinate system, and the planar coordinate information in the picture pixels; step S14 is to determine the rotation angle and movement coordinates of the camera based on the camera internal parameters of the camera, and determine the shooting distance between the camera and the graphic card during the next shooting according to the rotation angle and movement coordinates. Thus, the alignment accuracy during shooting can be improved, and the accuracy of image testing can be improved.

[0056] Specifically, in step S11, a picture taken by using the camera of the shooting device to aim at the chart is obtained, and at least one set of recognition and positioning points is set in the picture. Here, the shooting device is fixed at a fixed position of the robotic arm, then aimed at the chart, and the camera of the shooting device is used to take a picture to obtain the taken picture.

[0057] Specifically, in step S12, the three-dimensional coordinate information of the at least one set of recognition and positioning points in the chart coordinate system and the planar coordinate information in the picture pixels are determined. Here, the used chart has a physical coordinate system, which is the world coordinate system, and this coordinate system is a three-dimensional (3D) coordinate system, where z = 0 is set, and the 3D coordinate information of the marked points in the chart is determined. When using the camera to aim at the chart for shooting, at least one set of recognition and positioning points is determined. The at least one set of recognition and positioning points is within the picture and not too close to the edge of the picture. For example, one set of recognition and positioning points is four points, which are the points at the four corners of the picture respectively. The corresponding marked points on the chart are these four recognition and positioning points. The three-dimensional coordinate information of these four recognition and positioning points on the chart and the pixel information in the taken picture are determined respectively. This pixel information is the planar coordinate information of the picture, that is, a coordinate system of the picture pixels is established, and this coordinate system is a two-dimensional coordinate system, and the coordinate information of the four points in the pixels of the plane is determined to form 2D data. It is also possible to set N groups of four positioning points in the picture to form multiple groups of 2D data. In this application, single pictures or multiple pictures can be used for the alignment of the camera shooting function.

[0058] Specifically, in step S13, the internal parameters of the camera are determined according to the distance between the camera and the chart, the three-dimensional coordinate information of the at least one set of recognition and positioning points in the chart coordinate system, and the planar coordinate information in the picture pixels. Here, according to the shooting distance of the camera, that is, the distance between the camera and the chart, combined with the above-obtained 3D data and 2D data, the internal parameters of the camera are estimated. Thus, in step S14, based on the internal parameters of the camera, the rotation angle and the moving coordinates of the camera are determined, and according to the rotation angle and the moving coordinates, the shooting distance between the camera and the chart during the next shooting is determined. Here, the rotation angle and the moving coordinates of the camera are calculated by using the obtained internal parameters of the camera. This moving coordinate refers to the three-dimensional coordinate of the movement in the world coordinate system, that is, the moving coordinate in the chart coordinate system. Furthermore, the rotation angle and the moving coordinates of the current camera can be used to adjust the shooting distance between the camera and the chart during the next shooting, so as to align the camera and the chart and improve the shooting accuracy.

[0059] In some embodiments of the present application, the pattern card includes a regular dot pattern card or a checkerboard pattern card, and the method includes: establishing a three-dimensional coordinate system on the pattern card according to the dot pitch on the regular dot pattern card or the side length of the checkerboard on the checkerboard pattern card. Here, the pattern card used can be a regular dot pattern or a checkerboard. When establishing the pattern card coordinate system, three-dimensional coordinates are obtained according to the dot pitch of the dot pattern or the side length of the checkerboard, and then the three-dimensional coordinate system is constructed. When multiple pictures are taken, the dots in the dot pattern picture or the corner points of the checkerboard can be recognized to obtain pixel coordinates.

[0060] In some embodiments of the present application, in step S13, the horizontal direction coordinate information and the vertical direction coordinate information of each identification and positioning point in the pattern card are obtained from the three-dimensional coordinate information of the at least one group of identification and positioning points in the pattern card coordinate system; the picture pixel width, the picture pixel height, the horizontal direction pixel coordinate information, and the vertical direction pixel coordinate information of each identification and positioning point in each group are obtained from the plane coordinate information of the at least one group of identification and positioning points in the picture pixels; the horizontal pixel information of the camera focal length is determined according to the horizontal direction pixel coordinate information, the horizontal direction coordinate information, and the distance between the camera and the pattern card; the vertical pixel information of the camera focal length is determined according to the vertical direction pixel coordinate information, the vertical direction coordinate information, and the distance between the camera and the pattern card; the horizontal coordinate of the principal point pixel of the camera is determined according to the picture pixel width of each identification and positioning point, and the vertical coordinate of the principal point pixel of the camera is determined according to the picture pixel height of each identification and positioning point. Here, a single picture or a multi-picture matrix can be used when establishing the camera internal parameter matrix. Before establishing the matrix, the camera internal parameters are first determined. Specifically, the horizontal direction coordinate information and the vertical direction coordinate information of the positioning and identification points in the pattern card are obtained, that is, marker_X (the center distance of the outer identification points in the horizontal direction in the pattern card in mm) and marker_Y (the center distance of the outer identification points in the vertical direction in the pattern card in mm) are obtained. The shooting distance (distance) between the camera and the pattern card is measured. This shooting distance is the distance after the camera moves to the initial position. Pictures are taken at this fixed position, and then the pixel information of the positioning and identification points is obtained, including the picture pixel width, the picture pixel height, the horizontal direction pixel coordinate information, and the vertical direction pixel coordinate information. Among them, cols represents the picture pixel width (in pixels), rows represents the picture pixel height (in pixels), pixel_X represents the center pixel distance of the outer identification points in the horizontal direction of the captured pattern card, and pixel_Y represents the center pixel distance of the outer identification points in the vertical direction of the captured pattern card. Thus, the camera internal parameters are calculated according to this information. The camera internal parameters include the horizontal pixel information of the camera focal length, the vertical pixel information of the camera focal length, the horizontal coordinate of the principal point pixel of the camera, and the vertical coordinate of the principal point pixel of the camera; where:

[0061] fx = pixel_X / marker_X * distance;

[0062] fy = pixel_Y / marker_Y * distance;

[0063] cx = cols / 2;

[0064] cy = rows / 2;

[0065] fx represents how many horizontal pixels the focal length is equivalent to on the imaging plane with focal length f, fy represents how many vertical pixels the focal length is equivalent to on the imaging plane with focal length f; cx represents the pixel abscissa of the principal point, and cy represents the pixel ordinate of the principal point.

[0066] In some embodiments of the present application, in step S14, an intrinsic matrix of the camera is established based on the camera's internal parameters, and a pose matrix diagram of the camera is determined according to the intrinsic matrix of the camera; three-dimensional coordinate information of the angular rotation of the camera is determined based on the pose matrix diagram of the camera; the rotation angle and movement coordinates of the camera are determined according to the three-dimensional coordinate information of the angular rotation of the camera, the three-dimensional coordinate information of the at least one set of recognition and positioning points in the coordinate system of the card, and the plane coordinate information in the picture pixels. Here, an intrinsic matrix of the camera is constructed according to the calculated camera internal parameters, and then the intrinsic matrix is calculated to obtain a pose matrix diagram of the camera. Furthermore, the coordinates of the angular rotation of the camera are obtained using this pose matrix diagram. According to the coordinates of the angular rotation and single or multiple sets of 2D data and 3D data, they are transmitted to the EPnP algorithm, and the angular rotation and movement coordinates of the same angle are output. Specifically, when transmitted to the EPnP algorithm, the function solvePnP can be called to implement it, where

[0067] solvePnP(objPoints, imaPoints, camera_matrix, distortion_coefficents, rvec, tvec, false, CV_EPNP),

[0068] objPoints represents the actual coordinates of the center of the recognized Marker point in the world coordinate system, with the unit of mm; imaPoints: the pixel coordinates of the center of the corresponding Marker point in the image coordinate system, with the unit of pixels. camera_matrix: the intrinsic matrix of the camera. distortion_coefficents: the distortion coefficient of the camera. rvec: the output rotation vector. tvec: the output displacement vector. false: marks whether to output rvec and tvec, with the default being false. CV_EPNP: specifies the method for solving the PnP problem.

[0069] In some embodiments of the present application, an intrinsic matrix of the camera is established based on the horizontal pixel information of the camera focal length, the vertical pixel information of the camera focal length, the abscissa of the principal point pixel of the camera, and the ordinate of the principal point pixel of the camera. Among them, the intrinsic matrix of the camera satisfies the following form:

[0070]

[0071] Among them, fx represents the horizontal pixel information of the camera focal length, fy represents the vertical pixel information of the camera focal length, cx represents the abscissa of the principal point pixel of the camera, and cy represents the ordinate of the principal point pixel of the camera.

[0072] Here, when using the obtained camera intrinsics to construct the intrinsic matrix and the imaging device is a mobile phone, considering that the camera distortion is small, it can be defaulted that the camera distortion is distCoeffD[5] = {0, 0, 0, 0, 0}. At this time, the constructed camera intrinsic matrix is the above cam[9]. When the camera distortion is large, the distortion matrix needs to be used to adjust the initially constructed intrinsic matrix when constructing the intrinsic matrix.

[0073] Specifically, the calculation process can use:

[0074] calibrateCamera(object_points, image_points_seq, image_size, cameraMatrix, distCoeffs, rvecsMat, tvecsMat, 0); Thus, cameraMatrix and distCoeffs (the combination of the two is the camera intrinsics) can be directly obtained.

[0075] Among them, object_points represents the points in the world coordinate system, image_points_seq represents the set of pixel points of the corresponding points in the image, image_size represents the size of the image, cameraMatrix represents the camera intrinsic matrix, distCoeffs represents the camera distortion coefficient, rvec represents the output rotation vector, tvec represents the output displacement vector, and 0 represents the algorithm used for calibration, defaulting to 0.

[0076] In some embodiments of the present application, the photographing device is fixed on a robotic arm with an interface. The method includes: transmitting the rotation angle, movement coordinates of the camera, and the photographing distance between the camera and the graphic card during the next photographing into the interface, so that the robotic arm moves the photographing device to the corresponding position. Here, the obtained rotation angle of the camera, movement coordinates (xyz coordinates), and the new photographing distance are corresponded to the robotic arm that fixes the photographing device. This robotic arm has an interface and can transmit these values into the interface. Then, the robotic arm uses these values to move the photographing device to the corresponding position, thereby timely adjusting the photographing position of the camera, ensuring the accurate alignment between the camera and the graphic card, and being able to control the alignment accuracy during photographing. In a specific embodiment of the present application, use Figure 2 the device shown in the figure to perform photographing calibration of the camera. Among them, the camera platform (1) is used to place the camera. The camera platform includes a robotic arm (2). The robotic arm has a data interface and can receive the calculated information about the camera, and then adjust the spatial position of the camera; the frame (3) is used to install the graphic card. Among them, the frame (3) includes a bracket (31), a mounting frame (32), a driving member (35), and a rotating shaft. The graphic card is detachably arranged on the mounting frame (32); the frame (3) also includes a background board (34) and a storage rack (33). The storage rack (33) is fixedly arranged on the bracket (31), and the graphic card is detachably arranged on the storage rack (33). The storage rack (33) is used to store the graphic card. After using this device to fix the graphic card, establish the actual coordinate system of the graphic card, that is, the world coordinate system. Fix the photographing device on the camera platform and then take a photo. Then, use the method described in the present application to obtain the rotation angle, movement coordinates of the camera of the photographing device, and the photographing distance during the next photo-taking. Then, control the robotic arm to move the photographing device to the corresponding position according to these values. It should be noted that Figure 2 the device shown in the figure is only an example, including but not limited to other devices that can fix the graphic card and the photographing device and have a robotic arm with an interface.

[0077] In some embodiments of the present application, in step S12, use a training model to train all the recognition points in the picture; select at least one group of recognition and positioning points from all the recognition points. Here, it is necessary to set the positioning recognition points on the picture, and these recognition points can be trained by a training model. Then, select one or more groups of recognition points from all the available recognition points obtained by training, and then perform calibration of the photographing function.

[0078] Specifically, the process of training the model to obtain the recognition points is as follows: Determine the positive samples and negative samples, where the positive samples include the recognition point markings to be recognized, and the negative samples are obtained from the environment where the positive samples are located; After normalizing the positive samples, place them in the positive sample folder under the root file, and directly place the negative samples in the negative sample folder; Create a positive sample editing document in the positive sample folder and a negative sample editing document in the negative sample folder; Create a classifier folder under the root file, call the training execution program to train the positive sample editing document and the negative sample editing document to obtain a training model, and store the training model under the classifier folder; Input the picture into the training model under the classifier folder and output all the recognition points in the picture. Here, adaboost (combining general classifiers into a cascade classifier) and haar features (a commonly used feature description operator in computer vision) in OpenCV2.4 can be used. Create a new Adaboost folder and copy the OpenCV programs that need to be used into it. The specific process includes:

[0079] Step 1: Collect training samples and train the model. Two sample sets, positive and negative, need to be prepared. In this application, the ratio of positive to negative samples used is 1:2.5. Among them, the positive samples contain the Marker markers (recognition point markers) to be recognized, while the negative samples do not. The negative samples are taken from the environment where the positive samples appear. First, preprocess the samples. Normalize the positive samples to a unified size of 30*30. Create a Positive folder under the Adaboost folder to store all positive samples. Create a Negative folder under the Adaboost folder to store negative samples. The negative samples can be left without normalization, ensuring that they do not contain the marker to be detected. Next, create sample files. Enter the Positive folder and create pos.txt. Edit the file so that the format is expressed as "***.bmp 10 0 30 30". Enter the Negative folder and generate the nega.txt file. Subsequently, create the positive sample vec file. Enter the Adaboost folder and call opencv_createsamples.exe, execute the command line: opencv_createsamples.exe -vec pos.vec -info positive\pos.txt -bg negative\nega.txt -w 30 -h 30 -num 2000. Among them, opencv_createsamples.exe: the name of the execution program. -vec pos.vec: specify the target file to be generated. -info positive\pos.txt: the location of the positive sample information. -bg negative\nega.txt: the location of the negative sample information. -w 30 -h 30: the width and height of the normalized size of the positive samples. -num 2000: the number of positive samples.

[0080] After execution, pos.vec will be generated. This operation is not required for negative samples.

[0081] Step 2: Training Samples. Create a classifier folder under the Adaboost file. The finally trained model will be stored here. Call the OpenCV execution program opencv_traincascade.exe with the following parameters: opencv_traincascade -data classifier -vec pos.vec -bg nega\neg.txt -numPos 2000 -numNeg 5000 -numStages 15 -precalcValBufSize 1024 -precalcIdxBufSize 1024 -featureType HAAR -w 30 -h 30 -maxDepth 1. Here, opencv_traincascade is the name of the execution program, -data classifier specifies the storage location of the trained data, -vec pos.vec is the location of the positive sample file, -bg nega\neg.txt is the location of the negative sample information, -numPos 2000 -numNeg 5000 are the numbers of positive and negative samples, -numStages 15 is the number of stages of the trainer, -precalcValBufSize 1024 -precalcIdxBufSize 1024 are the cache sizes of the pre-computed feature values and feature indices, -featureType HAAR indicates that the type of image features extracted during training is HAAR, -w 30 -h 30 is the normalized size of the previous step, and -maxDepth 1 is the maximum depth of the weak classifier, set to 1 for a binary tree. Enter the classifier folder, obtain the cascade.xml file to get the required trained model, and input the image into this trained model to obtain the recognition points in the image.

[0082] Through the method described in this application, during the image test process, the camera can always be aligned with the shooting plane, the alignment accuracy during shooting can be controlled, and the parameters calculated after each current shooting are used to correct the position during the next shooting, thereby improving the correction accuracy of shooting and the accuracy of the test results.

[0083] In addition, the embodiment of this application also provides a computer-readable medium, on which computer-readable instructions are stored, and the computer-readable instructions can be executed by a processor to implement the foregoing method for image card shooting correction.

[0084] Corresponding to the method described above, this application also provides a terminal, which includes the ability to execute the above Figure 1 or Figure 2modules or units corresponding to the method steps of each embodiment. These modules or units can be implemented by hardware, software, or a combination of both, and this application does not limit it. For example, in an embodiment of this application, a device based on calibration of graphic card shooting is further provided. The device includes:

[0085] one or more processors; and

[0086] a memory storing computer-readable instructions, which when executed cause the processor to perform the operations of the method as described above.

[0087] For example, when the computer-readable instructions are executed, they cause the one or more processors to:

[0088] obtain a picture taken by using the camera of the shooting device to align with the graphic card, and set at least one set of identification and positioning points in the picture;

[0089] determine the three-dimensional coordinate information of the at least one set of identification and positioning points in the graphic card coordinate system and the planar coordinate information in the picture pixels;

[0090] determine the camera internal parameters of the camera according to the distance between the camera and the graphic card, the three-dimensional coordinate information of the at least one set of identification and positioning points in the graphic card coordinate system, and the planar coordinate information in the picture pixels;

[0091] determine the rotation angle and movement coordinates of the camera based on the camera internal parameters of the camera, and determine the shooting distance between the camera and the graphic card for the next shooting according to the rotation angle and movement coordinates.

[0092] Figure 3 The structural schematic diagram of a device based on calibration of graphic card shooting provided according to another aspect of this application is shown. The device includes: a setting device 11, a determining device 12, a calculating device 13, and a calibration device 14. Among them, the setting device 11 is used to obtain a picture taken by using the camera of the shooting device to align with the graphic card, and set at least one set of identification and positioning points in the picture; the determining device 12 is used to determine the three-dimensional coordinate information of the at least one set of identification and positioning points in the graphic card coordinate system and the planar coordinate information in the picture pixels; the calculating device 13 is used to determine the camera internal parameters of the camera according to the distance between the camera and the graphic card, the three-dimensional coordinate information of the at least one set of identification and positioning points in the graphic card coordinate system, and the planar coordinate information in the picture pixels; the calibration device 14 is used to determine the rotation angle and movement coordinates of the camera based on the camera internal parameters of the camera, and determine the shooting distance between the camera and the graphic card for the next shooting according to the rotation angle and movement coordinates.

[0093] It should be noted that the operations performed by the setting device 11, the determining device 12, the calculating device 13, and the correcting device 14 are the same as or correspondingly the same as those in the above steps S11, S12, S13, and S14. For the sake of brevity, they will not be elaborated here.

[0094] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these changes and modifications.

[0095] It should be noted that this application can be implemented in software and / or a combination of software and hardware. For example, it can be implemented using an application specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In one embodiment, the software program of this application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive, or a floppy disk and similar devices. Additionally, some steps or functions of this application can be implemented using hardware, for example, as a circuit that cooperates with the processor to execute each step or function.

[0096] In addition, a part of this application can be applied as a computer program product. For example, computer program instructions, when executed by a computer, can, through the operation of the computer, call or provide the methods and / or technical solutions according to this application. The program instructions for calling the methods of this application may be stored in a fixed or removable recording medium, and / or transmitted through a data stream in a broadcast or other signal-bearing medium, and / or stored in the working memory of a computer device running according to the program instructions. Here, an embodiment according to this application includes a device that includes a memory for storing computer program instructions and a processor for executing the program instructions. When the computer program instructions are executed by the processor, the device is triggered to run based on the methods and / or technical solutions according to the foregoing multiple embodiments of this application.

[0097] For those skilled in the art, it is obvious that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present application is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be construed as limiting the claims concerned. In addition, it is obvious that the term "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. The multiple elements or devices recited in the apparatus claims can also be implemented by one element or device through software or hardware. First, second, etc. are used to denote names and do not denote any particular order.

Claims

1. A method based on calibration by taking pictures of a figure card, characterized in that, the method includes: Obtaining a picture taken by aligning a camera of a photographing device with a figure card, and setting at least one set of recognition and positioning points in the picture. Among them, the photographing device is fixed on a robotic arm with an interface. The figure card includes a regular dot pattern figure card or a checkerboard figure card. The figure card has a three-dimensional coordinate system, which is established according to the point distance on the regular dot pattern figure card or the side length of the checkerboard on the checkerboard figure card; Among them, setting at least one set of recognition and positioning points in the picture includes: training all the recognition points in the picture using a training model, and selecting at least one set of recognition and positioning points from all the recognition points. Among them, training all the recognition points in the picture using the training model includes: determining positive samples and negative samples. Among them, the positive samples include the recognition point marks to be recognized, and the negative samples are obtained from the environment where the positive samples are located; normalizing the positive samples and then putting them into the positive sample folder under the root file, and directly putting the negative samples into the negative sample folder; creating a positive sample editing document in the positive sample folder and a negative sample editing document in the negative sample folder; creating a classifier folder under the root file, calling a training execution program to train the positive sample editing document and the negative sample editing document to obtain a training model, and storing the training model under the classifier folder; inputting the picture into the training model under the classifier folder and outputting all the recognition points in the picture; Determining the three-dimensional coordinate information of the at least one set of recognition and positioning points in the figure card coordinate system and the plane coordinate information in the picture pixels; Determining the camera internal parameters of the camera according to the distance between the camera and the figure card, the three-dimensional coordinate information of the at least one set of recognition and positioning points in the figure card coordinate system, and the plane coordinate information in the picture pixels; Determining the rotation angle and movement coordinates of the camera based on the camera internal parameters of the camera, and determining the shooting distance between the camera and the figure card during the next shooting according to the rotation angle and movement coordinates; Transmitting the rotation angle, movement coordinates of the camera, and the shooting distance between the camera and the figure card during the next shooting into the interface, so that the robotic arm moves the photographing device to the corresponding position.

2. The method according to claim 1, characterized in that, determining the camera internal parameters of the camera according to the distance between the camera and the figure card, the three-dimensional coordinate information of the at least one set of recognition and positioning points in the figure card coordinate system, and the plane coordinate information in the picture pixels includes: Obtaining the horizontal direction coordinate information and vertical direction coordinate information of each recognition and positioning point in each set in the figure card from the three-dimensional coordinate information of the at least one set of recognition and positioning points in the figure card coordinate system; Obtaining the picture pixel width, picture pixel height, horizontal direction pixel coordinate information, and vertical direction pixel coordinate information of each recognition and positioning point in each set from the plane coordinate information of the at least one set of recognition and positioning points in the picture pixels; Determine the lateral pixel information of the camera focal length based on the horizontal direction pixel coordinate information, the horizontal direction coordinate information, and the distance between the camera and the card; Determine the longitudinal pixel information of the camera focal length based on the vertical direction pixel coordinate information, the vertical direction coordinate information, and the distance between the camera and the card; Determine the abscissa of the principal point pixel of the camera according to the picture pixel width of each recognition and positioning point, and determine the ordinate of the principal point pixel of the camera according to the picture pixel height of each recognition and positioning point.

3. The method according to claim 2, wherein, Determining the rotation angle and movement coordinates of the camera based on the camera internal parameters of the camera, including: Establishing a camera internal parameter matrix based on the camera internal parameters of the camera, and determining the pose matrix diagram of the camera according to the camera internal parameter matrix; Determining the three-dimensional coordinate information of the angular rotation of the camera based on the pose matrix diagram of the camera; Determine the rotation angle and movement coordinates of the camera according to the three-dimensional coordinate information of the angular rotation of the camera, the three-dimensional coordinate information of at least one set of recognition and positioning points in the card coordinate system, and the plane coordinate information in the picture pixels.

4. The method according to claim 3, wherein, Establishing a camera internal parameter matrix based on the camera internal parameters of the camera, including: Establishing a camera internal parameter matrix according to the lateral pixel information of the camera focal length, the longitudinal pixel information of the camera focal length, the abscissa of the principal point pixel of the camera, and the ordinate of the principal point pixel of the camera, wherein the camera internal parameter matrix satisfies the following form: wherein, fx represents the lateral pixel information of the camera focal length, fy represents the longitudinal pixel information of the camera focal length, cx represents the abscissa of the principal point pixel of the camera, and cy represents the ordinate of the principal point pixel of the camera.

5. A device for calibration based on card shooting, wherein, The device includes: A setting device for acquiring a picture of the camera of the shooting device aligned with the card for shooting, and setting at least one set of recognition and positioning points in the picture, wherein the shooting device is fixed on a robotic arm with an interface, the card includes a regular dot pattern card or a checkerboard card, and the card has a three-dimensional coordinate system established according to the point distance on the regular dot pattern card or the side length of the checkerboard on the checkerboard card; A determining device for determining the three-dimensional coordinate information of at least one set of recognition and positioning points in the card coordinate system and the plane coordinate information in the picture pixels; A calculating device for determining the camera internal parameters of the camera according to the distance between the camera and the card, the three-dimensional coordinate information of at least one set of recognition and positioning points in the card coordinate system, and the plane coordinate information in the picture pixels; A calibration device for determining the rotation angle and movement coordinates of the camera based on the camera internal parameters of the camera, and determining the shooting distance between the camera and the card during the next shooting according to the rotation angle and movement coordinates; A transmitting device for transmitting the rotation angle of the camera, the movement coordinates, and the shooting distance between the camera and the card during the next shooting into the interface, so that the robotic arm moves the shooting device to the corresponding position.

6. An apparatus based on calibration by graphic card photographing, characterized in that, the apparatus comprises: one or more processors; and a memory storing computer-readable instructions that, when executed, cause the processors to perform the operations of the method according to any one of claims 1 to 4.

7. A computer-readable medium having stored thereon computer-readable instructions that are executable by a processor to implement the method according to any one of claims 1 to 4.

Images