Real-time feedback rigid body six-degree-of-freedom measurement method, device, equipment and medium

Abstract

The present application relates to rigid body measurement technology, and discloses a kind of real-time feedback rigid body six degrees of freedom measurement method, device, equipment and storage medium.The method comprises: according to checkerboard calibration plate, the initial camera internal and external parameter of monocular camera is acquired;Rigid body image is collected, and rigid body historical image sequence and rigid body real-time image are obtained;The feature extraction of rigid body historical image sequence is carried out, and each rigid body marker feature set is obtained;According to the pre-constructed minimum re-projection error algorithm and the rigid body marker feature set, the initial camera internal and external parameter is dynamically adjusted, and dynamic camera internal and external parameter is obtained;Image feature extraction is carried out to the rigid body real-time image, and rigid body real-time feature set is obtained, and according to the dynamic camera internal and external parameter, six degrees of freedom measurement is carried out to the rigid body real-time feature set, and rigid body measurement result is obtained.The present application can be dynamically accurately measured to rigid body by monocular vision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of rigid body measurement, and particularly relates to a real-time feedback rigid body six-degree-of-freedom measurement method, device, equipment and computer readable storage medium. BACKGROUND

[0002] In the field of machine vision and precision measurement, the six-degree-of-freedom measurement technology of rigid body has always been a research hotspot. The traditional measurement method configures the parameters of a monocular vision camera by using a checkerboard, which is more suitable for static measurement environment. However, in a dynamic environment, the drift of the camera internal and external parameters occurs with the movement of the rigid body and the camera, resulting in inaccurate measurement. SUMMARY

[0003] The present application provides a real-time feedback rigid body six-degree-of-freedom measurement method, device, equipment and storage medium, which mainly aims to dynamically and accurately measure the rigid body by using monocular vision.

[0004] To achieve the above purpose, the present application provides a real-time feedback rigid body six-degree-of-freedom measurement method, which comprises the following steps:

[0005] An initial camera internal and external parameter is obtained by performing an initialization calibration operation on a pre-constructed monocular camera using a pre-constructed checkerboard calibration plate.

[0006] A rigid body with a preset checkerboard marker is obtained, and a continuous photography of the rigid body is performed using the monocular camera to obtain a rigid body historical image sequence and a rigid body real-time image.

[0007] A rigid body historical image is extracted from the rigid body historical image sequence in sequence, and image feature extraction is performed on the rigid body historical image to obtain a rigid body marker feature set.

[0008] The initial camera internal and external parameter is dynamically adjusted according to a pre-constructed minimum re-projection error algorithm and the rigid body marker feature set to obtain a dynamic camera internal and external parameter, and the initial camera internal and external parameter is updated using the dynamic camera internal and external parameter.

[0009] When the rigid body historical image in the rigid body historical image sequence is traversed, image feature extraction is performed on the rigid body real-time image to obtain a rigid body real-time feature set, and six-degree-of-freedom measurement is performed on the rigid body real-time feature set according to the dynamic camera internal and external parameter to obtain a rigid body measurement result.

[0010] Optionally, the continuous photography of the rigid body using the monocular camera to obtain the rigid body historical image sequence and the rigid body real-time image comprises the following steps:

[0011] Real-time acquisition of the monocular camera real-time shooting image, and object recognition is performed on the real-time shooting image to obtain an object recognition result;

[0012] It is judged whether the object recognition result appears a preset checkerboard marker object;

[0013] When the checkerboard marker object appears in the object recognition result, the checkerboard marker object is recorded for target marking to obtain a wall history image video;

[0014] When receiving a photography instruction issued by a user, the rigid body history image video is intercepted to obtain a rigid body real-time image, and a video in a preset time period before the rigid body real-time image in the rigid body history image video is collected according to a preset interception frequency to obtain a rigid body history image sequence.

[0015] Optionally, the initial camera internal and external parameters are dynamically adjusted according to the pre-constructed minimum re-projection error algorithm and the rigid body marker feature set to obtain dynamic camera internal and external parameters, including:

[0016] The model size information of the rigid body is acquired;

[0017] The initial camera internal and external parameters and the model size information are used to perform image prediction on the rigid body according to a pre-constructed camera projection algorithm to obtain a rigid body prediction image;

[0018] The difference of the rigid body marker feature set to the rigid body prediction image is calculated according to a pre-constructed minimum re-projection error algorithm to obtain a re-projection error;

[0019] The re-projection error is minimized by using a pre-constructed least square method to obtain dynamic camera internal and external parameters.

[0020] Optionally, the initial camera internal and external parameters are updated by using the dynamic camera internal and external parameters, including:

[0021] The difference of the dynamic camera internal and external parameters to the initial camera internal and external parameters is calculated by using a pre-constructed incremental algorithm to obtain a parameter increment, wherein the incremental algorithm is represented as:

[0022] P updated =P initial +α(P dynamic -P initial )

[0023] In the formula, P updated represents the updated initial camera internal and external parameters, P initial represents the initial camera internal and external parameters, and P dynamicindicates the dynamic camera internal and external parameters, and a indicates a smooth factor ranging from 0 to 1;

[0024] According to the parameter increment, the initial camera internal and external parameters are calculated by weighting to obtain updated initial camera internal and external parameters.

[0025] Optionally, the pre-constructed checkerboard calibration board is used to initialize and calibrate the pre-constructed monocular camera to obtain initial camera internal and external parameters, including:

[0026] The pre-constructed monocular camera is used to obtain a calibration image sequence according to pre-set multi-angle position information of the pre-constructed checkerboard calibration board;

[0027] The checkerboard corner feature extraction operation is performed on the calibration image sequence to obtain a corner feature information set;

[0028] The size layout information of the checkerboard calibration board is obtained, and the size layout information is quantized to obtain a checkerboard layout feature vector;

[0029] The pre-constructed calibration algorithm is used to identify the shooting parameters of the corner feature information according to the checkerboard layout feature vector to obtain initial camera internal and external parameters.

[0030] Optionally, the image feature extraction is performed on the rigid body historical image to obtain a rigid body marker feature set, including:

[0031] The rigid body historical image is subjected to grayscale processing to obtain a rigid body grayscale image, and the rigid body grayscale image is subjected to Gaussian filter smoothing processing to obtain a rigid body denoising grayscale image;

[0032] The rigid body denoising grayscale image is subjected to feature extraction operation to obtain a rigid body image feature set;

[0033] The rigid body image feature set is subjected to feature classification judgment based on a pre-set marker to obtain a rigid body marker feature set.

[0034] Optionally, after the rigid body measurement result is obtained, the method further includes:

[0035] The real measurement result of the rigid body is obtained by using two cameras in the pre-constructed experimental platform;

[0036] The measurement difference is obtained by using the rigid body measurement result and the real measurement result to perform difference calculation;

[0037] It is judged whether the measurement difference is greater than a pre-set qualified threshold;

[0038] When the measurement difference is greater than the pre-set qualified threshold, an alarm prompt information is generated.

[0039] To solve the above problems, the application further provides a rigid body six-degree-of-freedom measurement device with real-time feedback, which comprises:

[0040] A camera parameter initialization module is configured to initialize and calibrate a pre-constructed monocular camera by using a pre-constructed checkerboard calibration plate to obtain initial camera internal and external parameters.

[0041] A rigid body image acquisition module is configured to acquire a rigid body with a pre-set checkerboard marker, continuously collect and photograph the rigid body by using the monocular camera, and obtain a rigid body historical image sequence and a rigid body real-time image.

[0042] A parameter real-time updating module is configured to sequentially extract a rigid body historical image from the rigid body historical image sequence, extract image features of the rigid body historical image to obtain a rigid body marker feature set, dynamically adjust the initial camera internal and external parameters according to a pre-constructed minimum re-projection error algorithm and the rigid body marker feature set, obtain dynamic camera internal and external parameters, and update the initial camera internal and external parameters by using the dynamic camera internal and external parameters.

[0043] A rigid body measurement module is configured to extract image features of the rigid body real-time image to obtain a rigid body real-time feature set when the rigid body historical image in the rigid body historical image sequence is traversed, and measure six degrees of freedom of the rigid body real-time feature set according to the dynamic camera internal and external parameters to obtain a rigid body measurement result.

[0044] To solve the above problems, the application further provides an electronic device, which comprises:

[0045] at least one processor; and

[0046] a memory in communication with the at least one processor; wherein

[0047] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the real-time feedback rigid body six-degree-of-freedom measurement method described above.

[0048] To solve the above problems, the application further provides a computer readable storage medium, which stores at least one computer program, and the at least one computer program is executed by a processor in an electronic device to implement the real-time feedback rigid body six-degree-of-freedom measurement method described above.

[0049] The embodiment of the present application firstly calibrates the parameters of the monocular camera through the chessboard calibration board, obtains the initial camera internal and external parameters, and then photographs the rigid body pasted with the chessboard identification object; the present application adopts the pre-aiming mode to take the calibration result in the shooting interval as the prior information, continuously calculates the dynamic camera internal and external parameters, and updates the initial camera internal and external parameters, so that the final measurement process is more accurate; in addition, when updating the initial camera internal and external parameters, the present application adopts the incremental algorithm instead of direct replacement, greatly improves the stability of the initial camera internal and external parameters in the updating process, makes the parameters more difficult to drift, and causes the measurement result deviation. Therefore, the real-time feedback rigid body six-degree-of-freedom measurement method, device, equipment and storage medium provided by the embodiment of the present application can dynamically and accurately measure the rigid body through monocular vision. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 The flowchart of the real-time feedback rigid body six-degree-of-freedom measurement method provided by an embodiment of the present application is shown in the figure.

[0051] Figure 2 The scene coordinate system introduction diagram in the real-time feedback rigid body six-degree-of-freedom measurement method provided by an embodiment of the present application is shown in the figure.

[0052] Figure 3 The scene diagram of an experimental platform in the real-time feedback rigid body six-degree-of-freedom measurement method provided by an embodiment of the present application is shown in the figure.

[0053] Figure 4 A test comparison diagram in the real-time feedback rigid body six-degree-of-freedom measurement method provided by an embodiment of the present application is shown in the figure.

[0054] Figure 5 A test comparison diagram in the real-time feedback rigid body six-degree-of-freedom measurement method provided by an embodiment of the present application is shown in the figure.

[0055] Figure 6 The functional module diagram of the real-time feedback rigid body six-degree-of-freedom measurement device provided by an embodiment of the present application is shown in the figure.

[0056] Figure 7 The structure diagram of the electronic device for implementing the real-time feedback rigid body six-degree-of-freedom measurement method provided by an embodiment of the present application is shown in the figure.

[0057] The implementation of the present application, functional features and advantages will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0059] The embodiment of the present application provides a real-time feedback rigid body six-degree-of-freedom measurement method. In the embodiment of the present application, the execution subject of the real-time feedback rigid body six-degree-of-freedom measurement method includes but is not limited to at least one of electronic devices such as a server, a terminal and the like which can be configured to execute the method provided by the embodiment of the present application. In other words, the real-time feedback rigid body six-degree-of-freedom measurement method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be a stand-alone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0060] Referring to Figure 1 FIG. 1 is a flowchart of a real-time feedback rigid body six-degree-of-freedom measurement method provided by an embodiment of the present application. In the embodiment, the real-time feedback rigid body six-degree-of-freedom measurement method includes:

[0061] S1, using a pre-constructed checkerboard calibration plate, performing an initial calibration operation on a pre-constructed monocular camera to obtain initial camera internal and external parameters.

[0062] The checkerboard calibration plate is a commonly used tool in the field of computer vision and photogrammetry, mainly used for camera calibration and geometric correction, and is composed of a plane plate (such as paper, plastic or metal) and a regular arrangement of black and white checkerboard patterns on its surface.

[0063] The monocular camera refers to an image capture device that uses only one lens to capture images, and acquires scene information through a single perspective.

[0064] The initial camera internal and external parameters refer to parameters used to describe the imaging characteristics of the camera in the camera model, which are usually divided into internal parameters and external parameters. The internal parameters are, for example, focal length, principal point and distortion coefficient, etc., and the external parameters are the position and direction of the camera in the world coordinate system, such as translation vector and rotation matrix.

[0065] In detail, in the embodiment of the present application, the initial calibration operation on the pre-constructed monocular camera using the pre-constructed checkerboard calibration plate to obtain the initial camera internal and external parameters includes:

[0066] Using the pre-constructed monocular camera, according to the pre-set multi-angle position information, a calibration image sequence of the pre-constructed checkerboard calibration plate is obtained;

[0067] Carry out a chessboard corner feature extraction operation on the calibration image sequence to obtain a corner feature information set;

[0068] Obtain size layout information of the chessboard calibration board, quantize the size layout information to obtain a chessboard layout feature vector;

[0069] Using a pre-constructed calibration algorithm, according to the chessboard layout feature vector, identify the shooting parameters of the corner feature information to obtain initial camera internal and external parameters.

[0070] The multi-angle position information can be adaptively adjusted according to the rigid body type.

[0071] The chessboard corner refers to an intersection formed at the junction of black and white squares in the chessboard.

[0072] Specifically, in the embodiment of the present application, a monocular camera with a resolution of 1280*720 is selected, the size of the chessboard calibration board is determined according to the actual situation, the chessboard calibration board is fixed in a stable position, and it is ensured that it is clearly visible during the calibration process, and the image of the calibration board is shot from multiple different angles using a monocular camera. When shooting, the distance, angle and lighting conditions between the camera and the calibration board are changed to obtain rich calibration data, the corner points of the chessboard are extracted from each calibration image using image processing technology, the internal and external parameters of the camera are calculated using the camera imaging principle, and the accuracy of the calibration result is verified according to the error calculated by the re-projection formula.

[0073] S2, obtain a rigid body with a preset chessboard marker pasted thereon, and continuously collect and photograph the rigid body using the monocular camera to obtain a rigid body historical image sequence and a rigid body real-time image.

[0074] Specifically, referring to Figure 2 The chessboard markers are pasted on the rigid body to be measured, and it is ensured that these markers are always clearly visible during the movement of the rigid body. The number and arrangement of the markers should be selected according to the size and shape of the rigid body to ensure that enough feature points can be captured during the measurement.

[0075] Further, the rigid body with the markers pasted thereon needs to be image collected.

[0076] In detail, in the embodiment of the present application, the monocular camera is used to continuously collect and photograph the rigid body to obtain a rigid body historical image sequence and a rigid body real-time image, including:

[0077] Real-time shooting images of the monocular camera are obtained in real time, and object recognition is performed on the real-time shooting images to obtain an object recognition result.

[0078] determine whether the object recognition result appears a preset checkerboard marker object;

[0079] When the checkerboard marker object appears in the object recognition result, the checkerboard marker object is marked as a target, and a wall history image video is obtained.

[0080] When a user issues a photography instruction, the rigid body history image video is intercepted to obtain a rigid body real-time image, and a video in a preset time period before the rigid body real-time image in the rigid body history image video is collected according to a preset interception frequency to obtain a rigid body history image sequence.

[0081] Specifically, in the embodiment of the present application, the object recognition result depends on the scene during rigid body detection, and specifically can include a flowerpot, a light supplement lamp, a table, a checkerboard calibration board and the like.

[0082] Specifically, in the embodiment of the present application, a monocular camera is started, a pre-look mode is opened, a sequence of images containing a checkerboard marker is continuously collected to obtain a rigid body history image sequence, and a frame received when a user photography instruction is received is taken as a rigid body real-time image. And when collecting, it should be ensured that the field of view of the camera can cover the entire rigid body, and the interference factors such as light change and occlusion should be reduced as much as possible.

[0083] S3, a rigid body history image is extracted from the rigid body history image sequence in sequence, and image feature extraction is performed on the rigid body history image to obtain a rigid body marker feature set.

[0084] In detail, in the embodiment of the present application, the image feature extraction is performed on the rigid body history image to obtain a rigid body marker feature set, including:

[0085] The rigid body history image is subjected to grayscale processing to obtain a rigid body grayscale image, and the rigid body grayscale image is subjected to Gaussian filter smoothing processing to obtain a rigid body noise reduction grayscale image;

[0086] The rigid body noise reduction grayscale image is subjected to feature extraction operation to obtain a rigid body image feature set;

[0087] The rigid body image feature set is subjected to feature classification judgment based on a preset marker to obtain a rigid body marker feature set.

[0088] The grayscale processing refers to the process of converting a color image into a grayscale image. The grayscale image is composed of pixels of different grayscale levels, and the value of each pixel represents its brightness without color information.

[0089] The Gaussian filter smoothing processing refers to a common image processing technology, mainly used for reducing noise and details in an image, and making the image smoother, and generally using the characteristics of a Gaussian function to perform weighted average on each pixel of the image, so as to realize the smoothing effect.

[0090] The preset marker is a black-and-white grid distribution structure.

[0091] Specifically, in the embodiment of the application, for each frame of image, the feature points of the checkerboard marker are extracted by using the above image processing technology, for subsequent matching with the feature points on the calibration board.

[0092] S4, dynamically adjusting the initial camera internal and external parameters according to the pre-constructed minimization reprojection error algorithm and the rigid marker feature set, obtaining dynamic camera internal and external parameters, and updating the initial camera internal and external parameters by using the dynamic camera internal and external parameters.

[0093] The minimization reprojection error algorithm is a method commonly used in the field of computer vision and photogrammetry, mainly used for camera calibration, three-dimensional reconstruction, target tracking and other tasks, and the basic idea is to optimize the reprojection error, so that the projection position of the three-dimensional point in the image is as close as possible to the corresponding image feature point.

[0094] In detail, in the embodiment of the application, the dynamic adjustment of the initial camera internal and external parameters according to the pre-constructed minimization reprojection error algorithm and the rigid marker feature set, to obtain dynamic camera internal and external parameters, comprises:

[0095] Obtaining model size information of the rigid body;

[0096] Using a pre-constructed camera projection algorithm, predicting the image of the rigid body according to the initial camera internal and external parameters and the model size information, to obtain a rigid body prediction image;

[0097] According to the pre-constructed minimization reprojection error algorithm, calculating the difference of the rigid marker feature set for the rigid body prediction image, to obtain a reprojection error;

[0098] Using a pre-constructed least square method to minimize the reprojection error, to obtain dynamic camera internal and external parameters.

[0099] Specifically, in the embodiment of the application, the model size information can be taken as a reference by a car model. Then, the internal and external parameters of the camera are dynamically adjusted based on the minimization reprojection error algorithm by using the matched rigid marker feature set and the known checkerboard size.

[0100] Further, in the embodiment of the present application, the updating of the initial camera internal and external parameters by the dynamic camera internal and external parameters comprises:

[0101] The difference between the dynamic camera internal and external parameters and the initial camera internal and external parameters is calculated by using a pre-constructed incremental algorithm to obtain a parameter increment, wherein the incremental algorithm is expressed as:

[0102] P updated =P initial +α(P dynamic -P initial )

[0103] In the formula, P updated represents the updated initial camera internal and external parameters, P initial represents the initial camera internal and external parameters, P dynamic represents the dynamic camera internal and external parameters, and a represents a smoothing factor ranging from 0 to 1.

[0104] The initial camera internal and external parameters are calculated by weighting according to the parameter increment to obtain updated initial camera internal and external parameters.

[0105] Specifically, in the embodiment of the present application, the incremental calibration technology is used to reduce the calculation complexity and improve the real-time performance.

[0106] The embodiment of the present application uses the calibration results of the previous frame or several previous frames as prior information, which can improve the accuracy and stability of the internal parameter adjustment. The adjusted camera internal parameters are recorded and updated to the processing of the next frame image. This can ensure that the camera internal parameters remain up-to-date during the subsequent measurement process, thereby improving the measurement accuracy.

[0107] S5, when the rigid body historical image in the rigid body historical image sequence is traversed, image feature extraction is performed on the rigid body real-time image to obtain a rigid body real-time feature set, and six-degree-of-freedom measurement is performed on the rigid body real-time feature set according to the dynamic camera internal and external parameters to obtain a rigid body measurement result.

[0108] First, the updated camera internal parameters are used to eliminate the distortion of each frame of image obtained, and the coordinate relationship before and after the correction of radial distortion is:

[0109]

[0110] The tangential distortion needs two additional parameters to describe, and the coordinate relationship before and after the correction is:

[0111]

[0112] In the formula, (x, y), Respectively, the ideal and distorted image coordinates, r is the distance from the image pixel point to the image center, that is: r 2 = x 2 +y 2 .

[0113] Obtain the feature point [u v] and the chessboard corner point [X Y] from each frame of image, combine the Zhang camera calibration model Solve the homography matrix H.

[0114] And then combine the updated camera internal parameter A, according to:

[0115]

[0116] Solve the rotation matrix R = [r1 r2 r1x r2] and the translation matrix T = [t X t Y t Z ] of the rigid body coordinate system relative to the camera coordinate system in the current frame.

[0117] Convert the calculated rotation matrix and translation matrix to the required spatial coordinate system, and calculate the rotation angle according to the formula:

[0118]

[0119] Finally, output the six-degree-of-freedom information.

[0120] Further, in the embodiment of the present application, after obtaining the rigid body measurement result, the method further comprises:

[0121] Obtain the real measurement result of the rigid body by using two cameras in the pre-constructed experimental platform;

[0122] Calculate the measurement difference by using the rigid body measurement result and the real measurement result;

[0123] Determine whether the measurement difference is greater than a preset qualified threshold;

[0124] When the measurement difference is greater than the preset qualified threshold, generate an alarm prompt information.

[0125] Wherein, the qualified threshold can be configured according to the specific situation, and the qualified threshold is configured as 1% in the present application.

[0126] Wherein, the alarm prompt information can be

re-determine the camera internal and external parameters

[0127] As Figure 3As shown, an experimental platform is constructed in the embodiment of the present application, which adopts a configuration of two cameras. The setting of the two cameras aims to verify the error performance of the method of the present application in practical application. Specifically, we use the method of the present application to measure the six-degree-of-freedom changes of the all-terrain vehicle during its movement from near to far in detail. After multiple tests: when the distance between the all-terrain vehicle and the camera changes, the relative error of the six-degree-of-freedom of the all-terrain vehicle is as follows Figure 4 and Figure 5 .

[0128] By measuring the relative error between the all-terrain vehicle and the camera at different distances, the accuracy of the method of the present application is evaluated. In the test, the all-terrain vehicle is placed at different distances in front of the camera, and the relative error of the vehicle coordinate system in the world coordinate system is recorded.

[0129] As can be seen from the image, as the distance between the all-terrain vehicle and the camera increases, the relative error also gradually increases. This indicates that when the distance between the target object (all-terrain vehicle) and the camera increases, the accuracy of the visual-based pose estimation method may decrease. This may be due to various factors, such as the reduction of image resolution, the change of lighting conditions, the distortion of camera lens, etc.

[0130] In practical application, the distance between the target object and the camera needs to be adjusted according to the accuracy requirement.

[0131] The embodiment of the present application first calibrates the parameters of the monocular camera through the checkerboard calibration plate to obtain the initial camera internal and external parameters, and then photographs the rigid body with the checkerboard markers. The present application uses the pre-look method to take the calibration results between shots as prior information, continuously calculates the dynamic camera internal and external parameters, and updates the initial camera internal and external parameters, so that the final measurement process is more accurate. In addition, when updating the initial camera internal and external parameters, the present application uses an incremental algorithm instead of direct replacement, which greatly improves the stability of the initial camera internal and external parameters in the updating process, so that the parameters are less likely to drift and cause measurement result deviation. Therefore, the real-time feedback rigid body six-degree-of-freedom measurement method provided by the embodiment of the present application can dynamically and accurately measure the rigid body through monocular vision.

[0132] As shown in Figure 6 , it is a functional module diagram of the real-time feedback rigid body six-degree-of-freedom measurement device provided by an embodiment of the present application.

[0133] The real-time feedback rigid body six-degree-of-freedom measurement device 100 can be installed in an electronic device. According to the implemented function, the real-time feedback rigid body six-degree-of-freedom measurement device 100 can include a camera parameter initialization module 101, a rigid body image acquisition module 102, a parameter real-time updating module 103, and a rigid body measurement module 104. The modules in the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, which are stored in the memory of the electronic device.

[0134] In the present embodiment, the functions of each module / unit are as follows:

[0135] The camera parameter initialization module 101 is configured to initialize and calibrate a pre-constructed monocular camera using a pre-constructed checkerboard calibration plate to obtain initial camera internal and external parameters.

[0136] The rigid body image acquisition module 102 is configured to acquire a rigid body with a pre-set checkerboard marker, and continuously collect and photograph the rigid body using the monocular camera to obtain a rigid body historical image sequence and a rigid body real-time image.

[0137] The parameter real-time updating module 103 is configured to sequentially extract a rigid body historical image from the rigid body historical image sequence, and extract image features of the rigid body historical image to obtain a rigid body marker feature set, and dynamically adjust the initial camera internal and external parameters according to a pre-constructed minimum re-projection error algorithm and the rigid body marker feature set to obtain dynamic camera internal and external parameters, and update the initial camera internal and external parameters using the dynamic camera internal and external parameters.

[0138] The rigid body measurement module 104 is configured to, when the rigid body historical image in the rigid body historical image sequence is traversed, extract image features of the rigid body real-time image to obtain a rigid body real-time feature set, and perform six-degree-of-freedom measurement on the rigid body real-time feature set according to the dynamic camera internal and external parameters to obtain a rigid body measurement result.

[0139] In detail, each module in the real-time feedback rigid body six-degree-of-freedom measurement device 100 in the present embodiment uses the same technical means as the real-time feedback rigid body six-degree-of-freedom measurement method described in the above Figures 1 to 5 , and can produce the same technical effects, which will not be described here.

[0140] As shown in Figure 7 , it is a structural schematic diagram of an electronic device 1 for implementing a real-time feedback rigid body six-degree-of-freedom measurement method according to an embodiment of the present application.

[0141] The electronic device 1 can include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and can further include a computer program stored in the memory 11 and executable on the processor 10, such as a real-time feedback rigid body six-degree-of-freedom measurement program.

[0142] The processor 10 can be composed of an integrated circuit in some embodiments, for example, can be composed of a single packaged integrated circuit, or can be composed of multiple packaged integrated circuits with the same function or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, combinations of various control chips, etc. The processor 10 is the control unit of the electronic device 1, which connects various components of the entire electronic device through various interfaces and lines, executes or runs programs or modules stored in the memory 11 (such as a real-time feedback rigid body six-degree-of-freedom measurement program, etc.), and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0143] The memory 11 includes at least one type of readable storage medium, including flash memory, mobile hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 11 can be an internal storage unit of the electronic device in some embodiments, such as a mobile hard disk of the electronic device. The memory 11 can also be an external storage device of the electronic device in other embodiments, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Further, the memory 11 can include both the internal storage unit and the external storage device of the electronic device. The memory 11 can be used not only to store application software and various data installed on the electronic device, such as the code of the real-time feedback rigid body six-degree-of-freedom measurement program, but also to temporarily store data that has been output or will be output.

[0144] The communication bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0145] The communication interface 13 is used for communication between the electronic device 1 and other devices, including a network interface and a user interface. Optionally, the network interface can include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used to establish a communication connection between the electronic device and other electronic devices. The user interface can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface can also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch screen, etc. Among them, the display can also be appropriately called a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0146] Figure 7 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 7 The structure shown does not constitute a limitation on the electronic device 1, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0147] For example, although not shown, the electronic device 1 can also include a power supply (such as a battery) for powering each component. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, so that functions such as charge management, discharge management, and power consumption management can be realized through the power management device. The power supply can also include one or more direct current or alternating current power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components. The electronic device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which are not described here.

[0148] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by this structure.

[0149] The real-time feedback rigid body six-degree-of-freedom measurement program stored in the memory 11 in the electronic device 1 is a combination of a plurality of instructions, which, when executed in the processor 10, can achieve:

[0150] An initial camera internal and external parameter is obtained by performing an initialization calibration operation on the pre-constructed monocular camera using a pre-constructed checkerboard calibration plate.

[0151] acquire a rigid body pasted with preset checkerboard markers, continuously collect and photograph the rigid body by using the monocular camera to obtain a rigid body historical image sequence and a rigid body real-time image;

[0152] extract a rigid body historical image from the rigid body historical image sequence in sequence, and perform image feature extraction on the rigid body historical image to obtain a rigid body marker feature set;

[0153] perform dynamic adjustment on the initial camera internal and external parameters according to a pre-constructed minimization re-projection error algorithm and the rigid body marker feature set to obtain dynamic camera internal and external parameters, and update the initial camera internal and external parameters by using the dynamic camera internal and external parameters;

[0154] when the rigid body historical image in the rigid body historical image sequence is traversed, perform image feature extraction on the rigid body real-time image to obtain a rigid body real-time feature set, and perform six-degree-of-freedom measurement on the rigid body real-time feature set according to the dynamic camera internal and external parameters to obtain a rigid body measurement result

[0155] Specifically, the specific implementation method of the processor 10 on the above instructions can refer to the description of the related steps in the corresponding embodiment of the drawings, which will not be described here.

[0156] Further, the modules / units integrated in the electronic device 1 are realized in the form of software function units and sold or used as independent products, which can be stored in a computer readable storage medium. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory).

[0157] The application also provides a computer readable storage medium, which stores a computer program, and the computer program can realize the following when executed by a processor of an electronic device:

[0158] perform an initial calibration operation on the pre-constructed monocular camera by using a pre-constructed checkerboard calibration board to obtain initial camera internal and external parameters;

[0159] acquire a rigid body pasted with preset checkerboard markers, continuously collect and photograph the rigid body by using the monocular camera to obtain a rigid body historical image sequence and a rigid body real-time image;

[0160] extract a rigid body historical image from the rigid body historical image sequence in sequence, and perform image feature extraction on the rigid body historical image to obtain a rigid body marker feature set;

[0161] According to the pre-constructed minimization reprojection error algorithm and the rigid marker feature set, the initial camera internal and external parameters are dynamically adjusted to obtain dynamic camera internal and external parameters, and the initial camera internal and external parameters are updated by using the dynamic camera internal and external parameters.

[0162] When the rigid historical image in the rigid historical image sequence is traversed, image feature extraction is performed on the rigid real-time image to obtain a rigid real-time feature set, and six-degree-of-freedom measurement is performed on the rigid real-time feature set according to the dynamic camera internal and external parameters to obtain a rigid measurement result

[0163] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other manners. For example, the embodiments of the apparatus described above are merely schematic; for example, the division of the modules is merely a logical function division; for example, an actual implementation can be another division manner.

[0164] The modules illustrated as separated components can or can not be physically separated, and the components illustrated as modules can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments.

[0165] In addition, each function module in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of hardware plus software function modules.

[0166] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0167] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be regarded as limiting the claims involved.

[0168] The blockchain referred to in the present application is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism and encryption algorithm. The blockchain is essentially a decentralized database, and is a series of data blocks associated using cryptographic methods. Each data block contains information of a batch of network transactions, and is used to verify the validity (anti-fake) of the information and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer and an application service layer.

[0169] Embodiments of the present application can acquire and process related data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0170] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or devices stated in the system claims can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, not any particular order.

[0171] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method of real-time feedback rigid body six degrees of freedom measurement, characterized by, The method comprises: initializing and calibrating a pre-constructed monocular camera to obtain initial camera internal and external parameters; acquiring a rigid body with a pre-set checkerboard marker, and continuously collecting and photographing the rigid body by using the monocular camera to obtain a rigid body historical image sequence and a rigid body real-time image; extracting a rigid body historical image from the rigid body historical image sequence in sequence, and performing image feature extraction on the rigid body historical image to obtain a rigid body marker feature set; dynamically adjusting the initial camera internal and external parameters according to a pre-constructed minimum re-projection error algorithm and the rigid body marker feature set to obtain dynamic camera internal and external parameters, and updating the initial camera internal and external parameters by using the dynamic camera internal and external parameters; when the rigid body historical image in the rigid body historical image sequence is traversed, performing image feature extraction on the rigid body real-time image to obtain a rigid body real-time feature set, and performing six-degree-of-freedom measurement on the rigid body real-time feature set according to the dynamic camera internal and external parameters to obtain a rigid body measurement result; wherein the updating of the initial camera internal and external parameters by using the dynamic camera internal and external parameters comprises: calculating a difference between the dynamic camera internal and external parameters and the initial camera internal and external parameters by using a pre-constructed incremental algorithm to obtain a parameter increment, wherein the incremental algorithm is represented as: wherein, denotes the updated initial camera intrinsic and extrinsic parameters, denotes the initial camera intrinsic and extrinsic parameters, denotes the dynamic camera intrinsic and extrinsic parameters, denotes a smooth factor ranging from 0 to 1. performing weighted calculation on the initial camera internal and external parameters according to the parameter increment to obtain updated initial camera internal and external parameters.

2. The real-time feedback rigid body six degree of freedom measurement method of claim 1, wherein, The continuous collection and photography of the rigid body by using the monocular camera to obtain the rigid body historical image sequence and the rigid body real-time image comprises: real-time acquisition of a real-time shooting image of the monocular camera, and object recognition on the real-time shooting image to obtain an object recognition result; judging whether the object recognition result appears a pre-set checkerboard marker object; when the checkerboard marker object appears in the object recognition result, target marking and recording are performed on the checkerboard marker object to obtain a wall body historical image video; when a photography instruction issued by a user is received, the rigid body historical image video is intercepted to obtain a rigid body real-time image, and a video in a pre-set time period before the rigid body real-time image in the rigid body historical image video is collected according to a pre-set interception frequency to obtain a rigid body historical image sequence.

3. The real-time feedback rigid body six degree of freedom measurement method of claim 1, wherein, The dynamic adjustment of the initial camera internal and external parameters according to the pre-constructed minimum re-projection error algorithm and the rigid body marker feature set to obtain dynamic camera internal and external parameters comprises: acquiring model size information of the rigid body; performing image prediction on the rigid body according to the initial camera internal and external parameters and the model size information by using a pre-constructed camera projection algorithm to obtain a rigid body predicted image; calculating a difference of the rigid body marker feature set for the rigid body predicted image according to a pre-constructed minimum re-projection error algorithm to obtain a re-projection error; performing minimum operation on the re-projection error by using a pre-constructed least square method to obtain dynamic camera internal and external parameters.

4. The real-time feedback rigid body six degree of freedom measurement method of claim 1, wherein, The pre-built checkerboard calibration plate is used to initialize and calibrate the pre-built monocular camera to obtain the initial camera internal and external parameters, including: Using a pre-built monocular camera, according to the preset multi-angle position information, a pre-built checkerboard calibration plate is used to obtain a calibration image sequence; Performing a checkerboard corner feature extraction operation on the calibration image sequence to obtain a corner feature information set; Acquiring size layout information of the checkerboard calibration plate, quantizing the size layout information, and obtaining a checkerboard layout feature vector; Using a pre-built calibration algorithm, shooting parameters of the corner feature information are identified according to the checkerboard layout feature vector to obtain initial camera internal and external parameters.

5. The real-time feedback rigid body six degree of freedom measurement method of claim 1, wherein, The step of extracting image features from the rigid body historical image to obtain a rigid body marker feature set includes: Grayscale processing is performed on the rigid body historical image to obtain a rigid body grayscale image, and Gaussian filtering and smoothing processing is performed on the rigid body grayscale image to obtain a rigid body denoised grayscale image; Performing a feature extraction operation on the rigid body denoised grayscale image to obtain a rigid body image feature set; The rigid body image feature set is subjected to feature classification judgment based on preset markers to obtain a rigid body marker feature set.

6. The real-time feedback rigid body six degree of freedom measurement method of claim 1, wherein, After obtaining the rigid body measurement result, the method further includes: Using two cameras in a pre-built experimental platform to obtain real measurement results of the rigid body; Performing a difference calculation using the rigid body measurement result and the true measurement result to obtain a measurement difference; Determining whether the measurement difference is greater than a preset qualification threshold; When the measurement difference is greater than a preset qualified threshold, an alarm prompt message is generated.

7. A real-time feedback rigid body six degree of freedom measurement device, characterized in that, The device comprises: The camera parameter initialization module is used to initialize and calibrate the pre-built monocular camera using the pre-built checkerboard calibration plate to obtain the initial camera internal and external parameters; A rigid body image acquisition module is used to acquire a rigid body with a preset checkerboard marker attached thereto, and continuously capture and photograph the rigid body using the monocular camera to obtain a rigid body historical image sequence and a rigid body real-time image; The real-time parameter update module is used to extract a rigid body historical image from the rigid body historical image sequence in sequence, perform image feature extraction on the rigid body historical image to obtain a rigid body marker feature set, and dynamically adjust the initial camera intrinsic and extrinsic parameters based on a pre-built minimization reprojection error algorithm and the rigid body marker feature set to obtain dynamic camera intrinsic and extrinsic parameters, and use the dynamic camera intrinsic and extrinsic parameters to update the initial camera intrinsic and extrinsic parameters using the following method: A pre-built incremental algorithm is used to calculate the difference between the dynamic camera intrinsic and extrinsic parameters and the initial camera intrinsic and extrinsic parameters to obtain parameter increments, wherein the incremental algorithm is expressed as: wherein, denotes the updated initial camera intrinsic and extrinsic parameters, denotes the initial camera intrinsic and extrinsic parameters, denotes the dynamic camera intrinsic and extrinsic parameters, denotes a smoothing factor ranging from 0 to 1. performing weighted calculation on the initial camera intrinsic and extrinsic parameters according to the parameter increment to obtain updated initial camera intrinsic and extrinsic parameters; A rigid body measurement module is configured to perform image feature extraction on the real-time rigid body image to obtain a real-time rigid body feature set after a rigid body history image in the rigid body history image sequence is traversed, and perform six-degree-of-freedom measurement on the real-time rigid body feature set according to the internal and external parameters of the dynamic camera to obtain a rigid body measurement result.

8. An electronic device, comprising: The electronic device includes: at least one processor; and a memory connected to the at least one processor in communication; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the real-time feedback rigid body six-degree-of-freedom measurement method of any one of claims 1 to 6.

9. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the real-time feedback rigid body six-degree-of-freedom measurement method of any one of claims 1 to 6.

Images