Method for measuring distance of sensitive target in construction site based on binocular attitude correction

Abstract

The invention relates to the technical field of building construction equipment, and provides a method for measuring the distance of a sensitive target in a construction site based on binocular attitude correction, and the method comprises the following steps: 1, carrying out the deployment of a camera; step 2, camera parameter calibration; step 3, correcting attitude parameters of the camera; step 4, target detection; and step 5, target distance measurement. According to the invention, by adjusting the angle of the camera deployed in the tower crane system, a more comprehensive field of view of a construction site can be obtained, and all-directional control of a construction environment is realized; a camera attitude parameter correction method is introduced, so that the reliability of a measurement result can still be maintained under the condition that the attitude of the camera is changed, and internal and external parameters of the camera are effectively prevented from being calibrated repeatedly; finally, the distance between the sensitive targets in the construction site is measured through binocular vision, the result is fed back to the site, and the safety of the construction environment is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of building construction equipment, and in particular to a sensitive target ranging method in a construction site based on binocular posture correction. Background Art

[0002] As one of the common lifting machines in the construction industry, tower cranes are responsible for horizontal and vertical lifting and site monitoring and measurement. At the same time, as the commanding height of the construction site, tower cranes have a global vision of the construction site. Therefore, using sensors deployed at high positions of tower cranes to monitor multiple sensitive targets in the construction site and calculate the target distance can effectively improve the safety of operations in the construction site. Currently, sensors deployed at high positions of tower cranes are mainly divided into two types according to their working principles: laser ranging and binocular vision ranging.

[0003] The laser ranging type realizes accurate construction of scene three-dimensional information by emitting laser beams and changing its reflection time or phase. For example, the Chinese invention patent with patent number 202311729531.X uses laser radar to obtain feature information such as point cloud data, visual angle, and distance of the tower crane, and combines the metric learning method to train the tower crane feature expression encoder to realize the estimation and tracking of the tower crane; the Chinese invention patent with patent number 202310217020.3 combines the camera and laser radar to realize the precise positioning of the hook, and performs three-dimensional spatial clustering and positioning of the hoist in the ground detection area to realize intelligent dynamic path planning; the Chinese invention patent with patent number 202010662401.9 uses laser radar to obtain the site point cloud, and sets the collision warning distance to realize anti-collision alarm by comparing the data differences of different time frames.

[0004] Binocular vision ranging method obtains the parallax information of two images and calculates the scene depth based on the principle of triangulation. For example, the Chinese invention patent with patent number 201810956766.5 uses a binocular camera to observe the position information of the tower hook and the workstation below to improve the construction efficiency of the tower crane; the Chinese invention patent with patent number 202010896334.7 uses a binocular camera to calculate the three-dimensional coordinates of the cargo in the base coordinate system, and combines the feedback value of the mechanical sensor to determine whether the lifting is successful, so as to realize automatic loading of the tower crane; the Chinese invention patent with patent number 201821782774.4 uses multiple sets of binocular cameras to collect images from different perspectives to generate three-dimensional point cloud data to construct a three-dimensional real-life map; the Chinese invention patent with patent number 201811426087.3 uses the binocular camera of the boom and the SGBM algorithm to realize the spatial position recognition of "people-objects" in the construction site.

[0005] The technical threshold and cost of laser ranging are relatively high, and it is easy to generate redundant data. The binocular vision ranging method only needs two cameras and related image processing algorithms to achieve high-precision measurement. It has the characteristics of wide application range and relatively cheap hardware equipment. However, for large scene measurement, it is still limited by the baseline distance between binocular cameras and the fixed direction of the field of view angle; that is, when the camera rotates freely, the posture relationship between the left and right cameras will change, and the posture calibration needs to be performed again, which to a certain extent limits its application in tower crane space measurement. Therefore, under the premise of ensuring the ranging accuracy, the posture between the cameras must be corrected to improve its accuracy and applicability. Summary of the invention

[0006] In response to the above problems and shortcomings, the present invention provides a method for measuring the distance to sensitive targets in a construction site based on binocular attitude correction. An integrated camera is deployed at a high position of a tower crane, and the integrated camera is equipped with a gyroscope and a 4G antenna to accurately measure the distance to sensitive targets in the construction site.

[0007] The technical solution of the present invention is as follows:

[0008] A method for measuring distance of sensitive targets in a construction site based on binocular posture correction comprises the following steps:

[0009] Step 1: Camera deployment

[0010] The first camera and the second camera are respectively deployed on the first tower crane and the second tower crane to ensure that the first camera and the second camera can observe the same area from different angles, providing necessary viewing angle differences for subsequent image processing; the first camera and the second camera are both embedded with gyroscopes, which can measure the deflection angle increment of the first camera and the second camera in real time; the first camera and the second camera have wireless data transmission devices, which can transmit image data to the data processing center in real time;

[0011] Step 2: Camera parameter calibration

[0012] Placing a checkerboard calibration plate in a common field of view of a first camera and a second camera, keeping the first camera and the second camera stationary, acquiring multiple sets of checkerboard calibration plate images at different angles and positions by moving the checkerboard calibration plate, and acquiring intrinsic parameters of the first camera and the second camera, as well as initial extrinsic parameters between the first camera and the second camera using a camera calibration algorithm; wherein the intrinsic parameters of the first camera include an equivalent focal length in an imaging plane of the first camera pointing in a direction of the second camera and an equivalent focal length along a direction perpendicular to an optical axis, and pixel coordinates of an optical center of the first camera in an imaging plane; the intrinsic parameters of the second camera include an equivalent focal length in an imaging plane of the second camera pointing in a direction of the first camera and an equivalent focal length along a direction perpendicular to an optical axis, and pixel coordinates of an optical center of the second camera in an imaging plane;

[0013] Step 3: Camera posture parameter correction

[0014] Under the premise of ensuring a dynamic public field of view, the first camera and the second camera are freely rotated to change the direction of their field of view angles, and the deflection angle increments of the first camera and the second camera are output by the built-in gyroscope. The initial external parameters between the first camera and the second camera are corrected by using the deflection angle increments to obtain the final external parameters, ensuring that measurement can still be performed even when the postures of the first camera and the second camera change;

[0015] Step 4: Target Detection

[0016] A dataset is created using site photos collected by the first camera and the second camera, and the photo content is annotated using annotation tools, where the annotated content includes target category information; the image collected by the first camera is processed using a target detection algorithm to locate the target frame, and the pixel coordinates of the centroid of the target frame are output;

[0017] Step 5: Target distance measurement

[0018] Based on the intrinsic parameters, the final extrinsic parameters and the pixel coordinates of the target frame's centroid, the three-dimensional coordinates corresponding to the pixel coordinates of the target frame's centroid are calculated in combination with the principle of binocular vision triangulation, thereby achieving accurate measurement of the distance between target objects and transmitting the results to the data processing center for feedback on the on-site situation.

[0019] Preferably, a method for measuring the distance of sensitive targets in a construction site based on binocular posture correction comprises the following steps:

[0020] Step 1: Camera deployment

[0021] The first camera and the second camera are respectively deployed on the first tower crane and the second tower crane to have a good field of view, and to ensure that the first camera and the second camera can observe the same area from different angles, so as to provide necessary perspective differences for subsequent image processing;

[0022] The coordinate axes of the camera coordinate system of the first camera include X cl Axis, Y cl Axis and Z cl Axis, X cl The Y axis is the direction from the first camera to the second camera; cl The axis is perpendicular to the optical axis of the first camera, Z cl The axis is the direction of the optical axis of the first camera, and the coordinate axes of the camera coordinate system of the second camera include X cr Axis, Y cr Axis and Z cr Axis, X cr Axis and X cl Axis in the same direction, Z cr The Y axis is the optical axis direction of the second camera. cr Axis is X cr Axis and Z cr Axis cross product direction;

[0023] The first camera and the second camera have built-in gyroscopes, which can measure the rotation of the first camera around X in real time. cl The deflection angle increment Δα of the axis _l , around Y cl The deflection angle increment Δβ of the axis _l , around Z cl The deflection angle increment Δγ produced by the axis _l , and real-time measurement of the second camera 4 around X cr The deflection angle increment Δα of the axis _r , around Y cr The deflection angle increment Δβ of the axis _r , around Z cr The deflection angle increment Δγ produced by the axis _r ; In addition, the first camera and the second camera have wireless data transmission devices, which can transmit image data to the data processing center in real time;

[0024] Step 2: Camera parameter calibration

[0025] Place the checkerboard calibration plate in the common field of view of the first camera and the second camera, keep the first camera and the second camera stationary, obtain multiple sets of checkerboard calibration plate images at different angles and positions by moving the checkerboard calibration plate, and use the camera calibration algorithm to obtain the intrinsic parameters f of the first camera and the second camera x 、f y 、u p0 and v p0 ;f x and f y They represent the X direction in the imaging plane of the first camera. cl Axis and Ycl The equivalent focal length in the X-axis direction or the equivalent focal length in the imaging plane of the second camera cr Axis and Y cr The equivalent focal length in the axial direction, (u p0 , v p0 ) corresponds to the pixel coordinates of the optical center of the first camera or the second camera in the imaging plane; and the initial external parameter M0 between the first camera and the second camera, M0 represents the relative position relationship between the first camera and the second camera, and its expression is as follows:

[0026]

[0027] Where: R0, T0 are the initial rotation matrix and initial translation matrix between the first camera and the second camera respectively; T =[0,0,0] is a 1×3 row vector whose elements are all 0;

[0028] Step 3: Camera posture parameter correction

[0029] Under the premise of ensuring a dynamic public field of view, the first camera and the second camera are freely rotated to change the direction of their field of view angles, and the built-in gyroscope is used to output the deflection angle increment Δα of the first camera. _l , Δβ _l and Δγ _l , and the second camera deflection angle increment Δα _r , Δβ _r and Δγ _r , use the deflection angle increment to correct the initial external parameters between the first camera and the second camera, and obtain the final external parameters M F , ensuring that measurement can be performed even when the postures of the first camera and the second camera change;

[0030] Final external parameter M F The acquisition process is as follows:

[0031] The coordinates of the target point to be measured are P(x c ,y c , z c ), the initial coordinates of the target point in the camera coordinate system of the first camera and the camera coordinate system of the second camera are P cl_0 (x cl_0 ,y cl_0 , z cl_0 ) and P cr_0 (x cr_0 ,y cr_0 , z cr_0 ), and the first camera and the second camera satisfy the following relationship:

[0032]

[0033] As the first camera and the second camera rotate, the gyroscope obtains the first camera around Z cl The axis rotation angle increment is Δγ _l , around Y cl The axis rotation angle increment is Δβ _l , around X cl The axis rotation angle increment is Δα _l ; The second camera orbits Z cr The axis rotation angle increment is Δγ _r , around Y cr The axis rotation angle increment is Δβ _r , around X cr The axis rotation angle increment is Δα _r ;

[0034] Then the rotation matrix of the first camera after rotation relative to the first camera during calibration is:

[0035]

[0036] The rotation matrices of the rotated second camera relative to the calibrated second camera are:

[0037]

[0038] Combining equations (2), (3) and (4), we can get the total rotation matrix R of the first camera: l for:

[0039] R l =R l (α)·R l (β)·R l (γ) (9)

[0040] Combining equations (5), (6) and (7), we can get the total rotation matrix R of the second camera: r for:

[0041] R r =R r (α)·R r (β)·R r (γ). (10)

[0042] The first coordinate change of the measured target point P relative to the first camera coordinate system is P cl_1 (x cl_1 ,y cl_1 , z cl_1 ), relative to the camera coordinate system of the second camera is P cr_1 (x cr_1 ,y cr_1 , zcr_1 );

[0043] Then P cl_1 (x cl_1 ,y cl_1 , z cl_1 ) and P cl_0 (x cl_0 ,y cl_0 , z cl_0 ) satisfies the following relationship:

[0044]

[0045] Where: R l_1 is the total rotation matrix of the first camera before and after the first rotation transformation, M l_1 YesR l_1 The homogeneous transformation matrix of ;

[0046] P cr_1 (x cr_1 ,y cr_1 , z cr_1 ) and P cr_0 (x cr_0 ,y cr_0 , z cr_0 ) satisfies the following relationship:

[0047]

[0048] Where: R r_1 is the total rotation matrix of the first camera before and after the first rotation transformation, M r_1 YesR r_1 The homogeneous transformation matrix of ;

[0049] Substituting equation (11) and equation (12) into equation (2), we can obtain:

[0050]

[0051] Similarly, the relationship between the first camera rotating n times and the second camera rotating m times satisfies:

[0052]

[0053] Where: M l_n is the nth homogeneous transformation matrix of the first camera, M l_m is the mth homogeneous transformation matrix of the second camera, M F are the final external parameters of the first camera and the second camera;

[0054] Step 4: Target Detection

[0055] The first camera and the second camera are used to collect site photos to create a data set, and the annotation tools are used to annotate the content of the photos, and the annotated content is the category information of the sensitive target; the image collected by the first camera is processed by the target detection algorithm to locate the sensitive target and output the pixel coordinates of the centroid of the target frame (u pl , v pl );

[0056] Step 5: Target distance measurement

[0057] Based on the internal parameter f in step 2 x 、f y 、u p0 and v p0 , the final external parameter M obtained in step 3 F , and the pixel coordinates of the centroid of the target frame output in step 4 (u pl , v pl ), combined with the binocular vision triangulation principle to calculate the pixel coordinates of the target frame center of mass (u pl , v pl ) corresponds to the three-dimensional coordinates (x c ,y c , z c ), thereby achieving distance measurement between sensitive targets and transmitting the results to the data processing center for feedback on on-site conditions;

[0058] Three-dimensional coordinates (x c ,y c , z c ) are specifically obtained as follows:

[0059] According to the similarity of triangles, calculate the three-dimensional coordinates of the target point P:

[0060]

[0061] Wherein: b is the baseline distance between the optical centers of the first camera and the second camera, that is, the modulus of the translation matrix T0; d is the disparity, which can be obtained by stereo matching.

[0062] Preferably, the wireless data transmission device includes a 4G antenna or a 5G antenna.

[0063] Preferably, the wireless data transmission device is externally or internally mounted on the first camera and the second camera.

[0064] Preferably, in step three, the misaligned imaging plane state of the first camera and the second camera can be converted to an aligned imaging plane state according to the stereo correction, that is, the posture correction of the first camera and the second camera is achieved.

[0065] Preferably, the sensitive targets include workers, excavators and transport vehicles.

[0066] Preferably, the labeling tool is a Labelimg tool, and the target detection algorithm is a YOLO algorithm.

[0067] The beneficial effects of the present invention are as follows: by adjusting the angle of the camera deployed in the tower crane system, a more comprehensive view of the construction site can be obtained, thereby achieving all-round control of the construction environment; a camera attitude parameter correction method is introduced to ensure that the reliability of the measurement results can be maintained when the camera attitude changes, thereby effectively avoiding repeated calibration of the camera's internal and external parameters; finally, the distance between sensitive targets in the construction site is measured through binocular vision, and the results are fed back to the site, thereby improving the safety of the construction environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0069] Figure 1 This is a schematic diagram of the calibration of the internal and external parameters of the camera in the present invention;

[0070] Figure 2 This is a schematic diagram of binocular vision measurement principle when the imaging planes are not aligned in the present invention;

[0071] Figure 3 This is a schematic diagram of binocular vision measurement principle when the imaging plane is aligned in the present invention;

[0072] Figure 4 Schematic diagram of sensitive target ranging after the camera attitude parameters are corrected in the present invention.

[0073] In the figure: 1-first tower crane; 2-second tower crane; 3-first camera; 4-second camera; 5-chessboard calibration board; 6-initial public field of view; 7-dynamic public field of view. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] Example 1

[0076] A method for measuring distance of sensitive targets in a construction site based on binocular posture correction comprises the following steps:

[0077] Step 1: Camera deployment

[0078] Reference Figure 1 , the first camera 3 and the second camera 4 are respectively deployed on the first tower crane 1 and the second tower crane 2 to ensure that the first camera 3 and the second camera 4 can observe the same area from different angles, providing necessary viewing angle differences for subsequent image processing; the first camera 3 and the second camera 4 are both embedded with gyroscopes, which can measure the deflection angle increments of the first camera 3 and the second camera 4 in real time; the first camera 3 and the second camera 4 have wireless data transmission devices, which can transmit image data to the data processing center in real time;

[0079] Step 2: Camera parameter calibration

[0080] Place the checkerboard calibration plate 5 in the common field of view of the first camera 3 and the second camera 4, keep the first camera 3 and the second camera 4 stationary, obtain multiple sets of images of the checkerboard calibration plate 5 at different angles and positions by moving the checkerboard calibration plate 5, and use the camera calibration algorithm to obtain the intrinsic parameters of the first camera 3 and the second camera 4, as well as the initial extrinsic parameters between the first camera 3 and the second camera 4;

[0081] Step 3: Camera posture parameter correction

[0082] Reference Figure 4 , under the premise of ensuring a dynamic public field of view 7, the first camera 3 and the second camera 4 are freely rotated to change the direction of their field of view angles, the built-in gyroscope is used to output the deflection angle increment of the first camera 3 and the second camera 4, and the deflection angle increment is used to correct the initial external parameters between the first camera 3 and the second camera 4 to obtain the final external parameters, ensuring that measurement can still be performed even when the postures of the first camera 3 and the second camera 4 change;

[0083] Step 4: Target Detection

[0084] A data set is created using the site photos collected by the first camera 3 and the second camera 4, and the photo contents are annotated using an annotation tool, where the annotated contents include target object category information; the images collected by the first camera 3 are processed using a target detection algorithm to locate the target frame, and the pixel coordinates of the centroid of the target frame are output;

[0085] Step 5: Target distance measurement

[0086] Based on the intrinsic parameters, the final extrinsic parameters and the pixel coordinates of the target frame's centroid, the three-dimensional coordinates corresponding to the pixel coordinates of the target frame's centroid are calculated in combination with the principle of binocular vision triangulation, thereby achieving accurate measurement of the distance between target objects and transmitting the results to the data processing center for feedback on the on-site situation.

[0087] Among them, the intrinsic parameters of the first camera include the equivalent focal length of the imaging plane of the first camera pointing in the direction of the second camera and the equivalent focal length along the direction perpendicular to the optical axis, and the pixel coordinates of the optical center of the first camera in the imaging plane; the intrinsic parameters of the second camera include the equivalent focal length of the imaging plane of the second camera pointing in the direction of the first camera and the equivalent focal length along the direction perpendicular to the optical axis, and the pixel coordinates of the optical center of the second camera in the imaging plane.

[0088] Example 2

[0089] A method for measuring distance of sensitive targets in a construction site based on binocular posture correction comprises the following steps:

[0090] Step 1: Camera deployment

[0091] Reference Figure 1 , deploy the first camera 3 and the second camera 4 on the first tower crane 1 and the second tower crane 2 respectively, so that they have a good field of view, and ensure that the first camera 3 and the second camera 4 can observe the same area from different angles, providing necessary perspective differences for subsequent image processing;

[0092] The coordinate axes of the camera coordinate system of the first camera 3 include X cl Axis, Y cl Axis and Z cl Axis, X cl The Y axis is the direction from the first camera 3 to the second camera 4; cl The axis is perpendicular to the optical axis of the first camera 3, cl The axis is the direction of the optical axis of the first camera 3, and the coordinate axes of the camera coordinate system of the second camera 4 include X cr Axis, Y cr Axis and Z cr Axis, X cr Axis and X cl Axis in the same direction, Z cr The Y axis is the optical axis direction of the second camera 4, cr Axis is X cr Axis and Z cr Axis cross product direction;

[0093] The first camera 3 and the second camera 4 have built-in gyroscopes, which can measure the rotation of the first camera 3 around X in real time. cl The deflection angle increment Δα of the axis _l , around Ycl The deflection angle increment Δβ of the axis _l , around Z cl The deflection angle increment Δγ produced by the axis _l , and real-time measurement of the second camera 4 around X cr The deflection angle increment Δα of the axis _r , around Y cr The deflection angle increment Δβ of the axis _r , around Z cr The deflection angle increment Δγ produced by the axis _r ; In addition, the first camera 3 and the second camera 4 have wireless data transmission devices, which can transmit image data to the data processing center in real time;

[0094] Step 2: Camera parameter calibration

[0095] Reference Figure 1 , place the checkerboard calibration plate 5 in the common field of view of the first camera 3 and the second camera 4, keep the first camera 3 and the second camera 4 stationary, obtain multiple sets of images of the checkerboard calibration plate 5 at different angles and positions by moving the checkerboard calibration plate 5, and use the camera calibration algorithm to obtain the intrinsic parameters fx, fy, u of the first camera 3 and the second camera 4 p0 and v p0 (The intrinsic parameter f of the first camera 3 x and f y They respectively represent the X direction in the imaging plane of the first camera 3 c Axis and Y cl The equivalent focal length in the axis direction, the intrinsic parameters of the first camera 3 (u p0 , v p0 ) is the pixel coordinate of the optical center of the lens of the first camera 3 in the imaging plane; the intrinsic parameter f of the second camera 4 x and f y They respectively represent the X direction in the imaging plane of the second camera 3 c Axis and Y cl The equivalent focal length in the axis direction, the intrinsic parameters of the second camera 3 (u p0 , v p0 ) is the pixel coordinate of the optical center of the lens of the first camera 3 in the imaging plane), and the initial external parameter M0 between the first camera 3 and the second camera 4; M0 represents the relative position relationship between the first camera 3 and the second camera 4, which is obtained by the camera calibration in step 2, and its expression is as follows:

[0096]

[0097] Where: R0, T0 are the initial rotation matrix and initial translation matrix between the first camera 3 and the second camera 4 respectively; T=[0,0,0] is a 1×3 row vector whose elements are all 0;

[0098] Step 3: Camera posture parameter correction

[0099] Reference Figure 4 , under the premise of ensuring a dynamic public field of view 7, the first camera 3 and the second camera 4 are freely rotated to change the direction of their field of view angles, and the built-in gyroscope is used to output the deflection angle increment Δα of the first camera 3 _l , Δβ _l and Δγ _l , and the deflection angle increment Δα of the second camera 4 _r , Δβ _r and Δγ _r , use the deflection angle increment to correct the initial external parameters between the first camera 3 and the second camera 4, and obtain the final external parameters M F , ensuring that measurement can still be performed even when the postures of the first camera 3 and the second camera 4 change;

[0100] Final external parameter M F The acquisition process is as follows:

[0101] Reference Figure 2 , the coordinates of the target point to be measured are P(x c ,y c , z c ), the initial coordinates of the target point in the camera coordinate system of the first camera 3 and the camera coordinate system of the second camera 4 are P cl_0 (x cl_0 ,y cl_0 , z cl_0 ) and P cr_0 (x cr_0 ,y cr_0 , z cr_0 ), and the first camera 3 and the second camera 4 satisfy the following relationship:

[0102]

[0103] As the first camera 3 and the second camera 4 rotate, the gyroscope obtains the first camera 3 around Z cl The axis rotation angle increment is Δγ _l , around Y cl The axis rotation angle increment is Δβ _l , around X cl The axis rotation angle increment is Δα _l ; The second camera 4 is around Z cr The axis rotation angle increment is Δγ _r , around Y cr The axis rotation angle increment is Δβ _r , around Xcr The axis rotation angle increment is Δα _r ;

[0104] Then the rotation matrix of the first camera 3 after rotation relative to the first camera 3 during calibration is:

[0105]

[0106]

[0107] The rotation matrices of the second camera 4 after rotation relative to the second camera 4 during calibration are:

[0108]

[0109] Combining equations (2), (3) and (4), the total rotation matrix of the first camera 3 can be obtained as:

[0110] R l =R l (α)·R l (β)·R l (γ). (9)

[0111] Combining equations (5), (6) and (7), the total rotation matrix of the second camera 4 can be obtained as:

[0112] R r =R r (α)·R r (β)·R r (γ). (10)

[0113] The first coordinate change of the measured target point P relative to the camera coordinate system of the first camera 3 is P cl_1 (x cl_1 ,y cl_1 , z cl_1 ); relative to the camera coordinate system of the second camera 4 is P cr_1 (x cr_1 ,y cr_1 , z cr_1 );

[0114] Then P cl_1 (x cl_1 ,y cl_1 , z cl_1 ) and P cl_0 (x cl_0 ,y cl_0 , z cl_0 ) satisfies the following relationship:

[0115]

[0116] Where: Rl_1 is the total rotation matrix of the first camera 3 before and after the first rotation transformation, M l_1 YesR l_1 The homogeneous transformation matrix of ;

[0117] P cr_1 (x cr_1 ,y cr_1 , z cr_1 ) and P cr_0 (x cr_0 ,y cr_0 , z cr_0 ) satisfies the following relationship:

[0118]

[0119] Where: R r_1 is the total rotation matrix of the first camera 4 before and after the first rotation transformation; M r_1 YesR r_1 The homogeneous transformation matrix of ;

[0120] Substituting equation (10) and equation (11) into equation (2), we can obtain:

[0121]

[0122] Similarly, the relationship between the first camera 3 rotating n times and the second camera 4 rotating m times satisfies:

[0123]

[0124] Where: M l_n is the nth homogeneous transformation matrix of the first camera 3; M l_m is the mth homogeneous transformation matrix of the second camera 4; M F are the final external parameters of the first camera 3 and the second camera 4;

[0125] According to the stereo correction, Figure 2 The state where the imaging planes of the first camera 3 and the second camera 4 are not aligned is converted to Figure 3 The imaging plane is aligned to a state, that is, the posture correction of the first camera 3 and the second camera 4 is achieved;

[0126] Step 4: Target Detection

[0127] The first camera 3 and the second camera 4 are used to collect site photos to create a data set, and the photo content is annotated using an annotation tool, and the annotated content is the category information of the sensitive target; the image collected by the first camera 3 is processed by the target detection algorithm to locate the sensitive target, and the pixel coordinates of the centroid of the target frame (u pl , v pl );

[0128] Step 5: Target distance measurement

[0129] Based on the internal parameter f in step 2 x 、f y 、u p0 and v p0 , the final external parameter M obtained in step 3 F , and the pixel coordinates of the centroid of the target frame output in step 4 (u pl , v pl ), combined with the binocular vision triangulation principle to calculate the pixel coordinates of the target frame center of mass (u pl , v pl ) corresponds to the three-dimensional coordinates (x c ,y c , z c ), thereby achieving distance measurement between sensitive targets and transmitting the results to the data processing center for feedback on on-site conditions;

[0130] Three-dimensional coordinates (x c ,y c , z c ) are specifically obtained as follows:

[0131] like Figure 3 As shown, according to the triangle PO cl O cr With triangle PP l P r The similarity of , then the three-dimensional coordinates of the target point P can be calculated, and the three-dimensional coordinates of the target point P can be calculated:

[0132]

[0133] Wherein: b is the baseline distance between the optical centers of the first camera 3 and the second camera 4, that is, the modulus of the translation matrix T0; d is the parallax, which can be obtained by stereo matching.

[0134] As a preferred technical solution, the wireless data transmission device includes a 4G antenna or a 5G antenna. The wireless data transmission device is externally or internally mounted on the first camera 3 and the second camera 4.

[0135] Among them, the sensitive targets include workers, excavators and transport vehicles, but are certainly not limited to the three types listed above. Depending on the project, sensitive targets can include a variety of different purposes.

[0136] Wherein, the chessboard calibration plate 5 is a black and white chessboard calibration plate.

[0137] Preferably, the labeling tool is a Labelimg tool. Labelline is a common tool for data labeling and training models. It can help us quickly and efficiently add labels to large-scale data sets, thereby providing a data basis for the application of machine learning and artificial intelligence. The target detection algorithm is the YOLO algorithm. The YOLO algorithm (You Only LookOnce) is a target detection algorithm based on deep learning. Its core idea is to achieve end-to-end target detection through a single convolutional neural network (CNN), thereby greatly improving the detection speed and efficiency.

[0138] The camera calibration algorithm, target detection algorithm and binocular vision measurement algorithm involved in the above technologies are existing technologies that have been widely used in various industries and have high reliability and accuracy.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A sensitive target ranging method in a construction site based on binocular attitude correction, characterized in that: The following steps are involved: Step 1: Camera deployment The first camera and the second camera are respectively deployed on the first tower crane and the second tower crane to ensure that the first camera and the second camera can observe the same area from different angles, providing necessary perspective differences for subsequent image processing; The first camera and the second camera are both embedded with a gyroscope, which can measure the deflection angle increment of the first camera and the second camera in real time; The first camera and the second camera have wireless data transmission devices, which can transmit image data to the data processing center in real time; Step 2: Camera parameter calibration Placing a checkerboard calibration plate in the common field of view of the first camera and the second camera, keeping the first camera and the second camera stationary, obtaining multiple sets of checkerboard calibration plate images at different angles and positions by moving the checkerboard calibration plate, and obtaining intrinsic parameters of the first camera and the second camera, as well as initial extrinsic parameters between the first camera and the second camera using a camera calibration algorithm; Step 3: Camera posture parameter correction Under the premise of ensuring a dynamic public field of view, the first camera and the second camera are freely rotated to change the direction of their field of view angles, and the deflection angle increments of the first camera and the second camera are output by the built-in gyroscope. The initial external parameters between the first camera and the second camera are corrected by using the deflection angle increments to obtain the final external parameters, ensuring that measurement can still be performed even when the postures of the first camera and the second camera change; Step 4: Target Detection A dataset is created using site photos collected by the first camera and the second camera, and the photo content is annotated using annotation tools, where the annotated content includes target category information; the image collected by the first camera is processed using a target detection algorithm to locate the target frame, and the pixel coordinates of the centroid of the target frame are output; Step 5: Target distance measurement Based on the intrinsic parameters, the final extrinsic parameters and the pixel coordinates of the target frame's centroid, the three-dimensional coordinates corresponding to the pixel coordinates of the target frame's centroid are calculated in combination with the principle of binocular vision triangulation, thereby achieving accurate measurement of the distance between target objects and transmitting the results to the data processing center for feedback on the on-site situation.

2. The sensitive target ranging method in a construction site based on binocular posture correction as claimed in claim 1, characterized in that: The following steps are involved: Step 1: Camera deployment The first camera and the second camera are respectively deployed on the first tower crane and the second tower crane to have a good field of view, and to ensure that the first camera and the second camera can observe the same area from different angles, so as to provide necessary perspective differences for subsequent image processing; The coordinate axes of the camera coordinate system of the first camera include X cl Axis, Y cl Axis and Z cl Axis, X cl The Y axis is the direction from the first camera to the second camera; cl The axis is perpendicular to the optical axis of the first camera, Z cl The axis is the direction of the optical axis of the first camera, and the coordinate axes of the camera coordinate system of the second camera include X cr Axis, Y cr Axis and Z cr Axis, X cr Axis and X cl Axis in the same direction, Z cr The Y axis is the optical axis direction of the second camera. cr Axis is X cr Axis and Z cr Axis cross product direction; The first camera and the second camera have built-in gyroscopes, which can measure the rotation of the first camera around X in real time. cl The deflection angle increment Δα of the axis _l , around Y cl The deflection angle increment Δβ of the axis _l , around Z cl The deflection angle increment Δγ produced by the axis _l , and real-time measurement of the second camera 4 around X cr The deflection angle increment Δα of the axis _r , around Y cr The deflection angle increment Δβ of the axis _r , around Z cr The deflection angle increment Δγ produced by the axis _r ; In addition, the first camera and the second camera have wireless data transmission devices, which can transmit image data to the data processing center in real time; Step 2: Camera parameter calibration Place the checkerboard calibration plate in the common field of view of the first camera and the second camera, keep the first camera and the second camera stationary, obtain multiple sets of checkerboard calibration plate images at different angles and positions by moving the checkerboard calibration plate, and use the camera calibration algorithm to obtain the intrinsic parameters f of the first camera and the second camera x 、f y 、u p0 and v p0 , and the initial external parameters M0 between the first camera and the second camera; f x and f y They represent the X direction in the imaging plane of the first camera. cl Axis and Y cl The equivalent focal length in the X-axis direction or the equivalent focal length in the imaging plane of the second camera cr Axis and Y cr The equivalent focal length in the axial direction, (u p0 , v p0 ) corresponds to the pixel coordinates of the optical center of the first camera or the second camera in the imaging plane; M0 represents the relative position relationship between the first camera and the second camera, and its expression is as follows: Where: R0, T0 are the initial rotation matrix and initial translation matrix between the first camera and the second camera respectively; T =[0,0,0] is a 1×3 row vector whose elements are all 0; Step 3: Camera posture parameter correction Under the premise of ensuring a dynamic public field of view, the first camera and the second camera are freely rotated to change the direction of their field of view angles, and the built-in gyroscope is used to output the deflection angle increment Δα of the first camera. _l , Δβ _l and Δγ _l , and the second camera deflection angle increment Δα _r , Δβ _r and Δγ _r , use the deflection angle increment to correct the initial external parameters between the first camera and the second camera, and obtain the final external parameters M F , ensuring that measurement can be performed even when the postures of the first camera and the second camera change; Final external parameter M F The acquisition process is as follows: The coordinates of the target point to be measured are P(x c ,y c , z c ), the initial coordinates of the target point in the camera coordinate system of the first camera and the camera coordinate system of the second camera are P cl_0 (x cl_0 ,y cl_0 , z cl_0 ) and P cr_0 (x cr_0 ,y cr_0 , z cr_0 ), and the first camera and the second camera satisfy the following relationship: As the first camera and the second camera rotate, the gyroscope obtains the first camera around Z cl The axis rotation angle increment is Δγ _l , around Y cl The axis rotation angle increment is Δβ _l , around X cl The axis rotation angle increment is Δα _l ; The second camera orbits Z cr The axis rotation angle increment is Δγ _r , around Y cr The axis rotation angle increment is Δβ _r , around X cr The axis rotation angle increment is Δα _r ; Then the rotation matrix of the first camera after rotation relative to the first camera during calibration is: The rotation matrices of the rotated second camera relative to the calibrated second camera are: Combining equations (2), (3) and (4), we can get the total rotation matrix R of the first camera: l for: R l =R l (a)·R l (b)·R l (c). (9) Combining equations (5), (6) and (7), we can get the total rotation matrix R of the second camera: r for: R r =R r (a)·R r (b)·R r (c). (10) The first coordinate change of the measured target point P relative to the first camera coordinate system is P cl_1 (x cl_1 ,y cl_1 , z cl_1 ), relative to the camera coordinate system of the second camera is P cr_1 (x cr_1 ,y cr_1 , z cr_1 ); Then P cl_1 (x cl_1 ,y cl_1 , z cl_1 ) and P cl_0 (x cl_0 ,y cl_0 , z cl_0 ) satisfies the following relationship: Where: R l_1 is the total rotation matrix of the first camera before and after the first rotation transformation, M l_1 YesR l_1 The homogeneous transformation matrix of ; P cr_1 (x cr_1 ,y cr_1 , z cr_1 ) and P cr_0 (x cr_0 ,y cr_0 , z cr_0 ) satisfies the following relationship: Where: R r_1 is the total rotation matrix of the first camera before and after the first rotation transformation, M r_1 YesR r_1 The homogeneous transformation matrix of ; Substituting equation (11) and equation (12) into equation (2), we can obtain: Similarly, the relationship between the first camera rotating n times and the second camera rotating m times satisfies: Where: M l_n is the nth homogeneous transformation matrix of the first camera, M l_m is the mth homogeneous transformation matrix of the second camera, M F are the final external parameters of the first camera and the second camera; Step 4: Target Detection The first camera and the second camera are used to collect site photos to create a data set, and the annotation tools are used to annotate the content of the photos, and the annotated content is the category information of sensitive targets; the images collected by the first camera are processed by the target detection algorithm to locate the sensitive targets and output the pixel coordinates of the centroid of the target frame (u pl , v pl ); Step 5: Target distance measurement Based on the internal parameter f in step 2 x 、f y 、u p0 and v p0 , the final external parameter M obtained in step 3 F , and the pixel coordinates of the centroid of the target frame output in step 4 (u pl , v pl ), combined with the binocular vision triangulation principle to calculate the pixel coordinates of the target frame center of mass (u pl , v pl ) corresponds to the three-dimensional coordinate (x c ,y c , z c ), thereby achieving distance measurement between sensitive targets and transmitting the results to the data processing center for feedback on on-site conditions; Three-dimensional coordinates (x c ,y c , z c ) are specifically obtained as follows: According to the similarity of triangles, calculate the three-dimensional coordinates of the target point P: Wherein: b is the baseline distance between the optical centers of the first camera and the second camera, that is, the modulus of the translation matrix T0; d is the disparity, which can be obtained by stereo matching.

3. The sensitive target ranging method in a construction site based on binocular posture correction as claimed in claim 1, characterized in that: The intrinsic parameters of the first camera include the equivalent focal length in the imaging plane of the first camera pointing in the direction of the second camera and the equivalent focal length along the direction perpendicular to the optical axis, and the pixel coordinates of the optical center of the first camera in the imaging plane; the intrinsic parameters of the second camera include the equivalent focal length in the imaging plane of the second camera pointing in the direction of the first camera and the equivalent focal length along the direction perpendicular to the optical axis, and the pixel coordinates of the optical center of the second camera in the imaging plane.

4. The sensitive target ranging method in a construction site based on binocular posture correction as claimed in claim 1 or 2, characterized in that: The wireless data transmission device includes a 4G antenna or a 5G antenna.

5. The sensitive target ranging method in a construction site based on binocular posture correction as claimed in claim 4, characterized in that: The wireless data transmission device is externally or internally mounted on the first camera and the second camera.

6. The sensitive target ranging method in a construction site based on binocular posture correction as claimed in claim 2, characterized in that: In step three, the misaligned imaging plane state of the first camera and the second camera can be converted to an aligned imaging plane state according to the stereo correction, that is, the posture correction of the first camera and the second camera is achieved.

7. The sensitive target ranging method in a construction site based on binocular posture correction as claimed in claim 1 or 2, characterized in that: The sensitive targets include workers, excavators and transport vehicles.

8. The sensitive target ranging method in a construction site based on binocular posture correction as claimed in claim 2, characterized in that: The labeling tool is the Labelimg tool, and the target detection algorithm is the YOLO algorithm.

Images