A positioning detection method based on deep learning and monocular large field of view plane measurement

Abstract

The present invention provides a positioning detection method based on deep learning and monocular large field of view plane measurement. First, a calibration system is established, and a measurement plane and a calibration plane are set respectively. The measurement plane is the plane where the object to be measured is located. The set calibration plane is parallel to the measurement plane, and the distance h between the two planes is recorded; a number of calibration chessboards are placed in the calibration plane, and the No. 1 calibration chessboard coordinate system is selected as the entire calibration plane coordinate system, and then the corresponding transformation relationship between the calibration plane, the measurement plane and the image plane is found. Finally, based on image recognition technology, the image of the object to be measured is collected, and the transformation relationship between the above planes is derived according to the calibration object placed in the image, and the pixel coordinates of the object to be measured in the image are transformed into real space coordinates based on this relationship to achieve the purpose of positioning detection. The hardware equipment required by the present invention is very common and can be applied to, for example, steel bar positioning inspection work, liberating labor.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition and civil engineering measurement, and specifically relates to a positioning detection method based on deep learning and monocular large field of view plane measurement. Background Art

[0002] In today's civil engineering construction, prefabricated structures have been widely used due to their advantages such as less traffic interruption, short on-site construction time, high overall construction quality and safety. Prefabricated components are manufactured in a factory near the construction site, then transported to the project site as a whole, and connected and assembled with the corresponding foundation. During the connection operation, the sleeve embedded in the bottom of the prefabricated component is aligned and sleeved down to the steel bars on the foundation, and then grouting is poured into the sleeve to make the prefabricated component firmly connected to the foundation. In order to ensure the smooth installation of prefabricated components, it is very important to check whether the parameters such as the position of the steel bars and the spacing of the steel bars meet the design requirements before construction. In current practice, the detection of the position and size of the steel bars mainly relies on the on-site workers to measure manually using a tape measure. This purely manual detection method is very time-consuming and labor-intensive, especially for the installation of prefabricated components of large or complex structures.

[0003] In the prior art, there are relatively accurate three-dimensional spatial data measurements based on laser scanning technology. For example, a laser scanning technology developed by Kim et al. can automatically evaluate the key dimensions of reinforced concrete elements. However, the method requires the diameter of the steel bar as an input parameter, which limits its application when the steel bar size is unknown. Nishio et al. proposed a method to extract core wires from steel bar point cloud data, but this study cannot further analyze the steel bar spacing. At the same time, the above laser scanning technology usually takes a long time to set up when it is actually used, and requires experts with relevant knowledge to operate it. In addition, the hardware cost of the scanner is still too high. Therefore, the above factors make it difficult to promote and apply laser scanning technology in simple matters such as steel bar spacing.

[0004] In comparison, the cost of digital photography is relatively low. At the same time, with the development of computer vision technology, it is now possible to detect, identify and extract objects in images through corresponding software algorithms. Therefore, how to use image recognition technology to achieve steel bar measurement and positioning is a more feasible research direction. Summary of the invention

[0005] In view of the shortcomings in the prior art, the present invention provides a positioning detection method based on deep learning and monocular large field of view plane measurement, so as to replace manual measurement of steel bar positions at a lower cost.

[0006] The present invention achieves the above technical objectives through the following technical means.

[0007] A positioning detection method based on deep learning and monocular large field of view plane measurement: establish a calibration system, set a measurement plane and a calibration plane respectively, the measurement plane is the plane where the object to be measured is located, the set calibration plane is parallel to the measurement plane, and the distance between the measurement plane and the calibration plane is denoted as h;

[0008] Place several calibration objects in the calibration plane, and assume that the coordinate system of the i-th calibration object is And the coordinate system of the first calibration object Selected as the entire calibration plane coordinate system, denoted as O T =X T Y T Z T , let the camera coordinate system be O C -X C Y C Z C , then any coordinate point (x ij ,y ij , 0) and its pixel coordinates in the camera image (u ij , v ij ) is as follows:

[0009]

[0010] The subscript ij represents the jth point on the i-th calibration object, s is the scale factor, and K is the intrinsic matrix of the camera. R T and T T The plane coordinate system O is calibrated respectively T =X T Y T Z T To the camera coordinate system O C -X C Y C Z C The rotation matrix and translation vector of the conversion, and are the coordinate systems of the i-th calibration object To the camera coordinate system O C -X C Y C Z C The rotation matrix and transformation vector of the transformation, and Respectively represented as the coordinate system of calibration object No. 1 To the camera coordinate system O C -X C Y C Z C The rotation matrix and transformation vector of the transformation;

[0011] Suppose any point P on the measuring plane M , then point P M In the calibration plane coordinate system O T =X T Y T Z T The coordinates below are (x M ,y M , h), point P M The corresponding pixel coordinates on the camera image are (u M , v M ), then:

[0012]

[0013] Collect the image of the object to be tested, and first use the calibration object in the image to infer the parameters s, K, and R T , T T Then, based on the image recognition technology, the neural network model is used to detect and identify the object to be tested in the image, and the pixel coordinates of the object to be tested (u M , v M ), and finally convert the pixel coordinates into spatial coordinates (x M ,y M , h), realize positioning detection.

[0014] Furthermore, the calibration object is a chessboard.

[0015] Furthermore, the number of the calibration objects is greater than 5, and the multiple calibration objects are evenly spread over the camera field of view.

[0016] Furthermore, after collecting the image, the Harris corner detector is used to detect the grid points on each chessboard in the image to obtain the pixel coordinates of the grid points (u ij , v ij ), the detected grid points are used as detection points, and the spatial coordinates (x ij ,y ij , 0), inversely deduce the parameters s, K, R T , T T The value of .

[0017] Furthermore, the least squares optimization algorithm is used to find the parameters s, K, and R T , T T The optimization goal is to minimize the total reprojection error between the detection point and the projection point in the corresponding image.

[0018] Furthermore, the error calculation formula between each calibration chessboard measurement value and the true value is:

[0019]

[0020] Where N X and N Y are the number of squares in the X and Y directions on the chessboard, d is the true value of the side length of the square on the chessboard, is the measured value of the side length of the ath square in the X direction of the small chessboard, is the measured value of the side length of the b-th square in the Y direction of the small chessboard.

[0021] Furthermore, it is applied to the steel bar positioning detection, the top plane of the steel bar is selected as the measuring plane, the ground is selected as the calibration plane, and the distance h between the measuring plane and the calibration plane is the height of the steel bar.

[0022] Furthermore, the neural network model is YOLO v5l.

[0023] Furthermore, in the stage of making data sets to train neural network models, Mosaic data enhancement technology is used to enrich the data sets.

[0024] Furthermore, when training the neural network model, the mini-batch size is set to 4, the initial learning rate is set to 0.01, and it is dynamically adjusted through the cosine annealing strategy, and the training cycle is fixed at 1000.

[0025] The beneficial effects of the present invention are:

[0026] (1) The present invention provides a positioning detection method based on deep learning and monocular large field of view plane measurement. By randomly scattering a number of calibration chessboards around the object to be measured, a calibration system for monocular large field of view measurement is constructed. Then, it is combined with image recognition technology to realize fully intelligent detection and positioning of the object to be measured.

[0027] (2) The detection method of the present invention is very suitable for the inspection of steel bars before the connection and installation of prefabricated parts in current civil engineering construction, which can greatly improve the inspection efficiency and also reduce the safety hazards of on-site workers during high-altitude inspection operations.

[0028] (3) The detection method of the present invention requires very common hardware equipment, such as unmanned aerial vehicles, drones or mobile phones, as long as they can take photos of the ground from a high place, and the subsequent measurement and positioning operations can also be automatically completed by the corresponding pre-written programs. Therefore, the training requirements for operators are also very simple. Therefore, the method of the present invention is highly applicable. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flow chart of the steps of the positioning detection method of the present invention;

[0030] Figure 2Take actual photos of the steel bars used to connect precast parts;

[0031] Figure 3 Schematic diagram of setting a large scale at the top of the object to be measured;

[0032] Figure 4 Schematic diagram of setting a large scale at the bottom of the object to be measured;

[0033] Figure 5 It is a schematic diagram of the calibration system of the present invention;

[0034] Figure 6 A photograph of the test site layout for testing the method of the present invention;

[0035] Figure 7 A layout diagram of 1 to 9 calibrated chessboards;

[0036] Figure 8 This is a diagram of the placement of 13 test chessboards;

[0037] Figure 9(a) is a graph of the measurement error under 1 to 5 calibration chessboards;

[0038] Figure 9(b) is a graph of the measurement error under 5 to 9 calibration chessboards;

[0039] Fig.10 Schematic photos of the site layout for testing different measurement heights;

[0040] Fig.11 It is the measurement error curve diagram at different measurement plane heights;

[0041] Fig.12 This is a typical sample example in the steel bar dataset;

[0042] Fig.13 Images taken to inspect the reinforcement of bridge pier caps;

[0043] Fig.14 Design drawings for the reinforcement of the pier cap;

[0044] Fig.15 This is the result of the inspection of the steel bar position of the pier cap;

[0045] Fig.16 It is a bar chart of the deviation of the measured position of the pier cap reinforcement relative to the designed position;

[0046] Fig.17 Images taken to inspect the reinforcement of bridge piers;

[0047] Fig.18 This is the result diagram of the pier reinforcement position detection;

[0048] Fig.19 This is a bar chart of the deviation of the measured position of the pier reinforcement relative to the designed position. DETAILED DESCRIPTION

[0049] Embodiments of the present invention are described in detail below, examples of the illustrated embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0050] 1. Technical solution

[0051] like Figure 1 The flowchart of the positioning detection method of the present invention is shown, which mainly includes two parts. One part is to select a suitable neural network model and train it, and then use the trained network model to detect and identify the images taken on site. The goal is to detect each object to be detected (i.e., steel bars) in the image. This part corresponds to Figure 1 The other part is to establish a suitable calibration system to obtain the conversion relationship between the measurement plane (the plane where the top of the steel bar is located) and the image plane captured by the camera, that is, the correspondence between the pixel points on the image and the real space position. This part corresponds to Figure 1 Step 3 in the figure; finally, the actual spatial position of each steel bar or the spacing between the steel bars is calculated by combining the detected object (top of the steel bar) and its pixel position in the image with the conversion relationship between the calibrated image plane and the measurement plane.

[0052] The specific steps are as follows:

[0053] 1. Network Model

[0054] The YOLO v5 network model is used. During the training phase, a camera is used to take multiple images of steel bars from different angles and different station locations to form a training set. The Mosaic data enhancement technology is then used to enrich the training set. Finally, the training set is used to train the YOLO v5 network model, so that it can accurately detect and identify the steel bars in the image and output the corresponding recognition results, including the bounding box of the steel bar, the pixel coordinates of the center of the steel bar cross section, etc.

[0055] The process of making a training set also includes steps such as preprocessing and labeling images. Making a training set to train a neural network model is a basic routine operation in the field of machine deep learning, so I will not go into details here.

[0056] 2. Calibration

[0057] In the image captured by the monocular camera, only the pixel spacing between two points can be obtained, that is, using the network model trained in the previous step, at most the pixel position of each steel bar in the image or the number of pixels between two steel bars in the image can be measured; and to calculate the real spatial distance from a single image, it is necessary to set a reference object in the image, such as placing a ruler of standard length next to the object being photographed as a reference, so that the technicians can obtain the corresponding real spatial distance information through a single image later. Therefore, the purpose of calibration is to obtain the corresponding relationship between image pixels and real spatial distance, and the basic principle is to set a reference object next to the object being photographed. However, the above theory is more complicated in actual operation: ① The camera is usually not facing the object being photographed, that is, there is an angle deviation between the image plane and the measurement plane, so the image and the surface to be measured are not a simple scaling relationship; ② It is impossible to place a reference ruler next to each dimension that needs to be measured.

[0058] 1) Measuring plane

[0059] like Figure 2 The picture shown is a photograph of foundation steel bars used to connect prefabricated parts. The heights of all steel bars exposed above the ground are the same or can be approximately considered the same, that is, the tops of all steel bars are in the same plane, and this plane is parallel to the ground. At the same time, compared with the bottom or any other part of the exposed part of the steel bar, when taking a photo to obtain an image, the top of the steel bar is not blocked or interfered and is easy to identify. Therefore, the present invention selects the top plane of the steel bar as the measurement plane. Figure 2 The picture shows the image plane. It is obvious that there is an angular deviation between the image plane and the measurement plane. Therefore, the calibration in the present invention is to obtain the homography matrix between the measurement plane and the image plane, so as to transform the pixel points on the image plane to the measurement plane through the homography matrix.

[0060] 2) Reference scale

[0061] The reference scale is also called the calibration object. When setting the reference scale, theoretically the best situation is to directly set a large scale on the measuring plane that can completely cover the entire measured object (i.e., steel bars). For example Figure 3 As shown in the figure, it is similar to a chessboard, hereinafter referred to as a chessboard. By knowing the length of the small squares, the corresponding top positions of each steel bar and the spacing between the steel bars can be calculated. However, it is not realistic to set up a large chessboard on the top of the steel bar. Figure 4 As shown, setting a large chessboard on the ground where the bottom of the steel bar is located is not completely applicable due to the different specific on-site construction environments. Moreover, if the large chessboard is deliberately laid with comprehensive accuracy, it will run counter to the original intention of simplifying the measurement and positioning of steel bars on the construction site. Therefore, the present invention proposes to establish a calibration system in the form of multiple small chessboards.

[0062] like Figure 5As shown, a number of small chessboards are randomly arranged on the ground. The chessboards do not need to be parallel horizontally or vertically. There can be angle deviations between them, and the spacing between them can also be unequal. The plane where the small chessboards are located is the calibration plane. The top of the steel bar is h above the ground, so the height of the measurement plane from the calibration plane is h. The camera shoots the image from a high place downward. The specific camera inclination angle is not fixed. Of course, it can also shoot vertically downward facing the ground. Based on this, let the spatial coordinate system where the camera shoots the image be O C -X C Y C Z C , the spatial coordinate system of the i-th chessboard is And the spatial coordinate system of chessboard No. 1 The coordinate system of the entire calibration plane is defined as O T =X T Y T Z T ; The specific numbering of each small chessboard can be arranged arbitrarily, and of course the spatial coordinate system of other numbered small chessboards can also be selected as the coordinate system of the entire calibration plane.

[0063] Of course, in addition to the chessboard, other forms of calibration objects can also be selected, such as the Halcon calibration plate, the circular grid, etc. The purpose of the calibration object is only to provide a known coordinate system and several known coordinate points on it to perform corresponding calibration work. Since the chessboard is the most commonly used camera calibration object, the calibration chessboard is specifically used as an example for illustrative explanation in this embodiment.

[0064] 3) Small chessboard merging

[0065] Suppose a point P in space is in the camera coordinate system O C -X C Y C Z C The coordinates under are (x, y, z), in the i-th chessboard coordinate system The spatial coordinates of the i ,y i , z i ), then the transformation relationship between point P in the two space coordinate systems is:

[0066]

[0067] Formula (1) can also be written as:

[0068]

[0069] Where R is the rotation matrix and T is the transformation vector. Correspondingly, and Represented as the spatial coordinate system of the i-th chessboard To the camera space coordinate system OC -X C Y C Z C The rotation matrix and transformation vector of the transformation. The specific matrix forms of R and T are:

[0070]

[0071] For the spatial coordinates of chessboard No. 1 To O C -X C Y C Z C The corresponding conversions are also:

[0072]

[0073] According to formula (1) and formula (2), we can deduce Towards The conversion relationship is:

[0074]

[0075] Separately Then for any i-th small chessboard, its coordinates can be obtained through their corresponding R Ti and T Ti The coordinates on each small chessboard can be uniformly converted to the spatial coordinate system of the calibration plane. T =X T Y T Z T In this way, multiple small chessboards are merged into a large chessboard with a larger coverage area.

[0076] 4) Calibration plane to image plane

[0077] Suppose a point W in space is in the calibration plane coordinate system O T =X T Y T Z T The coordinates below are (x w ,y w , z w ), the pixel coordinates in the image taken by the camera are (u, v); for the projection relationship of the current point in space on the camera imaging plane, the pinhole model is usually used, that is, through perspective transformation, the three-dimensional coordinates of the space (x w ,y w , z w ) is associated with the pixel coordinates (u, v), and the specific corresponding relationship formula is:

[0078]

[0079] Where s is a scale factor, K is the intrinsic matrix of the camera, and R T and T T The meanings are the same as above, respectively calibrating the plane coordinate system O T =X T Y T Z T To the camera coordinate system O C -X C Y C Z C The rotation matrix and transformation vector of the conversion, because the spatial coordinate system of the calibration plane in this embodiment is the spatial coordinate system of chessboard No. 1, so here The specific matrix form of K is:

[0080]

[0081] Combining equations (4) and (3), we can obtain the corresponding relationship between the coordinates of any point on the small chessboard and the pixel coordinates in the camera image:

[0082]

[0083] The subscript ij represents the jth point on the i-th small chessboard. Since the chessboard is a plane, the vertical coordinate z of the point on the chessboard is ij = 0, so the coordinates of point ij on the corresponding small chessboard are (x ij ,y ij , 0), the corresponding image pixel coordinates are (u ij , v ij ).

[0084] Because the size of the chessboard is known, that is, the coordinates (x ij ,y ij , 0) are known, and the corresponding image pixel coordinates (u ij , v ij ) is also known; therefore, based on formula (5), the parameters s, K, and R can be inferred by selecting several points on the chessboard as detection points. T , T T To ensure accuracy, the least squares optimization algorithm can be used to find the best value of the parameter so that the total reprojection error between the detected point and the projection point in the corresponding image is minimized.

[0085] 5) Calibration plane to measurement plane

[0086] As we know from the previous article, the measurement plane is located above the calibration plane, and the distance is h. The specific value of h can be obtained through field measurement. Because the measurement plane is parallel to the calibration plane, the rotation matrix between the calibration plane and the measurement plane is the unit matrix R TM =I, translation vector T TM =[0 0h] T Therefore, any point P on the measuring plane M In the calibration plane coordinate system O T =X T Y T Z T The spatial coordinates of the M ,y M , h), according to formula (4), we can get point P M The spatial coordinates (x M ,y M , h) and its pixel coordinates on the image (u M , v M ) is:

[0087]

[0088] 3. Steel bar measurement and positioning

[0089] According to the calibration method described above, several small chessboards are placed on the ground around the steel bars and images are taken. Then, in the computer background, several detection points are selected on the small chessboards to infer the parameters s, K, and R in equation (6). T , T T On the other hand, the network model obtained through the training mentioned above is used to detect and identify the top of the steel bar and output the corresponding pixel coordinates (u M , v M Finally, according to formula (6), the real space coordinates (x M ,y M , h); after obtaining the corresponding steel bar coordinates, we only need to calculate the coordinates of the two points on the plane (x a ,y a ) to (x b ,y b ) The distance between the two steel bars can be obtained.

[0090] 2. Effect Test

[0091] 1. The impact of the number of small chessboards on accuracy

[0092] like Figure 6As shown in the figure, a camera is set up on the floor of the laboratory, 1.9m above the ground. The camera is adjusted to shoot diagonally downward. Different numbers of small chessboards are placed in several groups as calibration chessboards within the ground range that can be captured by the camera's field of view. In order to minimize the impact of the position distribution of the calibration chessboards on the experiment, all placements are carried out in a way that evenly covers the camera's field of view, and try to ensure that the chessboards are parallel horizontally and vertically; all chessboards are exactly the same in shape and size, and the size of the white or black small squares on the chessboard is 45*45mm.

[0093] like Figure 7 As shown in the figure, in this test, 1 to 9 calibration chessboards were placed, and 9 sets of parameters s, K, R were obtained accordingly. T , T T The value of .

[0094] like Figure 8 As shown in the figure, the camera position angle is kept unchanged, and 13 chessboards are evenly placed in the camera field of view as test chessboards to simulate the top of the steel bar for measurement and calculate the corresponding error. At this time, the measurement plane and the calibration plane are the same, both are the ground, so h in formula (6) can be considered to be 0. Based on the 9 sets of parameter values ​​obtained previously, Figure 8 The size of each square on the 13 chessboards is measured and calculated in the image shown; in this test, the Harris corner detector is used to detect the grid points on each chessboard, and then the pixel coordinates of the grid points in the image are used in combination with formula (6) to obtain the real spatial position coordinates. Finally, the distance calculation formula is used to calculate the spacing between the grid points, that is, the side length of the small square. Finally, the following formulas are used to calculate the error between the measured values ​​and the real values ​​of the 13 chessboards:

[0095]

[0096] Where N X and N Y are the number of squares in the X direction and Y direction on the small chessboard (including white squares and black squares), d = 45 mm, is the measured value of the side length of the ath square in the X direction of the small chessboard, is the measured value of the side length of the b-th square in the Y direction of the small chessboard.

[0097] Figure 9(a) and Figure 9(b) show the corresponding test results. The horizontal axis "chessboard number" in the figure corresponds to Figure 8 The 13 test chessboards are numbered, and the vertical axis is the error. It can be seen intuitively from the figure that the error fluctuates between 0.1% and 6%. When the number of calibration chessboards is greater than 5, the error at any position in the camera field of view is less than 0.7%, which can achieve good measurement accuracy.

[0098] 2. The influence of measuring plane height on accuracy

[0099] like Fig.10 As shown in the figure, the measurement accuracy is tested for seven cases where the measurement plane height h is 41mm, 84mm, 128mm, 253mm, 502mm and 704mm. Fig.10 The figure is only used to intuitively illustrate the placement of the test chessboards at 7 heights. In actual testing, each height is tested separately, and it must be ensured that the variables such as placement position, placement angle, and placement quantity are consistent between each group of tests except for the height difference. Specifically, 9 test chessboards are placed at each height, and the 9 test chessboards are evenly spread over the camera field of view, and s, K, R T , T T The values ​​of other parameters are based on the data obtained using the 9 calibration chessboards.

[0100] like Fig.11 The test results of 7 groups of heights are shown. It can be seen from the figure that the measurement accuracy varies with the height of the measurement plane, but all errors are within 0.8%, which is within an acceptable range.

[0101] 3. Actual steel bar positioning test

[0102] First, we created a data set. In this test data set, we prepared 380 steel bar images with a total of 1,295 steel bar sections. All images were taken by a professional DJI Plantom 3 camera. The shooting angles and distances of each image are different. The number of steel bars in each image ranges from 6 to 38. In addition, the image also contains various interference information, such as uneven lighting, steel bar corrosion, and occlusion. The annotation tool used is LabelImg, which is used to mark the top section of the steel bar.

[0103] like Fig.12 The following are several representative data set samples selected in this test. The data sets are randomly divided into training set, validation set and test set, accounting for 80%, 10% and 10% respectively.

[0104] Based on GeForce RTX 2060Super 8GB GPU and Pytorch 4.0 framework, the neural network training test was carried out. Due to the limited GPU memory, the mini-batch size was set to 4; the initial learning rate was set to 0.01 and dynamically adjusted through the cosine annealing strategy; the training cycle (training epochs) was fixed at 1000. Through the above-made data set, the four network models YOLO v5s, YOLO v5m, YOLO v5l and YOLO v5x were trained and tested. The relevant training and test data are shown in Table 1. According to the test results, YOLO v5l has the best performance, with an accuracy of 0.9899 and a recall of 1. Therefore, the YOLO v5l network model was used for subsequent steel bar positioning inspection.

[0105] Table 1: Network model training and testing data

[0106] YOLO v5x YOLO v5l YOLO v5m YOLO v5s map@0.5 0.9985 0.9986 0.9956 0.9937 map@0.95 0.7536 0.7489 0.7158 0.6975 Accuracy 0.9834 0.9899 0.9756 0.9369 Recall 0.9933 1 0.9933 0.9867

[0107] 1) Check the spacing of steel bars during pier erection

[0108] like Fig.13 As shown in the figure, for two bridge piers with the same design parameters, several calibration chessboards are evenly placed around the bridge piers. Two placement methods are used, in which the chessboards are evenly placed on the ground inside pier 1, and for pier 2, the chessboards are placed on the ground outside the pier. After placing the calibration chessboards, a drone is used to shoot images downward from the air, that is, to obtain Fig.13 . Fig.14 Shown is the design drawing of the pier.

[0109] Fig.15 and Fig.16 is the corresponding test result, where Fig.15 The location diagram of the steel bars on the pier, including the designed location and the measured location; Fig.16 It is a bar graph of the deviation between the actual measured position of the steel bar and the designed position, including the deviation in the X and Y directions. The horizontal axis is the steel bar number and the vertical axis is the deviation.

[0110] 2) Check the spacing of steel bars before installing the cap beam

[0111] like Fig.17 As shown, the installation of the cap beam is also subject to corresponding steel bar positioning inspection. Unlike the inspection of the steel bars on the pier cap, the installation steel bars of the cap beam are located on the top of the pier, at a certain height from the ground. Therefore, the calibration chessboard can only be placed on the inside of the pier. Afterwards, a drone is also used to take photos from the air.

[0112] Fig.18 and Fig.19 The corresponding test results are shown in Fig.18 This is the location diagram of the steel bars on the pier. Fig.19 A bar graph showing the deviation between the actual measured position of the steel bars and the designed position.

[0113] In summary, 1) and 2) above verify that the steel bar positioning method based on deep learning and monocular large field of view plane measurement of the present invention can effectively complete the inspection of the steel bar installation position, and at the construction site, it is only necessary to place a positioning chessboard around the steel bars and use an ordinary monocular camera to take corresponding photo images, and the measurement and inspection of the steel bar position can be automatically completed through the background program algorithm, thereby greatly improving work efficiency, and the required equipment cost is relatively low, which is conducive to popularization and use.

[0114] In addition, although the present invention was born out of the work of steel bar positioning inspection, the positioning method of the present invention is not limited to steel bar positioning inspection in actual application. All positioning detections in similar situations can be completed using the method of the present invention. Therefore, the application field of steel bar inspection cannot be understood as a limitation on the purpose or technical solution of the present invention itself.

[0115] In the description of the present invention, it should be understood that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0116] The present invention is not limited to the above-mentioned embodiments. Any obvious improvement, substitution or deformation that can be made by those skilled in the art without departing from the essential content of the present invention belongs to the protection scope of the present invention.

Claims

1. A positioning detection method based on deep learning and monocular large field of view plane measurement, characterized in that: Establish a calibration system, set a measurement plane and a calibration plane respectively, the measurement plane is the plane where the object to be measured is located, the calibration plane is set to be parallel to the measurement plane, and the distance between the measurement plane and the calibration plane is denoted as h; Place several calibration objects in the calibration plane, and assume that the coordinate system of the i-th calibration object is And the coordinate system of the first calibration object Selected as the entire calibration plane coordinate system, denoted as O T =X T Y T Z T , let the camera coordinate system be O C -X C Y C Z C , then any coordinate point (x ij ,y ij , 0) and its pixel coordinates in the camera image (u ij , v ij ) is as follows: The subscript ij represents the jth point on the i-th calibration object, s is the scale factor, and K is the intrinsic matrix of the camera. R T and T T The plane coordinate system O is calibrated respectively T =X T Y T Z T To the camera coordinate system O C -X C Y C Z C The rotation matrix and transformation vector of the conversion; and are the coordinate systems of the i-th calibration object To the camera coordinate system O C -X C Y C Z C Transformed rotation matrix, transformation vector, and Respectively represented as the coordinate system of calibration object No. 1 To the camera coordinate system O C -X C Y C Z C The rotation matrix and transformation vector of the transformation; Suppose any point P on the measuring plane M , then point P M In the calibration plane coordinate system O T =X T Y T Z T The coordinates below are (x M ,y M , h), point P M The corresponding pixel coordinates on the camera image are (u M , v M ), then: Collect the image of the object to be tested, and first use the calibration object in the image to infer the parameters s, K, and R T , T T Then, based on the image recognition technology, the neural network model is used to detect and identify the object to be tested in the image, and the pixel coordinates of the object to be tested (u M , v M ), and finally convert the pixel coordinates into spatial coordinates (x M ,y M , h), realize positioning detection.

2. The positioning detection method based on deep learning and monocular large field of view plane measurement according to claim 1 is characterized in that: The calibration object is a chessboard.

3. The positioning detection method based on deep learning and monocular large field of view plane measurement according to claim 1 is characterized in that: The number of the calibration objects is greater than 5, and the multiple calibration objects are evenly spread over the camera field of view.

4. The positioning detection method based on deep learning and monocular large field of view plane measurement according to claim 2, characterized in that: After collecting the image, the Harris corner detector is used to detect the grid points on each chessboard in the image and obtain the pixel coordinates of the grid points (u ij , v ij ), the detected grid points are used as detection points, and the spatial coordinates (x ij ,y ij , 0), inversely deduce the parameters s, K, R T , T T The value of .

5. The positioning detection method based on deep learning and monocular large field of view plane measurement according to claim 4 is characterized in that: Use the least squares optimization algorithm to find the parameters s, K, R T 、T T The optimization goal is to minimize the total reprojection error between the detection point and the projection point in the corresponding image.

6. The positioning detection method based on deep learning and monocular large field of view plane measurement according to claim 5 is characterized in that: The error calculation formula between the measured value of each calibration chessboard and the true value is: Where N X and N Y are the number of squares in the X and Y directions on the chessboard, d is the true value of the side length of the square on the chessboard, is the measured value of the side length of the ath square in the X direction of the small chessboard, is the measured value of the side length of the b-th square in the Y direction of the small chessboard.

7. The positioning detection method based on deep learning and monocular large field of view plane measurement according to claim 1, characterized in that: Applied to steel bar positioning detection, the top plane of the steel bar is selected as the measuring plane, the ground is selected as the calibration plane, and the distance h between the measuring plane and the calibration plane is the height of the steel bar.

8. The positioning detection method based on deep learning and monocular large field of view plane measurement according to claim 1, characterized in that: The neural network model is YOLO v51.

9. The positioning detection method based on deep learning and monocular large field of view plane measurement according to claim 8, characterized in that: When preparing data sets and training neural network models, Mosaic data enhancement technology is used to enrich the data sets.

10. The positioning detection method based on deep learning and monocular large field of view plane measurement according to claim 9, characterized in that: When training the neural network model, the mini-batch size is set to 4, the initial learning rate is set to 0.01, and it is dynamically adjusted through the cosine annealing strategy. The training cycle is fixed at 1000.

Images