Feature extraction measurement and deviation correction device and method based on intelligent target

Abstract

A feature extraction measurement and deviation correction device and method based on an intelligent target. The present invention comprises intelligent target design, intelligent target recognition, target center point high-precision extraction and positioning, accurate measurement and real-time deviation correction. The present invention designs an intelligent target device that is easily recognized and capable of implementing quality inspection and self-circulation monitoring, provides a feature extraction and sub-pixel matching algorithm, and establishes a setting out data and real-time monitoring target online fusion and evaluation analysis system, provides an intelligent distribution and coordination mechanism based on a target instruction, effectively improves the measurement efficiency, saves the labor cost, reduces the target recognition error rate, achieves automatic recognition and automatic deviation correction of the target, and ensures high-precision measurement of the coordinates of a center point of the target, thereby effectively ensuring the precision of fieldwork measurement.

Description

A feature extraction, measurement and deviation correction device and method based on intelligent target Technical Field

[0001] The present invention relates to the technical field of photogrammetry, and in particular to a device and method for feature extraction, measurement and deviation correction based on an intelligent target. Background Art

[0002] Under multi-faceted construction conditions similar to those in narrow valleys, the principle of close-range photogrammetry is used to achieve simultaneous multi-point and multi-faceted measurement by setting up cameras on both sides and deploying control points on the ground or slopes, which can significantly improve measurement efficiency and save labor costs. Since the camera station is far away from the target detection point (generally between 80-120 meters), the construction conditions are complex, and the target installation is subject to significant human interference, the camera's built-in algorithm is easily affected by target tilt, human occlusion of the target, and other interference factors, resulting in misjudgment and affecting measurement accuracy. Therefore, it is necessary to propose a high-precision feature extraction, measurement, and correction technology system, method, and device based on intelligent targets to solve the problems of difficult target identification and matching, low accuracy, poor reliability, and inability to automatically correct errors, so as to achieve high-efficiency, high-precision, and low-recognition error rate measurement under multi-faceted construction conditions.

[0003] Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a feature extraction, measurement and correction device and method based on an intelligent target, so as to effectively improve measurement efficiency, save labor costs, reduce the target recognition error rate, realize automatic recognition and automatic correction of targets, ensure high-precision measurement of the target center point coordinates, and effectively ensure the accuracy of field measurement.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A feature extraction, measurement and correction device based on an intelligent target, wherein the target device includes a target disk, which is rotatably connected to the top of a target connecting rod, and the bottom of the target connecting rod is connected to a rotating target cloud platform. The target cloud platform is in sliding contact with the cloud platform base and is fixed by cloud platform fixing bolts, and the bottom of the cloud platform base is fixed to a support body.

[0007] The target cloud platform is a sphere, and the target cloud platform is in sliding contact with the circular groove on the base.

[0008] The bottom surface of the cloud platform base is fixedly connected to the spiral connecting rod.

[0009] In a preferred solution, the bottom surface of the cloud platform base is fixedly connected to the horizontally movable upper structure, and the horizontally movable upper structure is in sliding contact with the lower horizontally movable lower structure and is fixed by horizontal fixing bolts.

[0010] The above-mentioned horizontal movable lower structure is provided with a clamping plate mechanism, which includes a steel column at the bottom end of the horizontal movable lower structure, and a steel plate elastically clamped with the steel column is provided on one side of the horizontal movable lower structure.

[0011] A connecting rod is provided at the upper end of the above-mentioned steel plate. The connecting rod is in an inverted "L" shape and the horizontal end rod body is nested and connected with the horizontally movable lower structure. A spring is wrapped around the horizontal end rod body of the connecting rod. One end of the spring is in contact with the inner wall of the horizontally movable lower structure, and the other end of the spring is connected to the connecting rod. The end of the connecting rod extends out of the horizontally movable lower structure and the horizontal position is adjusted by a screw cap and a hollow nut sleeve.

[0012] The above-mentioned target disk is a rectangular disk with identical calibration components on both sides of the disk. The calibration components include multiple sets of concentric circles, the centers of the concentric circles coincide with the center of the disk, and the disk is also provided with crosshairs where diagonal lines intersect and lines connecting the midpoints of the sides intersect. A direction identification notch is provided at one corner of the rectangle, and a two-dimensional identification code is provided at one corner of the disk.

[0013] Using the target feature extraction method based on the above-mentioned intelligent target feature extraction measurement and correction device, the feature extraction steps are as follows:

[0014] Step 1: Construct a target image dataset. Collect target images in different field operation environments and use annotation software to create a target detection dataset. The dataset includes a training set, a validation set, and a test set. The annotated data is divided into a training set and a validation set according to a set ratio. The training images are enhanced by randomly rotating them by 90°, 180°, and 270° and flipping them horizontally. Target images are taken and annotated as a test set. Both the training set and the test set are preprocessed, and the original full-color RGB images are normalized and input into the network model.

[0015] Step 2: Build a target extraction model. Use the target detection algorithm based on YOLOv5. YOLOv5 consists of two parts: the backbone network and the neck network. The backbone network uses the Focus structure and the CSP structure to extract and improve image features. The neck network uses the FPN and PAN structures to process features of different scales.

[0016] Step 3: Train the target extraction model. Input the training set data processed in Step 1 into the target extraction model constructed in Step 2. The target extraction model randomly batches a specified number of images each time for learning. The training process is optimized using the Adam optimizer.

[0017] Step 4: Test the target extraction accuracy. Input the test data processed in Step 1 into the target extraction model trained in Step 3 to output the target location on the image. Measure the target extraction accuracy by calculating the accuracy, IoU (Intersection over Union), and positioning error evaluation indicators between the output image and the annotated image. If the accuracy does not meet the requirements, adjust it by increasing the amount of training data and adjusting the hyperparameters. Repeat Step 3 until the target extraction result meets the accuracy requirements.

[0018] Step 5: Target identification and position calculation: The photos taken on site are pre-processed and input into the target extraction model trained in Step 3. The image pixel coordinates of the target are calculated based on the target identification results.

[0019] After the above-mentioned target feature extraction, the target center point extraction and spatial positioning are performed on the target image after feature extraction, including:

[0020] Step 6.1: Image Enhancement

[0021] Step 6.1.1. Grayscale the image: Use the weighted average method to extract brightness information from the color channel; the calculation formula is as follows: Gray = 0.299*R + 0.587*G + 0.114*B

[0022] Where R represents the red channel, G represents the green channel, and B represents the blue channel;

[0023] Step 6.1.2, Image Binarization: Divide the image into two different regions according to the grayscale value of the pixels: foreground and background. Use the OTSU method to binarize the image to ensure the maximum variance between the foreground and background images.

[0024] Step 6.2: Hough transform line detection

[0025] The linear equation y=kx+b in the Cartesian coordinate system is transformed into b=-kx+y. k and b are regarded as the independent variable and dependent variable of the Cartesian parameter space. The Cartesian parameter space is then mapped to the polar coordinate parameter space. According to the duality of Hough space, the linear line in the Cartesian coordinate system is mapped to the corresponding single point in the parameter space in the polar coordinate system. The point with the most intersection of the curves in the parameter space is found. This point is the corresponding linear line in the original Cartesian coordinate system.

[0026] Based on the image enhancement results in Step 6.1, the equation of the intersecting line of the artificial target is obtained as:

[0027] Step 6.3, find the intersection of line segments

[0028] The intersection of the line equation of the artificial target detected by the Hough transform line in Step 6.2 is performed, and the coordinates of the intersection are obtained as follows:

[0029] The unit is pixel, and the obtained intersection point is the pixel coordinate value of the center point of the artificial target;

[0030] Step 6.4. Aerial triangulation

[0031] Calculate the exterior orientation elements of photo i using the bundle adjustment algorithm And the camera parameters (x0, y0, f, k1, k2, k3, p1, p2), where X i ,Y i ,Z i Represents the three-dimensional coordinate value of the center of the photo in the coordinate system, ω i , k i They represent the side tilt angle, heading tilt angle and photo rotation angle of the rotation system with the Y axis as the main axis respectively; x0, y0, f are the camera intrinsic parameters, where x0, y0 are the principal point coordinates, f is the focal length, k1, k2, k3, p1, p2 are the camera extrinsic parameters, where k1, k2, k3 are the radial distortion coefficients, and p1, p2 are the tangential distortion coefficients;

[0032] Step 6.5: Calculate the three-dimensional coordinates of the target center point by forward intersection

[0033] Due to the previous manual target recognition step, the target range, number and photo number have been obtained. The number can be used to filter out the center intersection coordinates (x1, y1), (x2, y2) of the target with the same name, as well as the external orientation elements of the corresponding photo.

[0034] By photogrammetric intersection algorithm:

[0035] Step 6.5.1. Calculate the rotation matrix R1, R2 of the photo using the exterior orientation elements.

[0036] Step 6.5.2. Calculate the photographic baseline component B using the exterior orientation elements u ,B v ,B w : B u =X2-X1 B v =Y2-Y1 B w =Z2-Z1;

[0037] Step 6.5.3. Calculate the auxiliary image space coordinates of the center point of the artificial target (u1, v1, w1), (u2, v2, w2):

[0038] Step 6.5.4. Calculate the projection coefficient of the target center point:

[0039] Step 6.5.5. Calculate the three-dimensional coordinates of the target center point: X = X1 + N1u1 = X2 + N2u2 Y = Y1 + N1v1 = Y2 + N2v2 Z = Z1 + N1w1 = Z2 + N2w2.

[0040] After extracting the target center point and spatially locating the target image in step 6 above, measurement and real-time correction are performed:

[0041] Measurement and real-time correction: The measurement and real-time correction system continuously obtains observation images of the target position and pushes the offset results to relevant on-site personnel as needed, including: automatically updating the real-time target data and calculating the target offset correction amount in real time. Based on the observation image photos and the calculated image exterior orientation elements, that is, the orientation attitude parameters, the object-side target offset vector is drawn and annotated in image-side vector or VR mode to form a correction real-scene photo or VR scene with the correction vector drawn, and form correction guidance instructions; guide the operation commander to perform correction operations: Step 7.1, real-time target data acquisition: regularly capture stereo image data of the target through the dual-camera station to obtain the real-time position and status of the target. Specific operations include:

[0042] Step 7.1.1. Set the shooting interval of the camera station and adjust it according to the accuracy requirements of the target;

[0043] Step 7.1.2: The camera automatically captures a set of high-precision image data at each shooting interval. The image includes the camera's position, attitude angle, and focal length parameters.

[0044] Step 7.1.3, the camera station transmits the captured image data to the backend server via wireless or wired network, waiting for the next step of processing;

[0045] Step 7.2, offset state calculation: Through the target recognition algorithm, the center point coordinates of the target are extracted from the image data and compared with the target design data coordinates to calculate the offset and offset direction of the target. The specific operations include:

[0046] Step 7.2.1. After receiving the image data transmitted by the camera station, the backend server stores it in the database and performs pre-processing such as denoising, enhancement, and cropping.

[0047] Step 7.2.2: The backend server calls the target recognition algorithm model to analyze the pre-processed image data and automatically identify the coordinates of the target;

[0048] Step 7.2.3: The backend server compares the calculated target coordinates with the target design data coordinates stored in the database to obtain the offset and offset direction of the target relative to the design position;

[0049] Step 7.3: Push the offset data. Push the calculated target offset information and target number to the front-end device manually or automatically so that the layout personnel can perform correction operations. The specific operations include:

[0050] Step 7.3.1. The backend server encapsulates the calculated target offset information and related parameters into a data packet and sends it to the front-end device via a wireless or wired network.

[0051] Step 7.3.2: After receiving the data packet, the front-end device selects manual or automatic push mode according to the user settings or system defaults. The manual mode means that the front-end device displays the data packet on the screen and waits for the user to confirm or modify it. After confirmation, it is sent to the visual correction platform. The automatic mode means that the front-end device directly transmits the data packet to the visualization platform.

[0052] Step 7.3.3, the front-end device feeds back the push results to the back-end server and records them in the database;

[0053] Step 7.4: Visual deviation correction: Through simulation technology, the target offset information is displayed in the 3D scene or the high-definition real-life image obtained by the camera station, and is superimposed and compared with the target design data, thereby providing real-time visual deviation correction assistance for the layout personnel. The specific operations include:

[0054] Step 7.4.1. The front-end device inputs the target offset information and related parameters into the visualization platform according to the user selection or system default;

[0055] Step 7.4.2: The visualization platform generates a virtual target based on the target design data and generates target X, Y, and Z offset annotation information based on the input real-time target data. The data is simultaneously displayed in the 3D scene or on the high-definition real-scene image obtained by the camera station.

[0056] Step 7.4.3. The visualization platform marks the target offset data with different colors or symbols to intuitively display the difference between the real-time target and the target design data, thereby helping the layout personnel to perform correction operations.

[0057] The present invention provides a feature extraction, measurement and correction device and method based on intelligent targets. In image data and target recognition, aerial triangulation technology and image recognition technology are integrated to propose a target intelligent recognition calculation method with aerial three-coordinate solution. It automatically performs image screening and aerial three-coordinate solution on multiple images taken by a fixed camera station. At the same time, it automatically identifies, classifies, and roughly and precisely solves the coordinates of the targets in the images, realizing functions such as automatic recognition, automatic measurement, and automatic correction. It proposes a new solution for traditional multi-faceted construction measurement that mainly relies on multi-station, multiple-time, and multi-person simultaneous measurement by total stations. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The present invention will be further described below with reference to the accompanying drawings and examples:

[0059] FIG1 is a design diagram of a target for safety monitoring according to the present invention;

[0060] FIG2 is a schematic diagram of the target cloud platform of the present invention;

[0061] FIG3 is a target design diagram for template lofting according to the present invention;

[0062] FIG4 is a side view of the target of FIG3;

[0063] FIG5 is a schematic diagram of the target horizontal movement structure in FIG3;

[0064] FIG6 is a schematic diagram of target tilt template layout adjustment;

[0065] Figure 7 is a flowchart of intelligent target identification;

[0066] FIG8 is a flowchart of center point extraction based on image features;

[0067] FIG9 is a framework diagram of the precise measurement and real-time correction system;

[0068] FIG10 is a schematic diagram of a test scenario in an embodiment;

[0069] FIG11 is a schematic diagram of the target extraction process.

[0070] In the figure: two-dimensional identification code 1, target center point 2, direction identification notch 3, target disk 4, target connecting rod 5, target cloud platform 6, cloud platform fixing bolt 7, cloud platform base 8, spiral connecting rod 9, horizontal fixing bolt 10, horizontal moving upper structure 11, horizontal moving lower structure 12, screw cap 13, hollow nut sleeve 14, spring 15, connecting rod 16, steel plate 17, steel column 18, groove line 19, level bubble 20. DETAILED DESCRIPTION

[0071] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0072] The feature extraction, measurement and correction device based on the intelligent target includes a target disk 4, which is rotatably connected to the top of the target connecting rod 5, and the bottom of the target connecting rod 5 is connected to the rotating target cloud platform 6. The target cloud platform 6 is in sliding contact with the cloud platform base 8 and is fixed by the cloud platform fixing bolts 7. The bottom of the cloud platform base 8 is fixed to the support body.

[0073] The target cloud platform 6 is a sphere, and the target cloud platform 6 is in sliding contact with the circular groove on the base 8 .

[0074] The bottom surface of the cloud platform base 8 is fixedly connected to the spiral connecting rod 9.

[0075] As shown in Figure 1, the target device with a spiral connecting rod 9 connected to the bottom is used to perform safety monitoring on the monitored area. The spiral connecting rod 9 is inserted into the monitored area, and the position change of the center of the target disk 4 is detected to check whether ground subsidence, landslide, etc. occur.

[0076] In a preferred solution, the bottom surface of the cloud platform base 8 is fixedly connected to the horizontally movable upper structure 11 , and the horizontally movable upper structure 11 is in sliding contact with the lower horizontally movable lower structure 12 and is fixed by horizontal fixing bolts 10 .

[0077] The above-mentioned horizontal movable lower structure 12 is provided with a clamping mechanism, which includes a steel column 18 at the bottom end of the horizontal movable lower structure 12 , and a steel plate 17 elastically clamped with the steel column 18 is provided on one side of the horizontal movable lower structure 12 .

[0078] A connecting rod 16 is provided at the upper end of the above-mentioned steel plate 17. The connecting rod 16 is in an inverted "L" shape and the horizontal end rod body is nested and connected to the horizontally movable lower structure 12. A spring 15 is wrapped around the horizontal end rod body of the connecting rod 16. One end of the spring 15 contacts the inner wall of the horizontally movable lower structure 12, and the other end of the spring 15 is connected to the connecting rod 16. The end of the connecting rod 16 extends out of the horizontally movable lower structure 12 and the horizontal position is adjusted by the screw cap 13 and the hollow nut sleeve 14.

[0079] As shown in FIG3 , in a preferred embodiment, a structural diagram of a target device for template layout is shown, which can be adjusted horizontally and the target connecting rod 5 can be rotated to adapt to clamping at different angles. The level bubble 20 on the target connecting rod 5 is used to keep the rod body vertical.

[0080] The above-mentioned target disk 4 is a rectangular disk, and the same calibration components are provided on both sides of the disk. The calibration components include multiple groups of concentric circles, the centers of the concentric circles coincide with the center of the disk body, and the disk is also provided with crosshairs where the diagonal lines and the midpoints of the sides intersect. A direction identification notch 3 is provided at one corner of the rectangle, and a two-dimensional identification code 1 is provided at one corner of the disk.

[0081] As shown in Figures 1, 3 and 6, the target disk 4 consists of three parts: a calibration component, a verification component, and a marking component:

[0082] 1) Calibration components: mainly composed of concentric circles, rectangles, crosshairs with the same intersection point, and rectangular frame gaps (for direction interpretation). They can be solved independently, eliminated during adjustment, or used for quality inspection;

[0083] 2) Verification component: It is mainly used for checking the accuracy of target center coordinates. It can realize the verification of positioning error values ​​of two groups of crosshair intersections, the verification of positioning error values ​​of crosshair intersections and concentric circle centers, and the verification of positioning error values ​​of crosshair intersections and rectangle centers.

[0084] 3) Identification symbol: Mainly used to identify basic target metadata and for on-site error correction guidance. The target is accompanied by a QR code image, which can be scanned to obtain detailed information about the target, mainly including identification interpretation, reference target number, target size, project name, production time, project area number and other information.

[0085] The target feature extraction algorithm of the feature extraction, measurement and correction device based on the intelligent target is used. The feature extraction steps are as follows: Step 1, constructing a target image data set; collecting target pictures under different field operation environments, and using annotation software to produce a target object detection data set; the data set includes a training set, a validation set and a test set, and the annotated data is divided into a training set and a validation set according to a set ratio. In the embodiment, the set ratio is 8:2, and the training image is enhanced by randomly rotating 90°, 180°, 270° and horizontally flipping; taking a target picture and annotating it as a test set; preprocessing both the training set and the test set, normalizing the original full-color RGB image, and inputting it into the network model. The detailed process is shown in Figure 8;

[0086] Step 2: Build a target extraction model. Use the target detection algorithm based on YOLOv5. YOLOv5 mainly consists of two parts: the backbone network and the neck network. The backbone network uses the Focus structure and the CSP structure to extract and improve image features. The neck network uses the FPN and PAN structures to process features of different scales.

[0087] Step 3: Train the target extraction model. Input the training set data processed in Step 1 into the target extraction model constructed in Step 2. The target extraction model randomly batches a specified number of images each time for learning. The training process is optimized using the Adam optimizer.

[0088] Step 4: Test the target extraction accuracy. Input the test set data processed in Step 1 into the target extraction model trained in Step 3 to output the target location on the image. Then, measure the target extraction accuracy by calculating objective evaluation indicators such as the accuracy between the output image and the annotated image, IoU (Intersection over Union), and positioning error. If the accuracy does not meet the requirements, adjust it by increasing the amount of training data, adjusting hyperparameters, etc. Repeat Step 3 until the target extraction result meets the accuracy requirements.

[0089] Step 5: Target identification and approximate position calculation: The photos taken on site are pre-processed and input into the target extraction model trained in Step 3. The pixel coordinates of the approximate image of the target are calculated based on the target identification results.

[0090] After the above-mentioned target feature extraction, the target center point extraction and spatial positioning are performed on the target image after feature extraction, including:

[0091] Step 6.1: Image Enhancement

[0092] Step 6.1.1. Grayscale the image: Use the weighted average method to extract brightness information from the color channel. Based on the different color sensitivities of the human eye, green is given the highest weight, followed by red, and blue the lowest. The calculation formula is as follows: Gray = 0.299*R + 0.587*G + 0.114*B

[0093] Where R represents the red channel, G represents the green channel, and B represents the blue channel;

[0094] Step 6.1.2. Image Binarization: Divide the image into two distinct regions based on the pixel grayscale values: foreground and background. The foreground typically contains the target object of interest, while the background is the surrounding environment. By separating the foreground and background into distinct pixel values, image processing tasks such as object detection, segmentation, and contour extraction can be performed more easily. The OTSU method is used for image binarization to ensure the maximum variance between the foreground and background images.

[0095] Step 6.2: Hough transform line detection

[0096] The linear equation y=kx+b in the Cartesian coordinate system is transformed into b=-kx+y. k and b are regarded as the independent variable and dependent variable of the Cartesian parameter space. The Cartesian parameter space is then mapped to the polar coordinate parameter space. According to the duality of Hough space, the linear line in the Cartesian coordinate system is mapped to the corresponding single point in the parameter space in the polar coordinate system. The point with the most intersection of the curves in the parameter space is found. This point is the corresponding linear line in the original Cartesian coordinate system.

[0097] Based on the image enhancement results in Step 6.1, the equation of the intersecting line of the artificial target is obtained as:

[0098] Step 6.3, find the intersection of line segments

[0099] The intersection of the line equation of the artificial target detected by the Hough transform line in Step 6.2 is performed, and the coordinates of the intersection are obtained as follows:

[0100] The unit is pixel, and the obtained intersection point is the pixel coordinate value of the center point of the artificial target;

[0101] Step 6.4. Aerial triangulation

[0102] Calculate the exterior orientation elements of photo i using the bundle adjustment algorithm And the camera parameters (x0, y0, f, k1, k2, k3, p1, p2), where X i ,Y i ,Z i Represents the three-dimensional coordinate value of the center of the photo in the coordinate system, ω i , k i They represent the side tilt angle, heading tilt angle and photo rotation angle of the rotation system with the Y axis as the main axis respectively; x0, y0, f are the camera intrinsic parameters, where x0, y0 are the principal point coordinates, f is the focal length, k1, k2, k3, p1, p2 are the camera extrinsic parameters, where k1, k2, k3 are the radial distortion coefficients, and p1, p2 are the tangential distortion coefficients;

[0103] Step 6.5: Calculate the three-dimensional coordinates of the target center point by forward intersection

[0104] Due to the previous manual target recognition step, the target range, number and photo number have been obtained. The number can be used to filter out the center intersection coordinates (x1, y1), (x2, y2) of the target with the same name, as well as the external orientation elements of the corresponding photo.

[0105] By photogrammetric intersection algorithm:

[0106] Step 6.5.1. Calculate the rotation matrix R1, R2 of the photo using the exterior orientation elements.

[0107] Step 6.5.2. Calculate the photographic baseline component B using the exterior orientation elements u ,B v ,B w : B u =X2-X1 Bv =Y2-Y1 B w =Z2-Z1;

[0108] Step 6.5.3. Calculate the auxiliary image space coordinates of the center point of the artificial target (u1, v1, w1), (u2, v2, w2):

[0109] Step 6.5.4. Calculate the projection coefficient of the target center point:

[0110] Step 6.5.5. Calculate the three-dimensional coordinates of the target center point: X = X1 + N1u1 = Y2 + N2u2 Y = Y1 + N1v1 = Y2 + N2v2 Z = Z1 + N1w1 = Z2 + N2w2.

[0111] After extracting and locating the target center point of the target image in step 6 above, measurement and real-time correction are performed:

[0112] Measurement and real-time correction: Through continuous observation of the target position, the measurement and real-time correction system pushes the deviation results to the on-site personnel on demand, including: automatic update of target real-time data and deviation status, automatic or manual push of specified target status information according to the observation position; based on the target data, correction is provided: the deviation posture and information are synchronized with the 3D scene or the high-definition real-life image captured by the camera station.

[0113] Step 7.1: Real-time target data acquisition: The camera station regularly captures the target's stereo image data to obtain the target's real-time position and status. The specific operations include:

[0114] Step 7.1.1. Set the shooting interval of the camera station and adjust it according to the accuracy requirements of the target;

[0115] Step 7.1.2: The camera automatically captures a set of high-precision image data at each shooting interval. The image includes the camera's position, attitude angle, and focal length parameters.

[0116] Step 7.1.3, the camera station transmits the captured image data to the backend server via wireless or wired network, waiting for the next step of processing;

[0117] Step 7.2, offset state calculation: Through the target recognition algorithm, the center point coordinates of the target are extracted from the image data and compared with the target design data coordinates to calculate the offset and offset direction of the target. The specific operations include:

[0118] Step 7.2.1. After receiving the image data transmitted by the camera station, the backend server stores it in the database and performs pre-processing such as denoising, enhancement, and cropping.

[0119] Step 7.2.2: The backend server calls the target recognition algorithm model to analyze the pre-processed image data and automatically identify the coordinates of the target;

[0120] Step 7.2.3: The backend server compares the calculated target coordinates with the target design data coordinates stored in the database to obtain the offset and offset direction of the target relative to the design position;

[0121] Step 7.3: Push the offset data. Push the calculated target offset information and target number to the front-end device manually or automatically so that the layout personnel can perform correction operations. The specific operations include:

[0122] Step 7.3.1. The backend server encapsulates the calculated target offset information and related parameters (such as target number, shooting time, shooting station number, etc.) into a data packet and sends it to the front-end device via a wireless or wired network;

[0123] Step 7.3.2: After receiving the data packet, the front-end device selects manual or automatic push mode according to the user settings or system defaults. The manual mode means that the front-end device displays the data packet on the screen and waits for the user to confirm or modify it. After confirmation, it is sent to the visual correction platform. The automatic mode means that the front-end device directly transmits the data packet to the visualization platform.

[0124] Step 7.3.3, the front-end device feeds back the push results to the back-end server and records them in the database;

[0125] Step 7.4: Visual deviation correction: Through simulation technology, the target offset information is displayed in the 3D scene or the high-definition real-life image obtained by the camera station, and is superimposed and compared with the target design data, thereby providing real-time visual deviation correction assistance for the layout personnel. The specific operations include:

[0126] Step 7.4.1. The front-end device inputs the target offset information and related parameters into the visualization platform according to the user selection or system default;

[0127] Step 7.4.2: The visualization platform generates a virtual target based on the target design data and generates target X, Y, and Z offset annotation information based on the input real-time target data. The data is simultaneously displayed in the 3D scene or on the high-definition real-scene image obtained by the camera station.

[0128] Step 7.4.3. The visualization platform marks the target offset data with different colors or symbols to intuitively display the difference between the real-time target and the target design data, thereby helping the layout personnel to perform correction operations.

[0129] Example:

[0130] (1) Intelligent target design

[0131] The intelligent target consists of three parts: calibration component, verification component, and identification component. See Figures 1 to 6 for details:

[0132] 1) Calibration components: mainly composed of concentric circles, rectangles, crosshairs with the same intersection point, and rectangular frame gaps (for direction interpretation). They can be solved independently, eliminated during adjustment, or used for quality inspection;

[0133] 2) Verification component: It is mainly used for checking the accuracy of target center coordinates. It can realize the verification of positioning error values ​​of two groups of crosshair intersections, the verification of positioning error values ​​of crosshair intersections and concentric circle centers, and the verification of positioning error values ​​of crosshair intersections and rectangle centers.

[0134] 3) Identification symbol: Mainly used to identify basic target metadata and for on-site error correction guidance. The target is accompanied by a QR code image, which can be scanned to obtain detailed information about the target, mainly including identification interpretation, reference target number, target size, project name, production time, project area number and other information.

[0135] This target has the following characteristics:

[0136] This target has the following characteristics:

[0137] a. This target is suitable for safety monitoring and routine construction surveying and setting out;

[0138] b. The target can be centered by adjusting the center point through structures 11 and 12 (see Figure 7 for details, where D is the horizontal adjustment distance), which is suitable for inclined or vertical template layout;

[0139] c. The target can be rotated 360° through 4, 5, 6, 7, and 8 structures.

[0140] (2) Intelligent target recognition

[0141] 1) Build a dataset

[0142] Target images were collected under conditions similar to those used in field operations, and a target detection dataset was created using annotation software. The dataset included a training set, a validation set, and a test set. The annotated data was divided into training and validation sets using an 8:2 ratio, and the training images were enhanced by randomly rotating them by 90°, 180°, and 270° and flipping them horizontally. Target images were taken and annotated as a test set. Both the training and test sets were preprocessed, and the original full-color RGB images were normalized and input into the network model. The detailed process is shown in Figure 7.

[0143] 2) Constructing a target extraction model

[0144] The object detection algorithm based on YOLOv5 is adopted. YOLOv5 mainly consists of two parts: the backbone network and the neck network. The backbone network uses the Focus structure and the CSP structure to extract and improve image features; the neck network uses the FPN and PAN structures to process features of different scales.

[0145] 3) Training target extraction model

[0146] The training set data processed in step 1) is input into the target extraction model constructed in step 2). The target extraction model randomly batches a specified number of images each time for learning, and the training process is optimized using the Adam optimizer.

[0147] 4) Test the accuracy of target extraction:

[0148] Input the processed test set data from step 1) into the target extraction model trained in step 3) to output the target's position on the image. The target extraction accuracy is then measured by calculating objective evaluation metrics such as the accuracy between the output image and the annotated image, Intersection over Union (IoU), and positioning error. If the accuracy does not meet the requirements, adjustments are made by increasing the amount of training data, adjusting hyperparameters, and repeating step 3) until the target extraction result meets the accuracy requirements.

[0149] 5) Target identification and approximate position calculation

[0150] The photos taken on site are pre-processed and input into the target extraction model trained in step 3), and the approximate image pixel coordinates of the target are calculated based on the target recognition results.

[0151] (3) High-precision extraction and positioning of target center points

[0152] The core algorithms of this module include intersection point extraction and refinement methods based on image analysis, and photogrammetry fast cluster computing technology for spatial measurement based on double-image forward intersection. The specific process is shown in Figure 8.

[0153] 1) Feature extraction and calculation adjustment of image intersection points.

[0154] 2) Forward intersection calculation service based on image exterior orientation parameters.

[0155] (4) Accurate measurement and real-time correction

[0156] The precise measurement and real-time correction system architecture is shown in Figure 9.

[0157] 1) Electronic layout and visual correction system

[0158] The electronic stakeout and visual deviation correction system primarily analyzes target results and intelligently delivers targeted information to frontline command personnel. Through continuous, intelligent observation of target positions, it delivers deviation results and correction instructions to relevant on-site personnel as needed. Its core functions include: automatically updating real-time target data and deviation status; automatically or manually delivering target status information based on operational objectives or observation locations; providing deviation correction or command instructions based on target data; and synchronizing deviation posture and information with the 3D scene or high-definition real-world imagery captured by the camera station.

[0159] 2) Comparative display of historical and designed target positions

[0160] The system will first load the data of historical target positions and designed target positions, which are usually stored in the system's database. Through AR technology, the system will overlay these position data in the user's field of view to create a virtual scene. The user can intuitively see the comparison between the historical and designed positions to understand the target's layout and error.

[0161] 3) Automatically calculate target layout error

[0162] Using AR technology, the system identifies the actual position image of the target captured by the acquisition equipment in real time, compares it with the designed position, automatically calculates the X, Y, and Z errors between the actual position and the designed position, and displays the error values ​​in the 3D scene or real-life image obtained by the camera station, helping users understand the accuracy of the sampling situation and decide whether adjustments are needed.

[0163] 4) Target change monitoring

[0164] The system continuously monitors the target's position and compares it to the designed position. If the target's actual position shifts significantly, the system automatically identifies the target change and displays the shifted target in a different color based on the set shift threshold.

[0165] 5) Target AR-assisted layout

[0166] If the system detects a significant error between the target position and the designed position, the system can display the offset value in real time in the AR scene to assist the user in adjusting the target position so that it is consistent with or close to the designed position, providing intuitive correction guidance to help users place the target more accurately.

[0167] 6) Update and enter the stakeout results data

[0168] If the user adjusts the target position, the system can support the user to manually input or automatically record the new target position data and save it to the database to ensure the real-time and accuracy of the survey results data for subsequent query and analysis.

[0169] 7) Scan the target QR code

[0170] Users can use their mobile phones to scan the QR code on the target, and the system will automatically extract the target's unique number and error calculation results. Users can quickly view target information and verify data.

[0171] 4. Test accuracy verification

[0172] Use directional software to achieve target positioning accuracy (5-8mm accuracy at a measurement distance of 70 meters)

[0173] (1) The real 3D environment of the test area is shown in Figure 10

[0174] (2) Target point extraction and measurement

[0175] The target point is measured by the intersection of the crosshairs, as shown in Figure 11.

[0176] (3) Accuracy and error check

[0177] The plane error is 4.9mm, and the elevation error is 4.5mm, as shown in Table 1:

[0178] Table 1 Accuracy error table

[0179] Conclusion: From the verification accuracy results, it can be seen that the technical method proposed in this patent is feasible, the accuracy can meet the measurement requirements, and it has the conditions for promotion and implementation.

Claims

1. A feature extraction, measurement and correction device based on intelligent targets, characterized in that: The target device comprises a target disk (4), the target disk (4) is rotatably connected to the top of a target connecting rod (5), the bottom of the target connecting rod (5) is connected to a rotating target cloud platform (6), the target cloud platform (6) is in sliding contact with a cloud platform base (8) and is fixed by cloud platform fixing bolts (7), and the bottom of the cloud platform base (8) is fixed to a support body.

2. The feature extraction, measurement and deviation correction device based on intelligent target according to claim 1 is characterized in that: The target cloud platform (6) is a spherical body, and the target cloud platform (6) is in sliding contact with the circular groove on the base (8).

3. The feature extraction, measurement and deviation correction device based on intelligent target according to claim 2 is characterized in that: The bottom surface of the cloud platform base (8) is fixedly connected to the spiral connecting rod (9).

4. The feature extraction, measurement and deviation correction device based on intelligent target according to claim 3 is characterized in that: The bottom surface of the cloud platform base (8) is fixedly connected to the horizontal movable upper structure (11), and the horizontal movable upper structure (11) is in sliding contact with the horizontal movable lower structure (12) below and is fixed by horizontal fixing bolts (10).

5. The feature extraction, measurement and deviation correction device based on intelligent target according to claim 4 is characterized in that: The horizontal movable lower structure (12) is provided with a clamping mechanism, which includes a steel column (18) at the bottom end of the horizontal movable lower structure (12), and a steel plate (17) elastically clamped with the steel column (18) is provided on one side of the horizontal movable lower structure (12).

6. The feature extraction, measurement and deviation correction device based on intelligent target according to claim 5, characterized in that: A connecting rod (16) is provided at the upper end of the steel plate (17). The connecting rod (16) is in an inverted "L" shape and the horizontal end rod body is nested and connected with the horizontal movable lower structure (12). A spring (15) is wrapped around the horizontal end rod body of the connecting rod (16). One end of the spring (15) contacts the inner wall of the horizontal movable lower structure (12), and the other end of the spring (15) is connected to the connecting rod (16). The end of the connecting rod (16) extends out of the horizontal movable lower structure (12) and is adjusted in horizontal position through a screw cap (13) and a hollow nut sleeve (14).

7. A feature extraction, measurement and correction device based on intelligent targets according to any one of claims 3 or 6, characterized in that: The target disk (4) is a rectangular disk, with identical calibration components on both sides of the disk. The calibration components include a plurality of groups of concentric circles, the centers of the concentric circles coincide with the center of the disk, and the disk is also provided with crosshairs where diagonal lines intersect and lines connecting the midpoints of the sides intersect. A direction identification notch (3) is provided at one corner of the rectangle, and a two-dimensional identification code (1) is provided at one corner of the disk.

8. A target feature extraction method using a feature extraction measurement and deviation correction device based on an intelligent target as claimed in claim 7, characterized in that: The steps of feature extraction are: Step 1, construct a target image dataset; collect target images in different field operation environments, and use annotation software to create a target object detection dataset; the dataset includes a training set, a validation set, and a test set. The annotated data is divided into a training set and a validation set according to a set ratio, and the training image is enhanced by randomly rotating 90°, 180°, 270°, and horizontally flipping; take target images and annotate them as a test set; preprocess both the training set and the test set, normalize the original full-color RGB image, and input it into the network model; Step 2: Build a target extraction model; use the target detection algorithm based on YOLOv5. YOLOv5 consists of a backbone network and a neck network. Part composition; The backbone network uses the Focus structure and CSP structure to extract and improve image features; The neck network processes features of different scales through FPN and PAN structures; Step 3, training the target extraction model; input the training set data processed in Step 1 into the target extraction model constructed in Step 2. The target extraction model randomly batches a specified number of images each time for learning. The training process is optimized using the Adam optimizer. Step 4, test the accuracy of target extraction; input the test set data processed in Step 1 into the target extraction model trained in Step 3, and the position of the target on the image can be output; then measure the target extraction accuracy by calculating the accuracy between the output image and the annotated image, IoU (Intersection over Union) and positioning error evaluation indicators. If the accuracy does not meet the requirements, adjust it by increasing the amount of training data and adjusting the hyperparameter method, and repeat Step 3 until the target extraction result meets the accuracy requirements; Step 5, target identification and position calculation: the photos taken on site are pre-processed and input into the target extraction model trained in Step 3, and the image pixel coordinates of the target are calculated based on the target identification results.

9. The target feature extraction method based on the feature extraction measurement and deviation correction device of the intelligent target according to claim 8 is characterized in that: After the target feature is extracted, the target center point is extracted and spatially positioned on the target image after the feature is extracted, including: Step 6.

1. Image enhancement Step 6.1.1, Image grayscale: Extract brightness information from the color channel by using the weighted average method; the calculation formula is as follows: Gray=0.299*R+0.587*G+0.114*B Where R represents the red channel, G represents the green channel, and B represents the blue channel; Step 6.1.2, Image binarization: Divide the image into two different areas according to the gray value of the pixel: foreground and background; Use the OTSU method to binarize the image to ensure that the variance between the foreground and background images is maximized; Step 6.2, Hough transform line detection The linear equation y=kx+b in the Cartesian coordinate system is transformed into b=-kx+y. k and b are regarded as the independent variable and dependent variable of the Cartesian parameter space, and then the Cartesian parameter space is mapped to the polar coordinate parameter space. According to the duality of Hough space, the linear line in the Cartesian coordinate system is mapped to the corresponding single point in the parameter space under the polar coordinate system. The point where the curves intersect the most in the parameter space is obtained, which is the corresponding line in the original Cartesian coordinate system. Based on the image enhancement results in Step 6.1, the equation of the intersecting straight line of the artificial target is obtained as follows: Step 6.3, find the intersection of line segments The equation of the artificial target line detected by the Hough transform line in Step 6.2 is intersected together to obtain the coordinates of the intersection point: The unit is pixel, and the obtained intersection point is the pixel coordinate value of the center point of the artificial target; Step 6.

4. Aerial triangulation The exterior orientation elements of photo i are calculated by using the bundle adjustment algorithm. And camera parameters (x0,y0,f,k1,k2,k3,p1,p2), where X i ,Y i ,Z i Represents the three-dimensional coordinate value of the center of the photo in the coordinate system, ω i , k i They represent the lateral inclination, heading inclination and photo rotation of the rotation system with the Y axis as the main axis respectively; x0, y0, f are the camera internal parameters, where x0, y0 are the principal point coordinates, f is the focal length, k1, k2, k3, p1, p2 are the camera external parameters, where k1, k2, k3 are the radial distortion coefficients, and p1, p2 are the tangential distortion coefficients; Step 6.5: Calculate the three-dimensional coordinates of the target center point by forward intersection Due to the previous manual target recognition step, the target range, number and photo number have been obtained. The number can be used to filter out the center intersection coordinates (x1, y1), (x2, y2) of the same target, as well as the external orientation elements of the corresponding photo. Through the photogrammetric intersection algorithm: Step 6.5.1, calculate the rotation matrix R1, R2 of the photo through the external orientation elements; Step 6.5.

2. Calculate the photographic baseline component B of the photo using the external orientation elements u ,B v ,B w : B u =X2-X1 B v =Y2-Y1 B w =Z2-Z1; Step 6.5.3, calculate the image space auxiliary coordinates of the center point of the artificial target (u1, v1, w1), (u2, v2, w2): Step 6.5.4, calculate the projection coefficient of the target center point: Step 6.5.

5. Calculate the three-dimensional coordinates of the target center point: X = X1 + N1u1 = X2 + N2u2 Y=Y1+N1v1=Y2+N2v2 Z=Z1+N1w1=Z2+N2w2.

10. The target feature extraction method based on the feature extraction measurement and deviation correction device of the intelligent target according to claim 9 is characterized in that: After the target center point of the target image is extracted and spatially positioned in step 6, measurement and real-time correction are performed: Measurement and real-time correction The measurement and real-time correction system continuously obtains observation images of the target position, and pushes the offset results to relevant on-site personnel as needed, including: automatically updating the real-time data of the target and calculating the target offset correction in real time. According to the observation image photos and the calculated image external orientation elements, that is, the directional attitude parameters, the object-side target offset vector is drawn and annotated in the image-side vector or VR mode to form a real-scene correction photo or VR scene with the correction vector drawn, and form correction guidance instructions; guide the operation commander to Line correction operation: Step 7.1, real-time target data acquisition: regularly capture the stereoscopic image data of the target through the dual-camera station to obtain the real-time position and status of the target; the specific operations include: Step 7.1.1, set the shooting interval of the camera station and adjust it according to the accuracy requirements of the target; Step 7.1.2, the camera automatically captures a set of high-precision image data at each shooting interval, and the image contains the position, attitude angle and focal length parameters of the camera; Step 7.1.3, the camera station transmits the captured image data to the back-end server via wireless or wired network, waiting for the next step of processing; Step 7.2, offset state calculation: through the target recognition algorithm, the center point coordinates of the target are extracted from the image data, and compared with the target design data coordinates, so as to calculate the offset and offset direction of the target; the specific operations include: Step 7.2.1, after receiving the image data transmitted by the camera station, the back-end server stores it in the database and performs pre-processing, such as denoising, enhancement, and cropping; Step 7.2.2, the back-end server calls the target recognition algorithm model, analyzes the pre-processed image data, and automatically identifies the coordinates of the target; Step 7.2.3, the back-end server compares the calculated target coordinates with the target design data coordinates stored in the database to obtain the offset and offset direction of the target relative to the design position; Step 7.3, push the offset data; push the calculated target offset information and target number to the front-end device manually or automatically so that the layout personnel can perform deviation correction operations; the specific operations include: Step 7.3.1, the back-end server encapsulates the calculated target offset information and related parameters into a data packet and sends it to the front-end device via a wireless or wired network; Step 7.3.2, after the front-end device receives the data packet, it selects manual or automatic mode for pushing according to the user settings or system defaults; the manual mode means that the front-end device displays the data packet on the screen and waits for the user to confirm or modify it, and then sends it to the visual correction platform after confirmation; the automatic mode means that the front-end device directly transmits the data packet to the visualization platform; Step 7.3.3, the front-end device feeds back the push results to the back-end server and records them in the database; Step 7.4, visual deviation correction: Through simulation technology, the target offset information is displayed in the three-dimensional scene or the high-definition real-life image obtained by the camera station, and is superimposed and compared with the target design data, thereby providing real-time visual deviation correction assistance for the layout personnel. The specific operations include: Step 7.4.1, the front-end device inputs the target offset information and related parameters into the visualization platform according to the user selection or system default; Step 7.4.2, the visualization platform generates a virtual target based on the target design data, and generates the target X, Y, and Z offset annotation information based on the input real-time target data, and simultaneously displays the data in the three-dimensional scene or on the high-definition real-scene image obtained by the camera station; Step 7.4.

3. The visualization platform marks the target offset data with different colors or symbols to intuitively display the difference between the real-time target and the target design data, thereby helping the layout personnel to perform correction operations.

Images