Center displacement measurement method with fusion of target detection and corner area screening

Abstract

The invention discloses a center displacement measurement method with fusion of target detection and corner area screening, which comprises the following steps of: performing target detection on a structure image containing a plurality of targets by using a YOLOv8 network, outputting a detection frame coordinate of each target, and calculating a center coordinate of each detection frame; for each obtained detection frame, carrying out sub-pixel-level corner detection in an image region of interest defined by the frame, and extracting four corner coordinates of the cross target; and carrying out geometric fitting or averaging calculation on the angular point coordinates. According to the method, real-time target detection and sub-pixel fine measurement are combined, on one hand, the detection efficiency and flexibility in a multi-target environment are improved (the mAP of YOLOv8 on a COCO data set reaches 28.6%, and images can be processed in real time), on the other hand, the high precision of measurement is ensured, and the detection accuracy is improved. A weight fusion algorithm and an anomaly rejection mechanism further enhance the stability and accuracy of the algorithm in a complex scene.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and structural health monitoring, and more specifically, to a center displacement measurement method that integrates target detection with corner point area screening. Background Art

[0002] Currently, optical methods are commonly used in structural health monitoring to measure tiny displacements of structures such as dams and bridges. For example, this is done by setting targets and obtaining displacement values ​​based on digital image processing. Traditional methods usually rely on camera calibration and image registration, completing measurements by converting pixel coordinate changes in the image into actual displacements.

[0003] However, the current measurement process is complicated in terms of camera calibration, and camera lens distortion can amplify measurement errors, making it difficult to quickly deploy and maintain on-site. Existing technologies that simply use deep learning target detection (such as the YOLO series) can achieve rapid positioning of multiple targets, but the center of the detection frame is only a rough position, with limited pixel-level accuracy, which does not meet the needs of high-precision measurement. At the same time, although sub-pixel corner detection methods can locate target corners in images with extremely high precision, due to the high requirements of sub-pixel algorithms for image clarity, they are prone to failure in blurred images, insufficient lighting, or occlusion. Existing technologies still lack measurement methods that combine deep learning target detection with high-precision corner extraction to balance detection robustness and positioning accuracy.

[0004] Therefore, a new method is urgently needed to achieve high-precision displacement measurement without or with reduced calibration conditions by integrating the advantages of target detection and corner point fine detection in multi-target situations. Summary of the Invention

[0005] To overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a center displacement measurement method that integrates target detection and corner area screening. By combining real-time target detection with sub-pixel fine measurement, on the one hand, it improves the detection efficiency and flexibility in multi-target environments (YOLOv8 achieves an mAP of 28.6% on the COCO dataset and can process images in real time). On the other hand, it ensures high measurement accuracy. The weight fusion algorithm and anomaly rejection mechanism further enhance the stability and accuracy of the algorithm in complex scenarios, thereby solving the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a center displacement measurement method integrating target detection and corner point area screening, comprising the following steps: Target detection and corner point screening: First, the YOLOv8 network is used to detect a structured image containing multiple targets. The approximate position of each cross target embedded in the circle is quickly located from the original image, and the corresponding detection frame is output. The region of interest (ROI) is cropped based on the spatial range of the detection frame, and only the target area is processed, eliminating interference information and improving the efficiency and accuracy of corner point extraction. Sub-pixel corner point extraction: In the ROI corresponding to each detection frame, image preprocessing (such as grayscale enhancement) is performed after corner point detection to extract the four corner points of the cross target. Sub-pixel level refinement algorithm (such as Harris corner point detection combined with cornerSubPix iterative optimization) is used to optimize the initial corner point coordinates so that the corner point position reaches the accuracy within the pixel; Center position optimization fusion algorithm: Calculate the center of the YOLO detection frame separately and the four corner points fitting the center , and then build a weight fusion strategy to get the optimization center. Specifically, weights are adaptively assigned based on indicators such as YOLO box confidence and corner fitting error. The fusion formula is as follows: . Design outlier rejection logic to remove corner points or detection frame data that obviously deviate from the geometric structure before fusion to improve robustness against blurred and interfering images; calibration-independent high-precision measurement: The measurement results of the present invention are calculated directly based on the change in the center position in the image plane, without the need for complex camera internal parameter calibration and distortion correction. By calculating the difference in pixel coordinates of the optimized center of the same target in adjacent frame images, high-precision two-dimensional displacement output is achieved, avoiding additional errors introduced by distortion and facilitating rapid on-site application.

[0007] In a preferred embodiment, in step S1, the YOLOv8 detection module trains a detection model for each target, and screens the detection results according to the confidence of the output detection box, setting a confidence threshold to eliminate false detections; The size of the detection frame can be appropriately expanded to ensure that the complete area of ​​the cross target is included, thereby limiting the ROI area for corner point extraction and improving the accuracy of corner point detection.

[0008] In a preferred embodiment, in step S2, the ROI image is pre-processed to enhance the contrast, the initial corners are detected using the Harris or Shi-Tomasi corner detection algorithm, and the corner coordinates are optimized using a sub-pixel refinement algorithm (such as OpenCV's cornerSubPix) to ensure that the extracted corner coordinates have sub-pixel accuracy. In a preferred embodiment, the weights in step S4 are By considering the confidence of the YOLO detection frame and corner fitting error To determine, for example, you can define The weighting function of is a function inversely proportional to the corner point fitting error, It is a function proportional to the confidence of the detection box, so that it is given greater weight when the corner detection is reliable, and relies more on the center of the YOLO box when the image is blurred or occluded; before fusion, the corner point coordinate set is also subjected to outlier removal, that is, outliers that are too far away from the intersection geometry or other corner points are removed.

[0009] In a preferred embodiment, the displacement measurement is the relative displacement of the target on the image plane. There is no need to use camera intrinsic parameters for calibration and distortion correction. The two-dimensional displacement of the structure is calculated by directly comparing the pixel coordinate difference of the target optimization center at different times, which directly avoids the error amplification introduced by distortion.

[0010] Technical effects and advantages of the present invention: This paper proposes an innovative fusion strategy that breaks through the limitations of traditional single-image measurement processes. By combining real-time target detection with sub-pixel fine measurement, it not only improves the detection efficiency and flexibility in multi-target environments (YOLOv8 achieves a mAP of 28.6% on the COCO dataset and can process images in real time), but also ensures high measurement accuracy. The weighted fusion algorithm and anomaly rejection mechanism further enhance the algorithm's stability and accuracy in complex scenarios. This method can achieve high-precision displacement measurement in all-weather operation without frequent calibration in structural health monitoring of dams, bridges, etc., showing significant innovation and practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of the overall structure of the present invention; Figure 2 Schematic diagram of the actual measured value of target x of the present invention; Figure 3 Schematic diagram of the actual measured value of target y of the present invention. DETAILED DESCRIPTION

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0013] As attached Figure 1 To the attached Figure 3The center displacement measurement method shown here integrates target detection and corner region screening. First, prefabricated circular targets with embedded crosses are attached to the surface of the monitored object. Each target is marked with a cross at the center, and the remaining area is a blank contrasting color. A high-resolution industrial camera is then used to capture monitoring images of the structure at different time points. For any frame of the image, the following processing steps are performed: Step 1: YOLOv8 target detection: Input the collected color image into the pre-trained YOLOv8 network model. YOLOv8 will output several detection boxes, each of which locates the position of a circular cross target and gives the coordinates of the four vertices of the detection box. and confidence , calculate the center coordinates of each detection box: In order to improve the inclusiveness of corner detection, the detection frame can be appropriately expanded and a larger ROI area can be defined for subsequent processing. By setting the confidence threshold, low-confidence frames can be filtered out to reduce false detections. Step 2: Sub-pixel corner detection. For the ROI area corresponding to each detection frame in step 1, image preprocessing (such as equalization, filtering, contrast enhancement, etc.) is first performed to improve corner features. Then, the Harris corner detection algorithm or the Shi-Tomasi algorithm is used to extract the initial corner points. In theory, the circular cross target should correspond to the four corner positions. The initial corner coordinates are sent to the sub-pixel optimization algorithm (such as cornerSubPix), and the corner positions are iteratively refined to achieve sub-pixel coordinate accuracy. The four corner coordinates are set as , If more than four corner points are detected, the redundant points can be removed based on the relative position relationship of the corner points or the threshold; if less than four, the target can be temporarily skipped or only the YOLO center can be used for processing; Step 3: Calculate the corner point center. Based on the obtained four corner point coordinates, calculate the corner point fitting center. For example, a simple arithmetic average can be used: In an ideal situation, the above average value is the coordinate of the intersection of the crosshairs. Alternatively, a more robust center can be obtained by fitting a straight line to the two sets of corner points and finding the intersection. During the calculation process, it is possible to detect whether each corner point is geometrically consistent with the intersection structure. If a corner point deviates too much (such as the distance from the average center exceeds the threshold), it is considered an outlier and removed, and then recalculated. Step 4: Center fusion and optimization, obtain two center positions of each target: the center of the YOLO detection frame and corner fitting center , considering the advantages and disadvantages of the two methods, the weight is introduced Fuse them, and define the fused center as: ; The fusion weight YOLO confidence and corner fitting error Together we decided that a feasible setting is to make the corner reliability , frame reliability , and then calculate the normalized weight , so that when the corner point fitting error is small tends to 1, the corner center is preferred; if the corner fitting quality is poor or the confidence If it is large, reduce it appropriately , giving more weight to the YOLO center. In addition, outlier removal is performed before fusion: for example, if or If the value exceeds the set threshold, the fusion result is determined to be unreliable. In this case, the corner point result can be ignored and only the YOLO center can be used, or the corner point extraction step can be re-evaluated. Step 5: Displacement calculation. For the same target, repeat the above steps in frames at different times to obtain the corresponding optimized center. , the center coordinates in the initial frame (reference frame) are marked as , the center of the frame at any time is recorded as , then the pixel displacement of the target in the horizontal and vertical directions is: , the pixel displacement is converted into actual physical displacement through a predetermined calibration or proportional relationship (the pixel length can be obtained through a one-time calibration), that is, a high-precision two-dimensional displacement measurement value is obtained. Because this method directly uses the image plane coordinate difference and the center accuracy after fusion is high, complex distortion correction is not required, and the error can be ignored within a small field of view. In step S1, the YOLOv8 detection module trains a detection model for each target, and screens the detection results based on the confidence of the output detection box, setting a confidence threshold to eliminate false detections; The size of the detection frame can be appropriately expanded to ensure that the complete area of ​​the cross target is included, thereby limiting the ROI area for corner point extraction and improving the accuracy of corner point detection; In step S2, the ROI image is pre-processed to enhance the contrast, the initial corners are detected using the Harris or Shi-Tomasi corner detection algorithm, and the corner coordinates are optimized using a sub-pixel refinement algorithm (such as OpenCV's cornerSubPix) to ensure that the extracted corner coordinates have sub-pixel accuracy. The weights in step S4 are By considering the confidence of the YOLO detection frame and corner fitting error To determine, for example, you can define The weighting function of is a function inversely proportional to the corner point fitting error, It is a function proportional to the confidence of the detection box, giving it greater weight when the corner detection is reliable, and relying more on the YOLO box center when the image is blurred or occluded; before fusion, the corner point coordinate set is also subjected to outlier removal, that is, outliers that are too far away from the intersection geometry or other corner points are removed; The displacement measurement is the relative displacement of the target on the image plane. There is no need to use camera intrinsic parameters for calibration and distortion correction. The two-dimensional displacement of the structure is calculated by directly comparing the pixel coordinate difference of the target optimization center at different times, avoiding the error amplification introduced by distortion.

[0014] Through the above steps, the present invention can achieve stable and accurate displacement measurement for multiple monitoring targets without the need for continuous calibration. In practical applications, the YOLOv8 model and parameters can be trained and adjusted specifically for each monitoring area to accommodate targets of varying sizes and materials. Furthermore, the corner detection algorithm can select appropriate thresholds and window sizes based on the target design. Experimental results demonstrate that this method can maintain sub-pixel displacement measurement accuracy despite mild image blur, illumination variations, and environmental clutter, offering significant advantages over traditional single-technology solutions.

[0015] In summary, this invention provides an innovative fusion strategy that uses object detection to screen corner regions and combines it with a weighted center fusion algorithm to achieve high-precision, calibration-free structural deformation and displacement measurement in complex environments. This technology offers simple implementation, excellent real-time performance, and high robustness, making it ideally suited for long-term online displacement monitoring in structural health monitoring applications such as dams and bridges.

[0016] The high-precision center position optimization displacement measurement device included in the center displacement measurement method of the above-mentioned target detection and corner area screening fusion includes: an image acquisition module, a YOLOv8 target detection module, a corner detection module, a center optimization fusion module and a displacement calculation module; the image acquisition module is used to obtain a structural deformation image containing multiple circular cross targets; the YOLOv8 target detection module is used to perform multi-target detection on the image and output the detection frame and the center coordinates of the detection frame for each target; the corner detection module is used to perform sub-pixel corner extraction in the ROI area corresponding to each detection frame, obtain the corner coordinates of the cross target and calculate the corner center; the center The optimization fusion module is used to compare the detection box center and the corner point center, assign weights based on confidence and fitting error, and fuse the two center positions to obtain the optimized center coordinates; the displacement calculation module is used to output the two-dimensional displacement measurement results of the target by comparing the position changes of the optimization center in different frame images, wherein the YOLOv8 target detection module realizes multi-scale and multi-category real-time detection; the corner point detection module can further include an image preprocessing submodule and a corner point refinement submodule to improve the stability and accuracy of corner point extraction; the center optimization fusion module includes an outlier filtering unit to identify and eliminate outliers that appear in the detection box or corner point calculation, thereby improving the robustness of the overall algorithm; One of the center displacement measurement methods that integrates target detection and corner point area screening also includes a computer-readable storage medium, which internally stores a program that enables the processor to execute the above five steps S1 to S5. When the program is executed, the processor completes the above steps in sequence, thereby realizing a high-precision displacement measurement function.

[0017] Finally, a few points should be explained: First, in the description of this application, it should be noted that, unless otherwise specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense, and may refer to mechanical or electrical connections, internal communication between two components, or direct connection. "Up," "down," "left," and "right" are only used to indicate relative positional relationships. When the absolute positions of the objects being described change, the relative positional relationships may also change. Secondly: The drawings of the embodiments disclosed in the present invention only involve structures related to the embodiments disclosed in the present invention. Other structures may refer to conventional designs. The same embodiment and different embodiments of the present invention may be combined with each other without conflict. Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A center displacement measurement method integrating target detection and corner point area screening, characterized in that: The following steps are involved: S1. Use the YOLOv8 network to perform target detection on a structured image containing multiple targets, output the detection frame coordinates of each target, and calculate the center coordinates of each detection frame. ; S2: For each detection frame obtained in step S1, perform sub-pixel corner detection within the image region of interest defined by the frame to extract the coordinates of the four corner points of the cross target. ; S3, by geometric fitting or averaging the corner point coordinates, obtain the corner point fitting center of the cross target ; S4, according to the center of the detection frame in step S1 and the center of the corner point in step S3 Position deviation between them, construct error weighted fusion algorithm, adaptively assign weights By fusing the two center positions, the optimized center position is calculated ; S5. Using the displacement change of the optimized target center in the continuous image frames, the two-dimensional displacement measurement value of the structural target is obtained.

2. The center displacement measurement method for integrating target detection and corner point area screening according to claim 1 is characterized in that: In step S1, the YOLOv8 detection module trains a detection model for each target, filters the detection results according to the confidence of the output detection box, and sets a confidence threshold to eliminate false detections; The size of the detection box can be expanded to ensure that the entire area of ​​the cross target is included, defining the ROI area for corner point extraction.

3. The center displacement measurement method for integrating target detection and corner point area screening according to claim 1 is characterized in that: In step S2, preprocessing is performed to enhance the contrast of the ROI image, the Harris or Shi-Tomasi corner detection algorithm is used to detect the initial corner points, and the sub-pixel refinement algorithm is used to optimize the corner point coordinates to ensure that the extracted corner point coordinates have sub-pixel accuracy.

4. The center displacement measurement method for integrating target detection and corner point area screening according to claim 1 is characterized in that: The weight in step S4 By considering the confidence of the YOLO detection frame and corner fitting error To determine, for example, you can define The weighting function of is a function inversely proportional to the corner point fitting error, It is a function proportional to the confidence of the detection box, giving it more weight when the corner detection is reliable, and relying more on the center of the YOLO box when the image is blurred or occluded; Before fusion, the corner point coordinate set is also subjected to outlier removal, that is, outliers that are too far away from the intersection geometry or other corner points are removed.

5. The center displacement measurement method for integrating target detection and corner point area screening according to claim 1 is characterized in that: The displacement measurement is the relative displacement of the target on the image plane. There is no need to use camera intrinsic parameters for calibration and distortion correction. The two-dimensional displacement of the structure is calculated by directly comparing the pixel coordinate difference of the target optimization center at different times, which directly avoids the error amplification introduced by distortion.

Images