Intelligent camera system for three-dimensional modeling, displacement detection and defect identification of bridge structure

By integrating an intelligent camera system and combining multiple AI algorithms, convenient 3D modeling and defect identification of bridge structures have been achieved, solving the problems of high monitoring costs and complex layout in existing technologies, and improving detection efficiency and accuracy.

CN121691640APending Publication Date: 2026-03-17ZHEJIANG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202511843917.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-12-09
Publication Date
2026-03-17

AI Technical Summary

Technical Problem

Existing technologies for bridge structural health monitoring suffer from high monitoring costs, complex layout methods, and difficulty in achieving convenient 3D modeling and defect identification.

Method used

An intelligent camera system integrating bridge structure displacement measurement, surface defect identification, and Gaussian splash modeling functions is adopted. The AI ​​algorithm can be switched in real time through a visual interface. Robust measurement and modeling are achieved by combining collaborative model tracking, YOLO algorithm, and 3DGS algorithm.

Benefits of technology

It enables convenient 3D modeling, displacement detection, and defect identification of bridge structures, meeting the multifunctional needs of different detection scenarios and improving detection efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121691640A_ABST
    Figure CN121691640A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent camera system for three-dimensional modeling, displacement detection and defect identification of a bridge structure, and belongs to the technical field of intelligent cameras. The system provided by the invention is used for carrying out instant switching of different AI algorithms through visual interface operation according to requirements in different use scenes; the system comprises a bridge structure displacement measurement module, a bridge structure surface defect identification module and a Gaussian splash modeling module. The intelligent camera applying the system comprises an image acquisition module, an image processing module, a storage module, an interactive LCD screen and a power supply module. According to the invention, multiple detection functions of a single intelligent camera are realized, and the detection requirements of bridge structures in different application scenes can be quickly met.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of intelligent camera technology, specifically to an intelligent camera system for three-dimensional modeling, displacement detection, and defect identification of bridge structures. Background Technology

[0002] Currently, methods such as displacement gauges and strain gauges are widely used to monitor changes in bridge structures. However, given the current demand for health monitoring of numerous bridge structures, the high cost and complex deployment of traditional sensor monitoring methods hinder the development of intelligent bridge structural health monitoring. With the promotion of BIM and digital twin technologies, there is an urgent need for equipment that can conveniently realize functions such as bridge structural monitoring and 3D modeling of bridge structures.

[0003] Machine vision camera and artificial intelligence algorithm technologies are developing rapidly, and a number of smart cameras based on embedded AI chips have emerged. These cameras can identify and judge different targets by using different algorithms on the smart AI chips, and are widely used in security monitoring, health monitoring, intelligent recognition and other fields.

[0004] In summary, the current technology foundation for camera systems based on intelligent AI chips is already well-established. In view of this, this invention addresses the need for convenient bridge structural health monitoring and 3D modeling by proposing a one-stop intelligent camera system that integrates functions such as 3D modeling, displacement detection, and defect identification. Summary of the Invention

[0005] The purpose of this invention is to propose an intelligent camera system for three-dimensional modeling, displacement detection, and defect identification of bridge structures to solve the problems mentioned in the background art. This invention can realize functions such as displacement detection, surface defect identification, and three-dimensional modeling of bridge structures in different application scenarios.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A smart camera system for 3D modeling, displacement detection, and defect identification of bridge structures. The system allows for real-time switching of different AI algorithms through a visual interface based on the needs of different usage scenarios. The system includes a bridge structure displacement measurement module, a bridge structure surface defect identification module, and a Gaussian splash modeling module. The bridge structure displacement measurement module uses a collaborative model tracking algorithm to robustly measure the displacement of the bridge structure based on video data. The bridge structure surface defect identification module, based on the YOLO algorithm, extracts bridge structure surface features from image data to identify bridge structure surface defects. The Gaussian splash modeling module, based on the 3DGS algorithm, applies a bridge structure image reconstruction model.

[0007] Preferably, the execution of the bridge structure displacement measurement module specifically includes the following: S101. Determine the target position and scale of adjacent frames: Search for target and scale information of adjacent frames in the input video data; S102. Extract HOG and color features: Apply the KCF model to extract HOG features to capture the shape and edge information of the target, and apply the CN model to extract color features to ensure the robustness of target recognition under occlusion and lighting conditions. S103. The final location of the bridge structure is obtained by fusing the KCF and CN response maps: The KCF algorithm is used to process HOG features to generate a KCF response map, and the CN algorithm is used to process color features to generate a CN response map (color information response map). The PSR values ​​of the KCF and CN response maps are calculated and weighted and fused to obtain the final location of the target. S104. Multi-scale sampling to estimate the optimal scale: Based on information from sampling at other scales of the input video data, estimate and improve the scale for target displacement recognition; S105. Divide the image region and determine occlusion: Divide the target region, calculate the weighted distance between the central block and the adjacent blocks, and quantify the similarity through the weighted distance to identify occlusion. S106. Output target displacement and scale: Based on the occlusion discrimination results, optimize the target recognition scale and accuracy, and output the bridge displacement measurement value with the best recognition accuracy.

[0008] Preferably, the execution of the bridge structure surface defect identification module specifically includes the following: S201. Image data preprocessing: The input image is cropped, rotated, scaled, and its brightness is adjusted to a uniform form through image preprocessing algorithms, and the image pixel values ​​are normalized. S202, Defect Feature Extraction: The YOLO algorithm is used to extract local defect features from the image, mark the defect areas in the image, form prediction anchor boxes, and output the defect recognition area; S203. Defect detection using feature thresholds: Set a confidence threshold for feature recognition and filter out predicted boxes with confidence levels below the threshold; suppress other duplicate predicted anchor boxes by using the intersection-union ratio of the predicted box with the highest confidence level to other predicted anchor boxes, and output the defect recognition area.

[0009] Preferably, the execution of the Gaussian splash modeling module specifically includes the following: S301. Image data input: According to the 3DGS modeling requirements, high-precision images with an overlap of more than 60% between adjacent viewpoints are acquired as input. S302, Sparse Point Cloud Generation: The SfM technique is used to extract feature points from the image, the algorithm matches the structure image and features, and the triangulation method is used to calculate the camera position and feature point coordinates to generate a sparse point cloud. S303, 3D Gaussian Distribution Initialization and Optimization: The sparse point cloud is initialized as Gaussian points, and the best fit of the bridge model is achieved through an iterative optimization algorithm based on the position coordinates, covariance matrix, opacity and color information. S304, Visual Model Output: Rasterization is performed using Gaussian point position, color, and opacity information to generate a high-quality model, which is then output to the device's storage space.

[0010] A smart camera for 3D modeling, displacement detection, and defect identification of bridge structures, the camera comprising an image acquisition module, an image processing module, a storage module, an interactive LCD screen, and a power supply module.

[0011] Preferably, the image acquisition module includes a main lens unit, an imaging unit, a focusing and zoom unit, and an environmental compensation unit; wherein: The main lens unit employs multiple low-dispersion lenses and a telecentric lens to reduce chromatic aberration and distortion in camera imaging. The imaging unit employs a CMOS optical sensor and an optical signal processing circuit. The CMOS optical sensor uses a global shutter to achieve high-definition image acquisition. The focusing and zoom unit uses an autofocus and zoom drive component, which is precisely positioned by a stepper motor drive to adjust the camera's focal length and magnification. The environmental compensation unit uses an LED light source and a vibration-resistant structure to achieve high-precision image acquisition under conditions of changing lighting and vibration.

[0012] Preferably, the image processing module employs an embedded AI chip for image data reception, processing, encoding, decoding, output, and deep learning algorithm acceleration; the image processing module includes a CPU processing unit, an image and video processing unit, and an artificial intelligence unit, wherein: The CPU processing unit is designed based on the ARM architecture and is used to manage and schedule various components within the intelligent AI chip to ensure efficient communication and interoperability between different components. The image and video processing unit integrates ISP technology to preprocess the image signals transmitted by the imaging unit and form high-definition image data; The artificial intelligence unit includes an NPU and a DPU, which are used to accelerate deep learning algorithms for image and video computation.

[0013] Preferably, the storage module is used for data storage in the smart camera and includes a cache unit, a built-in storage unit, and an external storage unit, wherein: The cache unit uses flash memory hardware such as DDR4 to serve as the environment and space for AI algorithm scheduling and execution, as well as the operation of the embedded Linux system. The built-in storage unit uses eMMC storage as storage space for locally deployed AI algorithms, embedded Linux systems and programs, and establishes a locally deployed AI algorithm repository based on this. The external storage unit uses hardware such as SD cards and USB storage to store image data and the results of AI algorithm processing.

[0014] Preferably, the interactive LCD screen is used for the visual interaction design of the smart camera, and a GUI interface is developed based on the Qt library of the embedded Linux system to realize interactive switching of AI algorithms and recognition data feedback.

[0015] Preferably, the power module is used for power supply to support the stability and durability of the smart camera in different usage scenarios.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention proposes an intelligent camera system for 3D modeling, displacement detection, and defect identification of bridge structures. It integrates multiple functional algorithms, including bridge structure displacement monitoring, surface defect identification, and 3D modeling. Through visual interaction, it enables one-stop switching of multi-functional algorithms, meeting the functional requirements of the equipment in different detection scenarios or the complex detection requirements of the equipment for multiple attributes of bridge structures. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings involved in the embodiments are now briefly described. Obviously, the drawings in the following description are merely illustrative of some embodiments of the present invention. For those skilled in the art, other forms of drawings can be constructed based on these drawings without creative effort.

[0018] Figure 1 This is a functional architecture diagram of the intelligent camera system for three-dimensional modeling, displacement detection, and defect identification of bridge structures proposed in this invention; Figure 2 This is an execution flowchart of the bridge structure displacement measurement module proposed in this invention; Figure 3 This is an execution flowchart of the bridge structure surface defect identification module proposed in this invention; Figure 4 This is the execution flowchart of the Gaussian splash modeling module proposed in this invention; Figure 5 This is a hardware architecture diagram of the intelligent camera for 3D modeling, displacement detection, and defect identification of bridge structures proposed in this invention; Figure 6 This is a hardware system architecture diagram of the image acquisition module proposed in this invention; Figure 7This is a hardware system architecture diagram of the intelligent AI chip module proposed in this invention.

[0019] Explanation of the labels in the diagram: 1. Image acquisition module; 11. Main lens unit; 12. Imaging unit; 13. Focusing and zoom unit; 14. Environmental compensation unit; 2. Image processing module; 21. CPU processing unit; 22. Image and video processing unit; 23. Artificial intelligence unit; 3. Storage module; 31. Cache unit; 32. Internal storage unit; 33. External storage unit; 4. Interactive LCD screen; 5. Power supply module. Detailed Implementation

[0020] The structure and working principle of the present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of the present application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0021] Example 1: like Figures 1-4 As shown, this example proposes an intelligent camera system for 3D modeling, displacement detection, and defect identification of bridge structures. It enables displacement detection of bridge structures under different application scenarios, including a bridge structure displacement measurement module, a bridge structure surface defect identification module, and a Gaussian splash modeling module. The implementation flow of each module is as follows: The implementation process of the bridge structure displacement measurement module includes: S101. Determine the target position and scale of adjacent frames. The specific implementation method is: search for target and scale information of adjacent frames in the input video data. S102. Extract HOG features and color features. The specific implementation method is as follows: apply the KCF model to extract HOG features to capture the shape and edge information of the target, and apply the CN model to extract color features to ensure the robustness of target recognition under occlusion and illumination change conditions. S103. The final location of the bridge structure is obtained by fusing the KCF and CN response maps. The specific implementation method is as follows: the KCF algorithm is used to process HOG features to generate a KCF response map, the CN algorithm is used to process color features to generate a color information response map, and the PSR values ​​of the KCF and CN response maps are calculated and weighted to obtain the final location of the target. S104. Multi-scale sampling estimates the optimal scale. The specific implementation method is as follows: based on the information of other scale sampling of the input video data, estimate and improve the scale of target displacement recognition. S105. Divide the image region and determine occlusion. The specific implementation method is as follows: divide the target region, calculate the weighted distance between the central block and the adjacent blocks, and quantify the similarity through the weighted distance to identify occlusion. S106. Output target displacement and scale. The specific implementation method is as follows: Based on the occlusion discrimination result, optimize the target recognition scale and accuracy, and output the bridge structure displacement measurement value with the best recognition accuracy.

[0022] The implementation process of the bridge structure surface defect identification module includes: S201. The process includes image data preprocessing, specifically implemented as follows: the input image is cropped, rotated, scaled, and its brightness is adjusted to a uniform format through an algorithm, and the image pixel values ​​are normalized. S202, Defect Feature Extraction S202, the specific implementation is as follows: apply the YOLO algorithm to mark the formation prediction anchor boxes of defect regions in the image, and output indicators such as anchor box center coordinates and defect existence probability; S203, Feature threshold detection of defects, the specific implementation method is as follows: set a confidence threshold for feature recognition, filter the predicted boxes with confidence below the threshold; suppress other repeated predicted anchor boxes by the intersection-union ratio of the predicted anchor box with the highest confidence and other predicted anchor boxes, and output the defect recognition area.

[0023] The implementation process of the Gaussian splash modeling module includes: S301, Image Data Input S301, the specific implementation is as follows: according to the 3DGS modeling requirements, high-precision images with adjacent viewpoints overlapping by more than 60% are acquired as input; S302, Generating a sparse point cloud. The specific implementation method is as follows: using SfM technology to extract feature points in the image, matching the structural image with the feature points through an algorithm, and calculating the coordinates of the camera at the feature points using triangulation to generate a sparse point cloud. S303, 3D Gaussian distribution initialization and optimization, the specific implementation method is as follows: the point cloud is initialized as Gaussian points, and the best fit of the structural model is achieved through iterative optimization algorithm based on information such as position coordinates, covariance matrix, opacity and color; S304, Visualization Model Output S304, is specifically implemented by rasterizing the model using attributes such as Gaussian point position, color, and opacity to generate a high-quality model, which is then output to the device's storage space.

[0024] like Figures 5-6 As shown, the proposed intelligent camera for 3D modeling, displacement detection, and defect identification of bridge structures includes an image acquisition module 1, an image processing module 2, a storage module 3, an interactive LCD screen 4, and a power supply module 5. The composition and working order of each module are as follows: Image acquisition module 1 includes a main lens unit 11, an imaging unit 12, a focusing and zoom unit 13, and an environmental compensation unit 14, used to acquire high-definition image signals in various scenes and transmit them to image processing module 2; wherein: The main lens unit 11 uses multiple low-dispersion lenses and a telecentric lens; the imaging unit 12 uses a CMOS optical sensor and an optical signal processing circuit, and the CMOS optical sensor uses a global shutter; the focusing and zoom unit 13 uses an autofocus and zoom drive assembly, which is precisely positioned by a stepper motor; the environmental compensation unit 14 uses an LED light source and an anti-vibration structure.

[0025] Image processing module 2 employs an embedded AI chip, including a CPU processing unit 21, an image and video processing unit 22, and an artificial intelligence unit 23; wherein: The CPU processing unit 21 is designed based on the ARM architecture. The CPU unit 21 is connected to and manages the image processing unit 22 and the artificial intelligence unit 23 through the chip bus. The image and video processing unit 22 integrates ISP technology to preprocess the image signals transmitted by the image acquisition module 1 and form high-definition image data; The artificial intelligence unit 23 includes an NPU and a DPU. After the image acquisition module 1 transmits the signal to the image processing unit 22 for preprocessing, it performs feature recognition and other processing through AI algorithms accelerated by the NPU and DPU. The storage module 3 includes a cache unit 21, an internal storage unit 22, and an external storage unit 23, which are connected and transmit data through a pre-installed interface of the intelligent AI chip. The cache unit 21 uses DDR4 flash memory hardware, the internal storage unit 22 uses eMMC storage, and the external storage unit 23 uses SD card, USB storage, and other hardware.

[0026] The interactive LCD screen 4 uses a GUI interface developed using the Qt library based on an embedded Linux system.

[0027] Power module 5 is used for power supply, supporting the stability and durability of the smart camera in different usage scenarios.

[0028] The above specific embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all similar technical solutions obtained by equivalent substitution or equivalent transformation are within the protection scope of the present invention.

Claims

1. An intelligent camera system for 3D modeling, displacement detection and defect identification of bridge structures, characterized in that, The system is used for instant switching of different AI algorithms through visual interface operation according to the needs in different use scenarios; the system comprises a bridge structure displacement measurement module, a bridge structure surface defect identification module and a Gaussian splash modeling module; The bridge structure displacement measurement module realizes robust measurement of displacement of the bridge structure according to video data of the bridge structure based on a cooperative model tracking algorithm; The bridge structure surface defect identification module extracts surface features of the bridge structure in image data to identify surface defects of the bridge structure based on a YOLO algorithm; The Gaussian splash modeling module applies a bridge structure image reconstruction model based on a 3DGS algorithm.

2. The bridge structure three-dimensional modeling, displacement detecting and defect identifying intelligent camera system according to claim 1, characterized in that, The bridge structure displacement measurement module specifically comprises the following contents: S101, determining adjacent frame target position and scale: searching target and scale information of adjacent frames in input video data; S102, extracting HOG features and color features: applying a KCF model to extract HOG features for capturing shape and edge information of the target, and applying a CN model to extract color features for ensuring robustness of target identification under conditions of occlusion and illumination change; S103, fusing KCF and CN response maps to obtain a final position of the bridge structure: using a KCF algorithm to process HOG features to generate a KCF response map, using a CN algorithm to process color features to generate a CN response map, calculating PSR values of the KCF and CN response maps for weighted fusion to obtain a final position of the target; S104, multi-scale sampling to estimate the best scale: estimating and improving the scale of target displacement identification according to information of other scale sampling of the input video data; S105, dividing image regions and judging occlusion: dividing the target region, calculating weighted distances of the center block and adjacent blocks, and quantifying similarity through the weighted distances to identify occlusion; S106, outputting target displacement and scale: optimizing target identification scale and accuracy according to the occlusion judgment result, and outputting a bridge displacement measurement value with the best identification accuracy.

3. The bridge structure three-dimensional modeling, displacement detecting and defect identifying intelligent camera system according to claim 2, characterized in that, The bridge structure surface defect identification module specifically comprises the following contents: S201, image data preprocessing: performing normalization processing on image pixel values by image preprocessing algorithms to crop, rotate, scale and adjust brightness of the input image into a unified form; S202, defect feature extraction: applying a YOLO algorithm to extract local defect features of the image, marking defect regions in the image, forming a prediction anchor box, and outputting a defect identification region; S203, realizing defect detection by using a feature threshold: setting a confidence threshold for feature recognition, filtering prediction boxes with a confidence lower than the threshold; and suppressing other repeated prediction anchor boxes through an intersection-over-union ratio of the prediction box with the highest confidence and other prediction anchor boxes, and outputting a defect identification region.

4. The bridge structure three-dimensional modeling, displacement detecting and defect identifying intelligent camera system according to claim 3, characterized in that, The Gaussian splash modeling module specifically comprises the following contents: S301, image data input: collecting high-precision images with an adjacent angle overlap of more than 60% as input according to 3DGS modeling requirements; S302, sparse point cloud generation: extracting feature points in the image, algorithmically matching structure images and features, calculating camera positions and feature point coordinates by using a triangulation method, and generating a sparse point cloud by using an SfM technology. S303, 3D Gaussian distribution initialization and optimization: initialize the sparse point cloud as Gaussian points, and realize the best fitting of the bridge model through iterative optimization algorithm based on position coordinates, covariance matrix, opacity and color information; S304, visual model output: rasterize the Gaussian point position, color and opacity information to generate a high-quality model and output to the device storage space.

5. The intelligent camera for 3D modeling, displacement detection and defect identification of bridge structures using the system of any of claims 1-4, characterized in that, The camera comprises an image acquisition module (1), an image processing module (2), a storage module (3), an interactive LCD screen (4) and a power module (5). 6.The bridge structure three-dimensional modeling, displacement detecting and defect identifying intelligent camera according to claim 5, characterized in that, The image acquisition module (1) comprises a main lens unit (11), an imaging unit (12), a focusing and zooming unit (13) and an environment compensation unit (14); wherein: The main lens unit (11) adopts multiple low-dispersion lenses and a telecentric lens to reduce the color difference and distortion of camera imaging; The imaging unit (12) adopts a CMOS optical sensor and an optical signal processing circuit, and the CMOS optical sensor adopts a global shutter to realize high-definition image acquisition; The focusing and zooming unit (13) adopts an automatic focusing and zooming driving assembly, which is driven by a stepping motor to accurately position, and is used to adjust the focal length and magnification of the camera; The environment compensation unit (14) adopts an LED light source and an anti-vibration structure to realize high-precision image acquisition under the conditions of light change and vibration environment. 7.The bridge structure three-dimensional modeling, displacement detecting and defect identifying smart camera according to claim 6, characterized in that, The image processing module (2) adopts an embedded AI chip for image data reception, processing, encoding, decoding, output and deep learning algorithm acceleration; the image processing module (2) comprises a CPU processing unit (21), an image video processing unit (22) and an artificial intelligence unit (23), wherein: The CPU processing unit (21) is designed based on ARM architecture and is used to manage and schedule various components in the intelligent AI chip to ensure efficient communication and mutual operability between different components; The image video processing unit (22) integrates ISP technology and is used to pre-process the image signals transmitted by the imaging unit (12) to form high-definition image data; The artificial intelligence unit (23) comprises an NPU and a DPU and is used to accelerate the deep learning algorithm for image and video calculation. 8.The bridge structure three-dimensional modeling, displacement detecting and defect identifying intelligent camera according to claim 7, characterized in that, The storage module (3) is used for intelligent camera data storage and comprises a cache unit (31), a built-in storage unit (32) and an external storage unit (33), wherein: The cache unit (31) adopts flash hardware as the environment and space for AI algorithm scheduling and running and embedded Linux system running; The built-in storage unit (32) adopts eMMC storage as the storage space for locally deploying AI algorithms and embedded Linux system and programs, and establishes an AI algorithm warehouse based on the local deployment; The external storage unit (33) adopts SD card and USB storage hardware as the storage space for image data and AI algorithm processed results. 9.The bridge structure three-dimensional modeling, displacement detecting and defect identifying smart camera according to claim 8, characterized in that, The interactive LCD screen (4) is used for visual interactive design of the intelligent camera, develops a GUI interface based on the Qt library of the embedded Linux system, and realizes interactive switching of AI algorithms and recognition data feedback. 10.The bridge structure three-dimensional modeling, displacement detecting and defect identifying intelligent camera according to claim 9, characterized in that, The power module (5) is used for power supply to support the stability and durability of the smart camera in different use scenarios.