Bridge disease detection and positioning method, device and equipment and storage medium

Abstract

The invention provides a bridge disease detection and positioning method, device and equipment and a storage medium, and relates to the technical field of bridge detection, and the bridge disease detection and positioning method comprises the steps: obtaining a bridge detection image of a to-be-detected region of a bridge, and carrying out the preprocessing of the bridge detection image; performing bridge disease identification judgment on the preprocessed bridge detection image based on a multi-scale convolutional neural network algorithm; when the bridge disease is identified, acquiring a multi-view disease image of an area where the bridge disease is located; feature points in the multi-view disease images are matched, the corresponding relation between the different-view disease images is determined, the position of the bridge disease in the three-dimensional space is calculated based on a triangulation method and the corresponding relation between the different-view disease images, and three-dimensional coordinates of the bridge disease are generated. According to the invention, efficient detection and accurate positioning of bridge diseases can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge detection, and in particular to a method, device, equipment and storage medium for detecting and locating bridge defects. Background Art

[0002] With the widespread application of bridge engineering in modern society, the safety and durability of bridges have become crucial factors in the transportation system. However, bridges will suffer from various defects such as cracks, corrosion, deformation, etc. during long-term service due to environmental factors, material aging, load effects, etc. Timely detection and location of these defects are crucial for the maintenance of bridges.

[0003] Traditional bridge defect detection methods mostly rely on manual inspection, which has the problems of low detection accuracy, low efficiency and significant influence of human subjective factors. At the same time, modern bridge structures are complex, and it is difficult to cover all key parts by relying solely on manual inspection, which poses a great safety hazard. Summary of the invention

[0004] The problem solved by the present invention is how to accurately identify and locate bridge defects.

[0005] In order to solve the above problems, the present invention provides a method, device, equipment and storage medium for detecting and locating bridge defects.

[0006] In a first aspect, the present invention provides a method for detecting and locating bridge defects, comprising:

[0007] Acquire a bridge detection image of the bridge to be detected area, and preprocess the bridge detection image;

[0008] Based on a multi-scale convolutional neural network algorithm, bridge disease identification and judgment are performed on the preprocessed bridge inspection image;

[0009] When a bridge defect is identified, a multi-view defect image of the area where the bridge defect is located is obtained;

[0010] The characteristic points in the multi-view defect images are matched to determine the correspondence between the defect images at different viewpoints, and the position of the bridge defect in three-dimensional space is calculated based on triangulation and the correspondence between the defect images at different viewpoints to generate the three-dimensional coordinates of the bridge defect.

[0011] Optionally, before acquiring the bridge detection image of the bridge to be detected area, the method further includes:

[0012] Acquire the ambient brightness data of the bridge to be tested area;

[0013] When the ambient brightness data is lower than a preset threshold, the fill light device is activated to fill light for the bridge area to be tested.

[0014] Optionally, the performing supplementary lighting on the bridge area to be tested includes:

[0015] The bridge area to be tested is illuminated in sections by a multi-light source array, wherein the brightness and lighting angle of each light source in the light source array are controllable;

[0016] Performing real-time detection of the ambient brightness of the bridge area to be tested, obtaining illumination distribution information of the bridge area to be tested, and analyzing illumination requirements in each area based on the illumination distribution information and reflectivity data of the bridge area to be tested;

[0017] Analyzing the reflectivity data and the lighting requirements based on a particle swarm algorithm to determine the illumination angle and lighting intensity of the multi-light source array;

[0018] The multi-light source array is controlled to perform fill light based on the illumination angle and light intensity of the multi-light source array.

[0019] Optionally, the performing bridge disease identification and judgment on the preprocessed bridge detection image based on a multi-scale convolutional neural network algorithm includes:

[0020] Performing feature extraction on the preprocessed bridge detection image based on a multi-scale convolutional neural network to obtain a first feature extraction image, wherein the multi-scale convolutional neural network uses convolution kernels of different sizes in multiple parallel paths to simultaneously extract local detail features and global structural features from the bridge detection image;

[0021] Dynamically adjusting a pooling window of an adaptive pooling layer according to the first feature extraction image, and performing pooling processing on the first feature extraction image based on the dynamically adjusted adaptive pooling layer;

[0022] Based on the feature fusion layer, the first feature extraction image after the pooling process is fused to obtain a high-dimensional feature vector;

[0023] Performing a linear transformation on the high-dimensional feature vector based on a fully connected layer, mapping the high-dimensional feature vector into a feature space representation for bridge disease classification;

[0024] Based on the feature space representation, using a Softmax function to perform binary classification on the first feature extraction image to determine whether there is a bridge disease;

[0025] If the output result of the Softmax function exceeds the preset judgment threshold, it is determined that the bridge disease exists; if the output result of the Softmax function is lower than or equal to the preset judgment threshold, it is determined that the bridge disease does not exist.

[0026] Optionally, when a bridge defect is identified, acquiring a multi-view defect image of an area where the bridge defect is located includes:

[0027] When a bridge defect is identified, determining a plurality of shooting angles of the bridge defect;

[0028] Determine the light intensity at each shooting angle, judge the light intensity, and configure the exposure time at each shooting angle based on the judgment result; wherein, if the light intensity is greater than a preset light intensity, set the exposure time of the shooting component to a first exposure time; if the light intensity is not greater than the preset light intensity, set the exposure time of the shooting component to a second exposure time, and the first exposure time is less than the second exposure time;

[0029] A multi-view defect image of the area where the bridge defect is located is acquired based on the exposure time corresponding to each shooting angle.

[0030] Optionally, matching the feature points in the multi-view defect images, determining the correspondence between the defect images at different viewpoints, calculating the position of the bridge defect in three-dimensional space based on triangulation and the correspondence between the defect images at different viewpoints, and generating the three-dimensional coordinates of the bridge defect includes:

[0031] Performing feature point detection on the multi-view defect image using a scale-invariant feature transformation algorithm to obtain descriptors corresponding to the feature points, wherein the feature points include corner points, edge points, and distinguishing points of the defect image at each view angle, and the descriptors are used to characterize local image information of the feature points;

[0032] Based on the Euclidean distance matching algorithm and the descriptor, the feature points in the multi-view disease images are matched to determine the corresponding relationship between the multi-view disease images to obtain feature point pairs;

[0033] Based on the characteristic point pairs, the three-dimensional coordinates of the bridge defect are determined using triangulation.

[0034] Optionally, the determining the three-dimensional coordinates of the bridge defect by using triangulation based on the feature point pair includes:

[0035] Acquire the internal parameter data and external parameter data of the shooting component, wherein the internal parameter data includes the focal length, the optical center coordinates and the distortion coefficient, and the external parameter data includes the rotation matrix and the translation vector of the shooting component, and each shooting component corresponds to a shooting angle;

[0036] Extracting pixel coordinates of the feature point pairs in the multi-view disease image;

[0037] The geometric relationship between the shooting components is determined based on the external parameter data, the position information of the feature points in three-dimensional space is determined based on the geometric relationship and the pixel coordinates of the feature point pairs, and the three-dimensional coordinates of the bridge defect are generated based on the position information.

[0038] In a second aspect, the present invention provides a bridge disease detection and positioning device, comprising:

[0039] An image acquisition module is used to acquire a bridge detection image of a bridge detection area and pre-process the bridge detection image;

[0040] A bridge disease recognition module, used for identifying and judging bridge diseases on the preprocessed bridge detection image based on a multi-scale convolutional neural network algorithm;

[0041] A multi-view image acquisition module, used for acquiring a multi-view image of the area where the bridge defect is located when a bridge defect is identified;

[0042] A positioning module is used to match the feature points in the multi-view defect images, determine the correspondence between the defect images at different viewpoints, calculate the position of the bridge defect in three-dimensional space based on triangulation and the correspondence between the defect images at different viewpoints, and generate the three-dimensional coordinates of the bridge defect.

[0043] In a third aspect, the present invention provides an electronic device, including a memory and a processor;

[0044] The memory is used to store computer programs;

[0045] The processor is used to implement the bridge disease detection and positioning method as described in the first aspect when executing the computer program.

[0046] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting and locating bridge defects as described in the first aspect is implemented.

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows: first, the bridge detection image of the bridge to be tested area is obtained. Since the multi-scale convolutional neural network has high sensitivity to small-scale bridge defects, the recognition and judgment of bridge defects by the multi-scale convolutional neural network algorithm can effectively improve the recognition accuracy of bridge defects and ensure the accuracy of recognition and judgment; then, the acquisition of multi-view defect images provides rich view information for the bridge defect area, making the subsequent matching of feature point pairs more reliable; finally, by combining the bridge defect feature points with the triangulation method, the bridge defects can be expanded to three-dimensional space while ensuring the accuracy of the three-dimensional coordinates of the bridge defects, thereby improving the positioning accuracy of the bridge defects. The present invention can achieve efficient detection and precise positioning of bridge defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a flow chart of a bridge disease detection and location method provided by an embodiment of the present invention;

[0049] Figure 2 A schematic flow chart of step S12 of a bridge defect detection and location method provided in one embodiment of the present invention;

[0050] Figure 3 A schematic diagram of the structure of a bridge defect detection and positioning device provided by an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be interpreted as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0053] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0054] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to"; the term "based on" means "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0055] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0056] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0057] Reference Figure 1 The present invention provides a method for detecting and locating bridge defects, comprising:

[0058] S11, obtaining a bridge detection image of a bridge detection area, and preprocessing the bridge detection image.

[0059] Specifically, the bridge inspection area is the area prone to bridge disease that is divided in advance by the staff based on past experience. It can be a single area of ​​the bridge or multiple areas of the bridge. The bridge inspection image can be collected by a high-resolution camera installed on a bridge inspection vehicle or a high-resolution camera installed on a drone. Preprocessing includes steps such as denoising, contrast enhancement and image correction of the bridge inspection image to improve the quality of the bridge inspection image and provide a clear image basis for subsequent bridge disease identification.

[0060] S12. Perform bridge disease identification and judgment on the preprocessed bridge detection image based on a multi-scale convolutional neural network algorithm.

[0061] Step S12 uses a multi-scale convolutional neural network algorithm to extract features and analyze the pre-processed image. The multi-scale convolutional neural network algorithm uses a multi-scale convolution kernel to extract local detail features and global structural features from the image. The trained multi-scale convolutional neural network algorithm can automatically identify the defect area in the bridge image and output the defect identification result, which can avoid the problems of difficulty and low recognition accuracy caused by manual identification.

[0062] S13. When a bridge defect is identified, a multi-view defect image of the area where the bridge defect is located is obtained.

[0063] Specifically, when the multi-scale convolutional neural network algorithm identifies bridge defects, it can control the shooting component to shoot the defect area from different preset angles, thereby obtaining defect images from multiple perspectives, providing a basis for the accurate identification of bridge defects.

[0064] S14, matching the feature points in the multi-view defect images, determining the correspondence between the defect images at different viewpoints, calculating the position of the bridge defect in three-dimensional space based on triangulation and the correspondence between the defect images at different viewpoints, and generating the three-dimensional coordinates of the bridge defect.

[0065] The feature points here refer to special pixel points that can reflect the significant features in the image. After determining the feature points at different perspectives, the matching algorithm can be used to determine the correspondence between the feature points between the defect images at different perspectives, so that the position of each feature point at different perspectives can be identified, and then the triangulation method is used to map the feature points in the two-dimensional image to the three-dimensional space, thereby achieving accurate positioning of bridge defects.

[0066] Compared with the prior art, the beneficial effects of the present invention are as follows: first, the bridge detection image of the bridge to be tested area is obtained. Since the multi-scale convolutional neural network has high sensitivity to small-scale bridge defects, the recognition and judgment of bridge defects by the multi-scale convolutional neural network algorithm can effectively improve the recognition accuracy of bridge defects and ensure the accuracy of recognition and judgment; then, the acquisition of multi-view defect images provides rich view information for the bridge defect area, making the subsequent matching of feature point pairs more reliable; finally, by combining the bridge defect feature points with the triangulation method, the bridge defects can be expanded to three-dimensional space while ensuring the accuracy of the three-dimensional coordinates of the bridge defects, thereby improving the positioning accuracy of the bridge defects. The present invention can achieve efficient detection and precise positioning of bridge defects.

[0067] In one embodiment, before acquiring the bridge detection image of the bridge to be detected area, the method further includes:

[0068] Acquire the ambient brightness data of the bridge to be tested area;

[0069] When the ambient brightness data is lower than a preset threshold, the fill light device is activated to fill light for the bridge area to be tested.

[0070] In the actual process of acquiring bridge defect images, the surrounding lighting environment is complex. When the lighting conditions are poor, such as at night and in foggy weather, or in the shadow area of ​​the bridge, the quality of the bridge inspection image can be improved by supplementing the light in the area to be tested, thereby improving the accuracy of bridge defect detection.

[0071] The ambient brightness data indicates the intensity or brightness level of light within the shooting area of ​​the detection device, which can be described by light illumination. Specifically, it can be collected in real time by a light sensor, and the fill light device can be a set of controllable light sources installed on the detection device.

[0072] Furthermore, the supplementary lighting of the bridge area to be tested includes:

[0073] The bridge area to be tested is illuminated in sections by a multi-light source array, and the brightness and lighting angle of each light source in the light source array are controllable.

[0074] Specifically, a multi-light source array is provided in the fill light device, and the brightness and lighting angle of each light source can be controlled individually; and the bridge area to be tested is divided into multiple independent lighting areas, each of which is illuminated by one or more light sources.

[0075] The ambient brightness of the bridge area to be tested is detected in real time to obtain the illumination distribution information of the bridge area to be tested, and based on the illumination distribution information and the reflectivity data of the bridge area to be tested, the illumination requirements in each area are analyzed.

[0076] Specifically, the ambient brightness of the bridge area to be tested can be obtained by a light sensor installed on the detection device. After obtaining the light distribution information of the bridge area to be tested, a light distribution map of the bridge area to be tested is drawn, and the reflectivity of the bridge area to be tested (i.e., the reflectivity of light in this area) is measured or estimated in advance, thereby determining the light requirements of each supplementary light area in the bridge area to be tested.

[0077] The reflectivity data and the illumination requirement are analyzed based on a particle swarm algorithm to determine an illumination angle and an illumination intensity of the multi-light source array.

[0078] It should be explained that according to the illumination distribution and reflectivity data of different illumination areas, the particle swarm algorithm can quickly obtain the optimal illumination parameters (i.e., the brightness and illumination angle of the light source) of each area in the test area, thereby ensuring that each area in the test area can obtain appropriate illumination.

[0079] The multi-light source array is controlled to perform fill light based on the illumination angle and light intensity of the multi-light source array.

[0080] Since it is necessary to collect multi-view defect images in the future and use them to identify and locate bridge defects, it is necessary to ensure the quality and consistency of the multi-view defect images. By using a multi-light source array to illuminate the bridge area to be tested, it is possible to effectively cope with complex lighting conditions, significantly improve the quality of bridge detection images and multi-view defect images, and provide reliable data support for the subsequent identification and positioning of bridge defects.

[0081] Reference Figure 2 The bridge disease identification and judgment based on the preprocessed bridge detection image based on the multi-scale convolutional neural network algorithm includes:

[0082] S121. Perform feature extraction on the preprocessed bridge detection image based on a multi-scale convolutional neural network to obtain a first feature extraction image, wherein the multi-scale convolutional neural network uses convolution kernels of different sizes in multiple parallel paths to simultaneously extract local detail features and global structural features from the bridge detection image.

[0083] This step uses a multi-scale convolutional neural network (MSCNN) to extract features from the preprocessed bridge detection image. The multi-scale convolutional neural network has multiple parallel convolution paths, each of which uses a convolution kernel of a different size to extract features of different scales from the bridge detection image. This embodiment can capture local details and global structural features in the image at the same time, improving the accuracy of bridge detection image recognition.

[0084] S122. Dynamically adjust a pooling window of an adaptive pooling layer according to the first feature extraction image, and perform pooling processing on the first feature extraction image based on the dynamically adjusted adaptive pooling layer.

[0085] The pooling layer of this embodiment dynamically adjusts the size of the pooling window to adapt it to the feature distribution of a specific image, thereby reducing the amount of data and the computational burden while maintaining important features. Step S122 reduces each feature image to a uniform size through the operation of the adaptive pooling layer while retaining the key features in the image.

[0086] S123. Perform fusion processing on the first feature extraction image after pooling processing based on a feature fusion layer to obtain a high-dimensional feature vector.

[0087] The first feature image after pooling is input into the feature fusion layer to fuse feature images from different convolution paths. By integrating feature information of different scales into a unified high-dimensional feature vector, the complex damage forms in the bridge inspection image can be captured.

[0088] S124, performing a linear transformation on the high-dimensional feature vector based on a fully connected layer, and mapping the high-dimensional feature vector into a feature space representation for bridge disease classification;

[0089] S125. Based on the feature space representation, use a Softmax function to perform binary classification on the first feature extraction image to determine whether there is any bridge disease.

[0090] This embodiment uses the Softmax function to calculate the probability value, which can effectively distinguish whether the bridge area to be tested has bridge defects and provide clear classification results; and combined with the preset threshold judgment, the reliability of the classification results can be guaranteed.

[0091] If the output result of the Softmax function exceeds the preset judgment threshold, it is determined that the bridge disease exists; if the output result of the Softmax function is lower than or equal to the preset judgment threshold, it is determined that the bridge disease does not exist.

[0092] This embodiment can improve the efficiency and accuracy of bridge defect detection by combining multi-scale feature extraction, dynamic pooling, feature fusion and accurate binary classification.

[0093] In one embodiment, when a bridge defect is identified, obtaining a multi-view defect image of the area where the bridge defect is located includes:

[0094] When a bridge defect is identified, a plurality of shooting angles of the bridge defect are determined.

[0095] After identifying the bridge defects, the system will select multiple shooting angles based on the shape and location of the defect area. By referring to the bridge structure diagram and the spatial distribution of the defect area, at least three different shooting angles can be selected, such as 0° (front), 45° (side) and 90° (top view), to ensure that the appearance characteristics of the defect are fully captured from multiple directions.

[0096] Determine the light intensity at each shooting angle, judge the light intensity, and configure the exposure time at each shooting angle based on the judgment result; wherein, if the light intensity is greater than a preset light intensity, set the exposure time of the shooting component to a first exposure time; if the light intensity is not greater than the preset light intensity, set the exposure time of the shooting component to a second exposure time, and the first exposure time is less than the second exposure time.

[0097] For each selected shooting angle, the light sensor can be used to detect the ambient light intensity at that angle in real time. For example, lighting data can be collected at multiple points in the shooting area to determine whether the lighting is sufficient to meet the shooting requirements. If the light intensity is uniform and higher than the preset value, it is judged that the lighting conditions are good; if the light intensity is lower than the preset value, it is judged that the light is insufficient and needs to be adjusted; according to the detection result of the light intensity, the corresponding exposure time is configured for each shooting angle. For shooting angles with higher light intensity, a shorter first exposure time is set, such as 1 / 1000 second, to prevent the loss of details of the multi-view defect image due to overexposure; and for shooting angles with lower light intensity, a longer second exposure time is set, such as 1 / 250 second, to ensure that the multi-view defect image can be fully exposed and retain the detailed characteristics of the bridge defect.

[0098] A multi-view defect image of the area where the bridge defect is located is acquired based on the exposure time corresponding to each shooting angle.

[0099] By capturing the appearance features of the diseased area from multiple angles through multi-angle shooting, the problem of missing information that may be caused by a single angle of view can be avoided, ensuring the comprehensiveness of disease information. Furthermore, the automatic configuration of exposure time based on the real-time detected light intensity can adapt to different lighting conditions and ensure the quality of multi-angle disease images. Finally, the automatic adjustment of exposure time reduces human intervention and can improve the shooting efficiency of multi-angle disease images.

[0100] In one embodiment, matching the feature points in the multi-view defect images, determining the correspondence between the defect images at different viewpoints, calculating the position of the bridge defect in three-dimensional space based on triangulation and the correspondence between the defect images at different viewpoints, and generating the three-dimensional coordinates of the bridge defect includes:

[0101] The multi-view defect image is subjected to feature point detection using a scale-invariant feature transformation algorithm to obtain descriptors corresponding to the feature points, wherein the feature points include corner points, edge points and distinguishing points of the defect image at each view angle, and the descriptors are used to characterize local image information of the feature points.

[0102] Using the scale-invariant feature transform (SIFT) algorithm for feature point detection can identify significant feature points in multi-view disease images, such as corner points, edge points, and texture points; then a descriptor is generated for each detected feature point (the descriptor is used to characterize the local image information of the feature point). The descriptor is generated based on the pixel intensity of the area around the feature point, and can remain consistent under different viewing angles, thereby facilitating subsequent feature point matching operations.

[0103] Based on the Euclidean distance matching algorithm and the descriptor, the feature points in the multi-view disease images are matched, the corresponding relationship between the multi-view disease images is determined, and the feature point pairs are obtained.

[0104] Specifically, the Euclidean distance between the feature points in one perspective and all the feature point descriptors in other perspectives is calculated, and the pair with the smallest distance is selected as the matching pair, which can accurately identify the corresponding feature points in the defect images from different perspectives; after the matching is completed, a corresponding relationship can be established for each pair of matched feature points, that is, the same feature points in the multi-perspective images are matched to form a set of feature point pairs, which provides a basis for the subsequent positioning of bridge defects.

[0105] Based on the characteristic point pairs, the three-dimensional coordinates of the bridge defect are determined using triangulation.

[0106] This embodiment uses the SIFT algorithm to perform feature point detection and descriptor generation, and combines it with the Euclidean distance matching algorithm to ensure the accuracy of feature point matching in defect images at different viewing angles, thereby reducing matching errors of feature points and enhancing the reliability of three-dimensional positioning; furthermore, the triangulation calculation based on feature points can achieve accurate positioning of bridge defects in three-dimensional space; finally, the multi-scale and rotational invariance of the SIFT algorithm can stably detect feature points under various viewing angles and lighting conditions, and perform accurate matching through the Euclidean distance matching algorithm, thereby improving the accuracy of bridge defect positioning.

[0107] Furthermore, the determining the three-dimensional coordinates of the bridge defect by using triangulation method based on the feature point pair includes:

[0108] The internal parameter data and external parameter data of the shooting component are obtained, wherein the internal parameter data include focal length, optical center coordinates and distortion coefficient, and the external parameter data include rotation matrix and translation vector of the shooting component, and each shooting component corresponds to a shooting angle.

[0109] Among them, the distortion parameters are used to correct the image distortion caused by the optical characteristics of the lens (such as radial and tangential distortion) to ensure the accuracy of the pixel coordinates; the rotation matrix is ​​used to describe the rotation relationship between the camera coordinate system and the world coordinate system to determine the orientation of the camera, and the translation vector is used to describe the position of the camera in the world coordinate system, that is, the displacement of the camera optical center relative to the origin of the world coordinate system.

[0110] The pixel coordinates of the feature point pairs in the multi-view disease image are extracted.

[0111] After completing the feature point matching of the bridge defect image, the pixel coordinates of the feature points in the respective bridge defect images are extracted from the matched feature point pairs. The pixel coordinates represent the position of the feature point on the two-dimensional image plane, usually represented by (u, v).

[0112] The geometric relationship between the shooting components is determined based on the external parameter data, the position information of the feature points in three-dimensional space is determined based on the geometric relationship and the pixel coordinates of the feature point pairs, and the three-dimensional coordinates of the bridge defect are generated based on the position information.

[0113] This step determines the geometric relationship of each shooting component in the world coordinate system based on the acquired external parameter data, that is, the position of the feature point in the camera coordinate system is converted to the world coordinate system using the camera's rotation matrix and translation vector; then the geometric relationship and the extracted pixel coordinates of the feature points are used to determine the position information of the feature points in the three-dimensional space through back projection calculation; for each pair of matching feature points, this step is repeated to determine the specific position information of the feature point pair in the three-dimensional space, and finally the three-dimensional coordinates of all feature points are combined to generate the three-dimensional coordinates of the bridge defect, thereby achieving accurate positioning of the bridge defect.

[0114] Reference Figure 3 The present invention provides a bridge disease detection and positioning device 20, comprising:

[0115] An image acquisition module 21 is used to acquire a bridge detection image of a bridge detection area and pre-process the bridge detection image;

[0116] A bridge disease identification module 22 is used to identify and judge bridge diseases on the pre-processed bridge detection image based on a multi-scale convolutional neural network algorithm;

[0117] A multi-view image acquisition module 23 is used to acquire a multi-view image of the area where the bridge defect is located when a bridge defect is identified;

[0118] The positioning module 24 is used to match the feature points in the multi-view defect images, determine the correspondence between the defect images at different viewpoints, calculate the position of the bridge defect in three-dimensional space based on the triangulation method and the correspondence between the defect images at different viewpoints, and generate the three-dimensional coordinates of the bridge defect.

[0119] The beneficial effects of the present invention are as follows: first, after the image acquisition module 21 acquires the bridge detection image of the bridge test area, the bridge disease recognition module 22 can effectively improve the recognition accuracy of bridge diseases by identifying and judging bridge diseases through the multi-scale convolutional neural network algorithm, because the multi-scale convolutional neural network has high sensitivity to small-scale bridge diseases, and ensure the accuracy of identification and judgment; then the multi-view image acquisition module 23 provides rich perspective information for the bridge disease area through the acquisition of multi-view disease images, making the subsequent matching of feature point pairs more reliable; finally, the positioning module 24 can expand the bridge disease to three-dimensional space while ensuring the accuracy of the three-dimensional coordinates of the bridge disease through the combination of bridge disease feature points and triangulation, thereby improving the positioning accuracy of the bridge disease. The present invention can achieve efficient detection and precise positioning of bridge diseases.

[0120] Reference Figure 4 , the present invention provides an electronic device 40, including a memory 41 and a processor 42;

[0121] The memory 41 is used to store computer programs;

[0122] The processor 42 is used to implement the above-mentioned bridge disease detection and positioning method when executing the computer program.

[0123] In other words, an electronic device 40 includes a memory 41 and a processor 42 coupled to the memory 41; the memory 41 is configured to store a computer program; and the processor 42 is configured to perform the following operations when executing the computer program:

[0124] Acquire a bridge detection image of the bridge to be detected area, and preprocess the bridge detection image;

[0125] Based on a multi-scale convolutional neural network algorithm, bridge disease identification and judgment are performed on the preprocessed bridge inspection image;

[0126] When a bridge defect is identified, a multi-view defect image of the area where the bridge defect is located is obtained;

[0127] The characteristic points in the multi-view defect images are matched to determine the correspondence between the defect images at different viewpoints, and the position of the bridge defect in three-dimensional space is calculated based on triangulation and the correspondence between the defect images at different viewpoints to generate the three-dimensional coordinates of the bridge defect.

[0128] The present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned method for detecting and locating bridge defects is implemented.

[0129] In other words, a non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor performs the following operations:

[0130] In other words, an electronic device 40 includes a memory 41 and a processor 42 coupled to the memory 41; the memory 41 is configured to store a computer program; and the processor 42 is configured to perform the following operations when executing the computer program:

[0131] Acquire a bridge detection image of the bridge to be detected area, and preprocess the bridge detection image;

[0132] Based on a multi-scale convolutional neural network algorithm, bridge disease identification and judgment are performed on the preprocessed bridge inspection image;

[0133] When a bridge defect is identified, a multi-view defect image of the area where the bridge defect is located is obtained;

[0134] The characteristic points in the multi-view defect images are matched to determine the correspondence between the defect images at different viewpoints, and the position of the bridge defect in three-dimensional space is calculated based on triangulation and the correspondence between the defect images at different viewpoints to generate the three-dimensional coordinates of the bridge defect.

[0135] An electronic device 40 that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device 40 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 40 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0136] The electronic device 40 includes a computing unit, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The computing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0137] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc. In the present application, the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present invention. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0138] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A method for detecting and locating bridge defects, characterized in that: include: Acquire a bridge detection image of the bridge to be detected area, and preprocess the bridge detection image; Based on a multi-scale convolutional neural network algorithm, bridge disease identification and judgment are performed on the preprocessed bridge inspection image; When a bridge defect is identified, a multi-view defect image of the area where the bridge defect is located is obtained; The characteristic points in the multi-view defect images are matched to determine the correspondence between the defect images at different viewpoints, and the position of the bridge defect in three-dimensional space is calculated based on triangulation and the correspondence between the defect images at different viewpoints to generate the three-dimensional coordinates of the bridge defect.

2. The bridge disease detection and positioning method according to claim 1 is characterized in that: Before obtaining the bridge detection image of the bridge to be tested area, the following steps are also included: Acquire the ambient brightness data of the bridge to be tested area; When the ambient brightness data is lower than a preset threshold, the fill light device is activated to fill light for the bridge area to be tested.

3. The bridge disease detection and positioning method according to claim 2 is characterized in that: The filling light to the bridge area to be tested comprises: The bridge area to be tested is illuminated in sections by a multi-light source array, wherein the brightness and lighting angle of each light source in the light source array are controllable; Performing real-time detection of the ambient brightness of the bridge area to be tested, obtaining illumination distribution information of the bridge area to be tested, and analyzing illumination requirements in each area based on the illumination distribution information and reflectivity data of the bridge area to be tested; Analyzing the reflectivity data and the lighting requirements based on a particle swarm algorithm to determine the illumination angle and lighting intensity of the multi-light source array; The multi-light source array is controlled to perform fill light based on the illumination angle and light intensity of the multi-light source array.

4. The bridge disease detection and positioning method according to claim 1 is characterized in that: The bridge disease identification and judgment based on the preprocessed bridge detection image based on the multi-scale convolutional neural network algorithm includes: Performing feature extraction on the preprocessed bridge detection image based on a multi-scale convolutional neural network to obtain a first feature extraction image, wherein the multi-scale convolutional neural network uses convolution kernels of different sizes in multiple parallel paths to simultaneously extract local detail features and global structural features from the bridge detection image; Dynamically adjusting a pooling window of an adaptive pooling layer according to the first feature extraction image, and performing pooling processing on the first feature extraction image based on the dynamically adjusted adaptive pooling layer; Based on the feature fusion layer, the first feature extraction image after the pooling process is fused to obtain a high-dimensional feature vector; Performing a linear transformation on the high-dimensional feature vector based on a fully connected layer, mapping the high-dimensional feature vector into a feature space representation for bridge disease classification; Based on the feature space representation, using a Softmax function to perform binary classification on the first feature extraction image to determine whether there is a bridge disease; If the output result of the Softmax function exceeds the preset judgment threshold, it is determined that the bridge disease exists; if the output result of the Softmax function is lower than or equal to the preset judgment threshold, it is determined that the bridge disease does not exist.

5. The bridge disease detection and positioning method according to claim 1 is characterized in that: When a bridge defect is identified, obtaining a multi-view defect image of the area where the bridge defect is located includes: When a bridge defect is identified, determining a plurality of shooting angles of the bridge defect; Determine the light intensity at each shooting angle, judge the light intensity, and configure the exposure time at each shooting angle based on the judgment result; wherein, if the light intensity is greater than a preset light intensity, set the exposure time of the shooting component to a first exposure time; if the light intensity is not greater than the preset light intensity, set the exposure time of the shooting component to a second exposure time, and the first exposure time is less than the second exposure time; A multi-view defect image of the area where the bridge defect is located is acquired based on the exposure time corresponding to each shooting angle.

6. The bridge disease detection and positioning method according to claim 1 is characterized in that: The matching of feature points in the multi-view defect images, determining the correspondence between the defect images at different view angles, calculating the position of the bridge defect in three-dimensional space based on triangulation and the correspondence between the defect images at different view angles, and generating the three-dimensional coordinates of the bridge defect includes: Performing feature point detection on the multi-view defect image using a scale-invariant feature transformation algorithm to obtain descriptors corresponding to the feature points, wherein the feature points include corner points, edge points, and distinguishing points of the defect image at each view angle, and the descriptors are used to characterize local image information of the feature points; Based on the Euclidean distance matching algorithm and the descriptor, the feature points in the multi-view disease images are matched to determine the corresponding relationship between the multi-view disease images to obtain feature point pairs; Based on the characteristic point pairs, the three-dimensional coordinates of the bridge defect are determined using triangulation.

7. The bridge disease detection and positioning method according to claim 6 is characterized in that: Determining the three-dimensional coordinates of the bridge defect by using triangulation based on the feature point pair includes: Acquire the internal parameter data and external parameter data of the shooting component, wherein the internal parameter data includes the focal length, the optical center coordinates and the distortion coefficient, and the external parameter data includes the rotation matrix and the translation vector of the shooting component, and each shooting component corresponds to a shooting angle; Extracting pixel coordinates of the feature point pairs in the multi-view disease image; The geometric relationship between the shooting components is determined based on the external parameter data, the position information of the feature points in three-dimensional space is determined based on the geometric relationship and the pixel coordinates of the feature point pairs, and the three-dimensional coordinates of the bridge defect are generated based on the position information.

8. A bridge disease detection and positioning device, characterized in that: include: An image acquisition module is used to acquire a bridge detection image of a bridge detection area and pre-process the bridge detection image; A bridge disease recognition module, used for identifying and judging bridge diseases on the preprocessed bridge detection image based on a multi-scale convolutional neural network algorithm; A multi-view image acquisition module, used for acquiring a multi-view image of the area where the bridge defect is located when a bridge defect is identified; A positioning module is used to match the feature points in the multi-view defect images, determine the correspondence between the defect images at different viewpoints, calculate the position of the bridge defect in three-dimensional space based on triangulation and the correspondence between the defect images at different viewpoints, and generate the three-dimensional coordinates of the bridge defect.

9. An electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the bridge defect detection and positioning method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting and locating bridge defects as described in any one of claims 1 to 7 is implemented.

Images