Bridge crack online monitoring method, device and bridge online automatic monitoring system

Abstract

The present invention discloses an online monitoring method and device for bridge cracks and an online automatic monitoring system for bridges, and relates to the technical field of bridge safety monitoring. The method is to first import a building surface image collected in real time by an online camera on the building surface showing bridge cracks into a pre-trained crack detection model, and output a crack detection result that can show whether there is a crack identification frame on the building surface image, and then if at least one crack identification frame is found, for each crack identification frame, the corresponding crack image is intercepted from the building surface image and the corresponding crack parameters are extracted, and finally, it is determined whether to trigger an alarm action according to the crack parameters, so that the problem of requiring cracks before targeted measurement can be perfectly avoided, and it is completely applicable to the complete online monitoring scene from the crack-free stage to the crack-presenting stage, which is convenient for practical application and promotion.

Description

Technical Field

[0001] The present invention belongs to the technical field of bridge safety monitoring, and in particular relates to an online monitoring method and device for bridge cracks and an online automatic monitoring system for bridges. Background Art

[0002] As an important carrier of public transportation, bridges play a vital role in the smooth operation of regional traffic and social and economic development. With the rapid development of social economy, the traffic volume on bridges has increased rapidly, and the impact of bridge safety hazards has intensified. Bridges are damaged and damaged due to factors such as harsh operating environment, load and long service life, which threatens bridge safety and causes bridge accidents.

[0003] Through the construction and operation of the bridge structure safety and health monitoring system, the environmental parameters and structural response parameters of the bridge during operation can be accurately and comprehensively grasped, and the use status and mechanical behavior of the bridge can be warned and preliminarily evaluated through data analysis, providing strong guarantees for the safe operation of the bridge. The bridge structure safety and health monitoring system is mainly divided into four functional data monitoring parts: structure monitoring, environment monitoring, load monitoring and video monitoring. Among them, structure monitoring can be further subdivided into deflection monitoring, crack monitoring, vibration monitoring, cable force monitoring, displacement monitoring and strain monitoring.

[0004] At present, the existing bridge crack monitoring scheme mainly uses crack meters to measure crack width. It usually consists of a moving measuring scale and a fixed base. When using the crack meter, it is necessary to place the measuring scale on the crack and move it to accurately measure the width of the crack. However, this also has the problem that cracks must be found before targeted measurements can be made. It cannot be applied to the complete online monitoring scenario from the crack-free stage to the crack-containing stage, so it needs further improvement. Summary of the invention

[0005] The purpose of the present invention is to provide a method and device for online monitoring of bridge cracks, an online automated bridge monitoring system, computer equipment, a computer-readable storage medium and a computer program product, so as to solve the problem that existing bridge crack monitoring schemes require cracks to be present before targeted measurements can be made, making them inapplicable to the complete online monitoring scenario from the no-crack stage to the crack stage.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, a method for online monitoring of bridge cracks is provided, comprising:

[0008] Acquire a building surface image collected in real time by an online camera of a building surface showing bridge cracks, wherein the building surface showing bridge cracks refers to an outer surface of a building located on a target bridge and expected to show bridge cracks when the target bridge undergoes structural deformation;

[0009] Importing the building surface image into a pre-trained crack detection model, and outputting a crack detection result that can display whether there is a crack identification box on the building surface image, wherein the crack detection model adopts a three-stage network structure including a first-stage network, a second-stage network, and a third-stage network, wherein the first-stage network, the second-stage network, and the third-stage network respectively use a Resnet50 network as a backbone network, and respectively use different dilated convolution algorithms with increasing dilated rates, so that the first-stage network, the second-stage network, and the third-stage network are reversely cascaded in sequence from coarse to fine according to the features of the deep convolution layer;

[0010] If at least one crack identification frame is found according to the crack detection result, for each crack identification frame in the at least one crack identification frame, a corresponding crack image is captured from the building surface image;

[0011] For each crack identification frame, extract corresponding crack parameters according to the corresponding crack image, wherein the crack parameters include crack width, crack length and / or crack area;

[0012] If any crack parameter of the crack identification frames is found to meet a preset alarm condition of a certain warning level, an alarm action corresponding to the certain warning level is triggered and executed.

[0013] Based on the above invention content, a new solution for complete monitoring of bridge cracks based on video surveillance images is provided, that is, first, the building surface image collected in real time by the online camera on the building surface presenting the bridge cracks is imported into a pre-trained crack detection model, and the crack detection result that can be displayed on the building surface image is obtained as output, and then if at least one crack identification frame is found, for each crack identification frame, the corresponding crack image is intercepted from the building surface image and the corresponding crack parameters are extracted, and finally, it is determined whether to trigger an alarm action according to the crack parameters. In this way, the problem of requiring cracks to be found before targeted measurements can be performed can be perfectly avoided, and it is fully suitable for the complete online monitoring scenario from the no-crack stage to the crack stage, which is convenient for practical application and promotion.

[0014] In a possible design, in the last layer of the first-stage network, M1 sliding windows with different sizes are used to perform window scanning on the building surface image to obtain a plurality of first marking frames, and according to the normalized results of the probabilities of the plurality of first marking frames containing the target cracks, N1 best first marking frames are screened out from the plurality of first marking frames using a first non-maximum suppression algorithm, wherein M1 represents a positive integer greater than or equal to 30, and N1 represents a positive integer greater than or equal to 3500;

[0015] In the last layer of the second-stage network, M2 sliding windows with different sizes are used to perform window scanning on the building surface image within the N1 best first marking frames to obtain a plurality of second marking frames, and according to the normalized results of the probabilities of the plurality of second marking frames containing target cracks, N2 best second marking frames are screened out from the plurality of second marking frames using a second non-maximum suppression algorithm, and N3 best third marking frames are aggregated and screened out using a third non-maximum suppression algorithm, wherein M2 represents a positive integer less than M1, N2 represents a positive integer less than N1, and N3 represents a positive integer less than N2;

[0016] In the last layer of the third-stage network, a structured random forest is trained using the convolutional features of the last layer to estimate the edge contour of the target crack, and based on the edge contour, a greedy iterative algorithm is used to search for a crack identification box with the maximum probability of containing the target crack at different positions and aspect ratios within the N3 best third marking boxes.

[0017] In a possible design, the first-stage network, the second-stage network, and the third-stage network also respectively adopt Focal Loss loss function as loss function.

[0018] In one possible design, the first-stage network adopts a dilated convolution algorithm with a dilation rate of 2, the second-stage network adopts a dilated convolution algorithm with a dilation rate of 8, and the third-stage network adopts a dilated convolution algorithm with a dilation rate of 16.

[0019] In a second aspect, a bridge crack online monitoring device is provided, comprising an image acquisition unit, a model application unit, an image interception unit, a parameter extraction unit and an alarm triggering unit which are sequentially communicatively connected;

[0020] The image acquisition unit is used to acquire a building surface image collected in real time by an online camera from a building surface showing bridge cracks, wherein the building surface showing bridge cracks refers to an outer surface of a building located on a target bridge and expected to show bridge cracks when the target bridge undergoes structural deformation;

[0021] The model application unit is used to import the building surface image into the pre-trained crack detection model, and output a crack detection result that can display whether there is a crack identification box on the building surface image, wherein the crack detection model adopts a three-stage network structure including a first-stage network, a second-stage network and a third-stage network, and the first-stage network, the second-stage network and the third-stage network respectively use the Resnet50 network as the backbone network, and respectively use different dilated convolution algorithms with increasing dilated rates, so that the first-stage network, the second-stage network and the third-stage network are reversely cascaded in sequence from coarse to fine according to the features of the deep convolution layer;

[0022] The image capture unit is configured to capture a corresponding crack image from the building surface image for each crack identification frame in the at least one crack identification frame if at least one crack identification frame is found according to the crack detection result;

[0023] The parameter extraction unit is used to extract corresponding crack parameters for each crack identification frame according to the corresponding crack image, wherein the crack parameters include crack width, crack length and / or crack area;

[0024] The alarm triggering unit is used to trigger and execute an alarm action corresponding to a certain warning level if any crack parameter is found to meet a preset alarm condition of a certain warning level according to the crack parameters of each crack identification frame.

[0025] In a third aspect, the present invention provides an online automated bridge monitoring system, comprising an online bridge crack monitoring subsystem and a monitoring data display board, wherein the online bridge crack monitoring subsystem comprises an edge computing device and a plurality of online cameras, and the plurality of online cameras are respectively arranged at a plurality of different positions of a target bridge;

[0026] The online camera is communicatively connected to the edge computing device, and is used to collect images of building surfaces where bridge cracks appear in real time, and transmit the collected results to the edge computing device in real time, wherein the building surfaces where bridge cracks appear refer to the outer surface of the building located on the target bridge and expected to show bridge cracks when the target bridge undergoes structural deformation;

[0027] The edge computing device is communicatively connected to the monitoring data display board, and is used to execute the bridge crack online monitoring method as described in the first aspect or any possible design of the first aspect, and transmit the real-time crack detection results and crack parameters of each crack identification frame to the monitoring data display board in real time;

[0028] The monitoring data display dashboard is used to output and display the crack detection results and the crack parameters, and when it is found that any crack parameter of the crack identification boxes meets the preset alarm condition of a certain warning level, the alarm content corresponding to the certain warning level is output and displayed.

[0029] In a possible design, it also includes a bridge deflection online monitoring subsystem, a bridge displacement online monitoring subsystem, a bridge vibration online monitoring subsystem, a bridge cable force online monitoring subsystem, a bridge strain online monitoring subsystem, a bridge working environment air temperature and humidity online monitoring subsystem, a bridge working environment wind online monitoring subsystem, a bridge working environment rainfall online monitoring subsystem and a bridge crossing vehicle online monitoring subsystem based on radar-visual fusion technology, which are respectively connected to the monitoring data display board;

[0030] The monitoring data display dashboard is also used to output and display the real-time values, change trends and change speeds of the monitoring data from the bridge deflection online monitoring subsystem, the bridge displacement online monitoring subsystem, the bridge vibration online monitoring subsystem, the bridge cable force online monitoring subsystem, the bridge strain online monitoring subsystem, the bridge working environment air temperature and humidity online monitoring subsystem, the bridge working environment wind online monitoring subsystem, the bridge working environment rainfall online monitoring subsystem and the bridge crossing vehicle online monitoring subsystem.

[0031] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the online monitoring method for bridge cracks as described in the first aspect or any possible design of the first aspect.

[0032] In a fifth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed on a computer, executes the method for online monitoring of bridge cracks as described in the first aspect or any possible design of the first aspect.

[0033] In a sixth aspect, the present invention provides a computer program product, comprising a computer program or instructions, which, when executed by a computer, implements the method for online monitoring of bridge cracks as described in the first aspect or any possible design of the first aspect.

[0034] Beneficial effects of the above scheme:

[0035] (1) The present invention creatively provides a new solution for complete monitoring of bridge cracks based on video surveillance images, namely, firstly, the building surface image collected in real time by the online camera showing the bridge cracks is imported into the pre-trained crack detection model, and the crack detection result that can be displayed on the building surface image is obtained as output, and then, if at least one crack identification frame is found, for each crack identification frame, the corresponding crack image is intercepted from the building surface image and the corresponding crack parameters are extracted, and finally, it is determined whether to trigger an alarm action according to the crack parameters. In this way, the problem of requiring cracks to be found before targeted measurements can be carried out can be perfectly avoided. It is fully applicable to the complete online monitoring scenario from the stage without cracks to the stage with cracks, and is convenient for practical application and promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A schematic flow chart of an online bridge crack monitoring method provided in an embodiment of the present application.

[0038] Figure 2 A schematic diagram of a three-stage network structure provided in an embodiment of the present application.

[0039] Figure 3 A schematic diagram of the structure of an online monitoring device for bridge cracks provided in an embodiment of the present application.

[0040] Figure 4 A schematic diagram of the structure of an online automatic monitoring system for bridges provided in an embodiment of the present application.

[0041] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0043] It should be understood that although the terms first and second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another object. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object without departing from the scope of the exemplary embodiments of the present invention.

[0044] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that there may be three relationships. For example, A and / or B can indicate three situations: A exists alone, B exists alone, or A and B exist at the same time. For another example, A, B and / or C can indicate the existence of any one of A, B and C or any combination of them. The term " / and" that may appear in this document describes another type of association object relationship, indicating that there may be two relationships. For example, A / and B can indicate two situations: A exists alone or A and B exist at the same time. In addition, the character " / " that may appear in this document generally indicates that the previous and next associated objects are in an "or" relationship.

[0045] Example:

[0046] like Figure 1 As shown, the bridge crack online monitoring method provided in the first aspect of this embodiment can be, but is not limited to, executed by a computer device having certain computing resources and communicating with an online camera, such as an edge computing device or a monitoring server. Figure 1 As shown, the bridge crack online monitoring method may include but is not limited to the following steps S1 to S5.

[0047] S1. Acquire a building surface image collected in real time by an online camera of a building surface showing bridge cracks, wherein the building surface showing bridge cracks refers to an outer surface of a building body located on a target bridge and expected to show bridge cracks when structural deformation occurs on the target bridge.

[0048] In step S1, the online camera refers to a monitoring camera that maintains online communication with the local machine, and can be installed conventionally to achieve the purpose of the camera's lens field of view covering the bridge crack presenting building surface, thereby real-time acquisition and enabling the local machine to receive and obtain the building surface image. In addition, the bridge crack presenting building surface is specifically but not limited to the side, top and / or bottom surface of the bridge building body.

[0049] S2. Import the building surface image into a pre-trained crack detection model, and output a crack detection result that can show whether there is a crack identification box on the building surface image, wherein the crack detection model adopts a three-stage network structure including a first-stage network, a second-stage network and a third-stage network, and the first-stage network, the second-stage network and the third-stage network respectively use Resnet50 network as the backbone network, and respectively use different void convolution algorithms with increasing void rates, so that the first-stage network, the second-stage network and the third-stage network are reverse cascaded in sequence from coarse to fine according to the characteristics of the deep convolution layer.

[0050] In step S2, the specific network structure of the three-stage network structure is as follows: Figure 2 As shown in the figure, the specific network structure of the Resnet50 network not only proposes a residual structure, but also proposes a skeleton network design paradigm, namely stem+nstage+cls head. For ResNet, its actual forward process is stem->4 stages->classification head, where the output stride (which represents the downsampling rate of the model, assuming that the input image size is 320×320, stride=10, then the output feature map size is 32×32, assuming that there are 9 anchors at each position, then the output feature map has a total of 32×32×9 anchors) of the stem (whose Input is a three-channel image, which first undergoes convolution, normalization, RELU, and then undergoes maximum pooling to obtain the Output) is 4, and the output strides of the four stages are 4, 8, 16, and 32 respectively. The dilated convolution algorithm refers to filling 0 in the middle of the 3×3 convolution kernel. The calculation method is the same as the standard convolution. The dilated convolution can expand the receptive field without losing the resolution problem caused by the previous downsampling step, and can capture multi-scale context information; similar to the YOLO algorithm, a dilated convolution with a small dilated rate is used to detect large targets, and a dilated convolution with a large dilated rate is used to detect small targets. Therefore, through the three-stage network structure design of the aforementioned crack detection model, the following crack detection process can be implemented: using the inverse cascade method from coarse to fine features of the deep convolution layer, possible object suggestions (that is, the detected target) are obtained in the image; dense suggestion sampling is performed in the last convolution layer of the first stage network, and then irrelevant frames are gradually filtered until the last layer of the third stage network; in the third stage network, the contour extracted from the last layer of the third stage network is used to refine the suggestion; the crack identification frame finally generated can be used in the object detection process; finally, the problem of imbalance between background noise and foreground object information is solved, and the performance of multi-scale target detection is improved.

[0051] In the step S2, specifically, in the last layer of the first-stage network, M1 sliding windows with different sizes are used to perform window scanning on the building surface image to obtain a plurality of first marking frames, and according to the normalized results of the probability of the plurality of first marking frames containing the target cracks, N1 best first marking frames are screened out from the plurality of first marking frames using a first non-maximum suppression algorithm, wherein M1 represents a positive integer greater than or equal to 30 (for example, 50), and N1 represents a positive integer greater than or equal to 3500 (for example, 4000); in the last layer of the second-stage network, M2 sliding windows with different sizes are used to perform window scanning on the building surface image within the N1 best first marking frames to obtain a plurality of second marking frames, and according to the normalized results of the probability of the plurality of second marking frames containing the target cracks, N1 best first marking frames are screened out from the plurality of first marking frames using a first non-maximum suppression algorithm. The normalized result of the probability of containing the target crack is obtained by using a second non-maximum suppression algorithm to select N2 best second marking frames from the multiple second marking frames, and also using a third non-maximum suppression algorithm to aggregate the N2 best second marking frames and select N3 best third marking frames, wherein M2 represents a positive integer less than M1 (for example, 10), N2 represents a positive integer less than N1 (for example, 3000), and N3 represents a positive integer less than N2 (for example, 1000); in the last layer of the third stage network, a structured random forest is trained using the convolutional features of the last layer to estimate the edge contour of the target crack, and according to the edge contour, a greedy iterative algorithm is used to search and try to find a crack identification frame with the maximum probability of containing the target crack at different positions and aspect ratios within the N3 best third marking frames.

[0052] In the step S2, specifically, the first stage network, the second stage network and the third stage network also respectively use the Focal Loss loss function as the loss function. The aforementioned Focal Loss loss function is introduced on the basis of the cross entropy (CE) loss function of the binary classification problem. It is a dynamically scaled cross entropy loss. Through a dynamic scaling factor, the weight of easily distinguishable samples can be dynamically reduced during the training process, thereby quickly focusing on those difficult-to-distinguish samples. The specific form is as follows:

[0053] FL(p t )=-α(1-p t ) γ log(p t )

[0054] Among them, α is used to balance positive and negative samples. p represents the probability when the predicted sample is equal to 1, and y represents the predicted value of the sample. In addition, specifically, the first-stage network adopts a dilated convolution algorithm with a dilation rate of 2, the second-stage network adopts a dilated convolution algorithm with a dilation rate of 8, and the third-stage network adopts a dilated convolution algorithm with a dilation rate of 16.

[0055] S3. If at least one crack identification frame is found according to the crack detection result, a corresponding crack image is captured from the building surface image for each crack identification frame in the at least one crack identification frame.

[0056] S4. For each crack identification frame, extract corresponding crack parameters according to the corresponding crack image, wherein the crack parameters include but are not limited to crack width, crack length and / or crack area.

[0057] S5. If any crack parameter of the crack identification frames is found to meet a preset alarm condition of a certain warning level, an alarm action corresponding to the certain warning level is triggered and executed.

[0058] Therefore, based on the online monitoring method for bridge cracks described in the aforementioned steps S1 to S5, a new solution for complete monitoring of bridge cracks based on video surveillance images is provided, namely, the building surface image collected in real time by the online camera on the building surface presenting the bridge cracks is first imported into the pre-trained crack detection model, and the crack detection result that can be displayed on the building surface image is obtained as output, and then if at least one crack identification frame is found, for each crack identification frame, the corresponding crack image is intercepted from the building surface image and the corresponding crack parameters are extracted, and finally, it is determined whether to trigger an alarm action based on the crack parameters. In this way, the problem of requiring cracks to be found before targeted measurements can be performed can be perfectly avoided, and it is completely suitable for the complete online monitoring scenario from the no-crack stage to the crack stage, which is convenient for practical application and promotion.

[0059] like Figure 3 As shown, the second aspect of this embodiment provides a virtual device for implementing the bridge crack online monitoring method described in the first aspect, comprising an image acquisition unit, a model application unit, an image interception unit, a parameter extraction unit and an alarm triggering unit which are sequentially connected in communication;

[0060] The image acquisition unit is used to acquire a building surface image collected in real time by an online camera from a building surface showing bridge cracks, wherein the building surface showing bridge cracks refers to an outer surface of a building located on a target bridge and expected to show bridge cracks when the target bridge undergoes structural deformation;

[0061] The model application unit is used to import the building surface image into the pre-trained crack detection model, and output a crack detection result that can display whether there is a crack identification box on the building surface image, wherein the crack detection model adopts a three-stage network structure including a first-stage network, a second-stage network and a third-stage network, and the first-stage network, the second-stage network and the third-stage network respectively use the Resnet50 network as the backbone network, and respectively use different dilated convolution algorithms with increasing dilated rates, so that the first-stage network, the second-stage network and the third-stage network are reversely cascaded in sequence from coarse to fine according to the features of the deep convolution layer;

[0062] The image capture unit is configured to capture a corresponding crack image from the building surface image for each crack identification frame in the at least one crack identification frame if at least one crack identification frame is found according to the crack detection result;

[0063] The parameter extraction unit is used to extract corresponding crack parameters for each crack identification frame according to the corresponding crack image, wherein the crack parameters include crack width, crack length and / or crack area;

[0064] The alarm triggering unit is used to trigger and execute an alarm action corresponding to a certain warning level if any crack parameter is found to meet a preset alarm condition of a certain warning level according to the crack parameters of each crack identification frame.

[0065] The working process, working details and technical effects of the aforementioned device provided in the second aspect of this embodiment can be found in the online monitoring method for bridge cracks described in the first aspect, and will not be repeated here.

[0066] like Figure 4 As shown, the third aspect of this embodiment provides a physical system for implementing the bridge crack online monitoring method described in the first aspect, including a bridge crack online monitoring subsystem and a monitoring data display board, wherein the bridge crack online monitoring subsystem includes an edge computing device and multiple online cameras, and the multiple online cameras are respectively arranged at multiple different positions of the target bridge;

[0067] The online camera is communicatively connected to the edge computing device, and is used to collect images of building surfaces where bridge cracks appear in real time, and transmit the collected results to the edge computing device in real time, wherein the building surfaces where bridge cracks appear refer to the outer surface of the building located on the target bridge and expected to show bridge cracks when the target bridge undergoes structural deformation;

[0068] The edge computing device is communicatively connected to the monitoring data display board, and is used to execute the bridge crack online monitoring method as described in the first aspect, and transmit the real-time crack detection results and crack parameters of each crack identification frame to the monitoring data display board in real time;

[0069] The monitoring data display dashboard is used to output and display the crack detection results and the crack parameters, and when it is found that any crack parameter of the crack identification boxes meets the preset alarm condition of a certain warning level, the alarm content corresponding to the certain warning level is output and displayed.

[0070] In a possible design, it also includes a bridge deflection online monitoring subsystem, a bridge displacement online monitoring subsystem, a bridge vibration online monitoring subsystem, a bridge cable force online monitoring subsystem, a bridge strain online monitoring subsystem, a bridge working environment air temperature and humidity online monitoring subsystem, a bridge working environment wind online monitoring subsystem, a bridge working environment rainfall online monitoring subsystem and a bridge crossing vehicle online monitoring subsystem based on radar-vision fusion technology, which are respectively communicatively connected to the monitoring data display board; the monitoring data display board is also used to output and display the real-time value, change trend and change speed of the monitoring data from the bridge deflection online monitoring subsystem, the bridge displacement online monitoring subsystem, the bridge vibration online monitoring subsystem, the bridge cable force online monitoring subsystem, the bridge strain online monitoring subsystem, the bridge working environment air temperature and humidity online monitoring subsystem, the bridge working environment wind online monitoring subsystem, the bridge working environment rainfall online monitoring subsystem and the bridge crossing vehicle online monitoring subsystem.

[0071] The working process, working details and technical effects of the aforementioned system provided in the third aspect of this embodiment can be found in the online monitoring method for bridge cracks described in the first aspect, and will not be described in detail here.

[0072] like Figure 5As shown, the fourth aspect of this embodiment provides a computer device for executing the bridge crack online monitoring method as described in the first aspect, including a memory, a processor and a transceiver that are sequentially connected in communication, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the bridge crack online monitoring method as described in the first aspect. For example, the memory may include, but is not limited to, a random access memory (Random-Access Memory, RAM), a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a first-in-first-out memory (First Input First Output, FIFO) and / or a first-in-last-out memory (First Input Last Output, FILO), etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power module, a display screen and other necessary components.

[0073] The working process, working details and technical effects of the aforementioned computer equipment provided in the fourth aspect of this embodiment can be found in the online monitoring method for bridge cracks described in the first aspect, and will not be repeated here.

[0074] In a fifth aspect of this embodiment, there is provided a computer-readable storage medium storing instructions including the method for online monitoring of bridge cracks as described in the first aspect, that is, the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the method for online monitoring of bridge cracks as described in the first aspect is executed. The computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to computer-readable storage media such as floppy disks, optical disks, hard disks, flash memories, USB flash drives, and / or memory sticks, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0075] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the fifth aspect of this embodiment can be referred to the online monitoring method for bridge cracks described in the first aspect, and will not be repeated here.

[0076] A sixth aspect of this embodiment provides a computer program product, including a computer program or instructions, wherein when the computer program or instructions are executed by a computer, the method for online monitoring of bridge cracks as described in the first aspect is implemented. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0077] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A bridge crack online monitoring method, characterized in that: include: Acquire a building surface image collected in real time by an online camera of a building surface showing bridge cracks, wherein the building surface showing bridge cracks refers to an outer surface of a building located on a target bridge and expected to show bridge cracks when the target bridge has structural deformation; The building surface image is imported into the pre-trained crack detection model, and a crack detection result that can show whether there is a crack identification frame on the building surface image is output, wherein the crack detection model adopts a three-stage network structure including a first-stage network, a second-stage network and a third-stage network, wherein the first-stage network, the second-stage network and the third-stage network respectively use Resnet50 network as the backbone network, and respectively use different hole convolution algorithms with increasing hole rates, so that the first-stage network, the second-stage network and the third-stage network are reversely cascaded in sequence from coarse to fine according to the features of the deep convolution layer; in the last layer of the first-stage network, M1 sliding windows with different sizes are used to perform window scanning on the building surface image to obtain multiple first marking frames, and according to the normalized results of the probabilities of the multiple first marking frames containing the target cracks, the first non-maximum suppression algorithm is used to screen out N1 best first marking frames from the multiple first marking frames, wherein M1 represents a positive integer greater than or equal to 30, and N1 represents a positive integer greater than or equal to 3500; in the last layer of the second-stage network, a sliding window with different sizes is used to scan the building surface image in a window ... the first non-maximum suppression algorithm is used to screen out N1 best first marking frames from the multiple first marking frames, wherein M1 represents a positive integer greater than or equal to 30, and N1 represents a positive integer greater than or equal to 3500. M2 sliding windows of different sizes are used to perform window scanning on the building surface image within the N1 best first marking frames to obtain multiple second marking frames, and according to the normalized results of the probabilities of the multiple second marking frames containing target cracks, a second non-maximum suppression algorithm is used to screen out N2 best second marking frames from the multiple second marking frames, and a third non-maximum suppression algorithm is also used to aggregate the N2 best second marking frames and screen out N3 best third marking frames, wherein M2 represents a positive integer less than M1, N2 represents a positive integer less than N1, N3 represents a positive integer less than N2; a structured random forest is trained in the last layer of the third-stage network using the convolutional features of the last layer to estimate the edge contour of the target crack, and a greedy iterative algorithm is used to search and try to find a crack identification frame with the maximum probability of containing the target crack at different positions and aspect ratios within the N3 best third marking frames according to the edge contour; the first-stage network adopts a dilated convolution algorithm with a dilated rate of 2, the second-stage network adopts a dilated convolution algorithm with a dilated rate of 8, and the third-stage network adopts a dilated convolution algorithm with a dilated rate of 16; If at least one crack identification frame is found according to the crack detection result, for each crack identification frame in the at least one crack identification frame, a corresponding crack image is captured from the building surface image; For each crack identification frame, extract corresponding crack parameters according to the corresponding crack image, wherein the crack parameters include crack width, crack length and / or crack area; If any crack parameter of the crack identification frames is found to meet a preset alarm condition of a certain warning level, then an alarm action corresponding to the certain warning level is triggered and executed.

2. The bridge crack online monitoring method according to claim 1 is characterized in that: The first-stage network, the second-stage network and the third-stage network also respectively adopt Focal Loss loss function as loss function.

3. An online monitoring device for bridge cracks, characterized in that: It includes an image acquisition unit, a model application unit, an image interception unit, a parameter extraction unit and an alarm triggering unit which are sequentially connected in communication; The image acquisition unit is used to acquire a building surface image collected in real time by an online camera from a building surface showing bridge cracks, wherein the building surface showing bridge cracks refers to an outer surface of a building located on a target bridge and expected to show bridge cracks when the target bridge undergoes structural deformation; The model application unit is used to import the building surface image into the pre-trained crack detection model, and output a crack detection result that can show whether there is a crack identification frame on the building surface image, wherein the crack detection model adopts a three-stage network structure including a first-stage network, a second-stage network and a third-stage network, the first-stage network, the second-stage network and the third-stage network respectively use Resnet50 network as the backbone network, and respectively use different void convolution algorithms with increasing void rates, so that the first-stage network, the second-stage network and the third-stage network are reversely cascaded in sequence from coarse to fine according to the features of the deep convolution layer; in the last layer of the first-stage network, M1 sliding windows with different sizes are used to perform window scanning on the building surface image to obtain multiple first marking frames, and according to the normalized results of the probabilities of the multiple first marking frames containing the target cracks, the first non-maximum suppression algorithm is used to screen out N1 best first marking frames from the multiple first marking frames, wherein M1 represents a positive integer greater than or equal to 30, and N1 represents a positive integer greater than or equal to 3500; in the last layer of the second-stage network A first layer uses M2 sliding windows with different sizes to perform window scanning on the building surface image within the N1 best first marking frames to obtain multiple second marking frames, and uses a second non-maximum suppression algorithm to filter out N2 best second marking frames from the multiple second marking frames according to the normalized results of the probabilities of the multiple second marking frames containing target cracks, and also uses a third non-maximum suppression algorithm to aggregate the N2 best second marking frames and filter out N3 best third marking frames, wherein M2 represents a positive integer less than M1, and N2 represents a positive integer less than N1. number, N3 represents a positive integer less than N2; in the last layer of the third-stage network, a structured random forest is trained using the convolutional features of the last layer to estimate the edge contour of the target crack, and according to the edge contour, a greedy iterative algorithm is used to search and try to have a crack identification frame with the maximum probability of containing the target crack at different positions and aspect ratios within the N3 best third marking frames; the first-stage network adopts a dilated convolution algorithm with a dilated rate of 2, the second-stage network adopts a dilated convolution algorithm with a dilated rate of 8, and the third-stage network adopts a dilated convolution algorithm with a dilated rate of 16; The image capture unit is configured to capture a corresponding crack image from the building surface image for each crack identification frame in the at least one crack identification frame if at least one crack identification frame is found according to the crack detection result; The parameter extraction unit is used to extract corresponding crack parameters for each crack identification frame according to the corresponding crack image, wherein the crack parameters include crack width, crack length and / or crack area; The alarm triggering unit is used to trigger and execute an alarm action corresponding to a certain warning level if any crack parameter is found to meet a preset alarm condition of a certain warning level according to the crack parameters of each crack identification frame.

4. A bridge online automatic monitoring system, characterized in that: It includes a bridge crack online monitoring subsystem and a monitoring data display board, wherein the bridge crack online monitoring subsystem includes an edge computing device and multiple online cameras, and the multiple online cameras are respectively arranged at multiple different positions of the target bridge; The online camera is communicatively connected to the edge computing device, and is used to collect images of building surfaces where bridge cracks appear in real time, and transmit the collected results to the edge computing device in real time, wherein the building surfaces where bridge cracks appear refer to the outer surface of the building located on the target bridge and expected to show bridge cracks when the target bridge undergoes structural deformation; The edge computing device is communicatively connected to the monitoring data display board, and is used to execute the bridge crack online monitoring method according to any one of claims 1 to 2, and transmit the real-time crack detection results and crack parameters of each crack identification frame to the monitoring data display board in real time; The monitoring data display dashboard is used to output and display the crack detection results and the crack parameters, and when it is found that any crack parameter of the crack identification boxes meets the preset alarm condition of a certain warning level, the alarm content corresponding to the certain warning level is output and displayed.

5. The bridge online automatic monitoring system according to claim 4, characterized in that: It also includes a bridge deflection online monitoring subsystem, a bridge displacement online monitoring subsystem, a bridge vibration online monitoring subsystem, a bridge cable force online monitoring subsystem, a bridge strain online monitoring subsystem, a bridge working environment air temperature and humidity online monitoring subsystem, a bridge working environment wind online monitoring subsystem, a bridge working environment rainfall online monitoring subsystem and a bridge crossing vehicle online monitoring subsystem based on a radar-vision fusion technology, which are respectively connected to the monitoring data display board in communication; The monitoring data display dashboard is also used to output and display the real-time values, change trends and change speeds of the monitoring data from the bridge deflection online monitoring subsystem, the bridge displacement online monitoring subsystem, the bridge vibration online monitoring subsystem, the bridge cable force online monitoring subsystem, the bridge strain online monitoring subsystem, the bridge working environment air temperature and humidity online monitoring subsystem, the bridge working environment wind online monitoring subsystem, the bridge working environment rainfall online monitoring subsystem and the bridge crossing vehicle online monitoring subsystem.

6. A computer device, characterized in that: The method comprises a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the online monitoring method for bridge cracks as described in any one of claims 1 to 2.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on the computer, the online monitoring method for bridge cracks as described in any one of claims 1 to 2 is executed.

8. A computer program product comprising a computer program or instructions, characterized in that When the computer program or the instruction is executed by a computer, the online monitoring method for bridge cracks as claimed in any one of claims 1 to 2 is implemented.

Images