AI-Based Bridge Digital Monitoring Method and System

Abstract

The present invention provides an AI-based digital monitoring method and system for bridges. By obtaining the structured feature information of bridge images and performing digital analysis, it can comprehensively and accurately extract key information in bridge images. The bridge image and its digital analysis data are encoded and represented through a bridge state recognition network, and the bridge structure state is recognized by fusing the image semantic coding vector and the digital coding vector, effectively generating bridge structure state data covering multiple target state space dimensions. This method not only improves the accuracy and comprehensiveness of bridge state recognition, but also optimizes the parameters of the basic bridge state recognition network through the sample bridge feature data sequence, further enhancing the reliability and practicality of bridge state recognition, contributing to the timely discovery of bridge structure anomalies and ensuring the safe operation of bridges.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology. Specifically, it relates to an AI-based digital monitoring method and system for bridges. Background Art

[0002] With the rapid development of the transportation industry, bridges, as key infrastructure connecting different regions, their safety and stability are of particular importance. However, during the long-term use of bridges, they will be affected by various natural factors (such as wind, rain, snow, earthquakes, etc.) and human factors (such as vehicle loads, construction and maintenance, etc.), resulting in the gradual aging, damage or even destruction of the bridge structure. Therefore, regular monitoring and evaluation of bridges, timely discovery and handling of potential safety hazards, are important measures to ensure the safe operation of bridges.

[0003] Traditional bridge monitoring methods mainly rely on manual inspections and regular detections. Manual inspections require professional personnel to regularly conduct on-site inspections of bridges. This method is not only time-consuming and laborious, but also difficult to cover all parts of the bridge, and it is easy to miss potential safety hazards. Regular detections usually need to be carried out when the bridge is closed or traffic is restricted, which will cause inconvenience to traffic and even economic losses.

[0004] With the progress of technology, digital monitoring technology has gradually been applied to the field of bridge monitoring. By installing sensors, cameras and other devices, various data of bridges can be collected in real time or regularly, such as images, vibrations, stresses, etc. These data provide more comprehensive and accurate information for the state assessment of bridges. However, how to effectively process and analyze these massive data and extract useful information for bridge state assessment is still a challenge faced by bridge digital monitoring technology. Summary of the Invention

[0005] In view of the above-mentioned problems, in combination with the first aspect of the present invention, embodiments of the present invention provide an AI-based digital monitoring method for bridges, and the method includes:

[0006] Obtain the first image structured feature information of the bridge image to be monitored;

[0007] Perform digital analysis on the bridge image to be monitored and the first image structured feature information to determine the digital analysis data of the bridge image to be monitored. The digital analysis data of the bridge image to be monitored includes at least one of the structure quantity information, structure trend feature information, structure standardization information of the first image structured feature information and the time span information of the bridge image to be monitored;

[0008] The bridge state recognition network encodes and represents the bridge image to be monitored and the first image structured feature information to generate an image semantic coding vector of the bridge image to be monitored, and encodes and represents the digital analysis data of the bridge image to be monitored through the bridge state recognition network to generate a digital coding vector of the bridge image to be monitored;

[0009] The bridge state recognition network performs bridge structure state recognition on the image semantic coding vector and the digital coding vector of the bridge image to be monitored to generate bridge structure state data of the bridge image to be monitored. The bridge structure state data includes a first state estimation result in a target state space dimension, and the target state space dimension includes at least one of a structural stability index dimension, a vibration frequency dimension, a load capacity dimension, a deformation monitoring duration dimension, a continuous stability dimension, a loss degree dimension, an average service life dimension, and a remaining service life dimension;

[0010] Among them, the bridge state recognition network is generated by optimizing the network parameters of the basic bridge state recognition network according to the second state estimation result and the labeled bridge structure state data of the template bridge image in the target state space dimension after the basic bridge state recognition network performs state recognition on the sample bridge feature data sequence to generate a second state estimation result of the template bridge image. The sample bridge feature data sequence includes the template bridge image, the second image structured feature information of the template bridge image, the digital analysis data of the template bridge image, and the labeled bridge structure state data of the template bridge image in the target state space dimension.

[0011] On the other hand, an embodiment of the present invention further provides an AI-based bridge digital monitoring system, including a processor and a machine-readable storage medium. The machine-readable storage medium is connected to the processor. The machine-readable storage medium is used to store programs, instructions, or codes, and the processor is used to execute the programs, instructions, or codes in the machine-readable storage medium to implement the above method.

[0012] Based on the above aspects, by obtaining the structured feature information of the bridge image and performing digital analysis, the embodiments of the present application can comprehensively and accurately extract the key information in the bridge image. The bridge state recognition network encodes and represents the bridge image and its digital analysis data, and fuses the image semantic coding vector and the digital coding vector to identify the bridge structure state, effectively generating bridge structure state data covering multiple target state space dimensions. This method not only improves the accuracy and comprehensiveness of bridge state recognition, but also optimizes the parameters of the basic bridge state recognition network through the sample bridge feature data sequence, further enhancing the reliability and practicality of bridge state recognition, contributing to the timely discovery of bridge structure anomalies and ensuring the safe operation of the bridge. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 FIG. is a schematic flowchart of the execution process of the AI-based bridge digital monitoring method provided by an embodiment of the present invention.

[0014] Figure 2 FIG. is a schematic diagram of the hardware architecture of the AI-based bridge digital monitoring system provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] The present invention will be specifically described below with reference to the accompanying drawings of the specification. Figure 1 FIG. is a schematic flowchart of the AI-based bridge digital monitoring method provided by an embodiment of the present invention, and the AI-based bridge digital monitoring method will be introduced in detail below.

[0016] Step S110, obtain the first image structured feature information of the bridge image to be monitored.

[0017] In this embodiment, assume that there is a large bridge located on a traffic artery that needs to be monitored. This bridge consists of multiple parts, including piers, bridge spans, abutments and other structures. In order to obtain the first image structured feature information of the bridge image to be monitored, a high-resolution image acquisition device is used, such as a professional drone equipped with a high-definition camera, to take pictures of the bridge at specific time intervals.

[0018] For the bridge images to be monitored, the acquisition of the first image structured feature information involves multiple aspects. Taking the bridge pier as an example, the image structured feature information may include the shape, size, surface texture, etc. of the bridge pier. From the perspective of shape, the bridge pier may be circular or square, and this information can be determined through the shape detection algorithm in the image recognition algorithm. For example, by analyzing the pixel points in the image, if the pixel points within a certain range show an approximate circular or square distribution pattern, the shape of the bridge pier can be determined. In terms of size, a scale can be established in the image, and the size of the bridge pier in the image can be determined by using known reference objects (such as standard length markings on the bridge or markers of known length set during shooting), and then the actual physical size can be deduced. The acquisition of the surface texture is determined by analyzing the gray-scale changes or color changes of the pixels on the surface of the bridge pier in the image. If there are cracks or corrosion on the surface of the bridge pier, it will be manifested as discontinuous texture or abnormal changes in gray-scale and color in the image.

[0019] Looking at the bridge span part again, its structured feature information may include the representation of the bridge span length in the image, the number of beam bodies in the bridge span structure, and the connection method between the beam bodies, etc. The representation of the bridge span length in the image can be determined by the distance from the known total bridge length information or the fixed markings at both ends of the bridge span in the image. The number of beam bodies can be counted through contour recognition in the image. Each beam body will present a specific contour shape in the image, and the number of beam bodies can be obtained by detecting and counting these contours. The connection method between the beam bodies, such as welding, bolt connection, etc., may be manifested as a specific shape or gray-scale change at the connection part in the image. For example, the welded part may have a relatively smooth transition, while the bolt connection will show the image of regularly arranged circular or square bolt heads in the image.

[0020] The image structured feature information of the abutment part is equally important. The position of the abutment, its connection relationship with the bridge pier and the bridge span, etc. are all key information. The position of the abutment in the image can be determined by its relative position relationship with the surrounding environment and the overall structure layout of the bridge. Its connection relationship with the bridge pier and the bridge span may be reflected in aspects such as the line orientation and structural continuity in the image. For example, if it is a rigid connection, there will be no obvious gap or dislocation at the connection part in the image; if it is a connection with an expansion device, a specific expansion joint structure will be shown in the image.

[0021] Thus, the first image structured feature information of the bridge images to be monitored can be obtained, providing basic data for subsequent bridge condition monitoring and analysis.

[0022] Step S120: Digitally analyze the to-be-monitored bridge image and the first image structured feature information to determine the digital analysis data of the to-be-monitored bridge image. The digital analysis data of the to-be-monitored bridge image includes at least one of the structural quantity information, structural trend feature information, structural standardization information, and time span information of the to-be-monitored bridge image in the first image structured feature information.

[0023] Continuing with the example of the large bridge on the previously mentioned important traffic artery. First, analyze the structural quantity information of the first image structured feature information. In the pier part, assume that it was previously determined through image analysis that the piers are square and there are reinforcement structures at the four corners of each pier. Through further analysis and statistics of the image, it is found that there are a total of 8 such reinforcement structures, which is a manifestation of the number of structural elements. Looking at the connection between the pier and the bridge foundation, each pier has 4 connection points, which is the number of structural connections. These structural quantity information can reflect certain aspects of the complexity and stability of the pier.

[0024] For the analysis of the structural trend feature information, the time factor needs to be considered. Assume that the bridge is imaged once every month for a year. Taking the corrosion condition on the surface of the pier as an example, in this one-year image sequence, first construct a time series set of feature information arranged in chronological order according to the pixel characteristics on the surface of the pier in each image. For example, for the corroded area on the surface of the pier, according to the pixel gray-scale change in the image, arrange the feature information such as the area or corrosion depth of the corroded area in each image acquisition according to the acquisition time.

[0025] Then, use the moving average method to smooth this time series set of feature information. Assume that the fluctuation of the corroded area in the initial image is large. After being processed by the moving average method, it can more clearly show the change trend of the corroded area over time. By calculating the slope and curvature of this time series set, it is found that the corroded area shows a gradually increasing trend in some time periods, which is the initially identified trend feature. Verify the rationality of these trend features. For example, after excluding abnormal fluctuations caused by accidental factors such as lighting or shooting angle during image acquisition, refine and classify the trend features that pass the verification. If it is found that the increasing trend of the corroded area is more obvious in summer and relatively slow in winter, this is the refined trend feature information.

[0026] Next, calculate the correlations between different trend features. For example, it is found that there is a significant correlation between the increasing trend of the corrosion area on the surface of the bridge pier and the humidity change trend near the bridge pier, and the humidity change has a positive impact on the increase in the corrosion area. By constructing a correlation relationship network between trend features, use network centrality indicators such as degree centrality to evaluate the importance and influence of each trend feature in this network. If it is found that the humidity change trend has a high degree centrality in this correlation relationship network, it indicates that it has a strong influence on other trend features (such as the increase in the corrosion area). Based on these evaluation results, standardize and normalize each trend feature, and then perform clustering analysis. For example, classify the corrosion trend features related to humidity into one category, and the structural deformation trend features related to temperature into another category. For each category, further analyze the internal feature similarities and differences, as well as the discrimination and correlation between different categories. Finally, through pattern recognition algorithms, identify representative and discriminative significant pattern features, such as the pattern feature that high humidity in summer leads to accelerated corrosion of the bridge pier, which is the structural trend feature information of the first image structured feature information.

[0027] For the structural standardization information, take the beam body in the middle of the bridge span as an example. In terms of the morphological standardization information of the beam body, if the beam body is designed as a straight beam, the central axis of the beam body in the image should be a straight line. Through the precise analysis of the beam body contour in the image, it is found that the central axis of some beam bodies is bent to a certain extent, which belongs to the information of morphological non-standardization. In terms of structural anomaly information, if there are protrusions or depressions that should not exist on the beam body, which are manifested as abnormal changes in the local pixel gray level or color in the image, this is the structural anomaly information.

[0028] Regarding the time span information of the bridge images to be monitored, since the bridge has been regularly imaged since a year ago, and each image acquisition is accompanied by an accurate time tag. Standardize and summarize these time tag data, for example, unify them into a format with years, months, and days as units, and the time span information of the bridge images to be monitored can be obtained as one year. Based on the above structural quantity information, structural trend feature information, structural standardization information, and time span information, the digital analysis data of the bridge images to be monitored is determined.

[0029] Step S130, encode and represent the bridge images to be monitored and the first image structured feature information through the bridge state recognition network to generate an image semantic coding vector of the bridge images to be monitored, and encode and represent the digital analysis data of the bridge images to be monitored through the bridge state recognition network to generate a digital coding vector of the bridge images to be monitored.

[0030] Taking this large bridge as an example, the bridge condition recognition network is a complex neural network structure. For the bridge image to be monitored and the first image structured feature information, first, the bridge image to be monitored is decomposed into regions. For example, the entire bridge image is divided into multiple bridge structure blocks such as pier regions, bridge span regions, and abutment regions according to the structure.

[0031] For these bridge structure blocks, image encoding representation is performed through the image encoder in the bridge condition recognition network. The image encoder may be based on the structure of a convolutional neural network (CNN), and it will perform operations such as multi-layer convolution and pooling on the pixel information in each bridge structure block. Taking the pier region as an example, the image encoder will learn features such as the texture, color, and shape of the pier in the image and convert these features into a specific encoding form. After being processed by multiple convolutional layers, for example, the first convolutional layer may extract the edge features of the pier, and the second convolutional layer further extracts more complex texture features, etc. The pooling layer will downsample the features, reducing the data volume while retaining the key features. After these operations, a bridge feature encoding vector of the bridge image to be monitored is generated.

[0032] At the same time, structured mapping output is performed on the first image structured feature information. Taking the structural features of the pier as an example, it is decomposed into multiple structured feature segments such as shape features, size features, and reinforcement structure features. These multiple structured feature segments are structurally encoded and represented through the digital encoder in the bridge condition recognition network. The digital encoder may convert the shape feature into a specific numerical code. For example, a circular pier is encoded as 1, and a square pier is encoded as 2; the size feature is normalized and encoded according to the actual size value; the reinforcement structure feature is encoded according to its quantity and type. In this way, a structured encoding vector of the first image structured feature information is generated.

[0033] Then, semantic interaction and fusion are performed on the bridge feature encoding vector and the structured encoding vector through the second interaction network layer in the bridge condition recognition network. This interaction and fusion process is based on the prior knowledge of the bridge structure. For example, it is known that the shape of the pier affects its force distribution, and the force distribution is related to the size and reinforcement structure of the pier. During semantic interaction and fusion, according to this internal relationship, the shape feature encoding vector will be fused with the encoding vectors of the size and reinforcement structure. If the force characteristics of a square pier are different from those of a circular pier, then this difference will be reflected in the interaction and fusion of the encoding vectors during the fusion process. Through this semantic interaction and fusion, an image semantic encoding vector of the bridge image to be monitored is generated.

[0034] For the digital analysis data of the bridge images to be monitored, they are also encoded and represented through the bridge state recognition network. For example, for the number of pier reinforcement structures in the structural quantity information, its value is converted into an encoding form suitable for network processing; for the change trend of the corrosion area over time in the structural trend feature information, it may be converted into an encoding vector containing time series features and change rate features; for the degree of non-standardization of the beam shape in the structural standardization information, it is also converted into a corresponding encoding form; the time span information is encoded according to specific time encoding rules. Through such an encoding process, a digital encoding vector of the bridge images to be monitored is generated.

[0035] Step S140, use the bridge state recognition network to perform bridge structure state recognition on the image semantic encoding vector and the digital encoding vector of the bridge images to be monitored, and generate bridge structure state data of the bridge images to be monitored. The bridge structure state data includes a first state estimation result in the target state space dimension, and the target state space dimension includes at least one of a structural stability index dimension, a vibration frequency dimension, a load capacity dimension, a deformation monitoring duration dimension, a continuous stability dimension, a loss degree dimension, an average service life dimension, and a remaining service life dimension.

[0036] Among them, the bridge state recognition network is generated by optimizing the network parameters of the basic bridge state recognition network according to the second state estimation result generated by performing state recognition on the sample bridge feature data sequence by the basic bridge state recognition network and the labeled bridge structure state data of the template bridge image in the target state space dimension. The sample bridge feature data sequence includes the template bridge image, the second image structured feature information of the template bridge image, the digital analysis data of the template bridge image, and the labeled bridge structure state data of the template bridge image in the target state space dimension.

[0037] Specifically, the first interaction network layer in the bridge state recognition network first performs semantic interaction and fusion on the image semantic encoding vector and the digital encoding vector. Taking the structural stability index dimension as an example, the image semantic encoding vector may contain image feature encodings of structures such as piers, bridge spans, and abutments, and these features reflect the appearance state of the structure; while the digital encoding vector contains structural quantity information, structural trend feature information, etc. For example, the change trend of the number of pier reinforcement structures may affect structural stability. During the semantic interaction and fusion process, the first interaction network layer will deeply fuse this information according to the mechanical principles and engineering experience of the bridge structure. For example, if the number of pier reinforcement structures decreases and there are signs of corrosion on the pier surface in the image, then the fused encoding vector will reflect a downward trend in the structural stability index.

[0038] Then, the fully connected mapping layer in the bridge state recognition network predicts the semantic interaction coding vector in the target state space dimension. Assume that the number of mapping spaces in the fully connected mapping layer is the same as the number of dimensions in the target state space. For the structural stability index dimension, the fully connected mapping layer will predict the value of the structural stability index based on the information in the previously fused coding vector and in combination with the patterns learned during training. For example, if there are many abnormalities in the structural features of the bridge piers and bridge spans shown in the coding vector, and the structural trend features indicate that these abnormalities tend to deteriorate, then the predicted structural stability index may be low.

[0039] For the vibration frequency dimension, the image semantic coding vector may contain features related to the vibration of the bridge structure in the image, such as the degree of blurriness of certain parts (possibly caused by vibration), and the structural trend feature information in the digital coding vector may include the trend of changes in the vibration amplitude of the bridge structure at different time periods, etc. The fully connected mapping layer will synthesize this information to predict the vibration frequency of the bridge. If the image shows periodic blurring of the bridge span part and the vibration amplitude has an increasing trend, then the predicted vibration frequency may be higher than the normal level.

[0040] In terms of the load capacity dimension, obtain the overall appearance information of the bridge structure from the image semantic coding vector, such as the deformation of the bridge span, and obtain the state information of the beam load-bearing structure in the structural standardization information from the digital coding vector, etc. The fully connected mapping layer predicts the load capacity of the bridge based on this information. If there are many structural abnormalities in the beam and the bridge span has obvious deformation, then the predicted load capacity may decrease.

[0041] For the deformation monitoring duration dimension, the fully connected mapping layer utilizes the time span information in the digital coding vector and the trend of structural deformation over time in the structural trend feature information, etc. If the structural deformation gradually increases from a certain point in time and lasts for a long time, then there will be corresponding estimation results in the deformation monitoring duration dimension.

[0042] In the continuous stability dimension, synthesize the long-term state information of each structure of the bridge in the image semantic coding vector and the long-term trends in the structural trend feature information in the digital coding vector, etc. If the states of each structure of the bridge are relatively stable over a long period of time and there is no obvious deterioration trend in the structural trends, then the estimation result of the continuous stability dimension will be better.

[0043] For the loss degree dimension, make predictions based on the manifestations of structural damage in the image semantic coding vector and the structural abnormalities in the structural standardization information in the digital coding vector, etc. If the image shows more structural damage and the structural abnormalities are serious, then the estimation result of the loss degree dimension will be higher.

[0044] In terms of the average service life dimension and the remaining service life dimension, the fully connected mapping layer comprehensively considers the information of all the above-mentioned target state space dimensions. For example, if the structural stability index is low, the vibration frequency is abnormal, the load capacity decreases, the deformation monitoring duration is long, the loss degree is high, etc. exist simultaneously, then the average service life will be predicted to be short, and the remaining service life will also be reduced accordingly. Determine the bridge structure state data of the bridge image to be monitored based on the first state estimation results of each candidate dimension included in the target state space dimension.

[0045] Throughout the process, the bridge state recognition network is generated by optimizing the network parameters of the basic bridge state recognition network based on the second state estimation result of generating the template bridge image by recognizing the state of the sample bridge feature data sequence by the basic bridge state recognition network and the labeled bridge structure state data of the template bridge image in the target state space dimension. For example, the initial basic bridge state recognition network may not accurately predict the structural stability index. When training with the sample bridge feature data sequence, the included template bridge image, the second image structured feature information of the template bridge image, the digital analysis data of the template bridge image, and the labeled bridge structure state data of the template bridge image in the target state space dimension will be used to adjust the parameters of the network. If in the template bridge image, the labeled structural stability index is high, but the basic bridge state recognition network predicts it to be low, then the weights and other parameters in the network will be adjusted according to this error. After multiple such adjustments, an optimized bridge state recognition network is generated, which can more accurately identify the bridge structure state of the bridge image to be monitored.

[0046] Based on the above steps, the embodiment of the present application can comprehensively and accurately extract the key information in the bridge image by obtaining the structured feature information of the bridge image and performing digital analysis. The bridge state recognition network encodes and represents the bridge image and its digital analysis data, and fuses the image semantic encoding vector and the digital encoding vector to perform bridge structure state recognition, effectively generating the bridge structure state data covering multiple target state space dimensions. This method not only improves the accuracy and comprehensiveness of bridge state recognition, but also optimizes the parameters of the basic bridge state recognition network through the sample bridge feature data sequence, further enhancing the reliability and practicality of bridge state recognition, helping to timely detect bridge structure anomalies and ensure the safe operation of the bridge.

[0047] In a possible implementation manner, the method further includes:

[0048] Step S101: Obtain a sample bridge feature data sequence, where the sample bridge feature data sequence includes a template bridge image, second image structured feature information of the template bridge image, digital analysis data of the template bridge image, and labeled bridge structure state data of the template bridge image in the target state space dimension.

[0049] In this embodiment, taking the large bridge mentioned before located on a traffic artery as an example, during the long-term monitoring of this bridge, a series of template bridge images have been accumulated. These template bridge images were taken at different times and under different environmental conditions and are representative. For each template bridge image, the method for obtaining its second image structured feature information is similar to the method for obtaining the first image structured feature information of the bridge image to be monitored before. For example, for the pier part in the template bridge image, features such as its shape, size, and surface texture will also be analyzed; for the bridge span part, the performance of the bridge span length in the image, the number of beam bodies, and the connection method, etc. will be analyzed; for the abutment part, its position and connection relationship, etc. will be analyzed. The digital analysis data of the template bridge image is also obtained based on these structured feature information, including structural quantity information, such as the number of pier reinforcement structures, the number of bridge span beam bodies, etc.; structural trend feature information, such as the change trend of pier surface corrosion over time, the time trend of bridge span deformation, etc.; structural standardization information, such as whether the shape of the beam body conforms to the standard and structural abnormal conditions; and the time span information of the template bridge image, that is, the time range from the start of collecting this template bridge image to the last one. And for each template bridge image, there is labeled bridge structure state data in the target state space dimension. For example, in the dimension of the structural stability index, a specific value is labeled, indicating the stability degree of the bridge structure corresponding to this template bridge image at that time; in the dimension of vibration frequency, the actually measured vibration frequency value is labeled; in the dimension of load capacity, the maximum load that the bridge can bear is labeled; in the dimension of deformation monitoring duration, the duration from the start of detectable deformation to the current is labeled; in the dimension of continuous stability, the stable condition of the bridge over a relatively long period of time is labeled; in the dimension of damage degree, the quantitative value corresponding to the damage condition of the bridge structure is labeled; in the dimension of average service life, the average service life estimated according to factors such as the design, construction materials, and use environment of the bridge is labeled; in the dimension of remaining service life, the estimated remaining available duration at the time of collecting the template bridge image is labeled.

[0050] Step S102: Use the encoder in the basic bridge state recognition network to perform encoding representation on the template bridge image, the second image structured feature information of the template bridge image, and the digital analysis data of the template bridge image, and generate a bridge feature encoding vector, a digital encoding vector, and a structured encoding vector of the template bridge image.

[0051] The encoder in the basic bridge state recognition network is also divided into parts such as an image encoder and a digital encoder. For the template bridge image, the image encoder will decompose it into multiple bridge structure blocks. For example, the entire bridge image is divided into parts such as the pier area, the bridge span area, and the abutment area. Then, each structure block is encoded. Taking the pier area as an example, the image encoder will process it through multiple convolutional layers and pooling layers. The convolutional layer will extract various image features of the pier, such as edge features and texture features, and the pooling layer will downsample the features to reduce the data volume. After these operations, a bridge feature encoding vector of the template bridge image is generated. For the second image structured feature information of the template bridge image, it is decomposed into multiple structured feature segments, such as the shape feature segment and the size feature segment of the pier. The digital encoder will encode these structured feature segments. For example, the shape feature of the pier is converted into a numerical value according to a specific encoding rule, such as 1 for circular and 2 for square, and the size feature is encoded by normalizing the actual numerical value, etc., so as to generate a structured encoding vector. For the digital analysis data of the template bridge image, the numerical values in the structure quantity information are encoded according to a certain rule, the trend change data in the structure trend feature information is converted into an encoding form suitable for network processing, and the abnormal conditions and morphological information in the structure normalization information are also encoded accordingly, and finally a digital encoding vector is generated.

[0052] Step S103: Load the bridge feature encoding vector, the digital encoding vector, and the structured encoding vector of the template bridge image into the fully connected mapping layer in the basic bridge state recognition network for bridge state prediction, and generate a second state estimation result of the template bridge image in the target state space dimension.

[0053] The fully connected mapping layer performs comprehensive processing based on the information in the previous encoded vectors. Taking the dimension of structural stability index as an example, the fully connected mapping layer analyzes the feature manifestations of structures such as bridge piers, bridge spans, and abutments in the bridge feature encoded vector in the image, combines the information such as the structural shape and size in the structured encoded vector and the information such as the number and trend of structures in the digital encoded vector. If there are many corrosion signs in the bridge pier shown in the bridge feature encoded vector, the size of the bridge pier in the structured encoded vector does not meet the standard, and the number of bridge pier reinforcement structures decreases and there is a corrosion trend in the digital encoded vector, then the fully connected mapping layer may predict a lower structural stability index. Similarly, for the vibration frequency dimension, the vibration frequency value is predicted by comprehensively considering the state of each part of the bridge in the image and the structure-related information; for the load capacity dimension, the load capacity is predicted by combining the beam structure characteristics and the load-related information; in the dimension of deformation monitoring duration, the prediction is made according to the structural deformation trend and the time-related information; in the dimension of continuous stability, the prediction is made by comprehensively considering the long-term information in all aspects; in the dimension of loss degree, the prediction is made according to the structural damage situation and the abnormal information; in the dimensions of average service life and remaining service life, the prediction is made by comprehensively considering the information in all the above dimensions, so as to generate the second state estimation result of the template bridge image in the target state space dimension.

[0054] Step S104, based on the second state estimation result of the template bridge image in the target state space dimension and the labeled bridge structure state data, train the basic bridge state recognition network to generate the bridge state recognition network.

[0055] In a possible implementation manner, step S104 includes:

[0056] Step S1041, for any one of the state space dimensions included in the target state space dimension, based on the second state estimation result of the template bridge image in the any one state space dimension and the labeled bridge structure state data, determine the first loss information of the any one state space dimension.

[0057] Step S1042, based on the first loss information of each state space dimension included in the target state space dimension, determine the second loss information of the template bridge image.

[0058] Step S1043, train the basic bridge state recognition network according to the second loss information to generate the trained basic bridge state recognition network.

[0059] Step S1044, if the trained basic bridge state recognition network meets the training termination requirement, output the trained basic bridge state recognition network as the bridge state recognition network.

[0060] For example, for any state space dimension included in the target state space dimension, taking the structural stability index dimension as an example, the second state estimation result of the template bridge image in this dimension is compared with the labeled bridge structure state data. If the labeled structural stability index is a higher value while the second state estimation result is a lower value, then the difference between the two is calculated as the first loss information for this dimension. The same operation is also performed for other state space dimensions such as the vibration frequency dimension and the load capacity dimension. The second loss information of the template bridge image is determined based on the first loss information of each state space dimension included in the target state space dimension. For example, a weighted summation method can be adopted. Weights are assigned to the first loss information of each dimension according to the importance of each dimension, and then the weighted first loss information of each dimension is added together to obtain the second loss information. The basic bridge state recognition network is trained based on the second loss information. During the training process, the parameters of the network are adjusted, such as adjusting the weights in the fully connected mapping layer, the convolution kernel parameters in the image encoder and the digital encoder, etc. After multiple such trainings, the trained basic bridge state recognition network is obtained. If the trained basic bridge state recognition network meets the training termination requirements, for example, when the second loss information drops below a preset threshold, or when the change in the second loss information is less than a certain value after consecutive multiple trainings, then the trained basic bridge state recognition network is output as the bridge state recognition network, and this bridge state recognition network can more accurately recognize the bridge structure state.

[0061] In a possible implementation manner, step S140 includes:

[0062] Step S141, performing semantic interaction and fusion on the image semantic encoding vector and the digital encoding vector of the bridge image to be monitored through the first interaction network layer in the bridge state recognition network, and generating a semantic interaction encoding vector of the bridge image to be monitored.

[0063] Step S142, predicting the semantic interaction encoding vector of the bridge image to be monitored in the target state space dimension through the fully connected mapping layer in the bridge state recognition network, and generating bridge structure state data of the bridge image to be monitored.

[0064] In a possible implementation manner, step S142 includes:

[0065] Step S1421, obtaining the number of dimensions included in the target state space dimension.

[0066] For the monitoring of the large bridges mentioned above, the target state space dimensions include the structural stability index dimension, vibration frequency dimension, load capacity dimension, deformation monitoring duration dimension, continuous stability dimension, loss degree dimension, average service life dimension, remaining service life dimension, etc. Suppose there are 8 dimensions in total for these dimensions, and this is the number of dimensions included in the target state space dimension.

[0067] Step S1422: According to the number of dimensions, perform vector connection conversion on the semantic interaction coding vector of the bridge image to be monitored through the fully connected mapping layer, and generate the first state estimation results of each candidate dimension included in the target state space dimension of the bridge image to be monitored. The number of mapping spaces of the fully connected mapping layer is the same as the number of dimensions.

[0068] The number of mapping spaces of the fully connected mapping layer is the same as the number of dimensions, which is 8 mapping spaces here. The fully connected mapping layer processes the semantic interaction coding vector according to the characteristics of each mapping space and the patterns learned during the training process. Taking the structural stability index dimension as an example, the fully connected mapping layer extracts the feature information related to structural stability from the semantic interaction coding vector. These information may include the characteristics related to stability after a series of previous coding and fusion of structures such as bridge piers, bridge spans, and abutments. If features such as a decrease in the number of bridge pier reinforcement structures, an increase in the surface corrosion trend, and an abnormality in the connection of the bridge span girders exist simultaneously in the semantic interaction coding vector, the fully connected mapping layer will predict a lower structural stability index result according to the patterns learned during training. Similarly, for the vibration frequency dimension, the fully connected mapping layer obtains the information related to vibration from the semantic interaction coding vector, such as the dynamic characteristics of structures such as bridge piers and bridge spans (which may be obtained from the dynamic blur part in the image semantic coding vector and the structural vibration trend in the digital coding vector), and predicts the vibration frequency value. For the load capacity dimension, the fully connected mapping layer makes a prediction by integrating relevant features such as the integrity of the beam structure and the load-bearing capacity of the bridge piers; in the deformation monitoring duration dimension, it makes a prediction based on relevant features such as the starting time and deformation speed of the structural deformation; in the continuous stability dimension, it makes a prediction by combining the long-term stability trends of various bridge structures; in the loss degree dimension, it makes a prediction according to features such as the degree and scope of structural damage; in the average service life dimension and the remaining service life dimension, it makes a prediction by comprehensively considering the information of all the above dimensions. Through such operations, the first state estimation results of each candidate dimension included in the target state space dimension of the bridge image to be monitored are generated.

[0069] Step S1423: Determine the bridge structure state data of the bridge image to be monitored according to the first state estimation results of each candidate dimension included in the target state space dimension of the bridge image to be monitored.

[0070] For example, if the prediction result of the structural stability index dimension is low, the prediction result of the vibration frequency dimension is high (indicating abnormal vibration), the prediction result of the load capacity dimension is low, the prediction result of the deformation monitoring duration dimension is long (indicating a long duration of deformation), the prediction result of the continuous stability dimension is poor, the prediction result of the loss degree dimension is high, the prediction result of the average service life dimension is short, and the prediction result of the remaining service life dimension is small, combining these results can determine the bridge structure state data of the bridge image to be monitored, indicating that this bridge is in a poor structural state and further inspection and maintenance measures are required to ensure the safe use of the bridge.

[0071] In a possible implementation manner, step S141 includes:

[0072] Step S1411, analyze the feature dimension structures of the image semantic coding vector and the digital coding vector, and determine the feature categories and corresponding dimension quantities included in the image semantic coding vector and the digital coding vector.

[0073] In this embodiment, for the large bridge mentioned above located on a traffic artery, in this process, it is necessary to analyze in detail the feature dimension structures of the image semantic coding vector and the digital coding vector. The image semantic coding vector is obtained by previously performing regional decomposition on the bridge image to be monitored, performing image coding on each structural block, and performing structured mapping output and coding fusion on the first image structured feature information, and it contains rich feature information. For example, in terms of structural features, the image semantic coding vector may contain feature encodings of the shape, size, surface texture, etc. of the pier part as shown in the image; for the bridge span part, it contains feature encodings of the bridge span length, number of beam bodies, beam body connection methods, etc. in the image; for the abutment part, it contains feature encodings of its position, connection relationship with other structures, etc. In terms of the number of dimensions, assuming that the pier shape feature encoding is within a specific dimension range, the size feature encoding is within another dimension range, the encodings of different structural parts and different features of the same structural part are distributed in different dimension intervals, and these dimensions together constitute the feature dimension structure of the image semantic coding vector.

[0074] Step S1412, based on the feature categories and corresponding dimension quantities included in the image semantic coding vector and the digital coding vector, according to the predefined feature dimension alignment rules, perform feature alignment on the image semantic coding vector and the digital coding vector to generate the image semantic coding vector and digital coding vector after feature dimension alignment.

[0075] The digital coding vector is obtained based on the digital analysis data of the bridge image to be monitored, which includes the quantity information of the structures containing the first image structure feature information, the structure trend feature information, the structure standardization information, and the time span information of the bridge image to be monitored, etc. For example, the numerical values such as the number of pier reinforcement structures and the number of bridge span girders in the quantity information of the structures have corresponding dimension representations in the digital coding vector after coding; the structure trend feature information such as the change trend of pier surface corrosion over time and the time trend of bridge span deformation also has its corresponding dimension representations; the beam shape in the structure standardization information whether it conforms to the standard and the structural abnormal conditions also have corresponding coding dimensions; the time span information also occupies a specific dimension. After determining the feature categories and the corresponding number of dimensions included in the image semantic coding vector and the digital coding vector, feature alignment is performed according to the pre-defined feature dimension alignment rules. This feature dimension alignment rule is determined based on the internal relationship between the features of the bridge structure and the logical relationship in bridge monitoring. For example, there is a certain correlation between the shape feature of the pier and the shape standardization in the structure standardization information, so when performing feature alignment, the coding dimensions corresponding to these two features are aligned, so that they can interact more reasonably in the subsequent fusion operation. After such an operation, the image semantic coding vector and the digital coding vector with feature dimension alignment are generated.

[0076] Step S1413, based on the prior knowledge and semantic information of bridge monitoring, construct the semantic interaction mapping relationship between the image semantic coding vector and the digital coding vector, and perform the fusion initialization operation on the features in the image semantic coding vector and the digital coding vector with feature dimension alignment according to the constructed semantic interaction mapping relationship. Specifically, for each pair of features that have corresponding associations in the semantic interaction mapping relationship, perform a preliminary fusion operation according to the set fusion strategy to obtain the first fusion feature data containing the preliminary fusion features.

[0077] In this embodiment, these prior knowledge includes the principles of bridge structural mechanics, the functional relationships of different structural parts, and the experience accumulated in long-term monitoring. For example, according to the principles of bridge structural mechanics, the structural stability of a pier is closely related to the number of reinforcement structures of the pier, the surface corrosion trend, etc. Then, in the semantic interaction mapping relationship, a mapping relationship is established between the features in the image semantic coding vectors such as the shape and size of the pier and the features in the digital coding vectors such as the number of reinforcement structures of the pier and the corrosion trend. Based on the constructed semantic interaction mapping relationship, a fusion initialization operation is performed on the features in the image semantic coding vector and the digital coding vector after the feature dimensions are aligned. For each pair of features that have corresponding associations in the semantic interaction mapping relationship, a preliminary fusion operation is performed according to the set fusion strategy. For example, for the pair of features of the pier shape and the number of reinforcement structures, if the set fusion strategy is weighted summation, different weights are assigned according to their importance in structural stability, and then the weighted feature values are added to obtain the preliminary fused feature value. After all such pairs of features are subjected to the preliminary fusion operation, the first fusion feature data containing the preliminary fused features is obtained.

[0078] Step S1414, according to the semantic rules and semantic association knowledge predefined in the field of bridge monitoring, perform semantic enhancement adjustment on the first fusion feature data to generate the second fusion feature data after semantic enhancement adjustment.

[0079] In the field of bridge monitoring, some semantic rules and association knowledge are determined based on engineering standards and long-term practical experience. For example, if in the first fusion feature data, the fusion feature of the number of reinforcement structures of the pier and the surface corrosion trend of the pier does not conform to the normal structural stability semantic rules (that is, when the number of reinforcement structures is sufficient, there should be no rapid corrosion trend), then this fusion feature is adjusted according to the semantic association knowledge. It may be corrected according to other relevant features (such as the coding features corresponding to the ambient humidity data, etc.) to generate the second fusion feature data after semantic enhancement adjustment.

[0080] Step S1415, by analyzing the correlation and redundancy between the features in the second fusion feature data, perform optimization selection or feature combination optimization on the second fusion feature data to generate the optimized third fusion feature data.

[0081] Step S1416, organize and encode the third fusion feature data to generate the semantic interaction coding vector of the bridge image to be monitored.

[0082] In the second fused feature data, there may be strong correlations between some features. For example, the surface corrosion trend of the bridge pier has a strong correlation with the encoded features corresponding to the humidity data near the bridge pier. Some features may be redundant. For example, there may be some redundancy between two features representing the local structural deformation of the bridge pier in different ways. Optimization selection is performed based on these correlations and redundancies. If the features with strong correlations are selected, the more representative feature for judging the structural state can be retained; for redundant features, they are removed or recombined. After such operations, the optimized third fused feature data is generated. Finally, the third fused feature data is sorted and encoded to generate the semantic interaction encoding vector of the bridge image to be monitored.

[0083] In a possible implementation manner, the encoder in the bridge state recognition network includes an image encoder and a digital encoder. Step S130 includes:

[0084] Step S131, perform regional decomposition on the bridge image to be monitored to generate multiple bridge structure blocks of the bridge image to be monitored, and perform image encoding representation on the multiple bridge structure blocks through the image encoder to generate the bridge feature encoding vector of the bridge image to be monitored.

[0085] In this embodiment, taking the large bridge mentioned above located on the traffic artery as an example, for this large bridge, its structure is complex and diverse, and the regional decomposition process has clear pertinence. For example, according to the main components of the bridge, it can be decomposed into multiple different bridge structure blocks such as the pier area, the bridge span area, and the abutment area. The pier area has unique visual features in the image, and its shape, texture, and relationship with the surrounding environment are different from other areas; the length of the bridge span area, the arrangement of the beam bodies, and the connection structure between the beam bodies also have specific forms of expression in the image; the position of the abutment area and the connection method with the piers and the bridge span also have recognizable image features. Through such regional decomposition, subsequent image encoding operations can be better performed for each structure block.

[0086] Next, the image encoder is a module specifically designed to process image data. It encodes each block of the bridge structure based on a specific encoding algorithm. Taking the pier area as an example, the image encoder first encodes the shape of the pier. If the pier is square, the image encoder will convert the features of the square into corresponding numerical codes according to the preset shape encoding rules. During the encoding process, detailed information such as the side length ratio of the square and the shape features of the corners will be considered. These information are extracted by the convolutional kernels in the convolutional layer. For example, when the convolutional kernel slides on the image, it will perform weighted calculations on the edge pixels of the pier shape, thereby extracting key features of the shape, such as the straightness and angle of the edge. At the same time, the image encoder will also encode the texture features of the pier. There may be different textures on the surface of the pier, such as the granular texture of concrete or the wear marks caused by long-term use. The image encoder analyzes features such as the gray-scale changes and directions of the textures, and encodes them by combining multiple convolutional layers and pooling layers. The convolutional layer gradually extracts higher-level texture features, and the pooling layer downsamples the features to reduce the data volume while retaining key texture feature information. For the color features of the pier area, although relatively less important in bridge structure analysis, they will also be encoded. For example, encoding is performed according to the RGB value distribution range of the color. Different color distributions may imply different states of the pier surface, such as whether there are color changes caused by corrosion products. Through the encoding of multiple features such as the shape, texture, and color of the pier area, and similar encoding operations on other bridge structure blocks such as the bridge span area and the abutment area, a bridge feature encoding vector of the bridge image to be monitored is finally generated.

[0087] Step S132: Structurally map and output the first image structured feature information of the bridge image to be monitored, generate multiple structured feature segments of the first image structured feature information, and perform structure encoding representation on the multiple structured feature segments through the digital encoder to generate the structured encoding vector of the bridge image to be monitored.

[0088] The first image structured feature information covers various feature information of each structural part of the bridge, and through the structured mapping output, it is transformed into multiple structured feature segments. Taking the bridge pier as an example, the pier-related features in the first image structured feature information can be decomposed into shape feature segments, dimension feature segments, reinforcement structure feature segments, etc. The shape feature segment further refines various features of the pier shape, such as the shape details of the four corners of a square pier, the exact numerical value of the side length, etc.; the dimension feature segment clarifies the actual dimensions of the pier in various directions, including height, bottom side length, etc.; the reinforcement structure feature segment details the reinforcement structure of the pier, such as the number of reinforcement bars, the distribution method, the connection method with the main structure of the pier, etc. For the bridge span part, the structured mapping output of the first image structured feature information may generate bridge span length feature segments, number of beam body feature segments, beam body connection method feature segments, etc. The bridge span length feature segment is accurate to the specific length value and the proportional relationship of the length in the image; the number of beam body feature segment clarifies the specific number of beam bodies in the bridge span; the beam body connection method feature segment details whether the beam bodies are connected by welding, bolt connection or other connection methods, and the specific structural features of the connection part, etc. After the structured mapping output of the first image structured feature information of the abutment part, abutment position feature segments, connection relationship feature segments with other structures, etc. will be generated. The abutment position feature segment accurately describes the geographical location of the abutment in the entire bridge structure, and the connection relationship feature segment with other structures details the connection structure and connection firmness degree between the abutment and the pier and the bridge span and other relevant information.

[0089] In this embodiment, the digital encoder adopts different coding strategies for different types of structured feature segments. For the pier shape in the shape feature segment, if it is square, the digital encoder may encode it as a specific value, such as 2, while a circle may be encoded as 1. This coding method is based on a pre-set shape coding dictionary. For the dimension feature segment, the digital encoder encodes the actual dimension values after normalization. For example, if the height of the pier is 10 meters, after normalization, according to the set dimension coding range, it may be encoded as a value between 0 and 1, and the specific value depends on the normalization algorithm and the value range of the overall dimension. For the reinforcement structure feature segment, such as the number of reinforcement bars in the pier being 20, the digital encoder assigns a corresponding coding value according to the range and importance of the number of reinforcement bars. For the bridge span length feature segment of the bridge span part, it is also encoded after normalization; the number of beam body feature segment is directly encoded according to the actual number; the beam body connection method feature segment is encoded according to different connection method types, for example, welded connection is encoded as 1, bolt connection is encoded as 2, etc. The feature segments of the abutment part are also encoded in a similar way. Through the encoding operation of each structured feature segment, a structured coding vector of the bridge image to be monitored is finally generated.

[0090] Step S133: Through the second interaction network layer in the bridge state recognition network, perform semantic interaction and fusion on the bridge feature encoding vector and the structured encoding vector of the bridge image to be monitored, and generate the image semantic encoding vector of the bridge image to be monitored.

[0091] In this embodiment, the role of the second interaction network layer is to fuse the information in the bridge feature encoding vector and the structured encoding vector based on the inherent semantic relationship of the bridge structure. Taking the bridge pier as an example, there are semantic associations between the image features such as the shape, texture, and color of the bridge pier in the bridge feature encoding vector and the shape feature segments, size feature segments, and reinforcement structure feature segments of the bridge pier in the structured encoding vector. From the perspective of structural mechanics, the shape and size of the bridge pier will affect its force distribution, and the force distribution is closely related to the reinforcement structure. These relationships are manifested as different features in the image. The second interaction network layer will perform semantic interaction and fusion according to these relationships. For example, if the shape of the bridge pier in the bridge feature encoding vector shows a square shape with a certain degree of wear at the corners, and at the same time, the size of the bridge pier in the structured encoding vector indicates that the side length ratio may cause uneven force distribution, and the reinforcement structure feature shows that the number of reinforcement bars is small, then in the process of semantic interaction and fusion, the second interaction network layer will combine these information and consider their mutual influence relationships. Similar operations are also performed on the bridge span and abutment parts. The image features of the beam body in the bridge span are semantically related to the structured features such as the number of beam bodies and the connection method, and there are also semantic interaction relationships between the image features of the abutment and the structured features such as the position of the abutment and the connection relationship. Through this comprehensive semantic interaction and fusion operation, the information in the bridge feature encoding vector and the structured encoding vector is deeply integrated, and finally the image semantic encoding vector of the bridge image to be monitored is generated. This encoding vector completely contains the information after the fusion of the bridge image to be monitored from both aspects of image features and structured features, providing a comprehensive data basis for the subsequent recognition of the bridge structure state.

[0092] In a possible implementation manner, step S120 includes:

[0093] Step S121: Perform structural feature statistics on the first image structured feature information to generate the structural quantity information of the first image structured feature information, where the structural quantity information includes at least one of the number of structural elements and the number of structural connections in the first image structured feature information.

[0094] In this embodiment, taking the large bridge located on the transportation artery mentioned above as an example, for the bridge images to be monitored of this large bridge, the first image structured feature information covers various features of each structural part of the bridge. Taking the bridge pier as an example, in terms of the number of structural elements, this embodiment considers the reinforcement structural elements of the bridge pier, such as reinforcement bars, support rib plates, etc. The number of these reinforcement structural elements is accurately identified and counted through image analysis technology. Suppose in the bridge pier image, after careful processing of the image recognition algorithm, the number of reinforcement bars can be accurately identified as 30, and the number of support rib plates is 4. These numbers constitute part of the number of structural elements. In terms of the number of structural connections, for the connection points between the bridge pier and the bridge foundation, the number of connection points is determined by analyzing the features of the connection parts in the image. If the bridge pier and the bridge foundation are connected by 8 bolts and these bolt connection points can be clearly identified in the image, then the number 8 of these connection points becomes part of the number of structural connections. For the bridge span part, the number of structural elements may include the number of beam bodies. For example, after image analysis, it is determined that the number of beam bodies in the bridge span is 5; the number of structural connections can be the number of connection points between the beam bodies. Suppose the beam bodies are connected by welding, and after image analysis, it is determined that there are a total of 20 welding connection points. Similar structural quantity statistics are also carried out for the abutment part, such as the number of connection structures between the abutment and the bridge pier and the bridge span. Through such comprehensive structural feature statistics, the structural quantity information of the first image structured feature information is generated, and these structural quantity information reflect the characteristics related to the complexity and stability of the bridge structure from one aspect.

[0095] Step S122: Perform a structural trend analysis on the first image structured feature information to generate the structural trend feature information and the structural standardization information of the first image structured feature information, where the structural standardization information includes at least one of the morphological standardization information and the structural anomaly information of the first image structured feature information.

[0096] In terms of the structural standardization information, taking the beam body in the bridge span as an example. For the morphological standardization information of the beam body, according to the design standard, the beam body should be straight and have a regular shape. The shape parameters of the beam body in the image, such as the straightness deviation and the degree of deformation of the cross-sectional shape, are accurately measured through image analysis technology. If the straightness deviation of the beam body in the image exceeds the range allowed by the design standard, this belongs to the morphological non-standardization information. For the structural anomaly information, such as cracks, depressions, or protrusions on the surface of the beam body, the manifestations of these abnormal structures in the image are identified through image analysis, including information such as their positions, sizes, and shapes.

[0097] Step S123: Obtain the time tag data of the bridge image to be monitored, and perform a standardized summary on the time tag data of the bridge image to be monitored to generate the time span information of the bridge image to be monitored.

[0098] For example, during the process of image acquisition of this large bridge, each bridge image to be monitored has an accurate time tag, such as "10:00:00, March 15, 2020". The time tag data is standardized and summarized in a unified format (such as year - month - day - hour - minute - second). Assuming that in this embodiment, the regular image acquisition of the bridge starts from January 1, 2019 and ends on December 31, 2021, then through the standardized summary of all image time tag data, the time span information of the bridge images to be monitored can be determined as from January 1, 2019 to December 31, 2021.

[0099] Step S124, based on the structure quantity information of the first image structured feature information, the structure trend feature information, the structure standardization information, and the time span information of the bridge image to be monitored, determine the digital analysis data of the bridge image to be monitored.

[0100] For example, the number of pier reinforcement structure elements, the number of structure connections, etc. in the structure quantity information, the corrosion trend of the pier, the telescopic trend of the beam body, etc. in the structure trend feature information, the morphological deviation of the beam body, the structural abnormal conditions, and the time span information, etc. in the structure standardization information. Combining these information forms comprehensive digital analysis data of the bridge image to be monitored, and these digital analysis data can provide an important data basis for the subsequent bridge state evaluation and analysis.

[0101] In a possible implementation manner, step S122 includes:

[0102] Step S1221, based on the first image structured feature information and the associated time interval data, generate a time series set of feature information arranged in chronological order. The specific operation is as follows: According to the time tag of each feature information in the first image structured feature information, assign this feature information to the corresponding time point. For the feature information without a clear time point, speculate and assign it according to the relative relationship or acquisition order with other information with time tags. Among them, for the periodic or continuously changing feature information, the sliding window method is used to convert this feature information into time series data changing with time. For the non - periodic or occasional feature information, record the time node when this feature information occurs and represent it in the form of an event marker in the time series.

[0103] For example, for the characteristic information of the corrosion condition of a bridge pier, assume that in this embodiment, there is a series of bridge images to be monitored taken at different times. According to the time tags of the corrosion characteristic information of the bridge pier in each image (this time tag is the exact time recorded during image acquisition), the corrosion characteristic information is assigned to the corresponding time points. If the corrosion characteristic information in some images has no clear time tag, but the time points can be speculated and assigned based on its relative relationship with the corrosion characteristics in other images with clear time tags (such as the order of corrosion degree) or the acquisition order (such as inferring its time order according to the order in which the acquisition device stores the images). For the characteristic information of the corrosion on the surface of the bridge pier, which may be a continuously changing characteristic, the sliding window method is used to convert it into time series data that changes over time. For example, with a fixed time window (such as every 3 months as a window), the change in the corrosion area on the surface of the bridge pier within each window is statistically calculated to form time series data. For non-periodic or sporadic characteristic information such as a sudden crack in the bridge pier, the time node when the crack appears is recorded and represented in the form of an event marker in the time series.

[0104] Step S1222, using the moving average method or the exponential smoothing method, smooth the time series set of the characteristic information, and identify the trend characteristics of each characteristic information changing over time by calculating the slope and curvature of the time series set of the characteristic information, and output the trend characteristics in the form of a candidate set. Among them, for the characteristic information with periodic or seasonal patterns, the seasonal decomposition algorithm is used to decompose it into a trend component, a seasonal component, and a random component.

[0105] Taking the time series data of the corrosion area of the bridge pier as an example, due to some measurement errors or short-term fluctuations that may exist in the actual acquisition process, these data can be effectively smoothed by the moving average method. Suppose the 3-period moving average method is adopted in this embodiment, that is, the average value of the corrosion areas at adjacent 3 time points is calculated each time as the new smoothed data point. After such smoothing processing, the fluctuations of the data are reduced, and the long-term trend can be better reflected. By calculating the slope and curvature of the time series set of feature information, the trend characteristics of each feature information changing with time are identified. For the time series data of the corrosion area of the bridge pier, the slope represents the change rate of the corrosion area with time. If the slope is positive and the value is large, it indicates that the corrosion area is increasing rapidly; the curvature can reflect the acceleration of this change. These trend characteristics are output in the form of a candidate set. For feature information with periodic or seasonal patterns, such as the expansion and contraction amount of some parts of the bridge may be related to seasonal temperature changes, the seasonal decomposition algorithm is used to decompose it into a trend component, a seasonal component, and a random component. For example, for the expansion and contraction amount data of the bridge expansion joint, through the seasonal decomposition algorithm, the total expansion and contraction amount can be decomposed into the regular expansion and contraction (seasonal component) caused by seasonal temperature changes, the long-term structural change trend (trend component), and the random fluctuation (random component) caused by accidental factors (such as short-term traffic load impacts, etc.).

[0106] Step S1223, verify the rationality of the candidate set, refine and classify the trend characteristics passed in the candidate set, and generate refined trend characteristic information.

[0107] Step S1224, adopt the correlation analysis algorithm, calculate the correlation between different trend characteristics in the refined trend characteristic information, and after identifying the target feature pairs with significant associations based on the calculated correlation, analyze the nature and strength of the associations of the target feature pairs to obtain the association analysis result. According to the association analysis result, construct the association relationship network between trend characteristics.

[0108] For example, for the trend feature of the increasing corrosion area of the pier, it is necessary to verify whether it is due to the actual corrosion situation rather than the error in the image acquisition process (such as the image color change caused by the illumination change being misjudged as corrosion). Refine and classify the trend features that pass the verification. If it is found that the trend of the increasing corrosion area of the pier has different manifestations in different regions, such as the corrosion area at the bottom of the pier increasing faster than that at the top, then refine and classify this difference. After generating the refined trend feature information, use the correlation analysis algorithm to calculate the correlation between different trend features. Assume that there is an association between the trend of the increasing corrosion area of the pier and the trend of the humidity change near the pier, and calculate the correlation coefficient between them through the correlation analysis algorithm. After identifying the target feature pairs with significant associations based on the calculated correlations, analyze the nature and strength of the associations of the target feature pairs. If the correlation coefficient is high and positive, it indicates that there is a positive correlation between the two, that is, the increase in humidity will lead to an increase in the corrosion area, and the association strength is high. Construct an association relationship network between the trend features according to the association analysis results. In this network, features such as the trend of the increasing corrosion area of the pier and the trend of the humidity change are used as nodes, and the association relationships between them are used as edges.

[0109] Step S1225, based on the association relationship network, use network centrality metrics to evaluate the importance and influence of each trend feature in the association relationship network, and according to the evaluation results, sort each of the trend features or assign weights to generate an importance ranking sequence or a weight sequence, where the network centrality metrics include at least one of degree centrality, closeness centrality, and betweenness centrality.

[0110] Step S1226, based on the importance ranking sequence or the weight sequence, perform standardization and normalization processing on each trend feature. After generating the target trend feature information set, use a clustering algorithm to perform clustering analysis on the target trend feature information set to generate a clustering result of feature groups or categories with similar trend features, and for each category in the clustering result, further analyze the feature similarities and differences within the category, as well as the discrimination and association between different categories, and use a pattern recognition algorithm to identify and classify the feature patterns in the clustering result, extract representative and discriminative significant pattern features, and output the clustering result and the significant pattern features in a structured form to generate the structural trend feature information of the first image structured feature information.

[0111] Taking degree centrality as an example, if a trend feature (such as the humidity change trend) is directly associated with many other trend features, then its degree centrality is relatively high, indicating that its influence on other trend features in this network is relatively extensive. Closeness centrality can measure the average distance from a trend feature to other trend features, while betweenness centrality reflects the importance of a trend feature when connecting other trend features. Sort or assign weights to each trend feature according to the evaluation results to generate an importance ranking sequence or a weight sequence. For example, due to its importance in the association relationship network, the humidity change trend is assigned a relatively high weight. Standardize and normalize each trend feature based on the importance ranking sequence or weight sequence to generate a target trend feature information set. Then, use a clustering algorithm to perform clustering analysis on the target trend feature information set. For example, cluster the pier corrosion trend features related to humidity, the bridge span expansion trend features related to temperature, etc. separately. For each category in the clustering result, further analyze the feature similarities and differences within the category. In the category of pier corrosion trend features related to humidity, analyze the similarities and differences in the influence of humidity on pier corrosion in different regions; analyze the discrimination and correlation between different categories. For example, the correlation between the category of pier corrosion trend features related to humidity and the category of bridge span expansion trend features related to temperature is generated through the influence of environmental factors on the overall structure of the bridge. Use a pattern recognition algorithm to identify and classify the feature patterns in the clustering result, and extract representative and discriminative significant pattern features. For example, identify the pattern feature of accelerated corrosion at the bottom of piers in high-humidity seasons. Output the clustering result and the significant pattern features in a structured form to generate the structural trend feature information of the first image structured feature information.

[0112] Figure 2 FIG. shows the hardware structure diagram of an AI-based bridge digital monitoring system 100 for implementing the above-described AI-based bridge digital monitoring method, as Figure 2 shown, the AI-based bridge digital monitoring system 100 may include a processor 110, a machine-readable storage medium 120, a bus 130, and a communication unit 140.

[0113] The machine-readable storage medium 120 can store data and / or instructions. In some embodiments, the machine-readable storage medium 120 can store data obtained from an external terminal. In some embodiments, the machine-readable storage medium 120 can store the data and / or instructions used by the AI-based bridge digital monitoring system 100 to execute or use to complete the exemplary methods described in the present invention.

[0114] In the specific implementation process, one or more processors 110 execute computer-executable instructions stored in the machine-readable storage medium 120, enabling the processors 110 to execute the AI-based bridge digital monitoring method of the above method embodiments. The processors 110, the machine-readable storage medium 120, and the communication unit 140 are connected through the bus 130, and the processors 110 can be used to control the transceiver actions of the communication unit 140.

[0115] For the specific implementation process of the processors 110, reference can be made to the respective method embodiments executed by the above AI-based bridge digital monitoring system 100. Their implementation principles and technical effects are similar, and will not be elaborated here in this embodiment.

[0116] In addition, an embodiment of the present invention further provides a readable storage medium, in which computer-executable instructions are preset. When the processor executes the computer-executable instructions, the above AI-based bridge digital monitoring method is implemented.

[0117] It should be noted that, in order to simplify the description of the present invention disclosure and thus help the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of the present invention, sometimes multiple features are merged into one embodiment, drawing, or description thereof.

Claims

1. A bridge digital monitoring method based on AI, characterized in that: The method comprises: Acquire first image structured feature information of the bridge image to be monitored; Performing digital analysis on the bridge image to be monitored and the first image structured feature information to determine digital analysis data of the bridge image to be monitored, wherein the digital analysis data of the bridge image to be monitored includes at least one of structural quantity information, structural trend feature information, structural standardization information of the first image structured feature information and time span information of the bridge image to be monitored; encoding and representing the image of the bridge to be monitored and the structural feature information of the first image through a bridge state recognition network to generate an image semantic encoding vector of the image of the bridge to be monitored, and encoding and representing the digital analysis data of the image of the bridge to be monitored through the bridge state recognition network to generate a digital encoding vector of the image of the bridge to be monitored; The image semantic coding vector and the digitized coding vector of the bridge image to be monitored are used to identify the bridge structure state through the bridge state recognition network, and the bridge structure state data of the bridge image to be monitored are generated, wherein the bridge structure state data includes a first state estimation result in a target state space dimension, and the target state space dimension includes at least one of a structural stability index dimension, a vibration frequency dimension, a load capacity dimension, a deformation monitoring time dimension, a continuous stability dimension, a loss degree dimension, an average service life dimension, and a remaining service life dimension; Wherein, the bridge state recognition network is generated by optimizing the network parameters of the basic bridge state recognition network according to the second state estimation result and the annotated bridge structure state data of the template bridge image in the target state space dimension after the state recognition of the sample bridge feature data sequence is performed according to the basic bridge state recognition network to generate the second state estimation result of the template bridge image, and the sample bridge feature data sequence includes the template bridge image, the second image structural feature information of the template bridge image, the digital analysis data of the template bridge image and the annotated bridge structure state data of the template bridge image in the target state space dimension; The step of digitally analyzing the image of the bridge to be monitored and the structural feature information of the first image to determine digital analysis data of the image of the bridge to be monitored includes: Performing structural feature statistics on the first image structural feature information to generate structural quantity information of the first image structural feature information, wherein the structural quantity information includes at least one of the number of structural elements and the number of structural connections in the first image structural feature information; Performing a structural trend analysis on the first image structural feature information to generate structural trend feature information and structural standardization information of the first image structural feature information, wherein the structural standardization information includes at least one of morphological standardization information and structural abnormality information of the first image structural feature information; Acquire time tag data of the image of the bridge to be monitored, and standardize and summarize the time tag data of the image of the bridge to be monitored to generate time span information of the image of the bridge to be monitored; Determine digital analysis data of the image of the bridge to be monitored based on the structural quantity information of the first image structured feature information, the structural trend feature information, the structural standardization information and the time span information of the image of the bridge to be monitored; The step of performing structural trend analysis on the first image structural feature information to generate structural trend feature information of the first image structural feature information includes: Based on the first image structured feature information and the time interval data associated therewith, a feature information time series set arranged in time order is generated; the specific operation is: according to the time tag of each feature information in the first image structured feature information, the feature information is assigned to the corresponding time point, and for feature information with unclear time points, it is inferred and assigned according to the relative relationship between the feature information and other information with time tags or the acquisition sequence; wherein, for periodic or continuously changing feature information, a sliding window method is used to convert the feature information into time series data that changes with time; for non-periodic or sporadic feature information, the time node when the feature information occurs is recorded and represented in the time series in the form of event markers; The feature information time series set is smoothed by using a moving average method or an exponential smoothing method, and the trend characteristics of each feature information changing over time are identified by calculating the slope and curvature of the feature information time series set, and the trend characteristics are output in the form of a candidate set; wherein, for feature information with periodic or seasonal patterns, a seasonal decomposition algorithm is used to decompose it into trend components, seasonal components and random components; Performing rationality verification on the candidate set, and refining and classifying the trend features that have passed the verification in the candidate set to generate refined trend feature information; A correlation analysis algorithm is used to calculate the correlation between different trend features in the refined trend feature information, and after identifying target feature pairs with significant correlation based on the calculated correlation, the nature and strength of the correlation between the target feature pairs are analyzed to obtain a correlation analysis result, and a correlation relationship network between the trend features is constructed according to the correlation analysis result; Based on the association relationship network, a network centrality index is used to evaluate the importance and influence of each trend feature in the association relationship network, and each trend feature is sorted or weighted according to the evaluation result to generate an importance sorting sequence or a weight sequence, wherein the network centrality index includes at least one of degree centrality, closeness centrality, and betweenness centrality; Based on the importance ranking sequence or weight sequence, each trend feature is standardized and normalized. After generating a target trend feature information set, a clustering algorithm is used to perform cluster analysis on the target trend feature information set to generate a clustering result of feature groups or categories with similar trend features, and, for each category in the clustering result, further analyze the feature similarity and difference within the category, as well as the distinction and correlation between different categories, and, using a pattern recognition algorithm, identify and classify the feature patterns in the clustering result, extract representative and discriminative significant pattern features, output the clustering results and significant pattern features in a structured form, and generate structural trend feature information of the first image structured feature information.

2. The AI-based bridge digital monitoring method according to claim 1 is characterized in that: The method further comprises: Acquire a sample bridge feature data sequence, wherein the sample bridge feature data sequence includes a template bridge image, second image structured feature information of the template bridge image, digital analysis data of the template bridge image, and annotated bridge structure state data of the template bridge image in the target state space dimension; The template bridge image, the second image structural feature information of the template bridge image and the digitized analysis data of the template bridge image are encoded and represented by an encoder in the basic bridge state recognition network to generate a bridge feature encoding vector, a digitized encoding vector and a structural encoding vector of the template bridge image; Loading the bridge feature coding vector, the digitized coding vector and the structured coding vector of the template bridge image into the fully connected mapping layer in the basic bridge state recognition network to perform bridge state prediction, and generating a second state estimation result of the template bridge image in the target state space dimension; The basic bridge state recognition network is trained based on the second state estimation result of the template bridge image in the target state space dimension and the annotated bridge structure state data to generate the bridge state recognition network.

3. The AI-based bridge digital monitoring method according to claim 2 is characterized in that: The training of the basic bridge state recognition network based on the second state estimation result of the template bridge image in the target state space dimension and the annotated bridge structure state data to generate the bridge state recognition network includes: For any one of the state space dimensions included in the target state space dimensions, based on the second state estimation result of the template bridge image in the any one of the state space dimensions and the annotated bridge structure state data, determining the first loss information of the any one of the state space dimensions; Determining second loss information of the template bridge image based on first loss information of each state space dimension included in the target state space dimension; Training the basic bridge state recognition network according to the second loss information to generate a trained basic bridge state recognition network; If the trained basic bridge state recognition network meets the training termination requirement, the trained basic bridge state recognition network is output as the bridge state recognition network.

4. The AI-based bridge digital monitoring method according to claim 1 is characterized in that: The bridge structure state recognition is performed on the image semantic coding vector and the digitized coding vector of the bridge image to be monitored by the bridge state recognition network to generate bridge structure state data of the bridge image to be monitored, including: Performing semantic interactive fusion on the image semantic coding vector and the digitized coding vector of the bridge image to be monitored through the first interactive network layer in the bridge state recognition network to generate the semantic interactive coding vector of the bridge image to be monitored; The semantic interaction encoding vector of the bridge image to be monitored is predicted on the target state space dimension through the fully connected mapping layer in the bridge state recognition network to generate bridge structure state data of the bridge image to be monitored.

5. The AI-based bridge digital monitoring method according to claim 4 is characterized in that: The method of predicting the semantic interaction encoding vector of the bridge image to be monitored on the target state space dimension through the fully connected mapping layer in the bridge state recognition network to generate bridge structure state data of the bridge image to be monitored includes: Obtaining the number of dimensions included in the target state space dimension; According to the number of dimensions, the semantic interaction coding vector of the bridge image to be monitored is subjected to vector connection conversion through the fully connected mapping layer to generate a first state estimation result of each candidate dimension of the bridge image to be monitored included in the target state space dimension, and the number of mapping spaces of the fully connected mapping layer is the same as the number of dimensions; The bridge structure state data of the bridge image to be monitored is determined according to the first state estimation results of each candidate dimension included in the target state space dimension of the bridge image to be monitored.

6. The AI-based bridge digital monitoring method according to claim 4 is characterized in that: The step of performing semantic interactive fusion on the image semantic coding vector and the digitized coding vector of the bridge image to be monitored through the first interactive network layer in the bridge state recognition network to generate the semantic interactive coding vector of the bridge image to be monitored includes: Analyze the feature dimension structure of the image semantic coding vector and the digital coding vector to determine the feature categories and corresponding dimension numbers contained in the image semantic coding vector and the digital coding vector; Based on the feature categories and corresponding dimension numbers contained in the image semantic coding vector and the digitized coding vector, according to a predefined feature dimension alignment rule, feature alignment is performed on the image semantic coding vector and the digitized coding vector to generate an image semantic coding vector and a digitized coding vector after feature dimension alignment; Based on the prior knowledge and semantic information of bridge monitoring, a semantic interaction mapping relationship between the image semantic coding vector and the digital coding vector is constructed, and according to the constructed semantic interaction mapping relationship, a fusion initialization operation is performed on the features in the image semantic coding vector and the digital coding vector after the feature dimension is aligned; specifically, for each feature pair that has a corresponding association in the semantic interaction mapping relationship, a preliminary fusion operation is performed according to the set fusion strategy to obtain the first fusion feature data containing the preliminary fusion features; According to predefined semantic rules and semantic association knowledge in the field of bridge monitoring, semantically enhancing and adjusting the first fused feature data to generate second fused feature data after semantically enhancing and adjusting; By analyzing the correlation and redundancy between the features in the second fused feature data, optimizing the selection or feature combination of the second fused feature data to generate optimized third fused feature data; The third fused feature data is sorted and encoded to generate a semantic interaction coding vector of the bridge image to be monitored.

7. The AI-based bridge digital monitoring method according to claim 1 is characterized in that: The encoder in the bridge state recognition network includes an image encoder and a digital encoder. The encoding and representing of the image of the bridge to be monitored and the structural feature information of the first image by the bridge state recognition network to generate an image semantic encoding vector of the image of the bridge to be monitored includes: Performing regional decomposition on the bridge image to be monitored to generate a plurality of bridge structure blocks of the bridge image to be monitored, and performing image coding representation on the plurality of bridge structure blocks by the image encoder to generate a bridge feature coding vector of the bridge image to be monitored; Performing structured mapping output on the first image structured feature information of the bridge image to be monitored to generate a plurality of structured feature segments of the first image structured feature information, and performing structural coding representation on the plurality of structured feature segments by the digital encoder to generate a structured coding vector of the bridge image to be monitored; The bridge feature coding vector and the structured coding vector of the bridge image to be monitored are semantically interactively fused through the second interactive network layer in the bridge state recognition network to generate an image semantic coding vector of the bridge image to be monitored.

8. An AI-based digital bridge monitoring system, characterized in that: The AI-based digital bridge monitoring system includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the AI-based digital bridge monitoring method described in any one of claims 1 to 7.

Images