AI-based digital bridge monitoring method and system

Abstract

The present invention provides an AI-based bridge digital monitoring method and system, which can comprehensively and accurately extract key information from bridge images by acquiring the structured feature information of bridge images and performing digital analysis. The bridge image and its digital analysis data are encoded and represented by a bridge state recognition network, and the image semantic coding vector and the digital coding vector are fused to perform bridge structure state recognition, effectively generating bridge structure state data covering multiple target state space dimensions. This method not only improves the accuracy and comprehensiveness of bridge state recognition, but also optimizes the parameters of the basic bridge state recognition network through sample bridge feature data sequences, further improving the reliability and practicality of bridge state recognition, helping to timely discover bridge structure anomalies and ensure the safe operation of bridges.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an AI-based bridge digital monitoring method and system. Background Art

[0002] With the rapid development of the transportation industry, the safety and stability of bridges, as key infrastructure connecting various places, are particularly important. However, during the long-term use of bridges, they will be affected by various natural factors (such as wind, rain, snow, earthquakes, etc.) and human factors (such as vehicle loads, construction and maintenance, etc.), causing the bridge structure to gradually age, become damaged, or even destroyed. Therefore, regular monitoring and evaluation of bridges to promptly discover and deal with potential safety hazards are important measures to ensure the safe operation of bridges.

[0003] Traditional bridge monitoring methods mainly rely on manual inspections and regular testing. Manual inspections require professionals to regularly inspect bridges on site, which is not only time-consuming and laborious, but also difficult to cover all parts of the bridge, and it is easy to miss potential safety hazards. Regular inspections usually need to be carried out when the bridge is closed or restricted, which will cause inconvenience to traffic and even cause economic losses.

[0004] With the advancement of science and technology, digital monitoring technology has gradually been applied to the field of bridge monitoring. By installing sensors, cameras and other equipment, various data of bridges, such as images, vibrations, stress, etc., can be collected in real time or regularly. These data provide more comprehensive and accurate information for bridge status assessment. However, how to effectively process and analyze these massive data and extract useful information for bridge status assessment is still a challenge facing bridge digital monitoring technology. Summary of the invention

[0005] In view of the above-mentioned problems, in combination with the first aspect of the present invention, an embodiment of the present invention provides an AI-based bridge digital monitoring method, the method comprising: Acquire first image structured feature information of the bridge image to be monitored; Performing digital analysis on the bridge image to be monitored and the first image structured feature information to determine digital analysis data of the bridge image to be monitored, wherein the digital analysis data of the bridge image to be monitored includes at least one of structural quantity information, structural trend feature information, structural standardization information of the first image structured feature information and time span information of the bridge image to be monitored; encoding and representing the image of the bridge to be monitored and the structural feature information of the first image through a bridge state recognition network to generate an image semantic encoding vector of the image of the bridge to be monitored, and encoding and representing the digital analysis data of the image of the bridge to be monitored through the bridge state recognition network to generate a digital encoding vector of the image of the bridge to be monitored; The image semantic coding vector and the digitized coding vector of the bridge image to be monitored are used to identify the bridge structure state through the bridge state recognition network, and the bridge structure state data of the bridge image to be monitored are generated, wherein the bridge structure state data includes a first state estimation result in a target state space dimension, and the target state space dimension includes at least one of a structural stability index dimension, a vibration frequency dimension, a load capacity dimension, a deformation monitoring time dimension, a continuous stability dimension, a loss degree dimension, an average service life dimension, and a remaining service life dimension; Among them, the bridge state recognition network is generated by optimizing the network parameters of the basic bridge state recognition network according to the second state estimation result and the annotated bridge structure state data of the template bridge image in the target state space dimension after the sample bridge feature data sequence is subjected to state recognition based on the basic bridge state recognition network to generate a second state estimation result of the template bridge image. The sample bridge feature data sequence includes the template bridge image, the second image structural feature information of the template bridge image, the digital analysis data of the template bridge image and the annotated bridge structure state data of the template bridge image in the target state space dimension.

[0006] On the other hand, an embodiment of the present invention also provides an AI-based digital bridge monitoring system, including a processor and a machine-readable storage medium, wherein the machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0007] Based on the above aspects, the embodiment of the present application can comprehensively and accurately extract key information from the bridge image by acquiring the structured feature information of the bridge image and performing digital analysis. The bridge image and its digital analysis data are encoded and represented by the bridge state recognition network, and the image semantic coding vector and the digital coding vector are fused to perform bridge structure state recognition, effectively generating bridge structure state data covering multiple target state space dimensions. This method not only improves the accuracy and comprehensiveness of bridge state recognition, but also optimizes the parameters of the basic bridge state recognition network through sample bridge feature data sequences, further improving the reliability and practicality of bridge state recognition, helping to timely detect bridge structure anomalies and ensure the safe operation of the bridge. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 It is a schematic diagram of the execution flow of the AI-based bridge digital monitoring method provided in an embodiment of the present invention.

[0009] Figure 2 It is a schematic diagram of the hardware architecture of the AI-based bridge digital monitoring system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0010] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 The figure is a flow chart of an AI-based digital bridge monitoring method provided by an embodiment of the present invention. The AI-based digital bridge monitoring method is introduced in detail below.

[0011] Step S110, obtaining first image structured feature information of the bridge image to be monitored.

[0012] In this embodiment, it is assumed that there is a large bridge located on a major traffic route that needs to be monitored. The bridge consists of multiple parts, including piers, spans, abutments and other structures. In order to obtain the first image structured feature information of the image of the bridge to be monitored, a high-resolution image acquisition device is used, such as a professional drone equipped with a high-definition camera, to shoot the bridge at specific time intervals.

[0013] For the collected bridge image to be monitored, the acquisition of the first image structured feature information involves multiple aspects. Taking the bridge pier as an example, the image structured feature information may include the shape, size, surface texture, etc. of the bridge pier. In terms of shape, the bridge pier may be round or square, and this information can be determined by the shape detection algorithm in the image recognition algorithm. For example, by analyzing the pixels in the image, if the pixels within a certain range show a distribution pattern that is approximately round or square, the shape of the bridge pier can be determined. In terms of size, the size of the bridge pier in the image can be determined by establishing a scale in the image and using known reference objects (such as standard length marks on the bridge or markers of known length set during shooting), and then the actual physical size can be calculated. The acquisition of surface texture is determined by analyzing the grayscale change or color change of the pixels on the surface of the bridge pier in the image. If there are cracks or corrosion on the surface of the bridge pier, it will appear as discontinuity of texture or abnormal changes in grayscale and color in the image.

[0014] Looking at the bridge span, its structured feature information may include the length of the bridge span in the image, the number of beams in the bridge span structure, and the connection method between the beams. The length of the bridge span in the image can be determined by the distance from the known total length of the bridge or the fixed marks at both ends of the bridge span in the image. The number of beams can be counted by contour recognition in the image. Each beam will present a specific contour shape in the image. The number of beams can be obtained by detecting and counting these contours. The connection method between beams, such as welding, bolt connection, etc., may appear in the image as a specific shape or grayscale change of the connection part. For example, the welding part may have a smoother transition, while the bolt connection will show regularly arranged round or square bolt head images in the image.

[0015] The structural feature information of the abutment image is equally important. The position of the abutment and its connection with the piers and spans are key information. The position of the abutment in the image can be determined by its relative position to the surrounding environment and the overall structural layout of the bridge. Its connection with the piers and spans may be reflected in the direction of the lines in the image, the continuity of the structure, etc. For example, if it is a rigid connection, there will be no obvious gap or displacement at the connection in the image; if it is a connection with a telescopic device, a specific expansion joint structure will be shown in the image.

[0016] In this way, the first image structured feature information of the bridge image to be monitored can be obtained, providing basic data for subsequent bridge status monitoring and analysis.

[0017] Step S120, digitally analyze the bridge image to be monitored and the structured feature information of the first image to determine the digital analysis data of the bridge image to be monitored, wherein the digital analysis data of the bridge image to be monitored includes at least one of structural quantity information, structural trend feature information, structural standardization information of the structured feature information of the first image and time span information of the bridge image to be monitored.

[0018] Let's continue with the example of the large bridge on the main traffic route mentioned earlier. First, the structural quantity information of the structured feature information of the first image is analyzed. In the pier part, it is assumed that the pier is square through image analysis, and each pier has a reinforcement structure at the four corners. Through further analysis and statistics of the image, it is found that there are 8 such reinforcement structures, which is a reflection of the number of structural elements. From the connection between the pier and the foundation, each pier has 4 connection points, which is the number of structural connections. These structural quantity information can reflect the complexity of the pier and certain aspects of its stability.

[0019] For the analysis of structural trend feature information, the time factor needs to be considered. Assume that the image of the bridge is collected every month for a year. Taking the corrosion of the pier surface as an example, in the image sequence of this year, the feature information time series set arranged in chronological order is first constructed based on the pixel features of the pier surface in each image. For example, for the corrosion area on the pier surface, according to the change of its pixel grayscale in the image, the feature information such as the area or corrosion depth of the corrosion area in each collected image is arranged according to the collection time.

[0020] Then, the moving average method is used to smooth the feature information time series set. Assuming that the area of ​​the corrosion region in the initial image fluctuates greatly, after being processed by the moving average method, the trend of the change of the area of ​​the corrosion region over time can be more clearly shown. By calculating the slope and curvature of this time series set, it is found that the area of ​​the corrosion region shows a trend of gradual increase in certain time periods, which is the trend feature preliminarily identified. The rationality of these trend features is verified. For example, after excluding abnormal fluctuations caused by accidental factors such as lighting or shooting angle during image acquisition, the verified trend features are refined and classified. If it is found that the increasing trend of the area of ​​the corrosion region is more obvious in summer and relatively slow in winter, this is the refined trend feature information.

[0021] Next, the correlation between different trend features is calculated. For example, it is found that the increasing trend of the corrosion area on the surface of the bridge pier is significantly correlated with the humidity change trend near the bridge pier, and the humidity change has a positive effect on the increase of the corrosion area. By constructing a correlation network between trend features, the importance and influence of each trend feature in this network are evaluated using network centrality indicators such as degree centrality. If it is found that the humidity change trend has a high degree centrality in this correlation network, it means that it has a strong influence on other trend features (such as the increase in corrosion area). Based on these evaluation results, each trend feature is standardized and normalized, and then cluster analysis is performed. For example, the corrosion trend features related to humidity are classified into one category, and the structural deformation trend features related to temperature are classified into another category. For each category, the internal feature similarities and differences, as well as the discrimination and correlation between different categories are further analyzed. Finally, through the pattern recognition algorithm, representative and distinguishing significant pattern features are identified, such as the pattern features of accelerated corrosion of bridge piers caused by high humidity in summer. This is the structural trend feature information of the first image structured feature information.

[0022] For structural standardization information, take the beam in the bridge span as an example. In terms of the morphological standardization information of the beam, if the beam is designed as a straight beam, the central axis of the beam should be a straight line in the image. Through the precise analysis of the contour of the beam in the image, it is found that the central axis of some beams is bent to a certain extent, which belongs to the information of non-standardized morphology. In terms of structural abnormality information, if there are bulges or depressions on the beam that should not exist, it will appear as abnormal changes in the grayscale or color of local pixels in the image, which is structural abnormality information.

[0023] Regarding the time span information of the bridge image to be monitored, since the bridge image has been regularly collected since a year ago, each collection has an accurate time tag. By standardizing and summarizing these time tag data, for example, unifying them into a format with years, months, and days as units, the time span information of the bridge image to be monitored can be obtained as one year. Based on the above-mentioned structural quantity information, structural trend characteristic information, structural standardization information, and time span information, the digital analysis data of the bridge image to be monitored is determined.

[0024] Step S130, encoding and representing the image of the bridge to be monitored and the structured feature information of the first image through a bridge state recognition network to generate an image semantic coding vector of the image of the bridge to be monitored, and encoding and representing the digital analysis data of the image of the bridge to be monitored through the bridge state recognition network to generate a digital coding vector of the image of the bridge to be monitored.

[0025] Still taking the large bridge as an example, the bridge state recognition network is a complex neural network structure. For the image of the bridge to be monitored and the structured feature information of the first image, the image of the bridge to be monitored is firstly regionally decomposed. For example, the entire bridge image is divided into multiple bridge structure blocks such as the pier area, the span area, and the abutment area according to the structure.

[0026] For these bridge structure blocks, image encoding representation is performed through the image encoder in the bridge state recognition network. The image encoder may be based on the structure of a convolutional neural network (CNN), which performs multi-layer convolution, pooling and other operations on the pixel information in each bridge structure block. Taking the bridge pier area as an example, the image encoder will learn the texture, color, shape and other features of the bridge pier in the image, and convert these features into a specific encoding form. After processing by multiple convolution layers, for example, the first convolution layer may extract the edge features of the bridge pier, and the second convolution layer further extracts more complex texture features. The pooling layer downsamples the features to reduce the amount of data while retaining the key features. After these operations, the bridge feature encoding vector of the bridge image to be monitored is generated.

[0027] At the same time, the structured feature information of the first image is structured mapped and output. Taking the structural features of the bridge pier as an example, it is decomposed into multiple structured feature fragments such as shape features, size features, and reinforcement structure features. These multiple structured feature fragments are structurally encoded by the digital encoder in the bridge state recognition network. The digital encoder may convert the shape feature into a specific numerical code, for example, a circular bridge pier is encoded as 1 and a square bridge pier is encoded as 2; the size feature is normalized and encoded according to the actual size value; and the reinforcement structure feature is encoded according to its quantity and type. In this way, a structured coding vector of the structured feature information of the first image is generated.

[0028] Then, the bridge feature coding vector and the structured coding vector are semantically interactively fused through the second interactive network layer in the bridge state recognition network. This interactive fusion process is based on the prior knowledge of the bridge structure. For example, it is known that the shape of the pier will affect its force distribution, and the force distribution is related to the size and reinforcement structure of the pier. During the semantic interactive fusion, the shape feature coding vector is fused with the coding vector of the size and reinforcement structure based on this inherent relationship. If the square pier has different characteristics from the circular pier in terms of force, then this difference will be reflected in the interactive fusion of the coding vector during the fusion process. Through this semantic interactive fusion, the image semantic coding vector of the bridge image to be monitored is generated.

[0029] The digital analysis data of the bridge image to be monitored is also encoded and represented by the bridge state recognition network. For example, for the number of bridge pier reinforcement structures in the structural quantity information, its numerical value is converted into a coding form suitable for network processing; the trend of the corrosion area over time in the structural trend feature information may be converted into a coding vector containing time series characteristics and change rate characteristics; the degree of non-standardization of the beam body morphology in the structural standardization information is also converted into a corresponding coding form; the time span information is encoded according to a specific time coding rule. Through such a coding process, a digital coding vector of the bridge image to be monitored is generated.

[0030] Step S140, performing bridge structure state recognition on the image semantic coding vector and the digital coding vector of the bridge image to be monitored through the bridge state recognition network, and generating bridge structure state data of the bridge image to be monitored, wherein the bridge structure state data includes a first state estimation result in a target state space dimension, and the target state space dimension includes at least one of a structural stability index dimension, a vibration frequency dimension, a load capacity dimension, a deformation monitoring time dimension, a continuous stability dimension, a loss degree dimension, an average service life dimension, and a remaining service life dimension.

[0031] Among them, the bridge state recognition network is generated by optimizing the network parameters of the basic bridge state recognition network according to the second state estimation result and the annotated bridge structure state data of the template bridge image in the target state space dimension after the sample bridge feature data sequence is subjected to state recognition based on the basic bridge state recognition network to generate a second state estimation result of the template bridge image. The sample bridge feature data sequence includes the template bridge image, the second image structural feature information of the template bridge image, the digital analysis data of the template bridge image and the annotated bridge structure state data of the template bridge image in the target state space dimension.

[0032] In detail, the first interactive network layer in the bridge state recognition network first performs semantic interactive fusion on the image semantic coding vector and the digital coding vector. Taking the structural stability index dimension as an example, the image semantic coding vector may contain image feature codes of structures such as piers, bridge spans and abutments, which reflect the appearance status of the structure; while the digital coding vector contains structural quantity information, structural trend feature information, etc. For example, the changing trend of the number of pier reinforcement structures may affect the structural stability. In the process of semantic interactive fusion, the first interactive network layer will deeply fuse these two aspects of information based on the mechanical principles and engineering experience of the bridge structure. For example, if the number of pier reinforcement structures decreases and the image shows signs of corrosion on the surface of the pier, the fused coding vector will reflect the downward trend of the structural stability index.

[0033] The semantic interaction encoding vector is then predicted in the target state space dimension by the fully connected mapping layer in the bridge state recognition network. It is assumed that the number of mapping spaces of the fully connected mapping layer is the same as the number of dimensions of the target state space dimension. For the structural stability index dimension, the fully connected mapping layer predicts the value of the structural stability index based on the information in the previously fused encoding vector and the patterns learned during the training process. For example, if the encoding vector shows that the structural features of the piers and spans have many anomalies, and the structural trend features indicate that these anomalies are tending to worsen, then the predicted structural stability index may be low.

[0034] For the vibration frequency dimension, the image semantic encoding vector may contain vibration-related features of the bridge structure in the image, such as the blurriness of certain parts (possibly caused by vibration), and the structural trend feature information in the digital encoding vector may contain the vibration amplitude change trend of the bridge structure in different time periods. The fully connected mapping layer will combine this information to predict the vibration frequency of the bridge. If the image shows that the span of the bridge has periodic blurring and the vibration amplitude tends to increase, the predicted vibration frequency may be higher than normal.

[0035] In terms of load capacity, the overall appearance information of the bridge structure, such as the deformation of the bridge span, is obtained from the image semantic coding vector, and the state information of the beam bearing structure in the structural standardization information is obtained from the digital coding vector. The fully connected mapping layer predicts the load capacity of the bridge based on this information. If there are many structural anomalies in the beam and the bridge span has obvious deformation, the predicted load capacity may be reduced.

[0036] For the deformation monitoring duration dimension, the time span information in the digital coding vector and the time trend of the structural deformation in the structural trend feature information will be used by the fully connected mapping layer. If the structural deformation gradually increases from a certain point in time and lasts for a long time, there will be corresponding estimation results in the deformation monitoring duration dimension.

[0037] In the dimension of continuous stability, the long-term state information of each bridge structure in the image semantic coding vector and the long-term trend of the structural trend feature information in the digital coding vector are integrated. If the state of each bridge structure is relatively stable over a long period of time and the structural trend does not show a significant deterioration trend, the estimation result of the continuous stability dimension will be better.

[0038] For the loss degree dimension, the prediction is made based on the representation of the structural damage in the image semantic coding vector and the structural anomaly in the structural normalization information in the digitized coding vector. If the image shows more structural damage and the structural anomaly is serious, the estimated result of the loss degree dimension will be higher.

[0039] In terms of the average service life dimension and the remaining service life dimension, the fully connected mapping layer will comprehensively consider the information of all the above target state space dimensions. For example, if the structural stability index is low, the vibration frequency is abnormal, the load capacity is reduced, the deformation monitoring time is long, and the loss degree is high, then the average service life will be predicted to be short and the remaining service life will be reduced accordingly. The bridge structure state data of the bridge image to be monitored is determined based on the first state estimation results of each candidate dimension included in the target state space dimension.

[0040] In the whole process, the bridge state recognition network is generated by optimizing the network parameters of the basic bridge state recognition network based on the second state estimation result and the labeled bridge structure state data of the template bridge image in the target state space dimension after the state recognition of the sample bridge feature data sequence is performed based on the basic bridge state recognition network to generate the second state estimation result of the template bridge image. For example, the initial basic bridge state recognition network may not accurately predict the structural stability index. When training with the sample bridge feature data sequence, the template bridge image, the second image structural feature information of the template bridge image, the digital analysis data of the template bridge image, and the labeled bridge structure state data of the template bridge image in the target state space dimension will be used to adjust the parameters of the network. If the labeled structural stability index is high in the template bridge image, but the basic bridge state recognition network predicts it to be low, then the weights and other parameters in the network will be adjusted according to this error. After multiple such adjustments, an optimized bridge state recognition network is generated, so that the bridge structure state of the monitored bridge image can be more accurately recognized.

[0041] Based on the above steps, the embodiment of the present application can comprehensively and accurately extract key information from the bridge image by acquiring the structured feature information of the bridge image and performing digital analysis. The bridge image and its digital analysis data are encoded and represented by the bridge state recognition network, and the image semantic coding vector and the digital coding vector are fused to perform bridge structure state recognition, effectively generating bridge structure state data covering multiple target state space dimensions. This method not only improves the accuracy and comprehensiveness of bridge state recognition, but also optimizes the parameters of the basic bridge state recognition network through sample bridge feature data sequences, further improving the reliability and practicality of bridge state recognition, helping to timely detect bridge structure anomalies and ensure the safe operation of the bridge.

[0042] In a possible implementation, the method further includes: Step S101, obtaining a sample bridge feature data sequence, wherein the sample bridge feature data sequence includes a template bridge image, second image structured feature information of the template bridge image, digital analysis data of the template bridge image, and annotated bridge structure state data of the template bridge image in the target state space dimension.

[0043] In this embodiment, taking the aforementioned large bridge located on a major traffic artery as an example, a series of template bridge images are accumulated during the long-term monitoring of the bridge. These template bridge images are taken at different times and under different environmental conditions and are representative. For each template bridge image, the method of obtaining the second image structured feature information is similar to the method of obtaining the first image structured feature information of the previously monitored bridge image. For example, for the pier part in the template bridge image, its shape, size, surface texture and other features will also be analyzed; the span part will analyze the performance of the span length in the image, the number of beams and the connection method, etc.; the abutment part will analyze its position and connection relationship, etc. The digital analysis data of the template bridge image is also obtained based on these structured feature information, including structural quantity information, such as the number of pier reinforcement structures, the number of span beams, etc.; structural trend feature information, such as the trend of pier surface corrosion over time, the time trend of span deformation, etc.; structural standardization information, such as whether the shape of the beam meets the standard and the structural abnormality; and the time span information of the template bridge image, that is, the time range from the beginning of the acquisition of the template bridge image to the last one. In addition, for each template bridge image, the bridge structure status data is annotated in the target state space dimension. For example, a specific value is annotated in the structural stability index dimension, indicating the stability of the bridge structure corresponding to the template bridge image at that time; the actual measured vibration frequency value is annotated in the vibration frequency dimension; the maximum load that the bridge can withstand is annotated in the load capacity dimension; the time from the beginning of detectable deformation to the current time is annotated in the deformation monitoring time dimension; the stability of the bridge over a long period of time is annotated in the continuous stability dimension; the quantitative value corresponding to the damage of the bridge structure is annotated in the loss degree dimension; the average service life dimension is annotated with the average service life estimated based on factors such as the bridge design, construction materials, and use environment; and the remaining service life dimension is annotated with the estimated remaining usable time when the template bridge image is collected.

[0044] Step S102, encoding and representing the template bridge image, the second image structured feature information of the template bridge image, and the digital analysis data of the template bridge image through an encoder in the basic bridge state recognition network, and generating a bridge feature coding vector, a digital coding vector, and a structural coding vector of the template bridge image.

[0045] The encoder in the basic bridge state recognition network is also divided into image encoder and digital encoder. For the template bridge image, the image encoder will decompose it into multiple bridge structure blocks, for example, the entire bridge image is divided into pier area, bridge span area, abutment area and other parts. Then each structure block is encoded. Taking the pier area as an example, the image encoder will process it through multiple convolutional layers and pooling layers. The convolutional layer will extract various image features of the pier, such as edge features, texture features, etc., and the pooling layer will downsample the features to reduce the amount of data. After these operations, the bridge feature encoding vector of the template bridge image is generated. For the second image structured feature information of the template bridge image, it is decomposed into multiple structured feature fragments, such as the shape feature fragments and size feature fragments of the pier. The digital encoder will encode these structured feature fragments, for example, the shape feature of the pier is converted into a numerical value according to a specific encoding rule, such as 1 for a circle and 2 for a square, and the size feature is normalized and encoded according to the actual numerical value, so as to generate a structured encoding vector. For the digital analysis data of the template bridge image, the numerical values ​​in the structural quantity information will be encoded according to certain rules, the trend change data in the structural trend feature information will be converted into a coding form suitable for network processing, and the abnormal conditions and morphological information in the structural standardization information will also be encoded accordingly, and finally a digital coding vector will be generated.

[0046] Step S103, loading the bridge feature coding vector, the digitized coding vector and the structured coding vector of the template bridge image into the fully connected mapping layer in the basic bridge state recognition network to perform bridge state prediction, and generate a second state estimation result of the template bridge image in the target state space dimension.

[0047] The fully connected mapping layer performs comprehensive processing based on the information in the previous coding vector. Taking the structural stability index dimension as an example, the fully connected mapping layer analyzes the characteristic performance of the bridge piers, bridge spans, abutments and other structures in the bridge feature coding vector in the image, combined with the structural shape, size and other information in the structured coding vector and the number and trend of structures in the digitized coding vector. If the bridge feature coding vector shows that the bridge piers have more signs of corrosion, the bridge pier size in the structured coding vector does not meet the standard, and the digitized coding vector shows that the number of bridge pier reinforcement structures is reduced and there is a corrosion trend, then the fully connected mapping layer may predict a low structural stability index. Similarly, for the vibration frequency dimension, the vibration frequency value is predicted by comprehensively considering the status of each part of the bridge in the image and the structural related information; for the load capacity dimension, the load capacity is predicted by combining the structural characteristics of the beam and the load-bearing related information; in the deformation monitoring duration dimension, the prediction is made based on the structural deformation trend and time-related information; in the continuous stability dimension, the prediction is made based on comprehensive long-term information from all aspects; in the loss degree dimension, the prediction is made based on the structural damage and abnormal information; in the average service life dimension and the remaining service life dimension, the information of all the above dimensions is comprehensively considered for prediction, thereby generating the second state estimation result of the template bridge image in the target state space dimension.

[0048] Step S104: training the basic bridge state recognition network based on the second state estimation result of the template bridge image in the target state space dimension and the annotated bridge structure state data to generate the bridge state recognition network.

[0049] In a possible implementation, step S104 includes: Step S1041, for any state space dimension included in the target state space dimension, based on the second state estimation result of the template bridge image in the any state space dimension and the annotated bridge structure state data, determine the first loss information of the any state space dimension.

[0050] Step S1042: Determine second loss information of the template bridge image based on first loss information of each state space dimension included in the target state space dimension.

[0051] Step S1043: training the basic bridge state recognition network according to the second loss information to generate a trained basic bridge state recognition network.

[0052] Step S1044: if the trained basic bridge state recognition network meets the training termination requirement, the trained basic bridge state recognition network is output as the bridge state recognition network.

[0053] For example, for any state space dimension included in the target state space dimension, taking the structural stability index dimension as an example, the second state estimation result of the template bridge image in this dimension is compared with the labeled bridge structure state data. If the labeled structural stability index is a higher value and the second state estimation result is a lower value, then the difference between the two is calculated as the first loss information of the dimension. The same operation is performed for other state space dimensions such as the vibration frequency dimension and the load capacity dimension. The second loss information of the template bridge image is determined based on the first loss information of each state space dimension included in the target state space dimension. For example, a weighted summation method can be used to assign weights to the first loss information of each dimension according to the importance of each dimension, and then the weighted first loss information of each dimension is added to obtain the second loss information. The basic bridge state recognition network is trained according to the second loss information, and the parameters of the network are adjusted during the training process, such as adjusting the weights in the fully connected mapping layer, the convolution kernel parameters in the image encoder and the digital encoder, etc. After multiple such trainings, the trained basic bridge state recognition network is obtained. If the trained basic bridge state recognition network meets the training termination requirements, for example, when the second loss information drops below a preset threshold, or the change in the second loss information is less than a certain value after multiple consecutive trainings, the trained basic bridge state recognition network is output as a bridge state recognition network, which can more accurately identify the state of the bridge structure.

[0054] In a possible implementation, step S140 includes: Step S141 , performing semantic interactive fusion on the image semantic coding vector and the digitized coding vector of the bridge image to be monitored through the first interactive network layer in the bridge state recognition network, so as to generate the semantic interactive coding vector of the bridge image to be monitored.

[0055] Step S142, predicting the semantic interaction coding vector of the bridge image to be monitored in the target state space dimension through the fully connected mapping layer in the bridge state recognition network, and generating bridge structure state data of the bridge image to be monitored.

[0056] In a possible implementation, step S142 includes: Step S1421, obtaining the number of dimensions included in the target state space dimension.

[0057] For the monitoring of the large bridge mentioned above, the target state space dimensions include structural stability index dimension, vibration frequency dimension, load capacity dimension, deformation monitoring duration dimension, continuous stability dimension, loss degree dimension, average service life dimension, remaining service life dimension, etc. Assuming that there are 8 dimensions in total, this is the number of dimensions included in the target state space dimension.

[0058] Step S1422, performing vector connection conversion on the semantic interaction coding vector of the bridge image to be monitored through the fully connected mapping layer according to the number of dimensions, generating a first state estimation result of each candidate dimension of the bridge image to be monitored included in the target state space dimension, and the number of mapping spaces of the fully connected mapping layer is the same as the number of dimensions.

[0059] The number of mapping spaces in the fully connected mapping layer is the same as the number of dimensions, which is 8 mapping spaces here. The fully connected mapping layer processes the semantic interaction encoding vector according to the characteristics of each mapping space and the patterns learned during the training process. Taking the structural stability index dimension as an example, the fully connected mapping layer extracts feature information related to structural stability from the semantic interaction encoding vector, which may include stability-related features of structures such as piers, spans, and abutments after a series of previous encoding fusions. If in the semantic interaction encoding vector, features such as a reduction in the number of pier reinforcement structures, an intensified surface corrosion trend, and abnormalities in the span beam connection exist at the same time, the fully connected mapping layer will predict a lower structural stability index based on the patterns learned during training. Similarly, for the vibration frequency dimension, the fully connected mapping layer will obtain vibration-related information from the semantic interaction encoding vector, such as the dynamic features of structures such as piers and spans (which may be obtained from the dynamic fuzzy part in the image semantic encoding vector and the structural vibration trend in the digitized encoding vector), and predict the value of the vibration frequency. For the load capacity dimension, the fully connected mapping layer will make predictions based on relevant features such as the structural integrity of the beam and the bearing capacity of the piers; for the deformation monitoring duration dimension, predictions are made based on relevant features such as the start time and deformation speed of the structural deformation; for the continuous stability dimension, predictions are made based on features such as the long-term stability trend of each bridge structure; for the loss degree dimension, predictions are made based on features such as the degree and scope of structural damage; for the average service life dimension and the remaining service life dimension, predictions are made based on information from all the above dimensions. Through such operations, the first state estimation results of each candidate dimension included in the target state space dimension of the bridge image to be monitored are generated.

[0060] Step S1423 , determining the bridge structure state data of the bridge image to be monitored according to the first state estimation results of each candidate dimension included in the target state space dimension of the bridge image to be monitored.

[0061] For example, if the prediction result of the structural stability index dimension is low, the prediction result of the vibration frequency dimension is high (indicating abnormal vibration), the prediction result of the load capacity dimension is low, the prediction result of the deformation monitoring time dimension is long (indicating that the deformation lasts for a long time), the prediction result of the continuous stability dimension is poor, the prediction result of the loss degree dimension is high, the prediction result of the average service life dimension is short, and the prediction result of the remaining service life dimension is small, these results can be combined to determine the bridge structure status data of the bridge image to be monitored, indicating that the bridge is in a poor structural state and further inspection and maintenance measures are needed to ensure the safe use of the bridge.

[0062] In a possible implementation, step S141 includes: Step S1411, analyzing the feature dimension structure of the image semantic coding vector and the digital coding vector, and determining the feature categories and corresponding dimension numbers contained in the image semantic coding vector and the digital coding vector.

[0063] In this embodiment, for the aforementioned large bridge located on a major traffic route, in this process, it is necessary to analyze in detail the characteristic dimension structure of the image semantic coding vector and the digital coding vector. The image semantic coding vector is obtained by performing regional decomposition on the image of the bridge to be monitored, image coding on each structural block, and structural mapping output and coding fusion on the structural feature information of the first image, which contains rich feature information. For example, in terms of structural features, the image semantic coding vector may contain the characteristic coding of its shape, size, surface texture, etc. on the image for the pier part; for the bridge span part, it contains the characteristic coding of the span length, the number of beams, the beam connection method, etc. in the image; for the abutment part, it contains the characteristic coding of its position, the connection relationship with other structures, etc. From the perspective of the number of dimensions, assuming that the shape feature coding of the pier is within a specific dimensional range, the size feature coding is within another dimensional range, and the coding of different structural parts and different features of the same structural part are distributed in different dimensional intervals. These dimensions together constitute the characteristic dimension structure of the image semantic coding vector.

[0064] Step S1412, based on the feature categories and corresponding dimension numbers contained in the image semantic coding vector and the digital coding vector, and according to a predefined feature dimension alignment rule, feature alignment is performed on the image semantic coding vector and the digital coding vector to generate an image semantic coding vector and a digital coding vector after feature dimension alignment.

[0065] The digital coding vector is obtained based on the digital analysis data of the bridge image to be monitored, which contains the structural quantity information, structural trend feature information, structural standardization information and time span information of the bridge image to be monitored of the first image structural feature information. For example, the numerical values ​​of the number of bridge pier reinforcement structures and the number of bridge span beams in the structural quantity information are encoded and represented by corresponding dimensions in the digital coding vector; structural trend feature information such as the trend of the change of bridge pier surface corrosion over time and the time trend of bridge span deformation also have their corresponding dimensional representations; whether the beam body morphology in the structural standardization information meets the standard and the structural abnormality also have corresponding coding dimensions; time span information also occupies a specific dimension. After determining the feature categories and corresponding dimension numbers contained in the image semantic coding vector and the digital coding vector, feature alignment is performed according to the pre-defined feature dimension alignment rule. This feature dimension alignment rule is determined based on the intrinsic relationship between the various features of the bridge structure and the logical relationship in bridge monitoring. For example, the shape feature of the bridge pier is related to the morphological standardization in the structural standardization information. Then, when aligning the features, the coding dimensions corresponding to the two features are aligned so that they can interact more reasonably in the subsequent fusion operation. After such operation, the image semantic coding vector and digital coding vector are generated after the feature dimensions are aligned.

[0066] Step S1413, based on the prior knowledge and semantic information of bridge monitoring, a semantic interaction mapping relationship between the image semantic coding vector and the digital coding vector is constructed, and according to the constructed semantic interaction mapping relationship, a fusion initialization operation is performed on the features in the image semantic coding vector and the digital coding vector after the feature dimension is aligned. Specifically, for each feature pair that has a corresponding association in the semantic interaction mapping relationship, a preliminary fusion operation is performed according to the set fusion strategy to obtain the first fusion feature data containing preliminary fusion features.

[0067] In this embodiment, these prior knowledge include the mechanical principles of bridge structures, the functional relationship of different structural parts, and the experience accumulated in long-term monitoring. For example, according to the mechanical principles of bridge structures, the structural stability of the pier is closely related to the number of reinforcement structures of the pier, the surface corrosion trend, etc., so in the semantic interaction mapping relationship, a mapping relationship between the features in the image semantic coding vector such as the shape and size of the pier and the features in the digital coding vector such as the number of reinforcement structures of the pier and the corrosion trend will be established. According to the constructed semantic interaction mapping relationship, the image semantic coding vector and the features in the digital coding vector after the feature dimension alignment are fused and initialized. For each feature pair that has a corresponding association in the semantic interaction mapping relationship, a preliminary fusion operation is performed according to the set fusion strategy. For example, for the feature pair of the pier shape and the number of reinforcement structures, if the set fusion strategy is weighted summation, different weights are assigned according to their importance in the structural stability, and then the weighted feature values ​​are added to obtain the preliminary fused feature values. After the preliminary fusion operation, all such feature pairs are subjected to the first fusion feature data containing the preliminary fusion features.

[0068] Step S1414: semantically enhance and adjust the first fused feature data according to predefined semantic rules and semantic association knowledge in the field of bridge monitoring, and generate second fused feature data after semantic enhancement and adjustment.

[0069] In the field of bridge monitoring, there are some semantic rules and association knowledge that are determined based on engineering standards and long-term practical experience. For example, if in the first fusion feature data, the fusion feature of the number of pier reinforcement structures and the corrosion trend of the pier surface does not conform to the normal structural stability semantic rule (that is, when the number of reinforcement structures is sufficient, there should be no rapid corrosion trend), then this fusion feature is adjusted according to the semantic association knowledge. This fusion feature may be corrected according to other relevant features (such as the encoding features corresponding to the surrounding humidity data, etc.) to generate the second fusion feature data after semantic enhancement and adjustment.

[0070] Step S1415, by analyzing the correlation and redundancy between the features in the second fused feature data, optimizing the selection or feature combination of the second fused feature data to generate optimized third fused feature data.

[0071] Step S1416: sort and encode the third fused feature data to generate a semantic interaction coding vector of the bridge image to be monitored.

[0072] In the second fused feature data, there may be some features that are highly correlated, such as the surface corrosion trend of the bridge pier and the coding features corresponding to the humidity data near the bridge pier. Some features may be redundant, such as two different ways of representing the local structural deformation of the bridge pier. There may be some redundancy. According to these correlations and redundancies, optimization selection is performed. If the features are highly correlated, the features that are more representative of the structural state judgment can be retained; the redundant features are removed or recombined. After such operations, the optimized third fused feature data is generated. Finally, the third fused feature data is sorted and encoded to generate the semantic interaction coding vector of the bridge image to be monitored.

[0073] In a possible implementation, the encoder in the bridge state recognition network includes an image encoder and a digital encoder, and step S130 includes: Step S131, performing regional decomposition on the bridge image to be monitored to generate multiple bridge structure blocks of the bridge image to be monitored, and performing image coding representation on the multiple bridge structure blocks through the image encoder to generate a bridge feature coding vector of the bridge image to be monitored.

[0074] In this embodiment, taking the previously mentioned large bridge located on a major traffic route as an example, the structure of this large bridge is complex and diverse, and the regional decomposition process has clear targeting. For example, according to the main components of the bridge, it can be decomposed into multiple different bridge structure blocks such as the pier area, the span area, and the abutment area. The pier area has unique visual features in the image, and its shape, texture, and relationship with the surrounding environment are different from other areas; the length of the span area, the arrangement of the beams, and the connection structure between the beams also have specific expressions in the image; the position of the abutment area and the connection with the piers and spans also have recognizable image features. Through such regional decomposition, subsequent image encoding operations can be better performed on each structural block.

[0075] Next, the image encoder is a module dedicated to processing image data. It encodes each bridge structure block based on a specific encoding algorithm. Taking the pier area as an example, the image encoder will first encode the shape of the pier. If the pier is square, the image encoder will convert the square features into corresponding numerical codes according to the preset shape encoding rules. During the encoding process, detailed information such as the ratio of the side length of the square and the shape features of the corners will be considered, and these details will be extracted through the convolution kernel in the convolution layer. For example, when the convolution kernel slides on the image, it will perform weighted calculations on the edge pixels of the pier shape to extract the key features of the shape, such as the straightness and angle of the edge. At the same time, the image encoder will also encode the texture features of the pier. There may be different textures on the surface of the pier, such as the granular texture of concrete or the wear marks caused by long-term use. The image encoder analyzes the grayscale changes of the texture, the direction of the texture, and other features, and encodes them by combining multiple convolution layers and pooling layers. The convolution layer gradually extracts more advanced texture features, and the pooling layer downsamples the features to reduce the amount of data while retaining key texture feature information. Although the color features of the pier area are relatively unimportant in bridge structure analysis, they are also encoded. For example, the color is encoded based on the distribution range of the RGB value. Different color distributions may imply different states of the pier surface, such as whether there are color changes caused by corrosion products. By encoding the shape, texture, color and other features of the pier area, as well as similar encoding operations for other bridge structure blocks such as the span area and abutment area, a bridge feature encoding vector of the bridge image to be monitored is finally generated.

[0076] Step S132, performing structured mapping output on the first image structured feature information of the bridge image to be monitored, generating multiple structured feature fragments of the first image structured feature information, and performing structural coding representation on the multiple structured feature fragments through the digital encoder to generate a structured coding vector of the bridge image to be monitored.

[0077] The first image structured feature information covers a variety of feature information of each structural part of the bridge, which is converted into multiple structured feature fragments through structured mapping output. Taking the bridge pier as an example, the bridge pier related features in the first image structured feature information can be decomposed into shape feature fragments, size feature fragments, reinforcement structure feature fragments, etc. The shape feature fragment further refines the various features of the bridge pier shape, such as the shape details of the four corners of the square bridge pier, the precise value of the side length, etc.; the size feature fragment clarifies the actual size of the bridge pier in all directions, including height, bottom side length, etc.; the reinforcement structure feature fragment describes the reinforcement structure of the bridge pier in detail, such as the number of reinforcing steel bars, distribution method, and connection method with the main structure of the bridge pier. For the bridge span, the structured mapping output of the first image structured feature information may generate bridge span length feature fragments, beam quantity feature fragments, beam connection method feature fragments, etc. The feature fragment of the bridge span length is accurate to the specific length value and the proportional relationship of the length in the image; the feature fragment of the number of beams clarifies the specific number of beams in the bridge span; the feature fragment of the beam connection method describes in detail whether the beams are connected by welding, bolts or other connection methods, as well as the specific structural characteristics of the connection parts. After the first image structured feature information of the abutment part is output through structured mapping, feature fragments of the abutment position and feature fragments of the connection relationship with other structures will be generated. The feature fragment of the abutment position accurately describes the geographical location of the abutment in the entire bridge structure, and the feature fragment of the connection relationship with other structures details the connection structure and connection firmness between the abutment and the bridge piers and the bridge span.

[0078] In this embodiment, the digital encoder adopts different encoding strategies for different types of structural feature fragments. For the shape of the pier in the shape feature fragment, if it is a square, the digital encoder may encode it as a specific value, such as 2, while the circle may be encoded as 1. This encoding method is based on a pre-set shape encoding dictionary. For the size feature fragment, the digital encoder will encode the actual size value after normalization. For example, the height of the pier is 10 meters. After normalization, it may be encoded as a value between 0 and 1 according to the set size encoding range. The specific value depends on the normalization algorithm and the value range of the overall size. For the reinforcement structure feature fragment, such as the number of pier reinforcement steel bars is 20, the digital encoder will assign a corresponding encoding value according to the range and importance of the number of steel bars. For the span length feature fragment of the span part, normalization encoding will also be performed; the beam quantity feature fragment is directly encoded according to the actual number; the beam connection mode feature fragment is encoded according to different connection types, for example, welding connection is encoded as 1, bolt connection is encoded as 2, etc. The feature fragment of the abutment part is also encoded in a similar way. By encoding each structured feature fragment, a structured coding vector of the bridge image to be monitored is finally generated.

[0079] Step S133, performing semantic interactive fusion on the bridge feature coding vector and the structured coding vector of the bridge image to be monitored through the second interactive network layer in the bridge state recognition network to generate an image semantic coding vector of the bridge image to be monitored.

[0080] In this embodiment, the role of the second interactive network layer is to fuse the information in the bridge feature coding vector and the structured coding vector based on the intrinsic semantic relationship of the bridge structure. Taking the bridge pier as an example, the image features such as the shape, texture, and color of the bridge pier in the bridge feature coding vector have semantic associations with the shape feature fragment, size feature fragment, and reinforcement structure feature fragment of the bridge pier in the structured coding vector. From the perspective of structural mechanics, the shape and size of the bridge pier will affect its force distribution, and the force distribution is closely related to the reinforcement structure. These relationships are manifested as different features in the image. The second interactive network layer will perform semantic interactive fusion based on these relationships. For example, if the shape of the bridge pier in the bridge feature coding vector is shown as a square and the corners have a certain degree of wear, and the size of the bridge pier in the structured coding vector indicates that its side length ratio may cause uneven force, and the reinforcement structure feature shows that the number of reinforcement steel bars is small, then in the semantic interactive fusion process, the second interactive network layer will integrate this information and consider the mutual influence relationship between them. Similar operations are performed for the bridge span and abutment parts. The image features of the beams in the bridge span are semantically related to the structural features such as the number of beams and the connection method. The image features of the abutments are also semantically interactive with the structural features such as the abutment position and connection relationship. Through this comprehensive semantic interactive fusion operation, the information in the bridge feature coding vector and the structural coding vector is deeply integrated, and finally the image semantic coding vector of the image of the bridge to be monitored is generated. This coding vector completely contains the information of the image of the bridge to be monitored after fusion from both the image features and the structural features, providing a comprehensive data basis for the subsequent recognition of the bridge structure status.

[0081] In a possible implementation, step S120 includes: Step S121 : performing structural feature statistics on the first image structural feature information to generate structural quantity information of the first image structural feature information, wherein the structural quantity information includes at least one of the number of structural elements and the number of structural connections in the first image structural feature information.

[0082] In this embodiment, taking the aforementioned large bridge located on a major traffic artery as an example, for the bridge image to be monitored of this large bridge, the first image structured feature information covers a variety of features of each structural part of the bridge. Taking the bridge pier as an example, in terms of the number of structural elements, this embodiment considers the reinforcement structural elements of the bridge pier, such as reinforcement steel bars, supporting ribs, etc. The number of these reinforcement structural elements is accurately identified and counted through image analysis technology. Assuming that in the bridge pier image, after careful image recognition algorithm processing, it can be accurately identified that the number of reinforcement steel bars is 30 and the number of supporting ribs is 4, these numbers constitute part of the number of structural elements. In terms of the number of structural connections, for the connection points between the bridge pier and the bridge foundation, the number of connection points is determined by analyzing the features of the connection parts in the image. If the bridge pier and the bridge foundation are connected by 8 bolts, and these bolt connection points can be clearly identified in the image, then the number of connection points 8 becomes part of the number of structural connections. For the bridge span, the number of structural elements may include the number of beams. For example, after image analysis, it is determined that the number of beams in the bridge span is 5. The number of structural connections can be the number of connection points between beams. Assuming that the beams are connected by welding, image analysis determines that there are 20 welding connection points. Similar structural quantity statistics are also performed on the abutment part, such as the number of connection structures between the abutment and the pier, and the span. Through such comprehensive structural feature statistics, the structural quantity information of the first image structured feature information is generated. These structural quantity information reflects the complexity and stability of the bridge structure from one aspect.

[0083] Step S122, performing a structural trend analysis on the first image structured feature information to generate structural trend feature information and structural standardization information of the first image structured feature information, wherein the structural standardization information includes at least one of morphological standardization information and structural abnormality information of the first image structured feature information.

[0084] In terms of structural standardization information, take the beam in the bridge span as an example. For the morphological standardization information of the beam, according to the design standard, the beam should be straight and regular in shape. The shape parameters of the beam in the image, such as straightness deviation, deformation degree of cross-sectional shape, etc., are accurately measured through image analysis technology. If the straightness deviation of the beam in the image exceeds the range allowed by the design standard, this belongs to morphological non-standardization information. For structural abnormal information, such as cracks, depressions or protrusions on the surface of the beam, image analysis is used to identify the performance of these abnormal structures in the image, including their position, size, shape and other information.

[0085] Step S123, obtaining the time tag data of the bridge image to be monitored, and standardizing and summarizing the time tag data of the bridge image to be monitored to generate time span information of the bridge image to be monitored.

[0086] For example, during the image acquisition process of this large bridge, each image of the bridge to be monitored has an accurate time tag, such as "March 15, 2020 10:00:00". These time tag data are standardized and aggregated in a unified format (such as year-month-day-hour-minute-second). Assuming that this embodiment starts regular image acquisition of the bridge from January 1, 2019 to December 31, 2021, then by standardizing and aggregating all image time tag data, it can be determined that the time span information of the image of the bridge to be monitored is from January 1, 2019 to December 31, 2021.

[0087] Step S124, determining digital analysis data of the image of the bridge to be monitored based on the structure quantity information of the first image structured feature information, the structure trend feature information, the structure standardization information and the time span information of the image of the bridge to be monitored.

[0088] For example, the structural quantity information includes the number of pier reinforcement structural elements and the number of structural connections, the structural trend feature information includes the pier corrosion trend and beam expansion trend, the structural standardization information includes the beam morphology deviation, structural anomalies and time span information, etc., and the comprehensive digital analysis data of the bridge image to be monitored are formed by combining this information. These digital analysis data can provide an important data basis for subsequent bridge status assessment and analysis.

[0089] In a possible implementation, step S122 includes: Step S1221, based on the first image structured feature information and the time interval data associated therewith, generate a feature information time series set arranged in chronological order. The specific operation is: according to the time tag of each feature information in the first image structured feature information, the feature information is assigned to the corresponding time point. For feature information with unclear time points, it is inferred and assigned based on the relative relationship between the feature information and other information with time tags or the acquisition sequence. Among them, for periodic or continuously changing feature information, the sliding window method is used to convert the feature information into time series data that changes with time. For non-periodic or sporadic feature information, the time node when the feature information occurs is recorded and represented in the time series in the form of event markers.

[0090] For example, for the characteristic information of the corrosion of bridge piers, it is assumed that there are a series of images of bridges to be monitored taken at different times in this embodiment. According to the time tag of the characteristic information of the corrosion of bridge piers in each image (this time tag is the precise time recorded when the image is collected), the corrosion characteristic information is assigned to the corresponding time point. If the corrosion characteristic information in some images does not have a clear time tag, the time point can be estimated and assigned based on its relative relationship with the corrosion characteristics in other images with clear time tags (such as the order of corrosion degree) or the acquisition sequence (such as inferring its time sequence according to the order in which the acquisition device stores the images). For characteristic information such as the corrosion of the surface of bridge piers, which may be continuously changing, the sliding window method is used to convert it into time series data that changes with time. For example, the change of the corrosion area of ​​the surface of the bridge pier in each window is counted in a fixed time window (such as a window every 3 months) to form time series data. For non-periodic or sporadic characteristic information such as cracks that suddenly appear on bridge piers, the time node of the crack is recorded and represented in the time series in the form of event markers.

[0091] Step S1222, using a moving average method or an exponential smoothing method, the feature information time series set is smoothed, and by calculating the slope and curvature of the feature information time series set, the trend characteristics of each feature information changing over time are identified, and the trend characteristics are output in the form of a candidate set. For feature information with periodic or seasonal patterns, a seasonal decomposition algorithm is used to decompose it into trend components, seasonal components, and random components.

[0092] Taking the time series data of the corrosion area of ​​bridge piers as an example, since there may be some measurement errors or short-term fluctuations in the actual collection process, the moving average method can effectively smooth these data. Assume that this embodiment adopts the three-period moving average method, that is, the average value of the corrosion area of ​​three adjacent time points is calculated each time as a new smoothed data point. After such smoothing, the fluctuation of the data is reduced, and the long-term trend can be better reflected. By calculating the slope and curvature of the feature information time series set, the trend characteristics of each feature information changing over time are identified. For the time series data of the corrosion area of ​​bridge piers, the slope represents the rate of change of the corrosion area over time. If the slope is positive and the value is large, it means that the corrosion area is increasing at an accelerated rate; the curvature can reflect the acceleration of this change. These trend features are output in the form of candidate sets. For feature information with periodic or seasonal patterns, such as the expansion and contraction of certain parts of the bridge may be related to seasonal temperature changes, the seasonal decomposition algorithm is used to decompose it into trend components, seasonal components and random components. For example, for the expansion data of bridge span expansion joints, the seasonal decomposition algorithm can be used to decompose the total expansion into regular expansion (seasonal component) caused by seasonal temperature changes, long-term structural change trends (trend components), and random fluctuations (random components) caused by accidental factors (such as short-term traffic load shocks, etc.).

[0093] Step S1223 , verifying the rationality of the candidate set, and refining and classifying the trend features that have passed the verification in the candidate set to generate refined trend feature information.

[0094] Step S1224, using a correlation analysis algorithm, calculates the correlation between different trend features in the refined trend feature information, and identifies target feature pairs with significant correlation based on the calculated correlation, analyzes the nature and strength of the correlation between the target feature pairs, obtains a correlation analysis result, and constructs a correlation relationship network between trend features based on the correlation analysis result.

[0095] For example, for the trend feature of the increase in the corrosion area of ​​the bridge pier, it is necessary to verify whether it is due to the actual corrosion situation rather than the error in the image acquisition process (such as the image color change caused by the change in illumination is misjudged as corrosion). The verified trend features are refined and classified. If it is found that the trend of the increase in the corrosion area of ​​the bridge pier has different manifestations in different areas, such as the corrosion area at the bottom of the bridge pier increases faster than the top, then this difference is refined and classified. After generating the refined trend feature information, the correlation analysis algorithm is used to calculate the correlation between different trend features. Assuming that there is a correlation between the trend of the increase in the corrosion area of ​​the bridge pier and the trend of humidity change near the bridge pier, the correlation coefficient between them is calculated by the correlation analysis algorithm. After identifying the target feature pairs with significant correlation based on the calculated correlation, the nature and strength of the correlation of the target feature pairs are analyzed. If the correlation coefficient is high and positive, it means that there is a positive correlation between the two, that is, the increase in humidity will lead to an increase in the corrosion area, and the correlation strength is high. According to the correlation analysis results, a correlation relationship network between trend features is constructed. In this network, the characteristics such as the trend of the increase in the corrosion area of ​​the bridge pier and the trend of humidity change are used as nodes, and the correlation between them is used as edges.

[0096] Step S1225, based on the association relationship network, use network centrality indicators to evaluate the importance and influence of each trend feature in the association relationship network, and according to the evaluation results, rank or assign weights to each trend feature to generate an importance ranking sequence or a weight sequence, and the network centrality indicators include at least one of degree centrality, closeness centrality, and betweenness centrality.

[0097] Step S1226, standardize and normalize each trend feature based on the importance ranking sequence or weight sequence, generate a target trend feature information set, use a clustering algorithm to perform cluster analysis on the target trend feature information set, generate a clustering result of feature groups or categories with similar trend features, and, for each category in the clustering result, further analyze the feature similarity and difference within the category, as well as the distinction and correlation between different categories, and, use a pattern recognition algorithm to identify and classify the feature patterns in the clustering result, extract representative and discriminative significant pattern features, output the clustering results and significant pattern features in a structured form, and generate the structural trend feature information of the first image structured feature information.

[0098] Taking degree centrality as an example, if a trend feature (such as humidity change trend) is directly related to many other trend features, then its degree centrality is high, indicating that it has a wide impact on other trend features in this network. Closeness centrality can measure the average distance from a trend feature to other trend features, and betweenness centrality reflects the importance of a trend feature in connecting other trend features. According to the evaluation results, each trend feature is sorted or weighted to generate an importance sorting sequence or weight sequence. For example, the humidity change trend is given a higher weight due to its importance in the association network. Based on the importance sorting sequence or weight sequence, each trend feature is standardized and normalized to generate a target trend feature information set. Then, a clustering algorithm is used to perform cluster analysis on the target trend feature information set. For example, the bridge pier corrosion trend feature related to humidity and the bridge span expansion trend feature related to temperature are clustered separately. For each category in the clustering results, the feature similarities and differences within the category are further analyzed. In the humidity-related pier corrosion trend feature category, the similarities and differences of the influence of humidity on pier corrosion in different regions are analyzed; the discrimination and correlation between different categories are analyzed, such as the correlation between the humidity-related pier corrosion trend feature category and the temperature-related bridge span expansion trend feature category is generated through the influence of environmental factors on the overall structure of the bridge. The pattern recognition algorithm is used to identify and classify the characteristic patterns in the clustering results, and extract representative and distinguishing significant pattern features. For example, the pattern features of accelerated corrosion at the bottom of the pier in high humidity seasons are identified. The clustering results and significant pattern features are output in a structured form to generate structural trend feature information of the first image structured feature information.

[0099] Figure 2 The hardware structure of the AI-based bridge digital monitoring system 100 for implementing the above-mentioned AI-based bridge digital monitoring method provided by an embodiment of the present invention is shown, as Figure 2 As shown, the AI-based digital bridge monitoring system 100 may include a processor 110 , a machine-readable storage medium 120 , a bus 130 , and a communication unit 140 .

[0100] The machine-readable storage medium 120 may store data and / or instructions. In some embodiments, the machine-readable storage medium 120 may store data acquired from an external terminal. In some embodiments, the machine-readable storage medium 120 may store data and / or instructions that the AI-based bridge digital monitoring system 100 uses to execute or use to complete the exemplary method described in the present invention.

[0101] During the specific implementation process, one or more processors 110 execute computer executable instructions stored in the machine-readable storage medium 120, so that the processor 110 can execute the AI-based bridge digital monitoring method of the above method embodiment. The processor 110, the machine-readable storage medium 120 and the communication unit 140 are connected via the bus 130, and the processor 110 can be used to control the sending and receiving actions of the communication unit 140.

[0102] The specific implementation process of the processor 110 can refer to the various method embodiments executed by the above-mentioned AI-based bridge digital monitoring system 100. The implementation principles and technical effects are similar, and this embodiment will not be repeated here.

[0103] In addition, an embodiment of the present invention further provides a readable storage medium, in which computer executable instructions are preset. When a processor executes the computer executable instructions, the above-mentioned AI-based digital bridge monitoring method is implemented.

[0104] It should be noted that in order to simplify the description of the present invention and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present invention, various features are sometimes combined into one embodiment, drawing or description thereof.

Claims

1. A bridge digital monitoring method based on AI, characterized in that: The method comprises: Acquire first image structured feature information of the bridge image to be monitored; Performing digital analysis on the bridge image to be monitored and the first image structured feature information to determine digital analysis data of the bridge image to be monitored, wherein the digital analysis data of the bridge image to be monitored includes at least one of structural quantity information, structural trend feature information, structural standardization information of the first image structured feature information and time span information of the bridge image to be monitored; encoding and representing the image of the bridge to be monitored and the structural feature information of the first image through a bridge state recognition network to generate an image semantic encoding vector of the image of the bridge to be monitored, and encoding and representing the digital analysis data of the image of the bridge to be monitored through the bridge state recognition network to generate a digital encoding vector of the image of the bridge to be monitored; The image semantic coding vector and the digitized coding vector of the bridge image to be monitored are used to identify the bridge structure state through the bridge state recognition network, and the bridge structure state data of the bridge image to be monitored are generated, wherein the bridge structure state data includes a first state estimation result in a target state space dimension, and the target state space dimension includes at least one of a structural stability index dimension, a vibration frequency dimension, a load capacity dimension, a deformation monitoring time dimension, a continuous stability dimension, a loss degree dimension, an average service life dimension, and a remaining service life dimension; Among them, the bridge state recognition network is generated by optimizing the network parameters of the basic bridge state recognition network according to the second state estimation result and the annotated bridge structure state data of the template bridge image in the target state space dimension after the sample bridge feature data sequence is subjected to state recognition based on the basic bridge state recognition network to generate a second state estimation result of the template bridge image. The sample bridge feature data sequence includes the template bridge image, the second image structural feature information of the template bridge image, the digital analysis data of the template bridge image and the annotated bridge structure state data of the template bridge image in the target state space dimension.

2. The AI-based bridge digital monitoring method according to claim 1 is characterized in that: The method further comprises: Acquire a sample bridge feature data sequence, wherein the sample bridge feature data sequence includes a template bridge image, second image structured feature information of the template bridge image, digital analysis data of the template bridge image, and annotated bridge structure state data of the template bridge image in the target state space dimension; The template bridge image, the second image structural feature information of the template bridge image and the digitized analysis data of the template bridge image are encoded and represented by an encoder in the basic bridge state recognition network to generate a bridge feature encoding vector, a digitized encoding vector and a structural encoding vector of the template bridge image; Loading the bridge feature coding vector, the digitized coding vector and the structured coding vector of the template bridge image into the fully connected mapping layer in the basic bridge state recognition network to perform bridge state prediction, and generating a second state estimation result of the template bridge image in the target state space dimension; The basic bridge state recognition network is trained based on the second state estimation result of the template bridge image in the target state space dimension and the annotated bridge structure state data to generate the bridge state recognition network.

3. The AI-based bridge digital monitoring method according to claim 2 is characterized in that: The training of the basic bridge state recognition network based on the second state estimation result of the template bridge image in the target state space dimension and the annotated bridge structure state data to generate the bridge state recognition network includes: For any one of the state space dimensions included in the target state space dimensions, based on the second state estimation result of the template bridge image in the any one of the state space dimensions and the annotated bridge structure state data, determining the first loss information of the any one of the state space dimensions; Determining second loss information of the template bridge image based on first loss information of each state space dimension included in the target state space dimension; Training the basic bridge state recognition network according to the second loss information to generate a trained basic bridge state recognition network; If the trained basic bridge state recognition network meets the training termination requirement, the trained basic bridge state recognition network is output as the bridge state recognition network.

4. The AI-based bridge digital monitoring method according to claim 1 is characterized in that: The bridge structure state recognition is performed on the image semantic coding vector and the digitized coding vector of the bridge image to be monitored by the bridge state recognition network to generate bridge structure state data of the bridge image to be monitored, including: Performing semantic interactive fusion on the image semantic coding vector and the digitized coding vector of the bridge image to be monitored through the first interactive network layer in the bridge state recognition network to generate the semantic interactive coding vector of the bridge image to be monitored; The semantic interaction encoding vector of the bridge image to be monitored is predicted on the target state space dimension through the fully connected mapping layer in the bridge state recognition network to generate bridge structure state data of the bridge image to be monitored.

5. The AI-based bridge digital monitoring method according to claim 4 is characterized in that: The method of predicting the semantic interaction encoding vector of the bridge image to be monitored on the target state space dimension through the fully connected mapping layer in the bridge state recognition network to generate bridge structure state data of the bridge image to be monitored includes: Obtaining the number of dimensions included in the target state space dimension; According to the number of dimensions, the semantic interaction coding vector of the bridge image to be monitored is subjected to vector connection conversion through the fully connected mapping layer to generate a first state estimation result of each candidate dimension of the bridge image to be monitored included in the target state space dimension, and the number of mapping spaces of the fully connected mapping layer is the same as the number of dimensions; The bridge structure state data of the bridge image to be monitored is determined according to the first state estimation results of each candidate dimension included in the target state space dimension of the bridge image to be monitored.

6. The AI-based bridge digital monitoring method according to claim 4 is characterized in that: The step of performing semantic interactive fusion on the image semantic coding vector and the digitized coding vector of the bridge image to be monitored through the first interactive network layer in the bridge state recognition network to generate the semantic interactive coding vector of the bridge image to be monitored includes: Analyze the feature dimension structure of the image semantic coding vector and the digital coding vector to determine the feature categories and corresponding dimension numbers contained in the image semantic coding vector and the digital coding vector; Based on the feature categories and corresponding dimension numbers contained in the image semantic coding vector and the digitized coding vector, according to a predefined feature dimension alignment rule, feature alignment is performed on the image semantic coding vector and the digitized coding vector to generate an image semantic coding vector and a digitized coding vector after feature dimension alignment; Based on the prior knowledge and semantic information of bridge monitoring, a semantic interaction mapping relationship between the image semantic coding vector and the digital coding vector is constructed, and according to the constructed semantic interaction mapping relationship, a fusion initialization operation is performed on the features in the image semantic coding vector and the digital coding vector after the feature dimension is aligned; specifically, for each feature pair that has a corresponding association in the semantic interaction mapping relationship, a preliminary fusion operation is performed according to the set fusion strategy to obtain the first fusion feature data containing the preliminary fusion features; According to predefined semantic rules and semantic association knowledge in the field of bridge monitoring, semantically enhancing and adjusting the first fused feature data to generate second fused feature data after semantically enhancing and adjusting; By analyzing the correlation and redundancy between the features in the second fused feature data, optimizing the selection or feature combination of the second fused feature data to generate optimized third fused feature data; The third fused feature data is sorted and encoded to generate a semantic interaction coding vector of the bridge image to be monitored.

7. The AI-based bridge digital monitoring method according to claim 1 is characterized in that: The encoder in the bridge state recognition network includes an image encoder and a digital encoder. The encoding and representing of the image of the bridge to be monitored and the structural feature information of the first image by the bridge state recognition network to generate an image semantic encoding vector of the image of the bridge to be monitored includes: Performing regional decomposition on the bridge image to be monitored to generate a plurality of bridge structure blocks of the bridge image to be monitored, and performing image coding representation on the plurality of bridge structure blocks by the image encoder to generate a bridge feature coding vector of the bridge image to be monitored; Performing structured mapping output on the first image structured feature information of the bridge image to be monitored to generate a plurality of structured feature segments of the first image structured feature information, and performing structural coding representation on the plurality of structured feature segments by the digital encoder to generate a structured coding vector of the bridge image to be monitored; The bridge feature coding vector and the structured coding vector of the bridge image to be monitored are semantically interactively fused through the second interactive network layer in the bridge state recognition network to generate an image semantic coding vector of the bridge image to be monitored.

8. The AI-based bridge digital monitoring method according to any one of claims 1 to 7, characterized in that: The step of digitally analyzing the image of the bridge to be monitored and the structural feature information of the first image to determine digital analysis data of the image of the bridge to be monitored includes: Performing structural feature statistics on the first image structural feature information to generate structural quantity information of the first image structural feature information, wherein the structural quantity information includes at least one of the number of structural elements and the number of structural connections in the first image structural feature information; Performing a structural trend analysis on the first image structural feature information to generate structural trend feature information and structural standardization information of the first image structural feature information, wherein the structural standardization information includes at least one of morphological standardization information and structural abnormality information of the first image structural feature information; Acquire time tag data of the image of the bridge to be monitored, and standardize and summarize the time tag data of the image of the bridge to be monitored to generate time span information of the image of the bridge to be monitored; Based on the structure quantity information of the first image structured feature information, the structure trend feature information, the structure standardization information and the time span information of the image of the bridge to be monitored, the digital analysis data of the image of the bridge to be monitored is determined.

9. The AI-based bridge digital monitoring method according to claim 8 is characterized in that: The step of performing structural trend analysis on the first image structural feature information to generate structural trend feature information of the first image structural feature information includes: Based on the first image structured feature information and the time interval data associated therewith, a feature information time series set arranged in time order is generated; the specific operation is: according to the time tag of each feature information in the first image structured feature information, the feature information is assigned to the corresponding time point, and for feature information with unclear time points, it is inferred and assigned according to the relative relationship between the feature information and other information with time tags or the acquisition sequence; wherein, for periodic or continuously changing feature information, a sliding window method is used to convert the feature information into time series data that changes with time; for non-periodic or sporadic feature information, the time node when the feature information occurs is recorded and represented in the time series in the form of event markers; The feature information time series set is smoothed by using a moving average method or an exponential smoothing method, and the trend characteristics of each feature information changing over time are identified by calculating the slope and curvature of the feature information time series set, and the trend characteristics are output in the form of a candidate set; wherein, for feature information with periodic or seasonal patterns, a seasonal decomposition algorithm is used to decompose it into trend components, seasonal components and random components; Performing rationality verification on the candidate set, and refining and classifying the trend features that have passed the verification in the candidate set to generate refined trend feature information; A correlation analysis algorithm is used to calculate the correlation between different trend features in the refined trend feature information, and after identifying target feature pairs with significant correlation based on the calculated correlation, the nature and strength of the correlation between the target feature pairs are analyzed to obtain a correlation analysis result, and a correlation relationship network between the trend features is constructed according to the correlation analysis result; Based on the association relationship network, a network centrality index is used to evaluate the importance and influence of each trend feature in the association relationship network, and each trend feature is sorted or weighted according to the evaluation result to generate an importance sorting sequence or a weight sequence, wherein the network centrality index includes at least one of degree centrality, closeness centrality, and betweenness centrality; Based on the importance ranking sequence or weight sequence, each trend feature is standardized and normalized. After generating a target trend feature information set, a clustering algorithm is used to perform cluster analysis on the target trend feature information set to generate a clustering result of feature groups or categories with similar trend features, and, for each category in the clustering result, further analyze the feature similarity and difference within the category, as well as the distinction and correlation between different categories, and, using a pattern recognition algorithm, identify and classify the feature patterns in the clustering result, extract representative and discriminative significant pattern features, output the clustering results and significant pattern features in a structured form, and generate structural trend feature information of the first image structured feature information.

10. An AI-based digital bridge monitoring system, characterized in that: The AI-based digital bridge monitoring system includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the AI-based digital bridge monitoring method described in any one of claims 1 to 9.

Images