Bridge crack monitoring method and system integrating multi-source sensor data

Abstract

The present application provides a bridge crack monitoring method and system that fuses multi-source sensor data, and relates to the field of data processing. The present application can construct a commonality measurement value matrix for determining the position of standard crack data in a bridge multimodal sensor monitoring data set based on the commonality measurement between each sensor monitoring sequence in a bridge multimodal sensor monitoring data set and each sensor monitoring sequence in standard crack data. Based on the constructed commonality measurement value matrix, similar sensor monitoring sequences included in the bridge multimodal sensor monitoring data set and the standard crack data can be directly displayed. At the same time, if the commonality measurement value matrix includes a locking arrangement pattern constructed by matrix units marked with significant features, it can be determined that the bridge multimodal sensor monitoring data set includes standard crack data, and based on the locking arrangement pattern included in the commonality measurement value matrix, the distribution of standard crack data in the bridge multimodal sensor monitoring data set can be accurately determined.

Description

Technical Field

[0001] The present application relates to the field of data processing, in particular to the field of bridge monitoring data processing, and specifically to a bridge crack monitoring method and system that integrates multi-source sensor data. Background Art

[0002] In the field of bridge health monitoring, crack detection is a crucial task, because cracks are a common manifestation of bridge structural damage and have a direct impact on the safety and service life of bridges. Traditional crack detection methods mainly rely on manual inspections, but this method is not only time-consuming and labor-intensive, but also difficult to achieve comprehensive and real-time monitoring of bridges. Therefore, in recent years, bridge crack monitoring technology based on multi-source sensor data has gradually attracted attention. Multi-source sensor data fusion technology provides more comprehensive and richer bridge status information by integrating data from different sensors, providing a new way for crack monitoring. However, how to accurately locate crack-related data in massive multi-source sensor data is still a problem that needs to be solved. Summary of the invention

[0003] The present application provides a bridge crack monitoring method and system that fuses multi-source sensor data.

[0004] According to one aspect of the present application, a bridge crack monitoring method for fusing multi-source sensor data is provided, the method comprising: obtaining P commonality measurement values ​​between a bridge multimodal sensor monitoring data set and standard crack data, wherein P≥2, the bridge multimodal sensor monitoring data set and the standard crack data both include multiple sensor monitoring sequences, a commonality measurement value represents a commonality measurement result between a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and a sensor monitoring sequence in the standard crack data; generating a commonality measurement value matrix based on the P commonality measurement values; a commonality measurement value is added to the commonality measurement result matrix; A matrix unit corresponds to a matrix unit in the measurement matrix, and the matrix unit corresponding to the commonality measurement value that meets the preset commonality requirement is a significant feature mark in the commonality measurement value matrix; the arrangement pattern of the matrix unit of the significant feature mark in the commonality measurement value matrix is ​​identified; if the commonality measurement value matrix includes a locking arrangement pattern, the position of the standard crack data in the bridge multimodal sensing monitoring data set is determined based on the matrix units of the significant feature marks that constitute the locking arrangement pattern; the commonality measurement value matrix includes the locking arrangement pattern to characterize that the bridge multimodal sensing monitoring data set includes the standard crack data.

[0005] As an implementation mode, the method of obtaining P commonality metrics between a bridge multimodal sensor monitoring data set and standard crack data includes: obtaining a quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set, and obtaining a quality classification array of each sensor monitoring sequence in the standard crack data; the quality classification array of any sensor monitoring sequence represents the confidence that the structural quality corresponding to any sensor monitoring sequence belongs to each quality type in a preset quality type set; and determining the P commonality metrics based on the error between the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the quality classification array of each sensor monitoring sequence in the standard crack data.

[0006] As an implementation mode, any sensor monitoring sequence in the bridge multimodal sensor monitoring data set is a target sensor monitoring sequence, and the target sensor monitoring sequence includes multiple data items obtained after performing a data splitting operation on the bridge multimodal sensor monitoring data set; the method of obtaining a quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set includes: extracting representation information of each data item included in the target sensor monitoring sequence to obtain data representation information of each data item included in the target sensor monitoring sequence; integrating representation information of multiple data representation information of multiple data items included in the target sensor monitoring sequence to generate integrated data representation information of the target sensor monitoring sequence; and inferring the quality classification of the target sensor monitoring sequence based on the integrated data representation information to obtain a quality classification array of the target sensor monitoring sequence.

[0007] As an implementation mode, the quality classification of the target sensor monitoring sequence is inferred based on the integrated data characterization information to obtain the quality classification array of the target sensor monitoring sequence, including: inferring the confidence that the structural quality corresponding to the target sensor monitoring sequence belongs to each quality type based on the integrated data characterization information through a debugged quality identification model; generating the quality classification array of the target sensor monitoring sequence based on the confidence that the structural quality corresponding to the target sensor monitoring sequence belongs to each quality type.

[0008] As an implementation mode, the method further includes: performing a data splitting operation on the bridge multimodal sensor monitoring data set to obtain Q data items included in the bridge multimodal sensor monitoring data set, Q>P; clustering the Q data items according to the data item values ​​to obtain a plurality of sensor monitoring sequences included in the bridge multimodal sensor monitoring data set; wherein, any sensor monitoring sequence of the bridge multimodal sensor monitoring data set includes a plurality of continuous data items among the Q data items.

[0009] As an implementation mode, the quality classification array of any sensor monitoring sequence is a confidence distribution array constructed by the confidence that the structural quality corresponding to any sensor monitoring sequence belongs to each quality type; the P commonality measurement values ​​are determined based on the error between the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the quality classification array of each sensor monitoring sequence in the standard crack data, including: determining the spatial spacing between the confidence distribution array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the confidence distribution array of each sensor monitoring sequence in the standard crack data, to obtain P array commonality measurement values; wherein, an array commonality measurement value represents the commonality measurement result between the confidence distribution array of a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the confidence distribution array of a sensor monitoring sequence in the standard crack data.

[0010] As an implementation mode, generating a commonality measurement value matrix based on the P commonality measurement values ​​includes: obtaining a two-dimensional matrix diagram formed based on a first time axis and a second time axis; the two-dimensional matrix diagram includes P elements, each element of the two-dimensional matrix diagram on the first time axis corresponds to each sensor monitoring sequence in the bridge multimodal sensor monitoring data set, each element of the two-dimensional matrix diagram on the second time axis corresponds to each sensor monitoring sequence in the standard crack data, and one commonality measurement value corresponds to one element in the two-dimensional matrix diagram; in the two-dimensional matrix diagram, the elements corresponding to the commonality measurement values ​​that meet the preset commonality requirements are displayed with the significant feature mark, and the elements corresponding to the commonality measurement values ​​that do not meet the preset commonality requirements are displayed in a display method other than the significant feature mark, so as to obtain the commonality measurement value matrix; wherein, the displayed element corresponding to a commonality measurement value belongs to the matrix unit corresponding to the commonality measurement value in the commonality measurement value matrix, and the locked arrangement pattern included in the commonality measurement value matrix represents the shape pattern of the element trajectory formed by multiple elements displayed with the significant feature mark.

[0011] As an implementation mode, the method further includes: taking a commonality metric value that is not less than a commonality metric reference value as a commonality metric value that meets the preset commonality requirement; taking a commonality metric value that is less than the commonality metric reference value as a commonality metric value that does not meet the preset commonality requirement.

[0012] As an implementation mode, generating a commonality metric value matrix based on the P commonality metric values ​​includes: performing feature imaging on each of the P commonality metric values ​​to obtain a control image node of each commonality metric value; a commonality metric value that meets the preset commonality requirement indicates that the value of the control image node meets the commonality metric value of the preset commonality requirement; obtaining a plane formed by a first time axis and a second time axis; the plane includes P node units, each node unit of the plane on the first time axis has a corresponding relationship with each sensor monitoring sequence in the bridge multimodal sensor monitoring data set, and each node unit of the plane on the second time axis has a corresponding relationship with each sensor monitoring sequence in the standard crack data. The columns have a corresponding relationship, a commonality metric value corresponds to a node unit in the plane; the RGB values ​​corresponding to the control image nodes of each commonality metric value are obtained; the node units corresponding to each commonality metric value in the plane are respectively assigned the RGB values ​​corresponding to the control image nodes of each commonality metric value, so as to obtain the commonality metric value matrix; wherein, a node unit corresponding to a commonality metric value after the RGB value is assigned belongs to the matrix unit corresponding to the commonality metric value in the commonality metric value matrix, the significant feature mark represents the marking method displayed as the target RGB value, and the locked arrangement mode included in the commonality metric value matrix represents the elements composed of multiple node units assigned the target RGB value. shape pattern of the trajectory; the matrix units of the significant feature marks constituting the locked arrangement pattern in the commonality metric value matrix include T matrix units displayed in sequence with the significant feature marks in the commonality metric value matrix, T>1; the position of the standard crack data in the bridge multimodal sensor monitoring data set is determined based on the matrix units of the significant feature marks constituting the locked arrangement pattern, including: obtaining the positions of the first sensor monitoring sequence and the second sensor monitoring sequence in the bridge multimodal sensor monitoring data set respectively; the first sensor monitoring sequence represents the sensor monitoring sequence of the commonality metric value corresponding to the initial matrix unit in the bridge multimodal sensor monitoring data set, and the initial matrix unit represents the The first matrix unit among the T matrix units, the second sensor monitoring sequence represents the sensor monitoring sequence of the commonality metric value corresponding to the end matrix unit in the bridge multimodal sensor monitoring data set, and the end matrix unit represents the last matrix unit among the T matrix units; the standard crack data in the bridge multimodal sensor monitoring data set is constructed by the sensor monitoring sequence between the first sensor monitoring sequence and the second sensor monitoring sequence in the bridge multimodal sensor monitoring data set; according to the positions of the first sensor monitoring sequence and the second sensor monitoring sequence in the bridge multimodal sensor monitoring data set, respectively, the position of the standard crack data in the bridge multimodal sensor monitoring data set is determined.

[0013] According to another aspect of the present application, a computer system is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described above.

[0014] This application has at least the following beneficial effects:

[0015] The present application obtains P commonality measurement values ​​between a bridge multimodal sensor monitoring data set and standard crack data. Both the bridge multimodal sensor monitoring data set and the standard crack data include multiple sensor monitoring sequences. A commonality measurement value represents the commonality measurement result between a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and a sensor monitoring sequence in the standard crack data. Then, a commonality measurement value matrix is ​​generated based on the P commonality measurement values. A commonality measurement value corresponds to a matrix unit in the commonality measurement value matrix. The matrix unit corresponding to the commonality measurement value that meets the preset commonality requirements is a significant feature mark in the commonality measurement value matrix. Then, the arrangement pattern of the matrix unit with the significant feature mark in the commonality measurement value matrix can be identified. In addition, if the commonality measurement value matrix includes a locking arrangement pattern, then the position of the standard crack data can be determined in the bridge multimodal sensor monitoring data set based on the matrix unit with the significant feature mark that constitutes the locking arrangement pattern. The commonality measurement value matrix includes the locking arrangement pattern, which represents that the bridge multimodal sensor monitoring data set includes standard crack data. Based on this, the present application can construct a commonality measurement value matrix for determining the position of standard crack data in a bridge multimodal sensor monitoring data set according to the commonality measurements between each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and each sensor monitoring sequence in the standard crack data, wherein the matrix units corresponding to similar sensor monitoring sequences in the bridge multimodal sensor monitoring data set and the standard crack data can be significant feature markers in the commonality measurement value matrix, then based on the constructed commonality measurement value matrix, the similar sensor monitoring sequences included between the bridge multimodal sensor monitoring data set and the standard crack data can be directly displayed, and at the same time, if the commonality measurement value matrix includes a locking arrangement pattern constructed by matrix units marked with significant features, it can be determined that the bridge multimodal sensor monitoring data set includes standard crack data, and based on the locking arrangement pattern included in the commonality measurement value matrix, the distribution of standard crack data in the bridge multimodal sensor monitoring data set can be accurately determined.

[0016] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0018] Figure 1 A schematic diagram of an application scenario of a bridge crack monitoring method that fuses multi-source sensor data according to an embodiment of the present application is shown.

[0019] Figure 2 A flow chart of a bridge crack monitoring method for fusing multi-source sensor data according to an embodiment of the present application is shown.

[0020] Figure 3 A schematic diagram of the functional module architecture of a bridge crack monitoring device according to an embodiment of the present application is shown.

[0021] Figure 4 A schematic diagram of the composition of a computer system according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0023] In this application, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the context of the description, they may also refer to different instances.

[0024] The terms used in the description of various examples in this application are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this application covers any one of the listed items and all possible combinations.

[0025] Figure 1A schematic diagram of an application scenario provided according to an embodiment of the present application is shown. The scenario includes one or more sensors 101, a computer system 120, and one or more networks 110 coupling the one or more sensors 101 to the computer system 120. The sensor 101 is used to collect various monitoring data of the target bridge.

[0026] exist Figure 1 In the configuration shown, the computer system 120 may include one or more components that implement the functions performed by the computer system 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. The sensor 101 may include a temperature sensor, a humidity sensor, a pressure sensor, a vibration sensor, and a sound sensor, among others.

[0027] The network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0028] Computer system 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Computer system 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, computer system 120 may run one or more services or software applications that provide the functionality described below.

[0029] The computing unit in the computer system 120 can run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The computer system 120 can also run any of various additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc. In some embodiments, the computer system 120 can be a server for a distributed system, or a server combined with a blockchain. The computer system 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The cloud server is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0030] In addition, one or more databases 130 may also be included in the scenario. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store data such as sensor data. The database 130 may reside in various locations. For example, the database used by the computer system 120 may be local to the computer system 120, or may be remote from the computer system 120 and may communicate with the computer system 120 via a network-based or dedicated connection. The database 130 may be of different types. In some embodiments, the database used by the computer system 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data from the database in response to a command.

[0031] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0032] Please refer to Figure 2 , is a flow chart of a bridge crack monitoring method for fusing multi-source sensor data provided in an embodiment of the present application. It includes the following steps:

[0033] Step S100: Obtain P commonality measurement values ​​between the bridge multimodal sensor monitoring data set and the standard crack data, where P≥2, the bridge multimodal sensor monitoring data set and the standard crack data both include multiple sensor monitoring sequences, and a commonality measurement value represents the commonality measurement result between a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and a sensor monitoring sequence in the standard crack data.

[0034] The purpose of step S100 is to obtain P commonality metrics between the bridge multimodal sensor monitoring data set and the standard crack data. These commonality metrics are used to characterize the degree of commonality between the sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the sensor monitoring sequence in the standard crack data. Wherein, P represents the number of commonality metrics, and P≥2, which means that at least two commonality metrics need to be calculated.

[0035] Specifically, both the bridge multimodal sensor monitoring data set and the standard crack data contain multiple sensor monitoring sequences. A sensor monitoring sequence corresponds to a set of data of multiple modes collected at a data collection time point. For example, in a bridge crack monitoring scenario, a sensor monitoring sequence may contain data of multiple modes such as temperature, humidity, sound, vibration, etc., which are collected from different sensors on the bridge at the same time point. The calculation of commonality metrics can be achieved in a variety of ways, such as using machine learning models, algorithms or neural networks. A specific implementation is that a computer system can use a feature extraction algorithm to extract feature vectors from each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the standard crack data. Then, a similarity metric function (such as Euclidean distance, cosine similarity, etc.) is used to calculate the commonality metric between each pair of feature vectors.

[0036] For example, suppose there is a sensor monitoring sequence A in the bridge multimodal sensor monitoring dataset, which contains data of three modes: temperature, humidity, and vibration, represented as vectors [23, 45, 67]. At the same time, there is a sensor monitoring sequence B in the standard crack data, which also contains data of these three modes, represented as vectors [25, 40, 65]. The computer system can use the Euclidean distance as the commonality metric function to calculate the commonality metric value between the two sequences. The calculated commonality metric value represents the degree of similarity between the sensor monitoring sequence A and the sensor monitoring sequence B. By calculating the commonality metric value between each sensor monitoring sequence in the bridge multimodal sensor monitoring dataset and each sensor monitoring sequence in the standard crack data, the computer system can obtain P commonality metric values ​​for subsequent crack monitoring and analysis.

[0037] Step S200: Generate a commonality measurement value matrix according to the P commonality measurement values; one commonality measurement value corresponds to one matrix unit in the commonality measurement value matrix, and the matrix unit corresponding to the commonality measurement value that meets the preset commonality requirement is a significant feature mark in the commonality measurement value matrix.

[0038] The purpose of step S200 is to generate a commonality measurement value matrix based on the P commonality measurement values ​​calculated previously. In this matrix, each commonality measurement value corresponds to a matrix unit (i.e., a matrix element), and the matrix units corresponding to the commonality measurement values ​​that meet the preset commonality requirements are marked as significant features in the commonality measurement value matrix.

[0039] Specifically, the computer system first creates a two-dimensional commonality metric value matrix, whose number of rows and columns correspond to the number of sensor monitoring sequences in the bridge multimodal sensor monitoring data set and the standard crack data, respectively. Then, the computer system fills each calculated commonality metric value into the corresponding position of the matrix, that is, each commonality metric value corresponds to a unit in the matrix.

[0040] Next, the computer system sets a preset commonality requirement, which can be a threshold value of the commonality metric value or a relative size relationship between the commonality metric values. The matrix cells corresponding to the commonality metric values ​​that meet the preset commonality requirement are marked as significant features in the commonality metric value matrix. For example, if the preset commonality requirement is that the commonality metric value is greater than a certain threshold, then all matrix cells corresponding to the commonality metric values ​​greater than the threshold can be marked as significant features.

[0041] For example, in the scenario of bridge crack monitoring, assuming that the bridge multimodal sensor monitoring data set contains 3 sensor monitoring sequences and the standard crack data also contains 3 sensor monitoring sequences, the computer system will create a 3×3 commonality metric value matrix. If the calculated commonality metric values ​​are 10, 20, 30, 40, 50, 60, 70, 80 and 90, respectively, and the preset commonality requirement is that the commonality metric value is greater than 50, then the computer system will mark the matrix cells corresponding to the four commonality metric values ​​of 60, 70, 80 and 90 as significant features.

[0042] By generating a commonality metric matrix and marking significant features, the computer system can more intuitively analyze the commonality relationship between the bridge multimodal sensor monitoring data set and the standard crack data, providing strong support for subsequent crack monitoring and early warning.

[0043] Step S300: Identify the arrangement pattern of the matrix units of the significant feature marks in the commonality metric matrix.

[0044] The purpose of step S300 is to identify the arrangement pattern of matrix cells marked as significant features in the commonality metric matrix. This arrangement pattern may reflect some inherent correlation or law between the bridge multimodal sensor monitoring data set and the standard crack data, which is of great significance for subsequent crack monitoring and early warning.

[0045] Specifically, the computer system scans the commonality measure matrix and locates all matrix cells marked as significant features. The computer then analyzes the distribution and arrangement of these significant feature matrix cells to try to identify whether there is a specific pattern or regularity among them.

[0046] For example, in the scenario of bridge crack monitoring, the computer system can find that the matrix units marked as significant features are mainly concentrated in a row or column of the commonality metric matrix, which may mean that there is a strong commonality between a specific sensor monitoring sequence and the standard crack data. Alternatively, the computer system can find that the significant feature matrix units present a certain symmetrical or periodic arrangement pattern, which may reveal some inherent correlation between different sensor monitoring sequences or data collected at different time points.

[0047] To illustrate more specifically, assume that the commonality metric matrix is ​​a 5×5 matrix in which the matrix cells marked as significant features are located in the 2nd row and the 4th column, as well as their symmetrical positions (i.e., the 4th row and the 2nd column). This symmetrical arrangement pattern may indicate that in the bridge multimodal sensor monitoring data set, there is a strong commonality between the 2nd sensor monitoring sequence and the 4th sensor monitoring sequence in the standard crack data, and there is a similar commonality between the 4th sensor monitoring sequence and the 2nd sensor monitoring sequence in the standard crack data. By identifying the arrangement pattern of the significant feature matrix cells, the computer system can further explore the potential associations and laws between the bridge multimodal sensor monitoring data set and the standard crack data, providing more in-depth and comprehensive support for the monitoring and early warning of bridge cracks.

[0048] Step S400: If the commonality metric value matrix includes a locking arrangement pattern, then the position of the standard crack data is determined in the bridge multimodal sensing monitoring data set based on the matrix units of the significant feature markers that constitute the locking arrangement pattern; the commonality metric value matrix includes the locking arrangement pattern to characterize that the bridge multimodal sensing monitoring data set includes the standard crack data.

[0049] The purpose of step S400 is to identify the locking arrangement pattern in the commonality metric matrix and determine the location of the standard crack data in the bridge multimodal sensor monitoring data set. The locking arrangement pattern refers to a specific, predefined arrangement of significant feature matrix units, which indicates that the bridge multimodal sensor monitoring data set contains data that matches or is similar to the standard crack data.

[0050] Specifically, the computer system first checks whether there is an arrangement of salient feature matrix units in the commonality metric matrix that matches the predefined locking arrangement pattern. This matching can be achieved by comparing the position and shape of the salient features in the matrix with the locking arrangement pattern.

[0051] If the computer system finds that there is indeed a locking arrangement pattern in the commonality metric matrix, it will determine the location of the standard crack data in the bridge multimodal sensor monitoring data set based on the matrix cells marked with significant features that constitute the locking arrangement pattern. This is because the existence of the locking arrangement pattern indicates that there is a strong commonality between certain sensor monitoring sequences in the bridge multimodal sensor monitoring data set and the standard crack data, so these sensor monitoring sequences are likely to be the corresponding positions of the standard crack data in the bridge multimodal sensor monitoring data set.

[0052] For example, in the scenario of bridge crack monitoring, the computer system may predefine a locking arrangement pattern, which means that when the 2nd row and 4th column, and the 4th row and 2nd column matrix cells in the commonality metric matrix are marked as significant features at the same time, it means that the bridge multimodal sensor monitoring data set contains data that matches the standard crack data. When the computer system detects this arrangement pattern in the actual commonality metric matrix, it determines that the 2nd and 4th sensor monitoring sequences in the bridge multimodal sensor monitoring data set are the locations of the standard crack data.

[0053] Through step S400, the computer system can accurately identify the location containing the standard crack data in the bridge multimodal sensing monitoring data set, which has important practical significance for subsequent crack monitoring, early warning and maintenance decision-making.

[0054] As an implementation manner, the step S100 of obtaining P commonality metrics between the bridge multimodal sensing monitoring data set and the standard crack data may specifically include:

[0055] Step S110: obtaining a quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set, and obtaining a quality classification array of each sensor monitoring sequence in the standard crack data; the quality classification array of any sensor monitoring sequence represents the confidence that the structural quality corresponding to any sensor monitoring sequence belongs to each quality type in the preset quality type set;

[0056] Step S120: determining the P commonality metrics according to the errors between the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the quality classification array of each sensor monitoring sequence in the standard crack data.

[0057] The implementation of step S100 specifically includes step S110 and step S120. The following is a detailed explanation of these two steps, and an example is given in conjunction with the scenario of bridge crack monitoring.

[0058] Step S110: Obtain the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set, and obtain the quality classification array of each sensor monitoring sequence in the standard crack data; the quality classification array of any sensor monitoring sequence represents the confidence that the structural quality corresponding to any sensor monitoring sequence belongs to each quality type in the preset quality type set.

[0059] In this step, the main task of the computer system is to extract the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the standard crack data. The quality classification array is a vector whose length is equal to the number of quality types in the preset quality type set, and each element in the vector represents the confidence that the corresponding sensor monitoring sequence belongs to a certain quality type.

[0060] For example, in the scenario of bridge crack monitoring, the preset quality type set may include four types: "no cracks", "small cracks", "medium cracks" and "severe cracks". For a sensor monitoring sequence in the bridge multimodal sensor monitoring data set, its quality classification array may be [0.1, 0.3, 0.4, 0.2], indicating that the confidence level of the structural quality corresponding to the sensor monitoring sequence belonging to "no cracks" is 0.1, the confidence level of belonging to "small cracks" is 0.3, the confidence level of belonging to "medium cracks" is 0.4, and the confidence level of belonging to "severe cracks" is 0.2.

[0061] The computer system obtains these quality classification arrays in any possible way. One possible method is to use a machine learning model, such as a neural network or a support vector machine (softmax), to classify each sensor monitoring sequence in the bridge multimodal sensor monitoring dataset and the standard crack data, and obtain the confidence level belonging to each quality type, thereby forming a quality classification array.

[0062] Step S120: determining the P commonality metrics according to the errors between the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the quality classification array of each sensor monitoring sequence in the standard crack data.

[0063] In this step, the computer system calculates P commonality metrics based on the errors between the quality classification arrays of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the standard crack data. These commonality metrics are used to characterize the degree of commonality between the bridge multimodal sensor monitoring data set and the standard crack data. The error can be calculated using a variety of methods, such as Euclidean distance, cosine similarity, cross entropy, etc. Taking the Euclidean distance as an example, for a sensor monitoring sequence A in the bridge multimodal sensor monitoring data set and a quality classification array of [0.1, 0.3, 0.4, 0.2], and a sensor monitoring sequence B in the standard crack data and a quality classification array of [0.2, 0.2, 0.5, 0.1], the Euclidean distance between them is 0.2. This calculated commonality metric represents the degree of similarity between sensor monitoring sequence A and sensor monitoring sequence B. By calculating the commonality metric value between each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and each sensor monitoring sequence in the standard crack data, the computer system can obtain P commonality metric values, where P represents the number of commonality metric values, and P≥2.

[0064] In practical applications, the computer system can select a variety of error calculation methods to obtain commonality metrics at different angles. For example, in addition to the Euclidean distance, cosine similarity can also be calculated as another commonality metric. In summary, the implementation method of step S100 obtains the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the standard crack data, and calculates P commonality metrics based on the errors between these arrays. These commonality metrics are used to characterize the degree of commonality between the bridge multimodal sensor monitoring data set and the standard crack data, providing strong support for subsequent crack monitoring and early warning. In the scenario of bridge crack monitoring, this implementation method can accurately evaluate the similarities and differences between the bridge multimodal sensor monitoring data set and the standard crack data, thereby helping engineers and decision makers to better understand the structural condition of the bridge and take necessary maintenance and repair measures in a timely manner.

[0065] As an implementation mode, any sensor monitoring sequence in the bridge multimodal sensor monitoring data set is a target sensor monitoring sequence, and the target sensor monitoring sequence includes multiple data items obtained after performing a data splitting operation on the bridge multimodal sensor monitoring data set. Based on this, the step S110, obtaining the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set, includes:

[0066] Step S111: extracting representation information of each data item included in the target sensing monitoring sequence to obtain data representation information of each data item included in the target sensing monitoring sequence;

[0067] Step S112: integrating the multiple data representation information of the multiple data items included in the target sensing monitoring sequence to generate integrated data representation information of the target sensing monitoring sequence;

[0068] Step S113: inferring the quality classification of the target sensing monitoring sequence according to the integrated data characterization information to obtain a quality classification array of the target sensing monitoring sequence.

[0069] In step S111, the task of the computer system is to extract data representation information from each data item in the target sensor monitoring sequence. Data representation information refers to information that can reflect the essential characteristics or attributes of the data item, which can be in the form of numerical values, vectors, labels, etc. In the scenario of bridge crack monitoring, data items may include data from different sensors, such as temperature, humidity, vibration, etc. These data items all contain certain representation information and can reflect the state of the bridge.

[0070] For example, suppose the target sensing monitoring sequence includes three data items: temperature, humidity and vibration. For the temperature data item, its data characterization information may be a specific temperature value, such as 25°C; for the humidity data item, its data characterization information may be a specific humidity value, such as 60%; for the vibration data item, its data characterization information may be characteristics such as the frequency and amplitude of the vibration signal.

[0071] The computer system extracts these data representation information in some way. One possible method is to use feature extraction algorithms, such as principal component analysis (PCA) and linear discriminant analysis (LDA), to convert data items into vectors or numerical values ​​that can reflect their essential characteristics. For example, for vibration data items, the PCA algorithm can be used to extract its main frequency and amplitude features to obtain a feature vector as data representation information.

[0072] In step S112, the task of the computer system is to integrate the data representation information extracted from each data item to generate integrated data representation information of the target sensor monitoring sequence. Integrated data representation information refers to fusing the data representation information of multiple data items together to form an information representation that can fully reflect the state of the target sensor monitoring sequence.

[0073] Continuing with the scenario of bridge crack monitoring, assume that data representation information has been extracted from the three data items of temperature, humidity and vibration, which are the temperature value of 25°C, the humidity value of 60%, and the vibration feature vector [f1, f2, f3]. In this step, the computer system needs to integrate these data representation information together to form an integrated data representation information. One possible way to integrate is to splice these data representation information into a vector, such as [25, 60, f1, f2, f3]. This vector contains the comprehensive status information of the target sensor monitoring sequence and can be used for subsequent quality classification.

[0074] In step S113, the task of the computer system is to infer the quality classification of the target sensor monitoring sequence based on the integrated data characterization information and generate a quality classification array. Quality classification refers to dividing the state of the target sensor monitoring sequence into a certain type in a preset quality type set, such as "no cracks", "minor cracks", "medium cracks" and "serious cracks". The quality classification array is a vector whose length is equal to the number of quality types in the preset quality type set, and each element in the vector represents the confidence that the target sensor monitoring sequence belongs to a certain quality type.

[0075] Continuing with the scenario of bridge crack monitoring, assume that the preset quality type set is {"no crack", "minor crack", "moderate crack", "serious crack"}, and the integrated data representation information is [25, 60, f1, f2, f3]. In this step, the computer system needs to use some method to infer the quality classification of the target sensor monitoring sequence based on the integrated data representation information. One possible method is to use a machine learning model, such as a neural network or a support vector machine, to take the integrated data representation information as input and output the confidence that the target sensor monitoring sequence belongs to each quality type, thereby forming a quality classification array.

[0076] For example, a computer system can use a trained neural network model, take the integrated data representation information [25, 60, f1, f2, f3] as input, and obtain the output vector [0.1, 0.3, 0.4, 0.2] after calculation by the neural network. This output vector is the quality classification array of the target sensor monitoring sequence, indicating that the confidence level of the target sensor monitoring sequence belonging to "no crack" is 0.1, the confidence level of the target sensor monitoring sequence belonging to "minor crack" is 0.3, the confidence level of the target sensor monitoring sequence belonging to "medium crack" is 0.4, and the confidence level of the target sensor monitoring sequence belonging to "serious crack" is 0.2.

[0077] In practical applications, computer systems can use more complex machine learning models and methods to improve the accuracy of quality classification. For example, deep learning models such as convolutional neural networks (CNN) or recurrent neural networks (RNN) can be used to perform deeper feature extraction and classification of integrated data representation information. In addition, ensemble learning methods such as random forests or gradient boosting trees can be used to combine the prediction results of multiple models to improve the stability and accuracy of quality classification.

[0078] In summary, the implementation method of step S110 extracts the data characterization information of each data item in the target sensor monitoring sequence, integrates these data characterization information to generate integrated data characterization information, and infers the quality classification array of the target sensor monitoring sequence based on the integrated data characterization information. This process realizes the acquisition of the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set, and provides a basis for subsequent commonality metric value calculation and crack monitoring. In the scenario of bridge crack monitoring, this implementation method can accurately evaluate the state of the bridge and divide it into different quality types, providing strong support for engineers and decision makers.

[0079] As an implementation manner, the step S113, inferring the quality classification of the target sensor monitoring sequence according to the integrated data representation information to obtain the quality classification array of the target sensor monitoring sequence, may include:

[0080] Step S1131: inferring the confidence level that the structural quality corresponding to the target sensing monitoring sequence belongs to each quality type based on the integrated data characterization information through the debugged quality identification model;

[0081] Step S1132: generating a quality classification array of the target sensing monitoring sequence according to the confidence that the structural quality corresponding to the target sensing monitoring sequence belongs to each quality type.

[0082] In the scenario of bridge crack monitoring, the core task of step S113 is to infer the quality classification of the target sensor monitoring sequence based on the integrated data characterization information and generate a corresponding quality classification array. This implementation is specifically divided into two steps: S1131 and S1132. The following is a detailed explanation of these two steps, and examples are given in combination with actual scenarios.

[0083] In step S1131, the task of the computer system is to use the quality recognition model that has been trained and debugged to infer the confidence that the structural quality corresponding to the target sensing monitoring sequence belongs to each preset quality type based on the integrated data characterization information. The quality recognition model is a machine learning model that can learn the mapping relationship between data and quality types from the input data, and infer the quality type of new data based on this mapping relationship.

[0084] Taking bridge crack monitoring as an example, suppose there is already a trained quality recognition model that can receive integrated data representation information as input and output the confidence that the target sensor monitoring sequence belongs to the four quality types of "no cracks", "minor cracks", "medium cracks" and "serious cracks". Now, the computer system receives the integrated data representation information of a new target sensor monitoring sequence, such as [25, 60, f1, f2, f3], where 25 represents the temperature value, 60 represents the humidity value, and f1, f2, and f3 represent vibration characteristics.

[0085] The computer system inputs this integrated data representation information into the quality identification model, and the model calculates the confidence that the target sensor monitoring sequence belongs to each quality type based on the mapping relationship that has been learned. For example, the model can output a vector [0.1, 0.3, 0.4, 0.2], which means that the confidence that the target sensor monitoring sequence belongs to "no crack" is 0.1, the confidence that it belongs to "minor crack" is 0.3, the confidence that it belongs to "medium crack" is 0.4, and the confidence that it belongs to "serious crack" is 0.2.

[0086] In step S1132, the task of the computer system is to generate a quality classification array of the target sensor monitoring sequence according to the confidence obtained in the previous step. The quality classification array is a vector whose length is equal to the number of quality types in the preset quality type set, and each element in the vector represents the confidence that the target sensor monitoring sequence belongs to a certain quality type.

[0087] Continuing with the example of bridge crack monitoring, in the previous step, the computer system has obtained the confidence that the target sensor monitoring sequence belongs to each quality type, that is, the vector [0.1, 0.3, 0.4, 0.2]. In this step, the computer system will directly use this vector as the quality classification array of the target sensor monitoring sequence. This array represents the possibility that the target sensor monitoring sequence belongs to each quality type, providing important reference information for subsequent crack monitoring and decision-making.

[0088] In practical applications, the training and debugging of quality identification models is a complex process that requires a large amount of bridge sensor monitoring data and professional machine learning technology. The following is a simplified example of quality identification model training and debugging:

[0089] Data preparation: Collect a large amount of bridge sensor monitoring data, including data from various sensors such as temperature, humidity, vibration, etc., and pre-process the data, such as cleaning and normalization. At the same time, invite professional bridge engineers to annotate these data and mark the quality type corresponding to each sensor monitoring sequence.

[0090] Model selection: Based on the characteristics of the data and the complexity of the problem, select a suitable machine learning model as the basis of the quality identification model, such as neural network, support vector machine, etc.

[0091] Model training: Use the prepared labeled data to train the model. During the training process, the model parameters and structure need to be adjusted continuously so that the model can better learn the mapping relationship between data and quality types. During the training process, cross-validation and other techniques are also needed to evaluate the performance of the model and select the model with the best performance as the final quality recognition model.

[0092] Model debugging: After model training is completed, the model needs to be debugged to ensure that the model output meets actual needs. For example, the output threshold of the model can be adjusted to control the model's sensitivity to different quality types.

[0093] Model deployment: The debugged quality identification model is deployed to the computer system for subsequent bridge crack monitoring.

[0094] During the training and debugging of the quality identification model, the computer system uses various machine learning algorithms and techniques to optimize the performance of the model. For example, the gradient descent algorithm can be used to optimize the parameters of the model, and regularization techniques can be used to prevent the model from overfitting. The specific implementation details of these algorithms and techniques depend on the selected machine learning model and the characteristics of the problem.

[0095] In summary, the implementation method of step S113 uses the quality recognition model that has been trained and debugged to infer the quality classification of the target sensor monitoring sequence based on the integrated data characterization information, and generates a corresponding quality classification array. This process realizes the automated processing and analysis of bridge sensor monitoring data, and provides strong technical support for bridge crack monitoring. In practical applications, the computer system will select appropriate quality recognition models and technologies to implement this process according to specific scenarios and requirements.

[0096] As an implementation mode, the present application can identify the arrangement pattern of the matrix units marked with significant features in the commonality metric value matrix based on the recognition model that has been debugged in advance. Wherein, the debugged recognition model can be obtained by debugging the recognition model using active training data and passive training data, the active training data can be sensor monitoring training data including a locked arrangement pattern (such as a trajectory consisting of multiple matrix units marked with significant features), the active training data can contain positive prior information (i.e., prior information characterizing that the sensor monitoring training data includes the locked arrangement pattern), the negative training data can be sensor monitoring training data that does not include the locked arrangement pattern (e.g., does not include a trajectory consisting of multiple matrix units marked with significant features), and the negative training data can contain negative prior information (i.e., prior information characterizing that the sensor monitoring training data does not include the locked arrangement pattern). The recognition model can identify whether each sensor monitoring training data includes a locking arrangement pattern (such as whether it includes a trajectory composed of matrix units marked with significant features) to obtain a recognition result. The recognition result can be the result of identifying that the sensor monitoring training data includes the locking arrangement pattern, and the identified position of the locking arrangement pattern in the sensor monitoring training data, or the recognition result can also be the result of identifying that the sensor monitoring training data does not include the locking arrangement pattern.

[0097] Then, the embodiment of the present application can adjust the model parameters of the recognition model based on the error between the recognition results of the recognition model for each sensor monitoring training data and the prior information contained in each sensor monitoring training data (for example, based on the cross entropy loss function), so that the recognition model can accurately identify that the positive training data includes the locking arrangement pattern and the position of the locking arrangement pattern in the positive training data, and the recognition model can accurately identify that the negative training data does not include the locking arrangement pattern. In this way, the model parameters of the recognition model are continuously iteratively adjusted. When the adjustment of the model parameters of the recognition model is completed, a debugged recognition model is obtained.

[0098] As an implementation mode, the bridge crack monitoring method for fusing multi-source sensor data provided by the present application may also include:

[0099] Step S10: performing a data splitting operation on the bridge multimodal sensor monitoring data set to obtain Q data items included in the bridge multimodal sensor monitoring data set, where Q>P;

[0100] Step S20: clustering the Q data items according to the data item values ​​to obtain a plurality of sensor monitoring sequences included in the bridge multimodal sensor monitoring data set;

[0101] Wherein, any sensor monitoring sequence of the bridge multimodal sensor monitoring data set includes a plurality of consecutive data items among the Q data items.

[0102] In step S10, the task of the computer system is to split the bridge multimodal sensor monitoring data set to obtain all data items contained in the data set. These data items may come from different sensors, such as temperature sensors, humidity sensors, vibration sensors, etc., and each data item contains the state information of the bridge at a certain moment.

[0103] Assume that the bridge multimodal sensor monitoring dataset is a large dataset that contains data from multiple sensors, which are collected at certain time intervals. In order to split this dataset, the computer system needs to first read the dataset file, and then split the data in the dataset into Q independent data items according to the data item division rule. Here, Q is an integer greater than P, indicating that the number of data items obtained after splitting is greater than the number of original sensor monitoring sequences.

[0104] For example, suppose a multimodal bridge sensor monitoring data set contains data from three sensors: temperature, humidity, and vibration. The data from each sensor is collected once per second. The computer system can split this data set into multiple data items, each of which contains the temperature value, humidity value, and vibration value at a certain moment. If the data set contains 1000 seconds of data, then the number of data items Q obtained after splitting is 3000, because there is data from three sensors at each moment.

[0105] In step S20, the task of the computer system is to cluster the Q data items obtained after the splitting according to the values ​​of the data items to form multiple sensor monitoring sequences. Each sensor monitoring sequence includes multiple continuous data items, and these data items have certain similarities in values.

[0106] Clustering operations can be implemented using various machine learning algorithms, such as the K-means algorithm, the DBSCAN algorithm, etc. These algorithms can divide data items into different groups based on their numerical features. The data items within each group have a high numerical similarity, while the data items between different groups have a low similarity.

[0107] For example, suppose that the computer system has used the K-means algorithm to cluster the 3,000 data items obtained after the split. The clustering results may show that these data items can be divided into 10 different sensor monitoring sequences, each of which contains 300 consecutive data items. These sensor monitoring sequences may represent the state changes of the bridge in different time periods, such as morning, noon, evening, etc.

[0108] In the specific implementation, the computer system needs to first select a suitable clustering algorithm and determine the clustering parameters, such as the number of clusters, distance measurement, etc. Then, the computer system uses the Q data items obtained after the splitting as input and runs the clustering algorithm to perform clustering operations. After the clustering is completed, the computer system will obtain multiple sensor monitoring sequences, each of which contains multiple continuous data items.

[0109] To illustrate the clustering process more concretely, consider a simple example using the K-means algorithm. Assume that the K-means algorithm has been selected and that the data items are to be clustered into 10 sensor monitoring sequences. The K-means algorithm works by first randomly selecting 10 data items as the initial cluster centers, then calculating the distance between each data item and these 10 cluster centers and assigning each data item to the closest cluster center. Next, the algorithm recalculates the center of each cluster by calculating the average of all the data items in the cluster and using this average as the new cluster center. This process is repeated until the cluster center no longer changes or the preset number of iterations is reached. In the end, 10 sensor monitoring sequences are obtained, each of which contains the data items that are closest to the corresponding cluster center.

[0110] In addition, the timestamp information of the data items can also be considered during the clustering process to ensure that the data items in each sensor monitoring sequence are continuous in time order. This can be achieved by adding a time constraint to the clustering algorithm, that is, only those data items with continuous timestamps are considered when assigning data items to cluster centers.

[0111] In summary, the implementation of step S10 and step S20 converts the bridge multimodal sensor monitoring data set into multiple sensor monitoring sequences through data splitting and clustering operations. These sensor monitoring sequences contain the state information of the bridge in different time periods, providing a basis for subsequent crack monitoring and data analysis. In practical applications, the computer system will select appropriate clustering algorithms and parameters according to specific scenarios and requirements to implement this process.

[0112] As an implementation mode, the quality classification array of any sensor monitoring sequence is a confidence distribution array constructed by the confidence that the structural quality corresponding to any sensor monitoring sequence belongs to each quality type. The step S120, determining the P commonality measurement values ​​according to the error between the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the quality classification array of each sensor monitoring sequence in the standard crack data, may include: determining the spatial spacing between the confidence distribution array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the confidence distribution array of each sensor monitoring sequence in the standard crack data, and obtaining P array commonality measurement values; wherein, an array commonality measurement value represents the commonality measurement result between the confidence distribution array of a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the confidence distribution array of a sensor monitoring sequence in the standard crack data.

[0113] In the scenario of bridge crack monitoring, the core task of step S120 is to determine P commonality metrics based on the error between the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the quality classification array of each sensor monitoring sequence in the standard crack data. These commonality metrics are used to quantify the similarity or consistency between the bridge multimodal sensor monitoring data set and the standard crack data, thereby providing an important basis for subsequent crack monitoring and evaluation.

[0114] Specifically, the quality classification array of any sensor monitoring sequence is a confidence distribution array constructed by the confidence that the structural quality corresponding to the sequence belongs to each quality type. This array reflects the distribution and confidence level of the sensor monitoring sequence on different quality types.

[0115] In step S120, the computer system calculates the spatial spacing between the confidence distribution array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the confidence distribution array of each sensor monitoring sequence in the standard crack data. This spatial spacing is used to measure the similarity or difference between the two confidence distribution arrays. By calculating P such spatial spacings, P array commonality measurement values ​​can be obtained, each value representing the commonality measurement result between the confidence distribution array of a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the confidence distribution array of a sensor monitoring sequence in the standard crack data.

[0116] For example, suppose there is a sensor monitoring sequence A in the bridge multimodal sensor monitoring data set, and its quality classification array (confidence distribution array) is [0.2, 0.5, 0.3], which means that the confidence of the sequence belonging to "no crack" is 0.2, the confidence of the sequence belonging to "minor crack" is 0.5, and the confidence of the sequence belonging to "serious crack" is 0.3. At the same time, there is a sensor monitoring sequence B in the standard crack data, and its quality classification array (confidence distribution array) is [0.1, 0.6, 0.3]. The computer system needs to calculate the spatial spacing between the confidence distribution arrays of sequence A and sequence B. A commonly used calculation method is to use the Euclidean distance formula, and the result obtained is about 0.14. This calculation result indicates that the spatial spacing between the confidence distribution arrays of sequence A and sequence B is about 0.14. In a similar way, the computer system can calculate the spatial spacing between each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and each sensor monitoring sequence in the standard crack data, thereby obtaining P array commonality measurement values.

[0117] These commonality metrics can be used to evaluate the consistency and similarity between the bridge multimodal sensor monitoring dataset and the standard crack data. For example, if a commonality metric is small, it means that the corresponding two confidence distribution arrays are close, that is, the sensor monitoring sequence in the bridge multimodal sensor monitoring dataset and the sensor monitoring sequence in the standard crack data have a high similarity in quality classification; conversely, if the commonality metric is large, it means that there is a large difference in quality classification between the two. These commonality metrics provide important quantitative basis for subsequent crack monitoring, data analysis and decision-making.

[0118] As an implementation manner, the step S200 of generating a commonality measurement value matrix according to the P commonality measurement values ​​may include:

[0119] Step S210: Obtain a two-dimensional matrix diagram constructed based on a first time axis and a second time axis; the two-dimensional matrix diagram includes P elements, each element of the two-dimensional matrix diagram on the first time axis corresponds to each sensor monitoring sequence in the bridge multimodal sensor monitoring data set, each element of the two-dimensional matrix diagram on the second time axis corresponds to each sensor monitoring sequence in the standard crack data, and a commonality metric value corresponds to one element in the two-dimensional matrix diagram.

[0120] In the scenario of bridge crack monitoring, step S210 is a key step in generating a commonality metric matrix. This step requires the computer system to obtain a two-dimensional matrix diagram based on the first time axis and the second time axis. This two-dimensional matrix diagram is the basis for generating a commonality metric matrix in subsequent steps.

[0121] Specifically, the computer system first determines the size of the two-dimensional matrix graph, that is, how many elements it contains. Since P commonality metrics have been calculated before, the two-dimensional matrix graph will be a P×P matrix containing P elements. The positions of these elements in the matrix are determined by the first time axis and the second time axis.

[0122] On the first time axis, each element of the two-dimensional matrix diagram has a corresponding relationship with each sensor monitoring sequence in the bridge multimodal sensor monitoring data set. This means that a row in the matrix represents a sensor monitoring sequence in the bridge multimodal sensor monitoring data set. Similarly, on the second time axis, each element of the two-dimensional matrix diagram has a corresponding relationship with each sensor monitoring sequence in the standard crack data. Therefore, a column in the matrix represents a sensor monitoring sequence in the standard crack data.

[0123] Through such a correspondence, an element in the two-dimensional matrix diagram corresponds to a commonality metric value, which characterizes the degree of commonality between a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and a sensor monitoring sequence in the standard crack data. In other words, the two-dimensional matrix diagram is a visual representation of the commonality metric value, which shows the commonality relationship between the bridge multimodal sensor monitoring data set and the standard crack data in each sensor monitoring sequence.

[0124] For example, suppose the bridge multimodal sensor monitoring dataset has 4 sensor monitoring sequences, labeled A, B, C, and D; the standard crack data also has 4 sensor monitoring sequences, labeled 1, 2, 3, and 4. Then, P is equal to 4, and the two-dimensional matrix diagram is a 4×4 matrix. In this matrix, the element (A, 1) represents the commonality metric between sensor monitoring sequence A in the bridge multimodal sensor monitoring dataset and sensor monitoring sequence 1 in the standard crack data. Similarly, the element (B, 2) represents the commonality metric between sensor monitoring sequence B and sensor monitoring sequence 2, and so on.

[0125] In this way, the computer system can construct a two-dimensional matrix diagram that contains all the common metric information and visually displays the common relationship between the bridge multimodal sensor monitoring data set and the standard crack data in each sensor monitoring sequence. This two-dimensional matrix diagram is the basis for generating the common metric matrix in the subsequent steps. It provides the necessary information and structure so that the computer system can further analyze and process the data to draw useful conclusions about bridge crack monitoring.

[0126] Step S220: In the two-dimensional matrix diagram, the elements corresponding to the commonality measurement values ​​that meet the preset commonality requirements are displayed with the significant feature marks, and the elements corresponding to the commonality measurement values ​​that do not meet the preset commonality requirements are displayed in a display method other than the significant feature marks, to obtain the commonality measurement value matrix; wherein, the displayed element corresponding to a commonality measurement value belongs to the matrix unit corresponding to the commonality measurement value in the commonality measurement value matrix, and the locked arrangement pattern included in the commonality measurement value matrix represents the shape pattern of the element trajectory composed of multiple elements after being displayed with the significant feature marks.

[0127] In the scenario of bridge crack monitoring, step S220 is one of the key steps to generate a commonality measurement value matrix. This step requires the computer system to specifically mark and display the commonality measurement values ​​in a two-dimensional matrix diagram according to preset commonality requirements, so as to obtain the final commonality measurement value matrix.

[0128] Specifically, the computer system first traverses each element in the two-dimensional matrix diagram, which represents the commonality measurement value between the bridge multimodal sensor monitoring data set and the standard crack data in each sensor monitoring sequence. For each element, the computer system will determine whether its corresponding commonality measurement value meets the preset commonality requirements.

[0129] This preset commonality requirement is a threshold, which represents the minimum metric value that the computer system considers to have sufficient commonality between two sensor monitoring sequences. This threshold can be set according to the actual application scenario and requirements. For example, in the scenario of bridge crack monitoring, a relatively high threshold can be set to ensure that only those very similar sensor monitoring sequences are considered to have commonality.

[0130] For commonality metrics that meet the preset commonality requirements, the computer system will display the corresponding elements with significant feature marks. This significant feature mark can be a color, shape or other visual mark, which can make these elements stand out in the commonality metric matrix for subsequent observation and analysis. For example, red can be selected as a significant feature mark, and all elements that meet the preset commonality requirements are marked in red.

[0131] For commonality metrics that do not meet the preset commonality requirements, the computer system will display the corresponding elements in a display method other than the significant feature mark. This means that these elements will not be highlighted with significant feature marks in the commonality metric matrix, but will be represented by other colors or shapes. For example, blue can be selected as the mark color for elements that do not meet the preset commonality requirements.

[0132] In this way, the computer system can obtain a commonality metric matrix, which contains all the commonality metric information and visually shows which sensor monitoring sequences have enough commonality and which sensor monitoring sequences do not have enough commonality. This commonality metric matrix is ​​an important basis for crack monitoring and data analysis in subsequent steps.

[0133] In addition, the commonality metric matrix also includes a locking arrangement pattern, which represents the shape pattern of the element trajectory composed of multiple elements after being displayed with significant feature markers. This shape pattern can reflect the overall distribution and trend of the commonality relationship between the multimodal bridge sensor monitoring data set and the standard crack data in each sensor monitoring sequence. By observing and analyzing this shape pattern, we can have a more comprehensive and in-depth understanding of the crack situation of the bridge, providing strong support for subsequent maintenance and repair work.

[0134] As an implementation mode, in step S200, the judgment criteria for the commonality metric value that meets the preset commonality requirements are as follows: a commonality metric value that is not less than a commonality metric reference value is regarded as a commonality metric value that meets the preset commonality requirements; a commonality metric value that is less than the commonality metric reference value is regarded as a commonality metric value that does not meet the preset commonality requirements.

[0135] During the implementation of step S200, the computer system determines which commonality metric values ​​meet the preset commonality requirements. To this end, the system sets a commonality metric reference value as a judgment standard. Specifically, the computer system will regard commonality metric values ​​that are not less than this commonality metric reference value as commonality metric values ​​that meet the preset commonality requirements; and will regard commonality metric values ​​that are less than this commonality metric reference value as commonality metric values ​​that do not meet the preset commonality requirements.

[0136] For example, in the scenario of bridge crack monitoring, suppose the computer system has calculated 10 commonality metrics, which represent the degree of commonality between the sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the sensor monitoring sequence in the standard crack data. These commonality metrics may range from 0 to 1, where 1 represents complete commonality and 0 represents no commonality.

[0137] If the commonality metric reference value set by the computer system is 0.5, then among the 10 commonality metric values, all commonality metric values ​​greater than or equal to 0.5 will be regarded as commonality metric values ​​that meet the preset commonality requirements. These values ​​may represent sensor monitoring sequences in the bridge multimodal sensor monitoring dataset that have significant commonality with sensor monitoring sequences in the standard crack data.

[0138] In contrast, all commonality metric values ​​less than 0.5 are considered as commonality metric values ​​that do not meet the preset commonality requirements. These values ​​may represent sensor monitoring sequences in the bridge multimodal sensor monitoring dataset that have a low degree of commonality with the sensor monitoring sequences in the standard crack data.

[0139] Through such judgment criteria, the computer system can clearly identify which commonality metric values ​​meet the preset commonality requirements and which do not. This is very important for the subsequent generation of commonality metric value matrix and crack monitoring and data analysis. Because only commonality metric values ​​that meet the preset commonality requirements are considered meaningful, they can reflect the actual commonality relationship between the bridge multimodal sensing monitoring data set and the standard crack data.

[0140] As another implementation of step S200, that is, generating a commonality measurement value matrix according to the P commonality measurement values, may include:

[0141] Step S210': feature-image each of the P commonality metrics to obtain a reference image node for each commonality metric value; a commonality metric value that meets the preset commonality requirement represents a commonality metric value whose value of the reference image node meets the preset commonality requirement.

[0142] Step S210' is the process of the computer system performing feature image conversion on P commonality metric values. In this process, the computer system converts each commonality metric value into an image containing multiple image nodes, and this image is the reference image node of the commonality metric value. The value in the reference image node will reflect the characteristics of the commonality metric value. If a commonality metric value meets the preset commonality requirements, then the value of its reference image node will also meet the corresponding preset conditions.

[0143] For example, in the scenario of bridge crack monitoring, the computer system has calculated P commonality metrics, which respectively represent the degree of commonality between the sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the sensor monitoring sequence in the standard crack data. Now, the computer system needs to perform feature image processing on these commonality metrics.

[0144] For each commonality metric, the computer system generates a reference image node. This node is an image containing multiple image nodes, each of which represents a feature of the commonality metric. For example, if the commonality metric represents the width of a crack, then one of the reference image nodes may represent the specific value of the crack width.

[0145] After the computer system generates the reference image nodes of all commonality metric values, it can determine which commonality metric values ​​meet the preset commonality requirements by checking the values ​​of these nodes. For example, if the preset commonality requirement is that the crack width is greater than a certain threshold, then the computer system will check the values ​​of the image nodes representing the crack width in all reference image nodes, and find those nodes that are greater than the threshold. The commonality metric values ​​corresponding to these nodes are the commonality metric values ​​that meet the preset commonality requirements.

[0146] Step S220': Acquire a plane constructed according to the first time axis and the second time axis; the plane includes P node units, each node unit of the plane on the first time axis has a corresponding relationship with each sensor monitoring sequence in the bridge multimodal sensor monitoring data set, each node unit of the plane on the second time axis has a corresponding relationship with each sensor monitoring sequence in the standard crack data, and a commonality metric value corresponds to a node unit in the plane.

[0147] Step S220' is a process in which the computer system obtains a plane composed of a first time axis and a second time axis. The plane is divided into P node units, each of which has a corresponding relationship with a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and a sensor monitoring sequence in the standard crack data. In this way, each node unit in the plane can represent a commonality metric value, indicating the degree of commonality between a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and a sensor monitoring sequence in the standard crack data.

[0148] Specifically, in the scenario of bridge crack monitoring, the computer system has obtained a bridge multimodal sensor monitoring data set and standard crack data, each of which contains multiple sensor monitoring sequences. In order to intuitively display the commonality between these sensor monitoring sequences, the computer system constructs a two-dimensional plane, in which the first time axis represents the sensor monitoring sequence in the bridge multimodal sensor monitoring data set, and the second time axis represents the sensor monitoring sequence in the standard crack data.

[0149] This plane is divided into P node units, each of which corresponds to a specific commonality metric value. For example, if the bridge multimodal sensor monitoring dataset has 5 sensor monitoring sequences and the standard crack data also has 5 sensor monitoring sequences, then the plane will be divided into 25 node units. The node unit at position (i, j) on the plane represents the commonality metric value between the i-th sensor monitoring sequence in the bridge multimodal sensor monitoring dataset and the j-th sensor monitoring sequence in the standard crack data.

[0150] In this way, the computer system can intuitively display the commonality relationship between the multimodal bridge sensor monitoring data set and the standard crack data on this plane. The position and color (or value) of each node unit represents the characteristics and size of the corresponding commonality metric value, allowing users to understand at a glance which sensor monitoring sequences have significant commonalities and the extent of these commonalities.

[0151] Step S230 ′: Obtain the RGB values ​​corresponding to the reference image nodes of each commonality metric value.

[0152] Step S230' is the process of the computer system obtaining the RGB values ​​corresponding to the reference image nodes of each commonality metric value. In this process, the computer system assigns a specific RGB value to the reference image node according to the characteristics of each commonality metric value. The RGB value is a color coding method that represents various colors through different combinations of red (R), green (G), and blue (B).

[0153] In the scenario of bridge crack monitoring, each commonality metric value represents the degree of commonality between a sensor monitoring sequence in the bridge multimodal sensor monitoring dataset and a sensor monitoring sequence in the standard crack data. In order to intuitively display the characteristics of these commonality metrics, the computer system assigns an RGB value to the control image node of each commonality metric value.

[0154] For example, the computer system can set a rule to assign RGB values ​​according to the size of the commonality metric value. If the commonality metric value is large, it means that the commonality between the two sensor monitoring sequences is high, so a brighter color, such as red (higher R value, lower G value and B value) can be assigned to its control image node. If the commonality metric value is small, it means that the commonality between the two sensor monitoring sequences is low, so a darker color, such as blue (higher B value, lower R value and G value) can be assigned to its control image node.

[0155] In this way, the computer system can assign a specific RGB value to each commonality metric comparison image node, which visually represents the characteristics of the commonality metric. In subsequent steps, the computer system will assign these RGB values ​​to the node units in the commonality metric matrix, thereby generating an intuitive, colorful commonality metric matrix, allowing users to understand at a glance the commonality relationship between the bridge multimodal sensor monitoring data set and the standard crack data.

[0156] Step S240': assign the RGB value corresponding to the control image node of each common metric value to the node unit corresponding to each common metric value in the plane, respectively, to obtain the common metric value matrix; wherein, the node unit corresponding to a common metric value after the RGB value is assigned belongs to the matrix unit corresponding to the common metric value in the common metric value matrix, the significant feature mark represents the marking method displayed as the target RGB value, and the locked arrangement pattern included in the common metric value matrix represents the shape pattern of the element trajectory formed by multiple node units assigned the target RGB value.

[0157] Step S240' is the process in which the computer system assigns RGB values ​​to the node units corresponding to each commonality metric value in the plane, thereby obtaining a commonality metric value matrix. In this process, the computer system assigns corresponding colors to each node unit in the plane according to the RGB values ​​corresponding to the reference image nodes of each commonality metric value, and finally forms a colorful commonality metric value matrix.

[0158] Specifically, in the scenario of bridge crack monitoring, the computer system has obtained the RGB value corresponding to each commonality metric value of the control image node, and has constructed a plane consisting of a first time axis and a second time axis, and each node unit in the plane has a corresponding relationship with a commonality metric value. Now, the computer system needs to assign a corresponding RGB value to each node unit in this plane to form a colored commonality metric value matrix.

[0159] The computer system will traverse each node unit in the plane. For each node unit, the computer system will find its corresponding commonality metric value and obtain the RGB value corresponding to the reference image node of the commonality metric value. Then, the computer system will assign this RGB value to the node unit so that it displays the corresponding color. For example, if a commonality metric value represents the commonality of crack width between a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and a sensor monitoring sequence in the standard crack data, and the RGB value corresponding to its reference image node is red (R=255, G=0, B=0), then the computer system will assign red to the node unit corresponding to this commonality metric value in the commonality metric value matrix.

[0160] In this way, the computer system can assign corresponding RGB values ​​to each node unit in the plane, and finally form a colored commonality metric value matrix. In this matrix, the color of each node unit represents the characteristics of its corresponding commonality metric value. Users can intuitively understand the commonality relationship between the bridge multimodal sensor monitoring data set and the standard crack data by observing this matrix.

[0161] In addition, the significant feature markers in the commonality metric matrix are presented by the shape pattern of the element trajectory composed of multiple node units assigned the target RGB value. For example, if the color of the node unit corresponding to a commonality metric in the matrix forms a specific shape or pattern, then this shape or pattern can be regarded as a significant feature marker, which represents the commonality between the bridge multimodal sensor monitoring dataset and the standard crack data in a certain aspect. Users can observe these significant feature markers to gain a deeper understanding of the monitoring of bridge cracks, thereby making more accurate judgments and decisions.

[0162] As an implementation mode, the matrix units of the significant feature marks constituting the locking arrangement pattern in the commonality metric value matrix include T matrix units sequentially displayed with the significant feature marks in the commonality metric value matrix, T>1. Based on this, in the step S400, determining the position of the standard crack data in the bridge multimodal sensing monitoring data set according to the matrix units of the significant feature marks constituting the locking arrangement pattern may include:

[0163] Step S410: Obtain the positions of the first sensor monitoring sequence and the second sensor monitoring sequence in the multimodal sensor monitoring data set of the bridge, respectively; the first sensor monitoring sequence represents the sensor monitoring sequence of the commonality metric values ​​corresponding to the initial matrix unit in the multimodal sensor monitoring data set of the bridge, the initial matrix unit represents the first matrix unit among the T matrix units, the second sensor monitoring sequence represents the sensor monitoring sequence of the commonality metric values ​​corresponding to the ending matrix unit in the multimodal sensor monitoring data set of the bridge, and the ending matrix unit represents the last matrix unit among the T matrix units; the standard crack data in the multimodal sensor monitoring data set of the bridge is constructed by the sensor monitoring sequence between the first sensor monitoring sequence and the second sensor monitoring sequence in the multimodal sensor monitoring data set of the bridge.

[0164] Step S410 is a key step for the computer system to obtain the positions of the first sensor monitoring sequence and the second sensor monitoring sequence in the bridge multimodal sensor monitoring data set when performing the bridge crack monitoring task. This process is crucial for the subsequent determination of the position of the standard crack data in the data set because it establishes a direct connection between the significant feature markers in the commonality metric matrix and the actual sensor monitoring sequence.

[0165] The computer system has constructed a commonality metric matrix that quantifies the degree of commonality between the sensor monitoring sequences in the bridge multimodal sensor monitoring data set and the sensor monitoring sequences in the standard crack data set by comparing them. Each node unit in this matrix corresponds to a commonality metric value, which represents the strength of commonality between a specific pair of sensor monitoring sequences. In addition, the computer system also identifies significant feature markers in the matrix, which are composed of a series of matrix units arranged in order, which appear in a specific shape or pattern, representing a certain important commonality relationship between the bridge multimodal sensor monitoring data set and the standard crack data. Now, let's explain step S410 in detail. The task of the computer system is to determine the position of the first sensor monitoring sequence and the second sensor monitoring sequence in the bridge multimodal sensor monitoring data set. These two sequences correspond to the initial matrix unit and the end matrix unit in the commonality metric matrix, respectively. The initial matrix unit is the first of the T matrix units that constitute the significant feature markers, and the end matrix unit is the last of these T matrix units.

[0166] In the scenario of bridge crack monitoring, the sensor monitoring sequence may include data from different sensors, such as temperature sensors, humidity sensors, sound sensors, etc. These sensors continuously collect information about the state of the bridge and generate corresponding sensor monitoring sequences. The computer system needs to analyze these sequences to detect possible cracks in the bridge. Assume that the significant feature markers in the commonality measure matrix form a diagonal line from the upper left corner to the lower right corner. This means that the initial matrix unit may correspond to the first sensor monitoring sequence in the multimodal sensor monitoring dataset of the bridge, such as data from the temperature sensor. The ending matrix unit may correspond to the last sensor monitoring sequence in the dataset, such as data from the sound sensor.

[0167] To obtain the positions of these two sequences in the data set, the computer system needs to do the following:

[0168] Identify the salient signatures in the commonality measure matrix. This typically involves visual analysis of the matrix or using image processing algorithms to identify specific shapes or patterns. Once the salient signature is identified, the computer system is able to determine the T matrix cells that make up the signature.

[0169] Determine the location of the initial matrix cell and the ending matrix cell in the commonality measure matrix. This is done by traversing the matrix and finding the start and end points of the significant feature markers. The start point corresponds to the initial matrix cell, and the end point corresponds to the ending matrix cell.

[0170] The locations of the initial matrix cells and the ending matrix cells are mapped into the multimodal sensor monitoring data set of the bridge. This typically involves finding identifiers or indices of the sensor monitoring sequences corresponding to the matrix cells. Once these identifiers or indices are found, the computer system is able to determine the exact locations of the first sensor monitoring sequence and the second sensor monitoring sequence in the data set.

[0171] All sensor monitoring sequences between the first sensor monitoring sequence and the second sensor monitoring sequence are extracted. These sequences constitute a subset of the bridge multimodal sensor monitoring data set, and they have some commonality with the standard crack data. The computer system will use these sequences to construct standard crack data for further crack monitoring and evaluation.

[0172] By performing the above operations, the computer system can accurately obtain the positions of the first sensor monitoring sequence and the second sensor monitoring sequence in the bridge multimodal sensor monitoring data set. This is a key step in determining the position of the standard crack data, because the standard crack data is constructed by fusing all the sensor monitoring sequences between the two sequences.

[0173] For example, suppose a multimodal bridge sensor monitoring dataset contains sensor monitoring sequences from five different sensors: temperature sensor, humidity sensor, pressure sensor, vibration sensor, and sound sensor. The computer system has constructed a commonality measure matrix and identified the significant feature signatures in it. This signature consists of a series of node cells in the matrix that form a diagonal line from the upper left corner to the lower right corner.

[0174] In step S410, the computer system determines that the starting point and the end point of the diagonal line correspond to the temperature sensor sequence and the sound sensor sequence, respectively. This means that the first sensing monitoring sequence is the temperature sensor sequence, which is the first one in the data set; and the second sensing monitoring sequence is the sound sensor sequence, which is the last one in the data set.

[0175] Once the locations of the two sequences are determined, the computer system will be able to extract all the sensor monitoring sequences between them: humidity sensor sequences, pressure sensor sequences, and vibration sensor sequences. These sequences will have some commonality with the standard crack data because they are all located between the first sensor monitoring sequence and the second sensor monitoring sequence. The computer system will use these sequences to construct the standard crack data and conduct further crack monitoring and evaluation work.

[0176] Step S420: determining the position of the standard crack data in the bridge multimodal sensor monitoring data set according to the positions of the first sensor monitoring sequence and the second sensor monitoring sequence in the bridge multimodal sensor monitoring data set respectively.

[0177] Step S420 is a key step for the computer system to determine the position of the standard crack data according to the position of the first sensor monitoring sequence and the second sensor monitoring sequence in the bridge multimodal sensor monitoring data set when performing the bridge crack monitoring task. This process involves the integration and analysis of the sensor monitoring sequence to construct standard crack data that can represent the characteristics of bridge cracks.

[0178] In step S410, the computer system has successfully obtained the positions of the first sensor monitoring sequence and the second sensor monitoring sequence in the bridge multimodal sensor monitoring data set. The two sequences correspond to the initial matrix unit and the end matrix unit in the commonality metric matrix, respectively, and all the sensor monitoring sequences between them have a certain commonality relationship with the standard crack data.

[0179] Now, the task of the computer system is to determine the location of the standard crack data in the bridge multimodal sensor monitoring data set based on the location of these two sequences. This usually involves the following steps:

[0180] All sensor monitoring sequences between the first sensor monitoring sequence and the second sensor monitoring sequence are extracted. These sequences constitute a subset of the bridge multimodal sensor monitoring dataset, which contains all the information related to the standard crack data.

[0181] Preprocess and integrate these sensor monitoring sequences. Preprocessing may include steps such as data cleaning, denoising, and normalization to ensure data accuracy and consistency. Integration involves merging these sequences into a comprehensive data set for subsequent analysis and processing.

[0182] Apply machine learning models or algorithms to analyze and process the integrated sensor monitoring sequence. These models or algorithms may include deep learning neural networks, support vector machines, random forests, etc., which can extract useful features from the data and use them to construct standard crack data.

[0183] Based on the output of the machine learning model or algorithm, determine the location of the standard crack data in the multimodal sensor monitoring data set of the bridge. This usually involves interpreting and analyzing the model output to find the sensor monitoring sequence or feature that is most correlated with the standard crack data.

[0184] For example, assuming that the bridge multimodal sensor monitoring data set includes sensor monitoring sequences from five different sensors: temperature sensor, humidity sensor, pressure sensor, vibration sensor and sound sensor. In step S410, the computer system has determined that the first sensor monitoring sequence is a temperature sensor sequence and the second sensor monitoring sequence is a sound sensor sequence.

[0185] In step S420, the computer system will extract all sensor monitoring sequences between the two sequences: humidity sensor sequence, pressure sensor sequence and vibration sensor sequence. Then, it will pre-process and integrate these sequences to form a comprehensive data set.

[0186] Next, the computer system will apply a deep learning neural network to analyze this comprehensive data set. This neural network has been trained to extract features related to bridge cracks from the sensor monitoring sequences. By analyzing this data set, the neural network will output a feature vector that represents the sensor monitoring sequences or features that are most related to the standard crack data.

[0187] Finally, the computer system will determine the location of the standard crack data in the bridge multimodal sensor monitoring data set based on the output of the neural network. This location may correspond to a specific sensor monitoring sequence or a set of related features that best represent the characteristics and status of bridge cracks. Through this process, the computer system can accurately locate the standard crack data in the bridge multimodal sensor monitoring data set and conduct further crack monitoring and evaluation work.

[0188] According to another aspect of the present application, a bridge crack monitoring device is also provided. Figure 3 , the device 300 comprises:

[0189] The data commonality determination module 310 is used to obtain P commonality measurement values ​​between the bridge multimodal sensor monitoring data set and the standard crack data, wherein P≥2, the bridge multimodal sensor monitoring data set and the standard crack data both include multiple sensor monitoring sequences, and one commonality measurement value represents the commonality measurement result between a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and a sensor monitoring sequence in the standard crack data;

[0190] A commonality matrix generating module 320 is used to generate a commonality measurement value matrix according to the P commonality measurement values; one commonality measurement value corresponds to one matrix unit in the commonality measurement value matrix, and the matrix unit corresponding to the commonality measurement value that meets the preset commonality requirement is a significant feature mark in the commonality measurement value matrix;

[0191] An arrangement pattern recognition module 330, used to recognize an arrangement pattern of the matrix units of the significant feature marks in the commonality metric value matrix;

[0192] The crack position determination module 340 is used to determine the position of the standard crack data in the bridge multimodal sensing monitoring data set based on the matrix units of the significant feature marks constituting the locking arrangement pattern if the commonality metric value matrix includes a locking arrangement pattern; the commonality metric value matrix includes the locking arrangement pattern to characterize that the bridge multimodal sensing monitoring data set includes the standard crack data.

[0193] According to an embodiment of the present application, a computer system is also provided. Figure 4 , is a block diagram of the computer system 120 of the present application, which includes a computing unit 121, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory, namely a ROM 122, or a computer program loaded from a storage unit 128 to a random access memory, namely a RAM 123. In the RAM 123, various programs and data required for the operation of the computer system 120 can also be stored. The computing unit 121, the ROM 122, and the RAM 123 are connected to each other via a bus 124. An input / output interface, namely an I / O interface 125, is also connected to the bus 124.

[0194] A number of components in the computer system 120 are connected to the I / O interface 125, including: an input unit 126, an output unit 127, a storage unit 128, and a communication unit 129. The input unit 126 may be any type of device capable of inputting information into the computer system 120, the input unit 126 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the computer system, and may include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 127 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 128 may include but is not limited to a disk, an optical disk. The communication unit 129 allows the computer system 120 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0195] The computing unit 121 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 121 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 121 performs the various methods and processes described above, such as methods. For example, in some embodiments, the method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 128. In some embodiments, part or all of the computer program may be loaded and / or installed on the computer system 120 via ROM 122 and / or a communication unit 129. When the computer program is loaded into RAM 123 and executed by the computing unit 121, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 121 may be configured to execute the method in any other appropriate manner (e.g., by means of firmware).

[0196] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0197] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0198] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0199] Although the embodiments or examples of the present application have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but is only limited by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present application. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present application.

Claims

1. A bridge crack monitoring method integrating multi-source sensor data, characterized in that: The method comprises: Obtaining P commonality metrics between a bridge multimodal sensor monitoring data set and standard crack data, wherein P≥2, the bridge multimodal sensor monitoring data set and the standard crack data both include multiple sensor monitoring sequences, and one commonality metric value represents a commonality metric result between a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and a sensor monitoring sequence in the standard crack data; Generate a commonality measurement value matrix according to the P commonality measurement values; one commonality measurement value corresponds to one matrix unit in the commonality measurement value matrix, and the matrix unit corresponding to the commonality measurement value that meets the preset commonality requirement is a significant feature mark in the commonality measurement value matrix; Identifying an arrangement pattern of matrix elements of the significant feature markers in the commonality metric matrix; If the commonality metric value matrix includes a locking arrangement pattern, the position of the standard crack data is determined in the bridge multimodal sensing monitoring data set based on the matrix units of the significant feature markers that constitute the locking arrangement pattern; the commonality metric value matrix includes the locking arrangement pattern to characterize that the bridge multimodal sensing monitoring data set includes the standard crack data.

2. The method according to claim 1, characterized in that The obtaining of P commonality metrics between the bridge multimodal sensing monitoring data set and the standard crack data includes: Obtaining a quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set, and obtaining a quality classification array of each sensor monitoring sequence in the standard crack data; the quality classification array of any sensor monitoring sequence represents the confidence that the structural quality corresponding to any sensor monitoring sequence belongs to each quality type in the preset quality type set; The P commonality metrics are determined according to the errors between the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the quality classification array of each sensor monitoring sequence in the standard crack data.

3. The method according to claim 2, characterized in that Any sensor monitoring sequence in the bridge multimodal sensor monitoring data set is a target sensor monitoring sequence, and the target sensor monitoring sequence includes a plurality of data items obtained after performing a data splitting operation on the bridge multimodal sensor monitoring data set; The step of obtaining a quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set includes: Extracting representation information of each data item included in the target sensing monitoring sequence to obtain data representation information of each data item included in the target sensing monitoring sequence; Integrating the representation information of a plurality of data representation information of a plurality of data items included in the target sensing monitoring sequence to generate integrated data representation information of the target sensing monitoring sequence; The quality classification of the target sensing monitoring sequence is inferred based on the integrated data characterization information to obtain a quality classification array of the target sensing monitoring sequence.

4. The method according to claim 3, characterized in that The method of inferring the quality classification of the target sensing monitoring sequence based on the integrated data characterization information to obtain a quality classification array of the target sensing monitoring sequence includes: The quality identification model completed through debugging infers the confidence level that the structural quality corresponding to the target sensing monitoring sequence belongs to each quality type based on the integrated data characterization information; A quality classification array of the target sensing monitoring sequence is generated according to the confidence that the structural quality corresponding to the target sensing monitoring sequence belongs to each quality type.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Performing a data splitting operation on the bridge multimodal sensor monitoring data set to obtain Q data items included in the bridge multimodal sensor monitoring data set, where Q>P; Clustering the Q data items according to data item values ​​to obtain a plurality of sensor monitoring sequences included in the bridge multimodal sensor monitoring data set; Wherein, any sensor monitoring sequence of the bridge multimodal sensor monitoring data set includes a plurality of consecutive data items among the Q data items.

6. The method according to claim 2, characterized in that The quality classification array of any sensing monitoring sequence is a confidence distribution array constructed by the confidence that the structural quality corresponding to any sensing monitoring sequence belongs to each quality type; The determining of the P commonality metrics according to the error between the quality classification array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the quality classification array of each sensor monitoring sequence in the standard crack data comprises: The spatial spacing between the confidence distribution array of each sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the confidence distribution array of each sensor monitoring sequence in the standard crack data is determined to obtain P array commonality measurement values; wherein an array commonality measurement value represents the commonality measurement result between the confidence distribution array of a sensor monitoring sequence in the bridge multimodal sensor monitoring data set and the confidence distribution array of a sensor monitoring sequence in the standard crack data.

7. The method according to claim 1, characterized in that Generating a commonality metric value matrix according to the P commonality metric values ​​includes: Acquire a two-dimensional matrix diagram formed according to a first time axis and a second time axis; the two-dimensional matrix diagram includes P elements, each element of the two-dimensional matrix diagram on the first time axis has a corresponding relationship with each sensor monitoring sequence in the bridge multimodal sensor monitoring data set, each element of the two-dimensional matrix diagram on the second time axis has a corresponding relationship with each sensor monitoring sequence in the standard crack data, and one commonality metric value corresponds to one element in the two-dimensional matrix diagram; In the two-dimensional matrix diagram, the elements corresponding to the commonality measurement values ​​that meet the preset commonality requirements are displayed with the significant feature marks, and the elements corresponding to the commonality measurement values ​​that do not meet the preset commonality requirements are displayed in a display method other than the significant feature marks, to obtain the commonality measurement value matrix; Among them, the displayed element corresponding to a common measurement value belongs to the matrix unit corresponding to the common measurement value in the common measurement value matrix, and the locked arrangement pattern included in the common measurement value matrix represents the shape pattern of the element trajectory composed of multiple elements after being displayed with the significant feature mark.

8. The method according to claim 7, characterized in that The method further comprises: Taking a commonality metric value that is not less than a commonality metric reference value as a commonality metric value that meets the preset commonality requirement; The commonality metric value that is smaller than the commonality metric reference value is regarded as the commonality metric value that does not meet the preset commonality requirement.

9. The method according to claim 1, characterized in that: Generating a commonality metric value matrix according to the P commonality metric values ​​includes: Performing feature imaging on each of the P commonality metric values ​​to obtain a reference image node for each commonality metric value; the commonality metric value that meets the preset commonality requirement represents the commonality metric value whose value of the reference image node meets the preset commonality requirement; Acquire a plane formed according to a first time axis and a second time axis; the plane includes P node units, each node unit of the plane on the first time axis has a corresponding relationship with each sensor monitoring sequence in the bridge multimodal sensor monitoring data set, each node unit of the plane on the second time axis has a corresponding relationship with each sensor monitoring sequence in the standard crack data, and one commonality metric value corresponds to one node unit in the plane; Obtain the RGB values ​​corresponding to the reference image nodes of each commonality metric value; Assigning the node units corresponding to each commonality metric value in the plane to the RGB values ​​corresponding to the control image nodes of each commonality metric value to obtain the commonality metric value matrix; The node unit after the RGB value is assigned to a commonality metric value belongs to the matrix unit corresponding to the commonality metric value in the commonality metric value matrix, the significant feature mark represents the marking method displayed as the target RGB value, and the locked arrangement pattern included in the commonality metric value matrix represents the shape pattern of the element trajectory formed by multiple node units assigned the target RGB value; The matrix units of the significant feature marks constituting the locking arrangement pattern in the commonality metric value matrix include T matrix units sequentially displayed with the significant feature marks in the commonality metric value matrix, where T>1; the position of the standard crack data in the bridge multimodal sensing monitoring data set is determined based on the matrix units of the significant feature marks constituting the locking arrangement pattern, including: Obtaining the positions of the first sensor monitoring sequence and the second sensor monitoring sequence in the multimodal sensor monitoring data set of the bridge, respectively; the first sensor monitoring sequence represents the sensor monitoring sequence of the commonality metric values ​​corresponding to the initial matrix unit in the multimodal sensor monitoring data set of the bridge, the initial matrix unit represents the first matrix unit among the T matrix units, the second sensor monitoring sequence represents the sensor monitoring sequence of the commonality metric values ​​corresponding to the ending matrix unit in the multimodal sensor monitoring data set of the bridge, the ending matrix unit represents the last matrix unit among the T matrix units; the standard crack data in the multimodal sensor monitoring data set of the bridge is constructed by the sensor monitoring sequence between the first sensor monitoring sequence and the second sensor monitoring sequence in the multimodal sensor monitoring data set of the bridge; The position of the standard crack data in the bridge multimodal sensing monitoring data set is determined according to the positions of the first sensing monitoring sequence and the second sensing monitoring sequence in the bridge multimodal sensing monitoring data set.

10. A computer system, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any one of claims 1 to 9.

Images