A multi-modal bridge data uploading method, device, equipment and medium
By calculating the collision probability and hash value for data sequence processing, the problem of poor rationality of multimodal bridge data upload is solved, the data integrity and upload stability are improved, and the redundancy and noise impact is reduced.
Patent Information
- Application Number
- CN202411166708.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-23
AI Technical Summary
The existing multimodal bridge data upload method has poor rationality, resulting in data loss, increased complexity of noise data, high network bandwidth usage and system stability during data collection and transmission.
By calculating the collision probability between data sequences, obtaining the initial data sequence, computing the hash value for missing sequence completion, and using the hash value to predict the exception level, filtering out complete and reliable data sequences for uploading.
It effectively reduces data redundancy, improves the integrity and reliability of data sequences, reduces untimely uploads and channel congestion, and improves the stability and rationality of multimodal bridge data upload.
Smart Images

Figure CN118690310B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a multi-modal bridge data uploading method, device, equipment and medium. Background Art
[0002] As an important part of infrastructure, bridges are subject to increasing traffic loads and the impact of the natural environment. Therefore, regular monitoring and evaluation are required to ensure their long-term reliability and safety. In order to gain a more comprehensive understanding of the health of bridges, researchers have begun to explore the application of multimodal data. Multimodal data refers to different types of data obtained from different data sources, such as engineering documents, inspection and maintenance documents, three-dimensional documents, sensors (accelerometers, sound sensors, cameras, etc.). By comprehensively utilizing these multimodal data, the structural health of bridges can be more accurately assessed and potential failures can be predicted.
[0003] At the same time, the traditional "end-to-cloud" data processing model has gradually shown its shortcomings in terms of real-time response, network stability, and traffic cost control. The main problems currently encountered are:
[0004] During the data collection and transmission process, some sensors may not work properly or data may be lost, which may reduce the accuracy and reliability of the analysis results. In addition, different sensors or different measurement modes of sensors may obtain the same or highly related information. This noisy data may take up storage space, increase the complexity of data processing and analysis, and have no effect on the final results.
[0005] Affected by complex internal and external factors such as natural environmental factors, traffic load, and structural aging, it is inevitable that abnormal data will be generated by edge devices. If abnormal data is not uploaded in time on the edge side, it will not only reduce the timeliness of data analysis on the cloud side, but also have a negative impact on the data analysis results on the cloud side.
[0006] Faced with massive amounts of multimodal data, uploading full data in real time on the edge side not only takes up a lot of network resources, but also puts a severe test on the quality of the upstream network bandwidth. Once the network crashes, it will not only cause data packet loss, but also affect the stability of the entire system.
[0007] It can be seen that the existing multimodal bridge data uploading method has the problem of poor rationality of multimodal bridge data uploading. Summary of the invention
[0008] The present application provides a multimodal bridge data uploading method, device, equipment and medium, which can solve the problem of poor rationality of multimodal bridge data uploading.
[0009] In a first aspect, the present application provides a multimodal bridge data uploading method, the multimodal bridge data uploading method comprising:
[0010] Acquire multiple data sequences of the target bridge; the multiple data sequences correspond to multiple modes;
[0011] For each data sequence, the collision probability between the data sequence and each other data sequence is calculated respectively, and multiple collision sequences of the data sequence are determined from all other data sequences according to all collision probabilities. When the number of collision sequences is greater than zero, the data sequence and the mean of all collision sequences are taken as an initial data sequence. When the number of collision sequences is equal to zero, the data sequence is taken as an initial data sequence. The collision probability is used to describe the probability that a data sequence is similar to other data sequences, and the collision sequence is other data sequences whose collision probability is greater than the probability threshold.
[0012] Calculate the hash value of each initial data sequence;
[0013] Filter out multiple missing sequences from all initial data sequences, and complete each missing sequence according to all hash values to obtain a complete sequence of each missing sequence; the missing sequence is an initial data sequence with missing data;
[0014] For each complete sequence, use the complete sequence and the corresponding hash value to predict the abnormality level of the complete sequence, and for each other initial data sequence except the missing sequence, use the hash value corresponding to the other initial data sequence to predict the abnormality level of the other initial data sequence;
[0015] Upload all complete sequences and all other initial data sequences according to all modalities, all anomaly levels to the target device.
[0016] Optionally, calculate the probability of collision between the data sequence and every other data sequence, including:
[0017] By formula:
[0018]
[0019] Calculate the The data series and The probability of collision between data sequences ;
[0020] in, Indicates The data series and The distance between the data sequences, Indicates the size of the hash bucket, , , represents the total number of data sequences, represents the probability density function, Represents the differential of distance.
[0021] Optionally, calculate the hash value of each initial data sequence, including:
[0022] By formula:
[0023]
[0024] Calculate the The hash value of the initial data sequence ;
[0025] in, Indicates The random vector corresponding to the initial data sequence, , represents the total number of initial data sequences, A vector representing a hash function, Represents a random number, , Indicates the size of the hash bucket.
[0026] Optionally, each missing sequence is completed according to all hash values to obtain a complete sequence of each missing sequence, including:
[0027] For each missing sequence, perform the following steps:
[0028] Determine at least one similar sequence corresponding to the missing sequence from all initial data sequences according to all hash values;
[0029] Obtain the implicit feature tensor between all similar sequences, perform feature extraction on the implicit feature tensor to obtain tensor features, and use the tensor features to predict each missing data in the missing sequence to obtain multiple predicted data;
[0030] All predicted data are integrated into the missing sequence to obtain the complete sequence of the missing sequence.
[0031] Optionally, the hash value corresponding to the complete sequence is used to predict the abnormality level of the complete sequence, including:
[0032] The hybrid expert network is used to predict anomalies of the complete sequence and obtain the reasoning expert distribution of the complete sequence;
[0033] According to the hash value corresponding to the complete sequence and the expert model, the outlier value of the complete sequence is calculated;
[0034] The abnormality level corresponding to the abnormal value is determined according to the abnormal value and the abnormality value ranges of the preset multiple abnormality levels.
[0035] Optionally, a hybrid expert network is used to perform anomaly prediction on the complete sequence to obtain the reasoning expert distribution of the complete sequence, including:
[0036] By formula:
[0037]
[0038] Calculate the The distribution of reasoning experts for a complete sequence ;
[0039] in, Indicates the impact of each expert model in the hybrid expert network on the The processing capability of a complete sequence, Reservation The front elements, , Represents the number of complete sequences.
[0040] Optionally, the outlier value of the complete sequence is calculated based on the hash value corresponding to the complete sequence and the inference expert distribution, including:
[0041] By formula:
[0042]
[0043] Calculate the Outliers for the complete series ;
[0044] in, represents the model parameters in the hybrid expert network, represents the transpose operation, Indicates the impact of each expert model in the hybrid expert network on the The inference result of a complete sequence, Indicates The hash value corresponding to the complete sequence.
[0045] In a second aspect, the present application provides a multimodal bridge data uploading device, comprising:
[0046] An acquisition module is used to acquire multiple data sequences of a target bridge; multiple data sequences correspond to multiple modes;
[0047] A determination module calculates the collision probability between each data sequence and each other data sequence respectively, and determines multiple collision sequences of the data sequence from all other data sequences according to all collision probabilities. When the number of collision sequences is greater than zero, the data sequence and the mean of all collision sequences are used as an initial data sequence. When the number of collision sequences is equal to zero, the data sequence is used as an initial data sequence. The collision probability is used to describe the probability that the data sequence is similar to other data sequences, and the collision sequence is other data sequences whose collision probability is greater than a probability threshold.
[0048] A calculation module calculates the hash value of each initial data sequence;
[0049] The completion module selects multiple missing sequences from all the initial data sequences, and completes each missing sequence according to all the hash values to obtain a complete sequence of each missing sequence; the missing sequence is the initial data sequence with missing data;
[0050] A prediction module, for each complete sequence, predicts the abnormality level of the complete sequence using the complete sequence and the corresponding hash value, and for each other initial data sequence except the missing sequence, predicts the abnormality level of the other initial data sequence using the hash value corresponding to the other initial data sequence;
[0051] The upload module uploads all complete sequences and all other initial data sequences according to all modalities and all anomaly levels to the target device.
[0052] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the multimodal bridge data uploading method when executing the computer program.
[0053] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned multimodal bridge data uploading method is implemented.
[0054] The above solution of the present application has the following beneficial effects:
[0055] In an embodiment of the present application, multiple data sequences of the target bridge are obtained, and then for each data sequence, the collision probability between the data sequence and each other data sequence is calculated, and multiple collision sequences of the data sequence are determined from all other data sequences according to all collision probabilities. When the number of collision sequences is greater than zero, the data sequence and the mean of all collision sequences are used as an initial data sequence. When the number of collision sequences is equal to zero, the data sequence is used as an initial data sequence, and then the hash value of each initial data sequence is calculated. Then, multiple missing sequences are screened out from all the initial data sequences, and each missing sequence is completed according to all hash values to obtain a complete sequence of each missing sequence. Then, for each complete sequence, the hash value corresponding to the complete sequence is used to predict the abnormality level of the complete sequence, and for each other initial data sequence except the missing sequence, the hash value corresponding to the other initial data sequence is used to predict the abnormality level of the other initial data sequences. Finally, all complete sequences and all other initial data sequences are uploaded to the target device according to all modes and all abnormality levels. Among them, obtaining the initial data sequence by calculating the collision probability and finding the average can effectively reduce the number of identical or highly correlated data sequences and reduce data redundancy. Using hash values to complete missing sequences can improve the integrity and reliability of data sequences. Using hash values to calculate the anomaly level can improve the accuracy of the anomaly level. Uploading reliable and complete data sequences based on accurate anomaly levels can reduce the situation of untimely uploading of multimodal bridge data and congestion of upload channels, improve the stability of upload, and thus improve the rationality of uploading multimodal bridge data.
[0056] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0058] Figure 1 A flowchart of a multi-modal bridge data uploading method provided in one embodiment of the present application;
[0059] Figure 2 A schematic diagram of the structure of a hybrid expert network provided in one embodiment of the present application;
[0060] Figure 3 A schematic diagram of the structure of a multi-modal bridge data uploading device provided in one embodiment of the present application;
[0061] Figure 4 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0062] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0063] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0064] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0065] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.
[0066] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0067] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0068] In response to the problem of poor rationality of existing multimodal bridge data upload, an embodiment of the present application provides a multimodal bridge data upload method, which obtains multiple data sequences of the target bridge, and then calculates the collision probability between the data sequence and each other data sequence for each data sequence, and determines multiple collision sequences of the data sequence from all other data sequences according to all collision probabilities. When the number of collision sequences is greater than zero, the data sequence and the mean of all collision sequences are used as an initial data sequence. When the number of collision sequences is equal to zero, the data sequence is used as an initial data sequence, and then the hash value of each initial data sequence is calculated. Then, multiple missing sequences are screened out from all the initial data sequences, and each missing sequence is completed according to all hash values to obtain a complete sequence of each missing sequence. Then, for each complete sequence, the hash value corresponding to the complete sequence is used to predict the abnormality level of the complete sequence, and for each other initial data sequence except the missing sequence, the hash value corresponding to the other initial data sequence is used to predict the abnormality level of the other initial data sequences. Finally, all complete sequences and all other initial data sequences are uploaded to the target device according to all modes and all abnormality levels. Among them, obtaining the initial data sequence by calculating the collision probability and finding the average can effectively reduce the number of identical or highly correlated data sequences and reduce data redundancy. Using hash values to complete missing sequences can improve the integrity and reliability of data sequences. Using hash values to calculate the anomaly level can improve the accuracy of the anomaly level. Uploading reliable and complete data sequences based on accurate anomaly levels can reduce the situation of untimely uploading of multimodal bridge data and congestion of upload channels, improve the stability of upload, and thus improve the rationality of uploading multimodal bridge data.
[0069] Next, the multimodal bridge data uploading method provided in this application is exemplified.
[0070] like Figure 1 As shown, the multimodal bridge data uploading method provided by the present application includes the following steps:
[0071] Step 11, obtaining multiple data sequences of the target bridge.
[0072] The above multiple data sequences correspond to multiple modes.
[0073] It should be noted that the target bridge mentioned above is a bridge that needs to use multimodal data to achieve analysis, monitoring and other tasks, such as monitoring of the structural health status of the bridge. The above-mentioned multiple modalities can be image data, text data, audio data, temperature data, height data, etc. The above-mentioned data sequence is composed of multiple data of the corresponding modalities, such as the temperature data sequence [23°C, 27°C, 19°C].
[0074] In some embodiments of the present application, multiple data sequences may be acquired using data acquisition devices such as cameras and sensors.
[0075] Step 12, for each data sequence, calculate the collision probability between the data sequence and each other data sequence, determine multiple collision sequences of the data sequence from all other data sequences according to all collision probabilities, when the number of collision sequences is greater than zero, take the data sequence and the mean of all collision sequences as an initial data sequence, when the number of collision sequences is equal to zero, take the data sequence as an initial data sequence.
[0076] The above collision probability is used to describe the probability that a data sequence is similar to other data sequences. The higher the probability, the higher the probability that the data sequence is similar to other data sequences. The collision sequence is other data sequences with a collision probability greater than the probability threshold.
[0077] Specifically, through the formula:
[0078]
[0079] Calculate the The data series and The probability of collision between data sequences .
[0080] in, Indicates The data series and The distance between the data sequences, Indicates the size of the hash bucket, , , represents the total number of data sequences, represents the probability density function that satisfies the normal distribution, Represents the differential of distance.
[0081] It should be noted that the above distance may be a Euclidean distance. After the initial data sequence is obtained, the data sequence corresponding to the initial data sequence and all collision sequences will be deleted and will no longer participate in the process of step 12 for the next data sequence.
[0082] Exemplarily, the collision probabilities between the first data sequence and the four other data sequences are 30%, 98%, 78%, and 96%, respectively, and the probability threshold is 90%. Then the collision sequence of the first data sequence is the second other data sequence and the fourth other data sequence. The mean between the first data sequence, the second other data sequence, and the fourth other data sequence is calculated. For example, if the three data sequences are [23°C, 12°C], [22°C, 10°C], and [24°C, 11°C], the mean is [23°C, 11°C], and the mean is used as an initial data sequence. At the same time, the first data sequence, the second other data sequence, and the fourth other data sequence are deleted. The collision probabilities corresponding to the second data sequence are 33%, 87%, and 12%, respectively, which are all less than the probability threshold, so the data sequence is directly used as an initial data sequence.
[0083] It is worth mentioning that by obtaining the collision sequence of the data sequence and calculating the average, and only retaining the initial data sequence, the number of similar data sequences can be reduced, data redundancy can be reduced, and the storage space occupied can be effectively reduced.
[0084] Step 13, calculate the hash value of each initial data sequence.
[0085] Specifically, through the formula:
[0086]
[0087] Calculate the The hash value of the initial data sequence .
[0088] in, Indicates The random vector corresponding to the initial data sequence, , represents the total number of initial data sequences, A vector representing a hash function, Represents a random number, , Indicates the size of the hash bucket.
[0089] It should be noted that the above hash values are locally sensitive hash values, which can describe the position and shape information of the initial data sequence in the implicit low-dimensional space.
[0090] Exemplarily, the above formula can be run using computer software such as MATLAB to calculate the hash value of each initial data sequence.
[0091] It is worth mentioning that by calculating the hash value of the initial data sequence, the position and shape information of the initial data sequence can be described, providing the position and shape information of the initial data sequence for subsequent calculations.
[0092] Step 14, multiple missing sequences are screened out from all initial data sequences, and each missing sequence is completed according to all hash values to obtain a complete sequence of each missing sequence.
[0093] The missing sequence mentioned above is an initial data sequence with missing data.
[0094] Exemplarily, the data in the initial data sequence are sorted according to the acquisition time. The first initial data sequence should contain temperature values at 9, 10, and 11 o'clock. However, due to sensor failure or other reasons, the temperature value at 10 o'clock was not collected. In this case, the initial data sequence is treated as a missing sequence, and the temperature data at 10 o'clock is marked as [fore].
[0095] In some embodiments of the present application, the above step of completing each missing sequence according to all hash values to obtain a complete sequence of each missing sequence is specifically:
[0096] For each missing sequence, perform the following steps:
[0097] In the first step, at least one similar sequence corresponding to the missing sequence is determined from all initial data sequences according to all hash values.
[0098] Specifically, an R-tree can be constructed using all hash values and all initial data sequences, with the nodes of the R-tree corresponding to the initial data sequences and the layers corresponding to the value range of the hash values. Then, a K-nearest neighbor query is used to query in the R-tree to obtain at least one similar sequence to the missing sequence.
[0099] The second step is to obtain the implicit feature tensor between all similar sequences, and perform feature extraction on the implicit feature tensor to obtain tensor features. The tensor features are used to predict each missing data in the missing sequence to obtain multiple predicted data.
[0100] Exemplarily, all similar sequences and the hash value corresponding to each similar sequence can be input into the attention layer to obtain an implicit feature tensor. The convolution layer can be used to extract features from the implicit feature tensor, and the fully connected layer can be used to predict each missing data.
[0101] In the third step, all predicted data are integrated into the missing sequence to obtain the complete sequence of the missing sequence.
[0102] For example, the missing sequence is [21, [fore], 15], and the predicted data is 7, then the complete sequence of the missing sequence is [21, 7, 15].
[0103] It is worth mentioning that this step effectively improves the reliability and completeness of the initial data sequence by completing the missing sequences.
[0104] Step 15, for each complete sequence, use the complete sequence and the corresponding hash value to predict the abnormality level of the complete sequence, and for each other initial data sequence except the missing sequence, use the hash value corresponding to the other initial data sequence to predict the abnormality level of the other initial data sequence.
[0105] The above anomaly level is used to describe the degree of data anomaly of a complete sequence or a missing sequence. The higher the anomaly level, the higher the degree of data anomaly. That is, sequences with high anomaly levels are more likely to be data when an abnormality occurs in the target bridge, such as picture data when the bridge collapses.
[0106] In some embodiments of the present application, the step of predicting the abnormality level of the complete sequence using the hash value corresponding to the complete sequence specifically includes:
[0107] In the first step, the hybrid expert network is used to predict anomalies on the complete sequence and obtain the reasoning expert distribution of the complete sequence.
[0108] Specifically, the hybrid expert network is constructed by the formula:
[0109]
[0110] Calculate the The distribution of reasoning experts for a complete sequence .
[0111] in, Indicates the impact of each expert model in the hybrid expert network on the The processing capability of a complete sequence, Reservation The front elements, , Represents the number of complete sequences.
[0112] In the second step, the outlier value of the complete sequence is calculated based on the hash value corresponding to the complete sequence and the inference expert distribution.
[0113] Specifically, through the formula:
[0114]
[0115] Calculate the Outliers for the complete series .
[0116] in, represents the model parameters in the hybrid expert network, represents the transpose operation, Indicates the impact of each expert model in the hybrid expert network on the The inference result of a complete sequence, Indicates The hash value corresponding to the complete sequence.
[0117] It should be noted that the above hybrid expert network is a network model that includes multiple expert models, and the above formulas for calculating the reasoning expert distribution and outliers can be understood as expressions of the hybrid expert network.
[0118] The third step is to determine the abnormal level corresponding to the abnormal value according to the abnormal value and the abnormal value ranges of the preset multiple abnormal levels.
[0119] Exemplarily, the multiple abnormality levels are: no abnormality, level 1 abnormality, level 2 abnormality, level 3 abnormality, level 4 abnormality, and level 5 abnormality. The abnormality value ranges are [0-10], [10-20], [20-30], [30-40], and [above 40], respectively. The abnormality value of the first complete sequence is 33, and the abnormality level of the complete sequence is level 4 abnormality.
[0120] It should be noted that the specific process of predicting the abnormality level of other initial data sequences using the hash values corresponding to other initial data sequences is the same as the process of predicting the abnormality level of the complete sequence mentioned above, that is, using the hybrid expert network to predict the abnormality of other initial data sequences, and obtain the inference expert distribution of other initial data sequences, and then calculate the abnormality level of other initial data sequences according to the hash values corresponding to other initial data sequences and the inference expert distribution.
[0121] It is worth mentioning that using hash values to calculate anomaly levels can improve the accuracy of anomaly levels. By calculating the anomaly levels of the complete sequence and other initial data sequences, the degree of anomaly of the complete sequence and other initial data sequences can be described.
[0122] The above hybrid expert network is illustrated below with reference to a specific example.
[0123] like Figure 2 As shown, the hybrid expert network includes: a gating model, multiple expert models, Figure 2 The completed sequence data (i.e., the complete sequence or other initial data sequence mentioned above) is input into the gating model, which divides the completed sequence data into multiple subsequences and assigns the subsequences to expert model 1, expert model 2, and expert model , Expert Model , the output of all expert models is combined with the local sensitive hash corresponding to the completed sequence data (i.e., the hash value mentioned above) to obtain the abnormality level.
[0124] Step 16, upload all complete sequences and all other initial data sequences according to all modalities and all anomaly levels to the target device.
[0125] The above-mentioned target device is a device that needs to use multimodal data to further process the target bridge, such as a cloud server.
[0126] In some embodiments of the present application, the priority of each modality can be preset according to the actual situation, and multiple other initial data sequences and multiple complete sequences corresponding to each modality can be uploaded according to the order of priority. At the same time, in the process of uploading each modality, all other initial data sequences and all complete sequences corresponding to the modality are uploaded according to the abnormality level from high to low.
[0127] Exemplarily, the priority of the picture is higher than that of temperature, the second other initial data sequence and the third other initial data sequence are both temperature modes, and the abnormality level of the second other initial data sequence is level five abnormality, and the abnormality level of the third other initial data sequence is no abnormality, then after all data corresponding to the picture mode are uploaded, the second other initial data sequence is started to be uploaded, and then the third other initial data sequence is uploaded.
[0128] It should be noted that the multimodal bridge data uploading method of the present application can be applied to edge devices. After uploading the multimodal bridge data to the target device using the method of the present application, the target device can use the received data to perform further processing such as prediction and monitoring of the target bridge, such as using the received multimodal data of the bridge to monitor the structural health status of the bridge.
[0129] It is worth mentioning that obtaining the initial data sequence by calculating the collision probability and finding the average can effectively reduce the number of identical or highly correlated data sequences and reduce data redundancy. Using hash values to complete missing sequences can improve the integrity and reliability of data sequences. Using hash values to calculate the anomaly level can improve the accuracy of the anomaly level. Uploading reliable and complete data sequences based on accurate anomaly levels can reduce the situation of untimely uploading of multimodal bridge data and congestion of upload channels, improve the stability of upload, and thus improve the rationality of multimodal bridge data upload.
[0130] The following is an exemplary description of the multimodal bridge data uploading device provided in this application.
[0131] like Figure 3 As shown, the embodiment of the present application provides a multi-modal bridge data uploading device, and the multi-modal bridge data uploading device 300 includes:
[0132] An acquisition module 301 acquires multiple data sequences of a target bridge; the multiple data sequences correspond to multiple modes;
[0133] The determination module 302 calculates the collision probability between each data sequence and each other data sequence respectively, and determines multiple collision sequences of the data sequence from all other data sequences according to all collision probabilities. When the number of collision sequences is greater than zero, the data sequence and the mean of all collision sequences are used as an initial data sequence. When the number of collision sequences is equal to zero, the data sequence is used as an initial data sequence. The collision probability is used to describe the probability that the data sequence is similar to other data sequences. The collision sequence is other data sequences whose collision probability is greater than a probability threshold.
[0134] A calculation module 303 calculates a hash value of each initial data sequence;
[0135] The completion module 304 selects multiple missing sequences from all the initial data sequences, and completes each missing sequence according to all the hash values to obtain a complete sequence of each missing sequence; the missing sequence is an initial data sequence with missing data;
[0136] The prediction module 305 predicts the abnormality level of each complete sequence using the complete sequence and the corresponding hash value, and predicts the abnormality level of each other initial data sequence except the missing sequence using the hash value corresponding to the other initial data sequence;
[0137] The upload module 306 uploads all complete sequences and all other initial data sequences to the target device according to all modalities and all abnormality levels.
[0138] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0139] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0140] like Figure 4 As shown, an embodiment of the present application provides a terminal device. The terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 4 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 implements the steps of any of the above-mentioned method embodiments when executing the computer program D102.
[0141] Specifically, when the processor D100 executes the computer program D102, it obtains multiple data sequences of the target bridge, and then calculates the collision probability between the data sequence and each other data sequence for each data sequence, and determines multiple collision sequences of the data sequence from all other data sequences based on all collision probabilities. When the number of collision sequences is greater than zero, the data sequence and the mean of all collision sequences are used as an initial data sequence. When the number of collision sequences is equal to zero, the data sequence is used as an initial data sequence, and then the hash value of each initial data sequence is calculated. Then, multiple missing sequences are screened out from all initial data sequences, and each missing sequence is supplemented according to all hash values to obtain a complete sequence of each missing sequence. Then, for each complete sequence, the hash value corresponding to the complete sequence is used to predict the abnormality level of the complete sequence, and for each other initial data sequence except the missing sequence, the hash value corresponding to the other initial data sequence is used to predict the abnormality level of the other initial data sequences. Finally, all complete sequences and all other initial data sequences are uploaded to the target device according to all modes and all abnormality levels. Among them, obtaining the initial data sequence by calculating the collision probability and finding the average can effectively reduce the number of identical or highly correlated data sequences and reduce data redundancy. Using hash values to supplement missing sequences can improve the integrity and reliability of data sequences. Using hash values to calculate the anomaly level can improve the accuracy of the anomaly level. Uploading reliable and complete data sequences based on accurate anomaly levels can reduce the situation of untimely uploading of multimodal bridge data and congestion of upload channels, improve the stability of upload, and thus improve the rationality of uploading multimodal bridge data.
[0142] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0143] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.
[0144] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.
[0145] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.
[0146] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the multimodal bridge data upload method device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk. The computer-readable medium cannot be an electric carrier signal and a telecommunication signal.
[0147] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0148] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0149] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A multi-modal bridge data uploading method, characterized in that: include: Acquire multiple data sequences of the target bridge; the multiple data sequences correspond to multiple modes; For each of the data sequences, respectively, the collision probability between the data sequence and each other data sequence is calculated, and multiple collision sequences of the data sequence are determined from all other data sequences according to all collision probabilities; when the number of collision sequences is greater than zero, the data sequence and the mean of all collision sequences are used as an initial data sequence; when the number of collision sequences is equal to zero, the data sequence is used as an initial data sequence; the collision probability is used to describe the probability that the data sequence is similar to other data sequences, and the collision sequences are other data sequences whose collision probability is greater than a probability threshold; Calculating a hash value of each of the initial data sequences; Filter out multiple missing sequences from all initial data sequences, and complete each missing sequence according to all hash values to obtain a complete sequence of each missing sequence; the missing sequence is an initial data sequence with missing data; For each of the complete sequences, respectively, predict the abnormality level of the complete sequence using the complete sequence and the corresponding hash value, and for each other initial data sequence except the missing sequence, respectively, predict the abnormality level of the other initial data sequence using the hash value corresponding to the other initial data sequence; Upload all complete sequences and all other initial data sequences according to all modalities, all anomaly levels to the target device.
2. The multimodal bridge data uploading method according to claim 1, characterized in that: The calculating the collision probability between the data sequence and each other data sequence comprises: By formula: Calculate the The data series and The probability of collision between data sequences ; in, Indicates the The data sequence is related to the The distance between the data sequences, Indicates the size of the hash bucket, , , represents the total number of data sequences, represents the probability density function, Represents the differential of distance.
3. The multi-modal bridge data uploading method according to claim 1, characterized in that: The calculating of the hash value of each initial data sequence comprises: By formula: Calculate the The hash value of the initial data sequence ; in, Indicates the The random vector corresponding to the initial data sequence, , represents the total number of initial data sequences, A vector representing a hash function, Represents a random number, , Indicates the size of the hash bucket.
4. The multi-modal bridge data uploading method according to claim 1, characterized in that: The step of completing each missing sequence according to all hash values to obtain a complete sequence of each missing sequence includes: For each of the missing sequences, perform the following steps: Determine at least one similar sequence corresponding to the missing sequence from all initial data sequences according to all hash values; Obtaining implicit feature tensors between all similar sequences, and performing feature extraction on the implicit feature tensors to obtain tensor features, and using the tensor features to predict each missing data in the missing sequence to obtain multiple predicted data; All predicted data are integrated into the missing sequence to obtain the complete sequence of the missing sequence.
5. The multi-modal bridge data uploading method according to claim 1, characterized in that: The predicting the abnormality level of the complete sequence by using the hash value corresponding to the complete sequence includes: Using a hybrid expert network to perform anomaly prediction on the complete sequence to obtain a reasoning expert distribution of the complete sequence; Calculating an outlier value of the complete sequence according to the hash value corresponding to the complete sequence and the inference expert distribution; The abnormality level corresponding to the abnormal value is determined according to the abnormal value and abnormality value ranges of a plurality of preset abnormality levels.
6. The multi-modal bridge data uploading method according to claim 5, characterized in that: The method of using a hybrid expert network to perform anomaly prediction on the complete sequence to obtain the inference expert distribution of the complete sequence includes: By formula: Calculate the The distribution of reasoning experts for a complete sequence ; in, Indicates the impact of each expert model in the hybrid expert network on the The processing capability of a complete sequence, Reservation The front elements, , Represents the number of complete sequences.
7. The multi-modal bridge data uploading method according to claim 6, characterized in that: The calculating the outlier value of the complete sequence according to the hash value corresponding to the complete sequence and the inference expert distribution includes: By formula: Calculate the Outliers for the complete series ; in, represents the model parameters in the mixture of experts network, represents the transpose operation, Indicates the impact of each expert model in the hybrid expert network on the The inference result of a complete sequence, Indicates the The hash value corresponding to the complete sequence.
8. A multi-modal bridge data uploading device, characterized in that: include: An acquisition module is used to acquire multiple data sequences of a target bridge; the multiple data sequences correspond to multiple modes; a determination module, for each of the data sequences, calculating the collision probability between the data sequence and each other data sequence, determining multiple collision sequences of the data sequence from all other data sequences according to all collision probabilities, and when the number of collision sequences is greater than zero, taking the data sequence and the mean of all collision sequences as an initial data sequence, and when the number of collision sequences is equal to zero, taking the data sequence as an initial data sequence; the collision probability is used to describe the probability that the data sequence is similar to other data sequences, and the collision sequence is other data sequences whose collision probability is greater than a probability threshold; A calculation module, calculating a hash value of each of the initial data sequences; A completion module, which selects multiple missing sequences from all initial data sequences, and completes each missing sequence according to all hash values to obtain a complete sequence of each missing sequence; the missing sequence is an initial data sequence with missing data; A prediction module, for each of the complete sequences, using the complete sequence and the corresponding hash value to predict the abnormality level of the complete sequence, and for each other initial data sequence except the missing sequence, using the hash value corresponding to the other initial data sequence to predict the abnormality level of the other initial data sequence; The upload module uploads all complete sequences and all other initial data sequences according to all modalities and all anomaly levels to the target device.
9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the multimodal bridge data uploading method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the multimodal bridge data uploading method according to any one of claims 1 to 7 is implemented.
