A Data Decoding Method for a Multi-Modal Physical Simulation System Based on Artificial Intelligence

Abstract

The present invention relates to the field of data decoding technology, and in particular to a data decoding method for a multi-mode physical simulation system based on artificial intelligence. The method comprises: obtaining different types of monitoring data sequences of physical objects; obtaining matching data pairs based on the correlation of data in a local window between each data in a reference sequence and each data in a sequence to be aligned, the difference in the fluctuation amplitude of data in the local window, and the difference in the acquisition time; classifying all time differences based on the time difference between the data in the matching data pairs, and determining the delay duration of the sequence to be aligned based on the comprehensive classification results; aligning the sequence to be aligned with the data in the reference sequence using the delay duration, and decoding the reference sequence and the aligned monitoring data sequence using a multi-mode physical simulation system. The present invention improves the accuracy of the decoding and fusion results of various monitoring data of the physical object.

Description

Technical Field

[0001] The present invention relates to the technical field of data decoding, and in particular to a data decoding method for a multi-mode physical simulation system based on artificial intelligence. Background Art

[0002] The multi-mode physical simulation system based on artificial intelligence is a simulation platform that integrates multiple sensors and intelligent algorithms, and is used for the development, testing and verification of complex physical systems. The data decoding of this system refers to the analysis and understanding of data from multiple sensors through intelligent algorithms, so as to reconstruct and simulate objects and behaviors in the real environment. Its role is to convert complex multi-source data into actionable information, so that the simulation system can more accurately reproduce the physical characteristics and interaction patterns of the real world, thereby improving simulation accuracy, supporting system optimization and decision-making, and promoting the development and application of new technologies.

[0003] In the data decoding process of the multi-mode physical simulation system, the various information collected by the physical object is mainly feature extracted and fused, so as to simulate the simulation components that are closer to the physical object's characteristic attributes through the simulation system. However, in the process of decoding and fusing various monitoring data collected by the physical object, there are many types of physical information collected, and these data may have certain differences in the acquisition and reception process due to the propagation speed of the collected data, resulting in different types of monitoring data having certain delay deviations in actual time, which in turn may lead to poor alignment of different data in time, and the accuracy of the decoding and fusion results of various information data of the physical object is low. Summary of the invention

[0004] In order to solve the problem that the existing methods have low accuracy in decoding and fusing the various information data of the physical object, the purpose of the present invention is to provide a multi-mode physical simulation system data decoding method based on artificial intelligence. The technical solution adopted is as follows:

[0005] The present invention provides a data decoding method for a multi-mode physical simulation system based on artificial intelligence, the method comprising the following steps:

[0006] Obtain different types of monitoring data series of physical objects;

[0007] Combine the correlation of data in the local window between each data in the reference sequence and each data in the sequence to be aligned, the difference in the fluctuation amplitude of the data in the local window, and the difference in the acquisition time to obtain a matching data pair; the reference sequence is one monitoring data sequence in every two monitoring data sequences, and the sequence to be aligned is the other monitoring data sequence; wherein the local window size is determined based on the local fluctuation characteristics of each data in the two monitoring data sequences and the overall fluctuation characteristics of the monitoring data sequences;

[0008] Classify all time differences based on the time differences between the data in the matching data pair, and determine the delay time of the sequence to be aligned based on the classification results; align the sequence to be aligned with the data in the reference sequence using the delay time;

[0009] A multi-mode physical simulation system is used to decode the reference sequence and the aligned monitoring data sequence.

[0010] Preferably, the acquisition of the local window size includes:

[0011] For any two types of monitoring data series:

[0012] One of the monitoring data sequences is recorded as the first sequence, and any data in the other monitoring data sequence is recorded as candidate data;

[0013] Determine a first coefficient based on the degree of dispersion of all data in the monitoring data sequence where the candidate data is located and the degree of dispersion of data in a time neighborhood of the candidate data, wherein the degree of dispersion of all data is positively correlated with the first coefficient, and the degree of dispersion of data in the time neighborhood is negatively correlated with the first coefficient;

[0014] Determine a second coefficient based on the number of extreme value points in the monitoring data sequence where the candidate data is located and the number of extreme value points in the first sequence, wherein the number of extreme value points in the monitoring data sequence where the candidate data is located is negatively correlated with the second coefficient, and the number of extreme value points in the first sequence is positively correlated with the second coefficient;

[0015] The local window size of the candidate data is obtained by combining the first coefficient and the second coefficient, and the first coefficient and the second coefficient are both positively correlated with the length of the local window.

[0016] Preferably, the step of combining the correlation of data in a local window between each data in the reference sequence and each data in the sequence to be aligned, the difference in the fluctuation amplitude of the data in the local window, and the difference in the acquisition time to obtain a matching data pair includes:

[0017] Obtaining a nonlinear factor between the data to be analyzed and each data in the sequence to be aligned according to the degree of correlation between the data in the local window of the data to be analyzed and the data in the local window of each data in the sequence to be aligned, wherein the degree of correlation is negatively correlated with the nonlinear factor;

[0018] Obtaining a matching degree between the data to be analyzed and each data in the sequence to be aligned according to a nonlinear factor between the data to be analyzed and each data in the sequence to be aligned, a difference between a fluctuation amplitude of the data in a local window of the data to be analyzed and a fluctuation amplitude of the data in a local window of each data in the sequence to be aligned, and a difference between a collection time of the data to be analyzed and a collection time of each data in the sequence to be aligned, wherein the nonlinear factor, the difference between the fluctuation amplitudes and the difference between the collection times are all negatively correlated with the matching degree;

[0019] A matching data pair is obtained based on the matching degree; the data to be analyzed is any data in the reference sequence.

[0020] Preferably, obtaining a matching data pair based on the matching degree comprises:

[0021] The data to be analyzed and the data with the greatest matching degree constitute a matching data pair.

[0022] Preferably, the classifying all time differences based on the time differences between the data in the matching data pair comprises:

[0023] The time difference between two data in all matching data pairs constitutes a time difference series;

[0024] Cluster the elements in the time difference sequence to obtain clusters.

[0025] Preferably, an iterative self-organizing clustering algorithm is used to cluster the elements in the time difference sequence.

[0026] Preferably, the comprehensive classification results determine the delay time of the sequence to be aligned, including:

[0027] The difference between the mean of all time differences in the candidate cluster and the mean of all time differences in the time difference sequence is recorded as the overall time interval; the difference between each time difference in the candidate cluster and the mean of all time differences in the candidate cluster is recorded as the first difference of each time difference in the candidate cluster; the confidence of the candidate cluster is obtained according to the overall time interval and the overall distribution level of the first difference of all time differences in the candidate cluster; the overall time interval and the overall distribution level of the first difference are negatively correlated with the confidence; the candidate cluster is any cluster;

[0028] The overall confidence of all clusters is combined to determine the delay length of the sequences to be aligned.

[0029] Preferably, the step of determining the delay time of the sequence to be aligned based on the overall confidence of all clusters includes:

[0030] The cluster with the largest confidence is taken as the target cluster;

[0031] The mean of all time differences within the target cluster is taken as the delay duration of the sequence to be aligned.

[0032] Preferably, the step of aligning the sequence to be aligned with the data in the reference sequence by using the delay time comprises:

[0033] Matching the data corresponding to the acquisition time in the sequence to be aligned with the data corresponding to the acquisition time in the reference sequence based on the delay time to obtain matching data corresponding to the data in the reference sequence;

[0034] If there is data in the reference sequence that has no corresponding matching data, the data prediction algorithm is used to complete the data.

[0035] Preferably, the acquiring of different types of monitoring data sequences of physical objects includes:

[0036] For any type of monitoring data:

[0037] All monitoring data of this type are sorted in order of collection time to obtain a monitoring data sequence of this type.

[0038] The present invention has at least the following beneficial effects:

[0039] The present invention takes into account that in the data decoding process of the multimodal physical simulation system, the various monitoring data collected by the physical object are mainly feature extracted and fused, so as to simulate the simulation components that are closer to the characteristic attributes of the physical object through the multimodal physical simulation system. However, the types of monitoring data collected may be large, and the collected monitoring data may have time delays, resulting in time differences between different types of monitoring data. Therefore, in order to improve the accuracy of the system in decoding the collected data and performing feature fusion and other operations, it is necessary to time align the collected monitoring data with delays. The present invention first combines the time difference between each data in the reference sequence and each data in the sequence to be aligned. Based on the correlation of data in the local window, the difference in the fluctuation amplitude of data in the local window and the difference in the collection time, multiple matching data pairs are obtained, and all time differences are classified based on the time difference between the data in the matching data pairs. A more accurate delay duration is obtained through the comprehensive classification results, and then the data in the sequence to be aligned and the reference sequence are aligned based on the delay duration. The reference sequence and the aligned monitoring data sequence are decoded and processed by the multi-mode physical simulation system, which improves the accuracy of the multi-mode physical simulation system in the process of collecting, decoding, fusion and analyzing the attribute information of the physical object, making the simulated physical attributes more accurate and ensuring the ultimate accuracy of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A flowchart of a data decoding method for a multi-mode physical simulation system based on artificial intelligence provided by an embodiment of the present invention;

[0042] Figure 2 A flowchart of a method for screening matching data pairs provided by an embodiment of the present invention;

[0043] Figure 3 A structural block diagram of a data decoding system for a multi-mode physical simulation system based on artificial intelligence provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following is a detailed description of a multi-mode physical simulation system data decoding method based on artificial intelligence proposed by the present invention in combination with the accompanying drawings and preferred embodiments.

[0045] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0046] The following is a detailed description of a data decoding method for a multi-mode physical simulation system based on artificial intelligence provided by the present invention in conjunction with the accompanying drawings.

[0047] An embodiment of a data decoding method for a multi-mode physical simulation system based on artificial intelligence:

[0048] The specific scenario targeted by this embodiment is: in the process of decoding and fusing different types of collected information data and realizing the simulation of the physical object, since the various sensors needed to collect the physical information collect different types of information data of the physical object, and these data may have certain differences in the collection and reception process due to the propagation speed of the collected data, etc., resulting in different types of data having certain delay deviations in actual time, and further resulting in different data being poorly aligned in time, resulting in deviations in the accuracy of the simulation system's simulation of the physical property information. This embodiment combines the specific distribution characteristics of different types of monitoring data, matches different types of monitoring data, determines the delay duration based on the matching results, and then aligns the delayed monitoring data with the non-delayed monitoring data, and uses the multi-mode physical simulation system to decode and process.

[0049] This embodiment proposes a data decoding method for a multi-mode physical simulation system based on artificial intelligence, such as Figure 1 As shown, a data decoding method for a multi-mode physical simulation system based on artificial intelligence in this embodiment includes the following steps:

[0050] Step S1, obtaining different types of monitoring data sequences of physical objects.

[0051] In the process of decoding the data of the multimodal physical simulation system, the main focus is on decoding and feature extraction of different types of information data collected by the physical object. Due to the different types of sensors and collected physical information, the collected data may have time delay deviations. For example, although visual information and sound information are collected at the same time, the propagation speed of the collected information may be different, resulting in poor time alignment of the collected different types of monitoring data.

[0052] When decoding and fusing the data of the physical simulation system, it is necessary to collect various information of the physical object. This embodiment takes the simulation of the properties of electronic components by the simulation system as an example for analysis. First, the relevant monitoring data of the physical object is collected by using sensors. For example, when simulating the influence of the current change of the simulated electronic component on the temperature change of the electronic component, the current sensor is used to collect the current data of the simulated component, and the temperature sensor is used to collect the temperature data of the simulated component, and so on. It is not limited to these two types of data. Temperature is one type of monitoring data, and current is another type of monitoring data. When collecting monitoring data, try to keep the sampling frequency consistency of different types of monitoring data. Otherwise, the monitoring data with a smaller sampling frequency can be interpolated according to needs. In this embodiment, the collection frequency of all types of monitoring data is the same, which is once per second. In specific applications, the implementer can select the type of monitoring data and set the collection frequency of monitoring data according to the specific situation.

[0053] For any type of monitoring data: sort all monitoring data of this type in the order of collection time to obtain the monitoring data sequence of this type.

[0054] So far, a monitoring data sequence consisting of each type of monitoring data is obtained, and one type of monitoring data corresponds to one monitoring data sequence.

[0055] Step S2, combining the correlation of data in a local window between each data in the reference sequence and each data in the sequence to be aligned, the difference in the fluctuation amplitude of the data in the local window, and the difference in the acquisition time, to obtain a matching data pair; the reference sequence is one monitoring data sequence in every two monitoring data sequences, and the sequence to be aligned is the other monitoring data sequence; wherein the local window size is determined based on the local fluctuation characteristics of each data in the two monitoring data sequences and the overall fluctuation characteristics of the monitoring data sequences.

[0056] By collecting the current data and temperature data of the electronic components and extracting and fusing these data features, the electronic components that meet the physical properties of the components can be simulated. When collecting the temperature data and current data of the electronic components, due to the time delay of the collection sensor in the data receiving process, different types of monitoring data may have a certain degree of time offset when collected.

[0057] When temporally matching any two types of monitoring data, the local features of each data are first analyzed and extracted, and then the feature data that may be correlated are matched. The time delay information between the two data is analyzed through the matching information between the data, and then the data with a larger time delay is interpolated and processed to make it correspond to the time of other data, thereby ensuring higher accuracy when decoding and fusing the two types of monitoring data later.

[0058] Based on this, we will first combine the local fluctuation characteristics of each data in the monitoring data sequence and the overall fluctuation characteristics of the monitoring data sequence to adaptively determine the local window of each data, and then combine the data characteristics within the local window of the data in the monitoring data sequence to screen matching data pairs.

[0059] Step S21, determining a local window of each data in every two monitoring data sequences according to the local fluctuation characteristics of each data in every two monitoring data sequences and the overall fluctuation characteristics of the monitoring data sequences.

[0060] When matching the time information of the two types of monitoring data, the data in the two types of monitoring data sequences are firstly linearly normalized. The linear normalization method is a prior art and will not be further processed here. It should be noted that the monitoring data mentioned later are all normalized data, and the monitoring data sequences mentioned later are also sequences composed of normalized data.

[0061] Since there may be many types of monitoring data, this embodiment takes any two types of monitoring data as examples for description, and other types of monitoring data may be processed using the method provided in this embodiment.

[0062] Specifically, for any two types of monitoring data sequences: one of the monitoring data sequences is recorded as the first sequence, and any data in the other monitoring data sequence is recorded as candidate data; based on the degree of discreteness of all data in the monitoring data sequence where the candidate data is located and the degree of discreteness of data in the time neighborhood of the candidate data, a first coefficient is determined, and the degree of discreteness of all data is positively correlated with the first coefficient, and the degree of discreteness of data in the time neighborhood is negatively correlated with the first coefficient; based on the number of extreme points in the monitoring data sequence where the candidate data is located and the number of extreme points in the first sequence, a second coefficient is determined, and the number of extreme points in the monitoring data sequence where the candidate data is located is negatively correlated with the second coefficient, and the number of extreme points in the first sequence is positively correlated with the second coefficient; the local window size of the candidate data is obtained by combining the first coefficient and the second coefficient, and the first coefficient and the second coefficient are both positively correlated with the length of the local window.

[0063] The method for obtaining the extreme value points in the monitoring data sequence is: using Newton's method to obtain the extreme value points in the monitoring data sequence. Newton's method is a prior art and will not be described in detail here.

[0064] The degree of dispersion of data can be represented by a variety of indicators, for example, it can be characterized by the standard deviation of the data, it can also be characterized by the variance of the data, and it can also be characterized by the range of the data. In this embodiment, the standard deviation of the data is used to characterize the degree of dispersion of the data, that is, the standard deviation of the data is calculated, and the obtained standard deviation is used as the degree of dispersion of the corresponding data.

[0065] A positive correlation means that the dependent variable will increase as the independent variable increases, and the dependent variable will decrease as the independent variable decreases. The specific relationship can be a multiplication relationship, an addition relationship, or the power of an exponential function with a base greater than 1. A negative correlation means that the dependent variable will decrease as the independent variable increases, and the dependent variable will increase as the independent variable decreases. It can be a subtraction relationship, a division relationship, etc., which is determined by actual application.

[0066] In this embodiment, a specific calculation formula for the length of the local window is given. The length of the local window of the i-th data in the a-th type of monitoring data sequence can be expressed as:

[0067]

[0068] in, represents the length of the local window of the i-th data in the a-th type of monitoring data sequence, represents the standard deviation of all data in the monitoring data sequence of type a, represents the standard deviation of all data in the time neighborhood of the ith data in the ath type of monitoring data sequence, represents the number of extreme points in the monitoring data sequence of type a, represents the number of extreme value points in the monitoring data sequence (first sequence) that matches the monitoring data sequence of type a, Indicates the preset first adjustment parameter, To preset the second adjustment parameter, represents the normalization function, represents the rounding up symbol, K represents the window selection hyperparameter when collecting data matching, and in this embodiment, the value of K is 4. In specific applications, the implementer can set it according to the specific situation.

[0069] For any data in any monitoring data sequence, the method for obtaining the data in the time neighborhood of the data is as follows: in the monitoring data sequence where the data is located, first, the data adjacent to the left side of the data is taken as the starting data, and n data are sequentially obtained on the left side of the data in order from right to left, and then, the data adjacent to the right side of the data is taken as the starting data, and n data are sequentially obtained on the right side of the data in order from left to right, and the 2n data obtained are all used as the data in the time neighborhood of the data. In this embodiment, the value of n is 5, and in specific applications, the implementer can set it according to specific circumstances.

[0070] The purpose of introducing the preset first adjustment parameter and the preset second adjustment parameter into the calculation formula of the local window length is to prevent the denominator from being 0. In this embodiment, the preset first adjustment parameter and the preset second adjustment parameter are both 0.001. In specific applications, the implementer can set them according to the specific situation.

[0071] It represents the first coefficient, which is used to characterize the ratio between the standard deviation of all data in the monitoring data sequence of the ath type and the standard deviation of all data in the time neighborhood of the ith data. The larger the value, the smaller the local variation of the local sequence of the ith data compared with the overall data, and a larger window may be needed to obtain the local features of the data point.

[0072] Represents the second coefficient, which is used to characterize the ratio between the number of extreme points in the monitoring data sequence that matches the a-th type of monitoring data sequence and the number of extreme points in the a-th type of monitoring data sequence. The smaller the value, the faster the a-th type of monitoring data sequence changes compared to the other data sequence. In this case, it is necessary to use a smaller window to extract features from the local changes of the sequence data in order to understand the local detailed changes of the sequence data points and improve the subsequent matching accuracy.

[0073] By adopting the above method, the length of the local window of each data in the monitoring data sequence can be obtained; for any data in any monitoring data sequence: taking the data as the center, a local window with a length equal to the length of the local window corresponding to it is made as the local window of the monitoring data. It should be noted that when the data in the same monitoring data sequence are matched with different monitoring data sequences, the length of the corresponding local window may be different.

[0074] Step S22, combining the correlation of data in the local window between each data in the reference sequence and each data in the sequence to be aligned, the difference in the fluctuation amplitude of the data in the local window, and the difference in the acquisition time, to obtain a matching data pair.

[0075] For any two types of monitoring data sequences: according to the actual delay of the two types of monitoring data, one of the monitoring data sequences with lower delay is artificially selected as the reference sequence, such as the temperature monitoring data sequence and the current monitoring data sequence mentioned above. When the two sequences are time-aligned and matched, since the temperature and changes of the resistor may have a certain time delay when they are transmitted, and the time delay of the current is lower, the current monitoring data sequence is used as the reference sequence when time alignment is performed. Then, when matching, any data point in the sequence is matched with the data point features in the window where any data of the matching sequence is located. In specific applications, the implementer sets the reference sequence according to the type of monitoring data and the actual situation of each monitoring data.

[0076] Next, one data in the reference sequence is analyzed as an example to obtain its corresponding matching data. The other data in the reference sequence can be processed using the method provided in this embodiment.

[0077] Specifically, any data in the reference sequence is recorded as the data to be analyzed. According to the correlation between the data in the local window of the data to be analyzed and the data in the local window of each data in the sequence to be aligned, the nonlinear factor of the data to be analyzed and each data in the sequence to be aligned is obtained, and the correlation is negatively correlated with the nonlinear factor; according to the nonlinear factor of the data to be analyzed and each data in the sequence to be aligned, the difference between the fluctuation amplitude of the data in the local window of the data to be analyzed and the fluctuation amplitude of the data in the local window of each data in the sequence to be aligned, and the difference between the acquisition time of the data to be analyzed and the acquisition time of each data in the sequence to be aligned, the matching degree of the data to be analyzed and each data in the sequence to be aligned is obtained, and the nonlinear factor, the difference between the fluctuation amplitudes and the difference between the acquisition times are all negatively correlated with the matching degree.

[0078] The correlation between data can be characterized by multiple indicators, for example, the absolute value of the Pearson correlation coefficient between data can be used to characterize it, or the negative correlation mapping value of the dynamic time warping distance between data can be used to characterize it. In this embodiment, the absolute value of the Pearson correlation coefficient between data is used to characterize it. The fluctuation range of data is characterized by the range between data.

[0079] A positive correlation means that the dependent variable will increase as the independent variable increases, and the dependent variable will decrease as the independent variable decreases. The specific relationship can be a multiplication relationship, an addition relationship, or the power of an exponential function with a base greater than 1. A negative correlation means that the dependent variable will decrease as the independent variable increases, and the dependent variable will increase as the independent variable decreases. It can be a subtraction relationship, a division relationship, etc., which is determined by actual application.

[0080] In this embodiment, a specific calculation formula for the matching degree is given. The i-th data and the The matching degree of the jth data in the monitoring data sequence of the type can be expressed as:

[0081]

[0082] in, Indicates The i-th data and the The matching degree of the jth data in the monitoring data sequence of the type, Indicates The data in the local window of the ith data in the monitoring data sequence of the type is The Pearson correlation coefficient between the data in the local window of the jth data in the type of monitoring data sequence, Indicates The range of all data in the local window of the ith data in the type of monitoring data sequence, Indicates The range of all data in the local window of the jth data in the type of monitoring data sequence, Indicates The collection time of the i-th data in the monitoring data sequence of the type is The difference between the collection time of the jth data in the monitoring data sequence of the type, represents the absolute value sign, and norm() represents the normalization function.

[0083] To characterize the The data in the local window of the ith data in the monitoring data sequence of the type is The correlation degree between the data in the local window of the jth data in the monitoring data sequence of the type. The larger the value, that is, the closer it is to 1, the The data in the local window of the ith data in the monitoring data sequence of the type is The more correlated the data in the local window of the jth data in the type of monitoring data sequence is; the smaller the value is, that is, the closer it is to 0, the more correlated the data in the jth data in the local window is; The data in the local window of the ith data in the monitoring data sequence of the type is The less correlated the data in the local window of the jth data in the type of monitoring data sequence are. Indicates The data in the local window of the ith data in the monitoring data sequence of the type is The nonlinear factor of the jth data in the monitoring data sequence of the type. The smaller the value, the The data in the local window of the ith data in the monitoring data sequence of the type is The stronger the correlation between the data in the local window of the jth data in the monitoring data sequence of the same type. The collection time of the i-th data in the monitoring data sequence of the type is The smaller the difference between the collection times of the jth data in the monitoring data sequence of the same type, the closer the two data may be in real time performance, and the greater the probability that the two data will match. The range of all data in the local window of the ith data in the monitoring data sequence of the type is The smaller the difference between the ranges of all data in the local window of the jth data in the type of monitoring data sequence, the smaller the difference between the ranges of all data in the local window of the jth data. The collection time of the i-th data in the monitoring data sequence of the type is The smaller the difference between the collection times of the jth data in the monitoring data sequence of the same type, the smaller the difference between the collection times of the jth data in the monitoring data sequence of the same type. The i-th data and the The more the jth data in the monitoring data sequence of the same type matches, the better the The i-th data and the The greater the matching degree of the jth data in the type of monitoring data sequence.

[0084] By adopting the above method, the matching degree between the data to be analyzed and each data in the first sequence can be obtained, and the data to be analyzed and the data with the greatest matching degree therewith constitute a matching data pair.

[0085] So far, using the above method, multiple matching data pairs can be obtained, such as Figure 2 As shown, the figure is a flow chart of the screening method for matching data pairs, each matching data pair contains two data, and the two data belong to different types of monitoring data, and each data in the reference sequence corresponds to a matching data pair.

[0086] Step S3, classifying all time differences based on the time differences between the data in the matching data pair, and determining the delay time of the sequence to be aligned based on the classification results; and aligning the sequence to be aligned with the data in the reference sequence using the delay time.

[0087] The time difference between the two data in each matching data pair is calculated respectively. The time difference is the difference between the data collection moments. For the reference sequence, there is a time difference between each data in the sequence and its matching data. Because there may be a certain matching error in the matching process, in order to obtain more accurate time delay information of the two sequences, the time difference between the two data in all matching data pairs is sorted in order from small to large or from large to small to obtain a time difference sequence. In this embodiment, the time difference is sorted in order from small to large. Then, the iterative self-organizing clustering (ISODATA) algorithm is used to cluster the elements in the time difference sequence to obtain multiple clusters. The iterative self-organizing clustering algorithm is a well-known technology and will not be described in detail here.

[0088] The following description will be given by taking one cluster as an example. The method provided in this embodiment can be used to process other clusters.

[0089] Specifically, any cluster is recorded as a candidate cluster, and the difference between the mean of all time differences in the candidate cluster and the mean of all time differences in the time difference sequence is recorded as the overall time interval; the difference between each time difference in the candidate cluster and the mean of all time differences in the candidate cluster is recorded as the first difference of each time difference in the candidate cluster; the confidence of the candidate cluster is obtained according to the overall time interval and the overall distribution level of the first difference of all time differences in the candidate cluster; the overall time interval and the overall distribution level of the first difference are negatively correlated with the confidence.

[0090] The overall distribution level of the first differences may be represented by an indicator such as an average value, a mode, a median, etc. of all the first differences. In this embodiment, the average value of the first differences is used to characterize the overall distribution level of the first differences.

[0091] A negative correlation means that the dependent variable will decrease as the independent variable increases, and the dependent variable will increase as the independent variable decreases. It can be a subtraction relationship, a division relationship, etc., which is determined by actual application.

[0092] In this embodiment, a specific calculation formula for confidence is given, and the confidence of the rth cluster can be expressed as:

[0093]

[0094] in, represents the confidence of the rth cluster, represents the mean of all time differences within the rth cluster, represents the mean of all time differences in the time difference series, represents the sth time difference within the rth cluster, represents the number of time differences within the rth cluster, represents the normalization function, Indicates the absolute value sign.

[0095] It represents the overall time interval of the r-th cluster, which is used to reflect the difference between the mean of all time differences in the r-th cluster and the mean of all time differences in the time difference sequence. The larger the value, the greater the difference between the two. The first difference of the sth time difference in the rth cluster is used to reflect the difference between the sth time difference in the rth cluster and the mean of all time differences in the rth cluster. The larger the value, the greater the difference between the two.

[0096] The average value of the first difference of all time differences in the rth cluster is used to reflect the overall distribution level of the first difference of all time differences in the rth cluster. When the overall time interval of the rth cluster is smaller and the average value of the first difference of all time differences in the rth cluster is smaller, it means that the cluster is more likely to meet the level of the overall time difference series, and the confidence that the data in the cluster is the time delay difference level of the two data series is greater.

[0097] By adopting the above method, the confidence of each cluster can be obtained. The cluster with the largest confidence is taken as the target cluster, and the mean of all time differences in the target cluster is taken as the delay time of the sequence to be aligned. Based on the delay time, the data corresponding to the acquisition time in the sequence to be aligned is matched with the data corresponding to the acquisition time in the reference sequence to obtain the matching data corresponding to the data in the reference sequence; specifically: assuming that the delay time is u seconds, for the jth data in the reference sequence, the j+uth data in the first sequence is the matching data of the jth data in the reference sequence; for the j+1th data in the reference sequence, the j+u+1th data in the first sequence is the matching data of the j+1th data in the reference sequence; for the j+2th data in the reference sequence, the j+u+2th data in the first sequence is the matching data of the j+2th data in the reference sequence; and so on; when the reference data has no corresponding matching data at the end, the data prediction algorithm is used to predict the subsequent part of the matching sequence, and then the missing part is matched with the reference data, that is, the alignment of the two sequences is completed. The data prediction algorithm is a prior art and will not be described in detail here. It should be noted that: if the number of collected monitoring data sequences is greater than 2, a reference sequence is manually set according to the type of monitoring data, and the method provided in this embodiment is used to align each monitoring data sequence except the reference sequence with the reference sequence in turn.

[0098] Step S4: Decoding the reference sequence and the aligned monitoring data sequence using a multi-mode physical simulation system.

[0099] In the above steps, the data alignment processing of the sequence to be aligned and the reference sequence is completed, and the aligned monitoring data sequence is obtained, that is, the preprocessing of the collected monitoring data sequence is completed, and then the multi-mode physical simulation system is used to perform analysis operations such as convolution, principal component analysis and change correlation on the preprocessed data information. According to the needs of the multi-mode physical simulation system, the preprocessed data is subjected to feature extraction and decoding operations to obtain useful information of the data. The extracted information is fused using a multi-modal neural network or other neural networks, and an appropriate model is selected in combination with the simulation task, such as a physical engine model, a data-driven model (such as a deep learning model), a hybrid model, etc., to obtain the final actual simulation model.

[0100] So far, the decoding process of the monitoring data is completed using the method provided by this embodiment.

[0101] This embodiment takes into account that in the data decoding process of the multimodal physical simulation system, the various monitoring data collected by the physical object are mainly feature extracted and fused, so as to simulate the simulation components that are closer to the characteristic attributes of the physical object through the multimodal physical simulation system. However, the types of monitoring data collected may be large, and the collected monitoring data may have time delays, resulting in time differences between different types of monitoring data. Therefore, in order to improve the accuracy of the system in decoding the collected data and performing feature fusion and other operations, it is necessary to time align the collected monitoring data with delays. This embodiment first combines each data in the reference sequence with each data in the sequence to be aligned. Based on the correlation of data in local windows between the two, the difference in the fluctuation amplitude of data in local windows and the difference in collection time, multiple matching data pairs are obtained, and all time differences are classified based on the time difference between the data in the matching data pairs. A more accurate delay duration is obtained through the comprehensive classification results, and then the data in the sequence to be aligned and the reference sequence are aligned based on the delay duration. The reference sequence and the aligned monitoring data sequence are decoded by the multi-mode physical simulation system, which improves the accuracy of the multi-mode physical simulation system in the process of collecting, decoding, fusion and analyzing the attribute information of the physical object, making the simulated physical attributes more accurate and ensuring the ultimate accuracy of the system.

[0102] An embodiment of a data decoding system for a multi-mode physical simulation system based on artificial intelligence:

[0103] See also Figure 3 , which shows a structural block diagram of a data decoding system for a multi-mode physical simulation system based on artificial intelligence provided by an embodiment of the present invention. The system may include a data acquisition module, a matching data pair screening module, a data alignment module and a data decoding module.

[0104] Among them, the data acquisition module is used to obtain different types of monitoring data sequences of physical objects;

[0105] A matching data pair screening module, used to obtain matching data pairs by combining the correlation of data in a local window between each data in the reference sequence and each data in the sequence to be aligned, the difference in the fluctuation amplitude of the data in the local window, and the difference in the acquisition time; the reference sequence is one monitoring data sequence in every two monitoring data sequences, and the sequence to be aligned is the other monitoring data sequence; wherein the local window size is determined based on the local fluctuation characteristics of each data in the two monitoring data sequences and the overall fluctuation characteristics of the monitoring data sequences;

[0106] A data alignment module is used to classify all time differences based on the time differences between the data in the matching data pair, determine the delay time of the sequence to be aligned based on the classification results, and align the sequence to be aligned with the data in the reference sequence using the delay time;

[0107] The data decoding module is used to decode the reference sequence and the aligned monitoring data sequence using the multi-mode physical simulation system.

[0108] It should be understood that Figure 3 The structural block diagram of a data decoding system of a multi-mode physical simulation system based on artificial intelligence and its modules can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented by hardware, software, or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art can understand that the above-mentioned method and system can be implemented using computer executable instructions and / or included in a processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The system and its modules of this specification can not only be implemented by hardware circuits such as ultra-large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., but can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software (for example, firmware).

[0109] For more details about the above modules, please refer to other places in this manual (for example Figure 1 , Figure 2 part and its related description), which will not be repeated here.

[0110] In other embodiments, a device is also provided, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the device executes the above-mentioned multi-mode physical simulation system data decoding method based on artificial intelligence. The device can be specifically a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute the multi-mode physical simulation system data decoding method based on artificial intelligence provided in the above embodiment.

[0111] In other embodiments, a computer program product is also provided. When the computer program product runs on a computer, the computer executes the above-mentioned related steps to implement a multi-mode physical simulation system data decoding method based on artificial intelligence provided in the above embodiment.

[0112] In other embodiments, a computer-readable storage medium is also provided, in which a computer program code is stored. When the computer program code is run on a computer, the computer executes the above-mentioned related method steps to implement a multi-mode physical simulation system data decoding method based on artificial intelligence provided in the above embodiment.

[0113] Among them, the provided systems, devices, computer program products, and computer-readable storage media are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0114] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A data decoding method for a multi-mode physical simulation system based on artificial intelligence, characterized in that: The method comprises the following steps: Obtain different types of monitoring data series of physical objects; Combine the correlation of data in the local window between each data in the reference sequence and each data in the sequence to be aligned, the difference in the fluctuation amplitude of the data in the local window, and the difference in the acquisition time to obtain a matching data pair; the reference sequence is one monitoring data sequence in every two monitoring data sequences, and the sequence to be aligned is the other monitoring data sequence; wherein the local window size is determined based on the local fluctuation characteristics of each data in the two monitoring data sequences and the overall fluctuation characteristics of the monitoring data sequences; Classify all time differences based on the time differences between the data in the matching data pair, and determine the delay time of the sequence to be aligned based on the classification results; align the sequence to be aligned with the data in the reference sequence using the delay time; Decoding the reference sequence and the aligned monitoring data sequence using a multi-mode physical simulation system; The step of combining the correlation of data in a local window between each data in the reference sequence and each data in the sequence to be aligned, the difference in the fluctuation amplitude of the data in the local window, and the difference in the acquisition time to obtain a matching data pair includes: Obtaining a nonlinear factor between the data to be analyzed and each data in the sequence to be aligned according to the degree of correlation between the data in the local window of the data to be analyzed and the data in the local window of each data in the sequence to be aligned, wherein the degree of correlation is negatively correlated with the nonlinear factor; Obtaining a matching degree between the data to be analyzed and each data in the sequence to be aligned according to a nonlinear factor between the data to be analyzed and each data in the sequence to be aligned, a difference between a fluctuation amplitude of the data in a local window of the data to be analyzed and a fluctuation amplitude of the data in a local window of each data in the sequence to be aligned, and a difference between a collection time of the data to be analyzed and a collection time of each data in the sequence to be aligned, wherein the nonlinear factor, the difference between the fluctuation amplitudes and the difference between the collection times are all negatively correlated with the matching degree; Obtaining a matching data pair based on the matching degree; the data to be analyzed is any data in the reference sequence; The classifying all time differences based on the time differences between the data in the matching data pair includes: The time difference between two data in all matching data pairs constitutes a time difference series; Cluster the elements in the time difference sequence to obtain clusters; The comprehensive classification results determine the delay time of the sequence to be aligned, including: The difference between the mean of all time differences in the candidate cluster and the mean of all time differences in the time difference sequence is recorded as the overall time interval; the difference between each time difference in the candidate cluster and the mean of all time differences in the candidate cluster is recorded as the first difference of each time difference in the candidate cluster; the confidence of the candidate cluster is obtained according to the overall time interval and the overall distribution level of the first difference of all time differences in the candidate cluster; the overall time interval and the overall distribution level of the first difference are negatively correlated with the confidence; the candidate cluster is any cluster; The overall confidence of all clusters is combined to determine the delay length of the sequences to be aligned.

2. The data decoding method of a multi-mode physical simulation system based on artificial intelligence according to claim 1 is characterized in that: The acquisition of the local window size includes: For any two types of monitoring data series: One of the monitoring data sequences is recorded as the first sequence, and any data in the other monitoring data sequence is recorded as candidate data; Determine a first coefficient based on the degree of dispersion of all data in the monitoring data sequence where the candidate data is located and the degree of dispersion of data in a time neighborhood of the candidate data, wherein the degree of dispersion of all data is positively correlated with the first coefficient, and the degree of dispersion of data in the time neighborhood is negatively correlated with the first coefficient; Determine a second coefficient based on the number of extreme value points in the monitoring data sequence where the candidate data is located and the number of extreme value points in the first sequence, wherein the number of extreme value points in the monitoring data sequence where the candidate data is located is negatively correlated with the second coefficient, and the number of extreme value points in the first sequence is positively correlated with the second coefficient; The local window size of the candidate data is obtained by combining the first coefficient and the second coefficient, and the first coefficient and the second coefficient are both positively correlated with the length of the local window.

3. The data decoding method of a multi-mode physical simulation system based on artificial intelligence according to claim 1 is characterized in that: The obtaining of a matching data pair based on the matching degree comprises: The data to be analyzed and the data with the greatest matching degree constitute a matching data pair.

4. The data decoding method of a multi-mode physical simulation system based on artificial intelligence according to claim 1 is characterized in that: An iterative self-organizing clustering algorithm is used to cluster the elements in the time difference series.

5. The data decoding method of a multi-mode physical simulation system based on artificial intelligence according to claim 1 is characterized in that: The step of determining the delay time of the sequence to be aligned by combining the overall confidence of all clusters includes: The cluster with the largest confidence is taken as the target cluster; The mean of all time differences within the target cluster is taken as the delay duration of the sequence to be aligned.

6. The data decoding method of a multi-mode physical simulation system based on artificial intelligence according to claim 1 is characterized in that: The step of aligning the sequence to be aligned with the data in the reference sequence by using the delay time comprises: Matching the data corresponding to the acquisition time in the sequence to be aligned with the data corresponding to the acquisition time in the reference sequence based on the delay time to obtain matching data corresponding to the data in the reference sequence; If there is data in the reference sequence that has no corresponding matching data, the data prediction algorithm is used to fill in the data.

7. The data decoding method of a multi-mode physical simulation system based on artificial intelligence according to claim 1 is characterized in that: The obtaining of different types of monitoring data sequences of physical objects includes: For any type of monitoring data: All monitoring data of this type are sorted in order of collection time to obtain a monitoring data sequence of this type.

Images