A self-tuning data fusion method for MEMS array accelerometer based on reinforcement learning

Abstract

The application provides a MEMS array accelerometer self-tuning data fusion method based on reinforcement learning, in a working mode, the MEMS array accelerometer opens a specific data channel under the control of a channel decision switching model, obtains to-be-fused data through a data acquisition and preprocessing module, the to-be-fused data is logarithmically mapped and transformed to serve as input of the channel decision switching model and a fusion weight generation model, the channel decision switching model updates the channel enablement of the MEMS array accelerometer according to the input, the fusion weight generation model generates fusion weights according to the input, and a data fusion module generates fusion output data according to the fusion weights and the to-be-fused data; in a calibration mode, the method further comprises that the fusion output data and calibration data generate rewards through a reward mechanism module, and the channel decision switching model and the fusion weight generation model adjust their own strategy models according to the rewards. The application has the advantages of high adaptability and self-learning.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of multi-sensor data fusion, in particular to a MEMS array accelerometer self-tuning data fusion method based on reinforcement learning. BACKGROUND

[0002] With the continuous development of precision manufacturing and precision measurement technology, a single sensor often cannot meet the detection requirements. For example, a high-end grinding and polishing robot for large-aperture optical devices, the vibration amplitude of the SAGEM grinding machine is between tens of nanometers and microns, and the frequency width is from Hz to hundreds of Hz. The acceleration sensor is required to have a wide dynamic range of 7 orders of magnitude and a wide frequency response characteristic of 1 to 500 Hz. The existing piezoelectric, resonant, capacitive, and other MEMS acceleration sensors cannot meet the requirements.

[0003] Based on multiple performance complementary MEMS accelerometers, a higher performance array sensor is constructed, which is an effective solution to the above problems. Multi-sensor data fusion is a key technology in this scheme, so it is required to have high precision (fusion precision is not lower than the precision of any sub-sensor) and low delay (less than 2ms). Most of the existing data-level fusion algorithms cannot meet the above requirements. Although a few methods such as adaptive weighted fusion and fuzzy logic reasoning can meet the requirements, the premise is to have a proper strategy model. However, the design of the strategy model often needs to rely on intelligent optimization algorithms, which has the problems of long time consumption and difficulty in convergence. When there are characteristic differences among the same level measurement units of multiple array sensors, a strategy model needs to be customized for each array sensor to avoid the problem of reduced precision due to inaccurate data fusion model. This is obviously not the best solution. SUMMARY

[0004] The purpose of the present application is to provide a MEMS array accelerometer self-tuning data fusion method based on reinforcement learning.

[0005] The technical solution for achieving the purpose of the present application is: a MEMS array accelerometer self-tuning data fusion method based on reinforcement learning, comprising the following steps:

[0006] Step 1: According to the dynamic range of each channel of the MEMS array accelerometer, establish a reward playback pool and a channel switching threshold, initialize a channel decision switching model and a fusion weight generation model, and generate an initial channel enable signal.

[0007] Step 2: Collect data from the enabled channels and perform necessary preprocessing operations. The preprocessed data, on the one hand, waits for data fusion, and on the other hand, after logarithmic mapping transformation, serves as the input of the channel decision switching model and the weight generation model. The channel decision switching model will generate channel decisions based on the input and update the channel enable signal. The fusion weight generation model will generate data fusion weights based on the input. The data fusion module fuses the preprocessed data according to the weights to obtain fused output data. If in calibration mode, execute step 3, otherwise repeat step 2 until the operation ends.

[0008] In step 3, the fusion output data obtained in step 2 and the calibration reference data are mapped through the reward mechanism to generate rewards. For the channel decision switching model, the current action and the reward obtained for this action are stored in pairs in the reward replay pool. Based on the statistics of the reward replay pool, the channel switching threshold and the parameters of the trial action generation network are updated using an offline update method. For the fusion weight generation model, the reward value will be used to update the parameters of the fusion weight generation network, and then jump to step 2 until the self-tuning is completed or the calibration is completed.

[0009] A MEMS array accelerometer self-tuning data fusion system based on reinforcement learning realizes MEMS array accelerometer self-tuning data fusion based on reinforcement learning based on the MEMS array accelerometer self-tuning data fusion method.

[0010] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the self-tuning data fusion of the MEMS array accelerometer based on the self-tuning data fusion method is implemented.

[0011] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the self-tuning data fusion method of the MEMS array accelerometer is used to implement the self-tuning data fusion of the MEMS array accelerometer based on reinforcement learning.

[0012] Compared with the existing technology, the present invention has the following significant advantages: 1) It adopts a channel decision switching strategy to collect data from two adjacent channels each time. Compared with the existing multi-sensor data fusion method, it avoids data redundancy, makes the data fusion speed faster and is not affected by the number of sub-sensors; 2) It adopts a logarithmic mapping transformation strategy to improve the recognition of small signal states, improve the accuracy of the model, and enhance the robustness of the algorithm; 3) It adopts a self-tuning method based on reinforcement learning to realize online self-tuning of the data fusion model during the calibration process, improves the problem of data fusion model inaccuracy caused by the difference in characteristics of the same-level measurement units of the array accelerometer, and has the advantages of high adaptability and self-learning compared with the existing multi-sensor data fusion method. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is the overall architecture diagram of the MEMS array accelerometer self-tuning data fusion method based on reinforcement learning in the present invention.

[0014] Figure 2 This is an architectural diagram of the channel decision switching model in the MEMS array accelerometer self-tuning data fusion method based on reinforcement learning in the present invention.

[0015] Figure 3 This is an architectural diagram of the fusion weight generation model in the MEMS array accelerometer self-tuning data fusion method based on reinforcement learning in the present invention.

[0016] Figure 4 This is a schematic diagram of the dynamic range complementarity and channel switching threshold setting of MEMS accelerometers in the MEMS array accelerometer self-tuning data fusion method based on reinforcement learning of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0018] The overall architecture of the MEMS array accelerometer self-tuning data fusion method based on reinforcement learning of the present invention is as follows: Figure 1 As shown in Figure 2, it includes five parts: channel decision switching model, fusion weight generation model, reward mechanism module, data fusion module, data acquisition and preprocessing module. The channel decision switching model architecture is shown in Figure 2. Figure 2 As shown, the architecture of the fusion weight generation model is as follows Figure 3 shown.

[0019] The online self-tuning data-level fusion method has two modes: calibration and working. In the calibration mode, the MEMS array accelerometer opens a specific data channel under the control of the channel decision switching model, and obtains the data to be fused through the data acquisition and preprocessing module. The data to be fused is transformed by logarithmic mapping and used as the input of the channel decision switching model and the fusion weight generation model. The channel decision switching model will update the MEMS array accelerometer channel enable according to the input, and the fusion weight generation model will generate fusion weights according to the input. The data fusion module generates fusion output data according to the fusion weight and the data to be fused. The fusion output data and the calibration data are rewarded by the reward mechanism module. The channel decision switching model and the fusion weight generation model adjust their own strategy models according to the reward value until the self-tuning is completed or the calibration is completed; in the working mode, Figure 1 The signal flow represented by the dotted line in the figure will be closed, and the fused output data will be directly used as the output of the MEMS array accelerometer.

[0020] This embodiment assumes that the MEMS array accelerometer includes three data channels. The implementation of this method specifically includes the following steps:

[0021] Step 1: Initialize the algorithm model according to the parameters of the MEMS array accelerometer

[0022] The MEMS array accelerometer hardware platform of this embodiment is composed of three MEMS accelerometers with complementary dynamic ranges. Figure 4 As shown, Figure 4 H in i and L i Represent the upper and lower limits of the operating range of the i-th MEMS accelerometer (H i and L i Take the logarithmic scale, for example, the lower limit of the dynamic range of channel 3 is 10mg, that is, 10 -2 g, so L3 = -2), and initialize the algorithm model according to the above parameters.

[0023] The initialization process of the algorithm model specifically includes the following three sub-steps:

[0024] Step 11: Initialize the channel switching decision model. Sort the three data channels in a logarithmic coordinate system according to the center of their dynamic range and group them in pairs. This results in two channel combinations: "channel 1 + channel 2" and "channel 2 + channel 3". Therefore, set a channel switching threshold T1 and a reward replay pool. Initialize T1 to the mean of L3 and H1, and randomly initialize the parameters of the trial action generation network.

[0025] Step 12: Initialize the fusion weight generation model and pre-train the fusion weight generation network so that the weight of channel 1 data tends to 1 when the input is L1~L2, the weight of channel 3 data tends to 1 when the input is H1~H3, and the output weight tends to 0.5 when the input is L2~H1;

[0026] Step 13: Initialize the channel enable signal E0 and select the channel combination with a larger dynamic range, that is, enable channel 2 and channel 3;

[0027] Step 2: Data collection and preprocessing, forward propagation, and data fusion

[0028] In channel enable E t Under the control of the signal, the data of the current channel combination is synchronously collected and sorted according to step 11, respectively expressed as d H and d L , for d H and d L Perform necessary preprocessing operations such as outlier correction and missing value filling, and convert log(d H ) and log(d L ) Input the fusion weight generation model and the channel decision switching model to obtain the new channel enable signal E t+1 and data fusion weight ω t , the data fusion module is based on d H d L and ω t Get the fusion output data d out If in calibration mode, proceed to step 3, otherwise repeat step 2 until the operation is completed.

[0029] An example of the data collection and preprocessing implementation method in step 2 is as follows:

[0030] Each channel of the MEMS array accelerometer communicates with the microcontroller unit (MCU) through the SPI protocol. A dual SPI channel is established and the DMA function is enabled. The two channels in each channel combination are connected to different SPI ports to achieve synchronous data acquisition. Then, outlier detection is performed and four historical fusion outputs are recorded. out , where the maximum value d Max , the second largest value d max , the next smallest value d min , minimum value d Min , then it is greater than 0.75×(d max +d min )+d Max -d Min or less than 0.25×(d max +d min )-d Max +d Minis an abnormal value, when a single channel has missing values or abnormal values, the data of another channel is used for filling or replacing, and when both channels have missing values or abnormal values, the last d out is filled or replaced.

[0031] An example of the forward propagation and data fusion implementation method in step 2 is as follows:

[0032] log(d H ) and log(d L ) are input into the channel decision switching model, and a new channel decision is generated by using the channel decision mapping module therein, and the representation of the channel decision mapping is as formula (1),

[0033]

[0034] In the formula, D t represents the generated channel decision, which is used to generate a new channel enable signal E t+1 , if D t =1, channel 1 and channel 2 are enabled, and if D t =2, channel 2 and channel 3 are enabled; log(d H ) and log(d L ) are input into the fusion weight generation model, and the data fusion weight ω t is obtained by forward propagation of the fusion weight generation network, and the fusion output data d H is obtained by the data fusion module according to d L , d t and ω out , and the representation of the data fusion is as formula (2).

[0035] d out =ω t d L +(1-ω t )d H Formula (2)

[0036] Step 3: Algorithm model self-tuning

[0037] The fusion output data d out obtained in step 2 and the calibration reference data d ref are mapped to generate a reward R by using the reward mechanism, for the channel decision switching model, the action A t and the reward R t obtained by the action are stored in pairs in the reward replay pool, and the parameters W1 of the channel switching threshold T and the exploration action generation network are updated according to the situation of the experience replay pool, for the fusion weight generation model, the parameters W2 of the fusion weight generation network are updated according to the reward value R t , and then jump back to step 2 until the algorithm model self-tuning is completed or the calibration is ended.

[0038] Step 3 includes the following two sub-steps:

[0039] Step 31: Reinforcement Learning Agent 1 Perform channel decision switching model self-tuning. When the data enters the range of L3 to H1, log(d H ) and log(d L ) is input to the tentative action generation network and passed through f s The (softmax) function obtains three outputs P1, P2 and P3, which represent the probabilities of the three actions of "decrease", "maintain" and "increase" for the threshold T1 respectively. The trial action Tr is selected according to the probability. t Increase ΔT, decrease ΔT or keep it unchanged on the current threshold, and then generate a temporary threshold T'1 and use it for channel decision mapping, and use the reward mechanism module to obtain the reward R t+1 , Agent 1 The reward mechanism is expressed as formula (3),

[0040]

[0041] Where ε is the maximum relative error allowed. t When (the "state" at time t) is between T1 and T'1, {Tr t ,S t ,R t+1} is stored in the reward replay pool of T1. When the number of records of the three actions in the reward replay pool reaches the threshold M, the average rewards obtained by the three actions of "decrease", "maintain" and "increase" are calculated. By performing an action with the highest average reward To update the threshold T1, and use the average reward to create the training label L = [l1, l2, l3]. The specific process is expressed as formula (4).

[0042]

[0043] Utilize actions The corresponding S t and label L to form a training sample, train the trial action generation network, and delete the action from the reward replay pool In step 31, the lower limit of probability P is set by modifying the softmax function to solve the problem of sample imbalance. The recommended setting is 20%; the initial value of ΔT is generally 1 / 4 of the distance between L3 and H1, and it decays continuously with the self-tuning process to enable the model to obtain higher convergence accuracy; the recommended setting of M is 6 to 30. A larger M will reduce the model convergence speed, and a smaller M will reduce the accuracy and stability of the model convergence. The cross entropy loss function should be used when training the trial action generation network. The specific parameter update process is expressed as formula (5), where α is the learning rate, and the recommended setting is 10 -1 ~10 -4 .

[0044]

[0045] Step 32: Reinforcement Learning Agent 2 Perform fusion weight generation model self-tuning. H ) and log(d L ) is input to the fusion weight generation network and normalized by the function f n Map the output to the interval [0,1] to obtain the data fusion weight ω t , normalization function f n The expression of is as shown in formula (6),

[0046]

[0047] The data fusion module is based on d H d L and ω t Get the fusion output data d out , use the reward mechanism module to obtain the reward R t+1 , Agent 2 The reward mechanism is expressed as formula (3), based on the gradient ascent method using R t+1 Update the parameters of the fusion weight generation network. The specific parameter update process is expressed as formula (7).

[0048]

[0049] The present invention also proposes a MEMS array accelerometer self-tuning data fusion system based on reinforcement learning, and realizes MEMS array accelerometer self-tuning data fusion based on reinforcement learning based on the MEMS array accelerometer self-tuning data fusion method.

[0050] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the processor implements reinforcement learning-based self-tuning data fusion of the MEMS array accelerometer based on the self-tuning data fusion method of the MEMS array accelerometer.

[0051] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the self-tuning data fusion method of the MEMS array accelerometer is used to implement the self-tuning data fusion of the MEMS array accelerometer based on reinforcement learning.

[0052] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0053] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A self-tuning data fusion method for MEMS array accelerometer based on reinforcement learning, characterized in that: It has two modes: calibration and working. In working mode, the MEMS array accelerometer opens a specific data channel under the control of the channel decision switching model, obtains the data to be fused through the data acquisition and preprocessing module, and uses the data to be fused as the input of the channel decision switching model and the fusion weight generation model after logarithmic mapping transformation. The channel decision switching model will update the MEMS array accelerometer channel enable according to the input, and the fusion weight generation model will generate fusion weights according to the input. The data fusion module generates fusion output data according to the fusion weights and the data to be fused, and the fusion output data is directly used as the output of the MEMS array accelerometer; In calibration mode, it also includes a model self-tuning process, in which the obtained fusion output data and calibration reference data are mapped through a reward mechanism to generate rewards. For the channel decision switching model, the action and the reward obtained for the action are stored in pairs in the reward replay pool, and the channel switching threshold and the parameters of the trial action generation network are updated according to the situation in the experience replay pool. For the fusion weight generation model, the parameters of the fusion weight generation network are updated according to the reward value until the algorithm model self-tuning is completed or the calibration is completed.

2. The MEMS array accelerometer self-tuning data fusion method based on reinforcement learning according to claim 1 is characterized in that: It includes a model initialization process. The MEMS array accelerometer hardware platform consists of three MEMS accelerometers with complementary dynamic ranges, H i and L i respectively represent the upper and lower limits of the working range of the i-th MEMS accelerometer, and H1 < H2 < H3, L1 < L2 < L3. Then the model initialization method is as follows: (11) Initialize the channel switching decision model, sort the three data channels in a logarithmic coordinate system according to the center of the dynamic range, and group them in pairs in order to obtain two channel combinations: "channel 1 + channel 2" and "channel 2 + channel 3". Set a channel switching threshold T1 and a reward replay pool, initialize T1 to the mean of L3 and H1, and randomly initialize the parameters of the trial action generation network; (12) Initializing the fusion weight generation model and pre-training the fusion weight generation network; (13) Initialize the channel enable signal E0 and select the channel combination with a larger dynamic range.

3. The MEMS array accelerometer self-tuning data fusion method based on reinforcement learning according to claim 2, characterized in that: The MEMS array accelerometer opens a specific data channel under the control of the channel decision switching model, and obtains the data to be fused through the data acquisition and preprocessing module. The specific method is as follows: Each channel of the MEMS array accelerometer communicates with the microcontroller unit MCU through the SPI protocol, establishes dual SPI channels and turns on the DMA function, connects the two channels in each channel combination to different SPI ports, and then realizes data synchronous acquisition. Then, outlier detection is performed and the four historical fusion outputs are recorded. out , where the maximum value d Max , the second largest value d max , the next smallest value d min , minimum value d Min , then it is greater than 0.75×(d max +d min )+d Max -d Min or less than 0.25×(d max +d min )-d Max +d Min When a single channel has missing values ​​or abnormal values, it is filled or replaced with the data of another channel. When both channels have missing values ​​or abnormal values, the last channel is used to fill or replace them. out Fill or replace.

4. The MEMS array accelerometer self-tuning data fusion method based on reinforcement learning according to claim 3 is characterized in that: After the data to be fused undergoes logarithmic mapping transformation, it serves as the input of the channel decision switching model and the fusion weight generation model. The channel decision switching model will update the MEMS array accelerometer channel enable according to the input, and the fusion weight generation model will generate fusion weights according to the input. The data fusion module generates fusion output data based on the fusion weights and the data to be fused. The specific method is as follows: In channel enable E t Under the control of the signal, the data of the current channel combination is collected synchronously and expressed as d according to the order. H and d L , for d H and d L After the outlier correction and missing value filling preprocessing operations, log(d H ) and log(d L ) Input the fusion weight generation model and the channel decision switching model, where the channel decision mapping module generates a new channel decision. The channel decision mapping is expressed as formula (1). Where D t Represents the generated channel decision, which is used to generate a new channel enable signal E t+1 , if D t =1, then enable channel 1 and channel 2. If D t =2 enables channel 2 and channel 3; Log(d H ) and log(d L ) Input the fusion weight generation model, and forward propagate through the fusion weight generation network to obtain the data fusion weight ω t , the data fusion module is based on d H d L and ω t Get the fusion output data d out , the data fusion is expressed as formula (2), d out = ω t d L + (1 - ω t )d H Equation (2).

5. The MEMS array accelerometer self-tuning data fusion method based on reinforcement learning according to claim 4 is characterized in that: The fusion output data and calibration data are rewarded by the reward mechanism module. The channel decision switching model and the fusion weight generation model adjust their own strategy model according to the reward value. 1 Perform channel decision switching model self-tuning. The specific method is as follows: When the data enters the range of L3 to H1, log(d H ) and log(d L ) is input to the trial action generation network, and then s The (softmax) function obtains three outputs P1, P2 and P3, which represent the probabilities of the three actions of "decrease", "maintain" and "increase" for the threshold T1 respectively. The trial action is selected according to the probability. t Increase ΔT, decrease ΔT or keep it unchanged on the current threshold, and then generate a temporary threshold T'1 and use it for channel decision mapping, and use the reward mechanism module to obtain the reward R t+1 , Agent 1 The reward mechanism is expressed as formula (3), Where ε is the maximum relative error allowed. When the state S at time t is t When {Tr t ,S t ,R t+1 } is stored in the reward replay pool of T1. When the number of records of the three actions in the reward replay pool reaches the threshold M, the average rewards obtained by the three actions of "decrease", "maintain" and "increase" are calculated. By performing an action with the highest average reward To update the threshold T1, and use the average reward to create the training label L = [l1, l2, l3]. The specific process is expressed as formula (4). Utilize actions The corresponding S t and label L to form a training sample, train the trial action generation network, and delete the action from the reward replay pool records; The cross entropy loss function should be used when training the trial action generation network. The specific parameter update process is expressed as formula (5), where α is the learning rate, which is set to 10 -1 ~10 -4 ; 6. The MEMS array accelerometer self-tuning data fusion method based on reinforcement learning according to claim 4, characterized in that: The fusion output data and calibration data are rewarded by the reward mechanism module. The channel decision switching model and the fusion weight generation model adjust their own strategy model according to the reward value. 2 Perform self-tuning of the fusion weight generation model. The specific method is as follows: Log(d H ) and log(d L ) is input to the fusion weight generation network and normalized by the function f n Map the output to the interval [0,1] to obtain the data fusion weight ω t , normalization function f n The expression of is as shown in formula (6), The data fusion module is based on d H d L and ω t Get the fusion output data d out , use the reward mechanism module to obtain the reward R t+1 , Agent 2 The reward mechanism is expressed as formula (3), based on the gradient ascent method using R t+1 Update the parameters of the fusion weight generation network. The specific parameter update process is expressed as formula (7).

7. A MEMS array accelerometer self-tuning data fusion system based on reinforcement learning, characterized in that: Based on the MEMS array accelerometer self-tuning data fusion method according to any one of claims 1 to 6, MEMS array accelerometer self-tuning data fusion based on reinforcement learning is realized.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for self-tuning data fusion of a MEMS array accelerometer according to any one of claims 1 to 6 is used to implement self-tuning data fusion of a MEMS array accelerometer based on reinforcement learning.

9. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for self-tuning data fusion of a MEMS array accelerometer according to any one of claims 1 to 6 is used to implement self-tuning data fusion of a MEMS array accelerometer based on reinforcement learning.

Images