A cutting state prediction method, system and device based on mechanical signals

Abstract

The application discloses a cutting state prediction method, system and device based on mechanical signals, and the method comprises the following steps: preprocessing the sampling data of cutting force; inputting the first time sequence data into the first data embedding layer of the space-time prior network to obtain the first embedding sequence; inputting the first embedding sequence into the encoder network layer of the space-time prior network to obtain the encoder output feature sequence; inputting the second time sequence data into the second data embedding layer to obtain the second embedding sequence; inputting the second embedding sequence and the encoder output feature sequence into the decoder network layer of the space-time prior network to obtain the decoder output feature sequence; inputting the decoder output feature sequence into the full connection layer of the space-time prior network to obtain the output sequence data, and taking the last l p elements in the output sequence data as the final prediction result. The application has high prediction accuracy, low operation complexity, fast operation speed and strong real-time performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of predicting the cutting state of surgical instruments, and in particular relates to a cutting state prediction method, system and device based on mechanical signals. BACKGROUND

[0002] Laminectomy is a high-difficulty spine surgery. Since robots have the advantages of high precision and can combine preoperative medical images for preoperative surgical path planning, they have good application prospects in laminectomy. The spine is an area with very dense nerve distribution, so laminectomy puts high requirements on the cutting state recognition of robots. Although the cutting state recognition based on mechanical, sound, impedance and other modal signals has high accuracy, high-temperature liquid splashing during cutting may still cause thermal damage to nerves. Therefore, to avoid damage to nerves during cutting, the cutting state of the robot needs to be recognized and predicted.

[0003] When a robot performs laminectomy, accurate and efficient signal prediction is needed to ensure safety and cutting efficiency. When only using the collected signals to infer the cutting state of the robot, the signals will only show obvious characteristic changes when the laminectomy is penetrated, and the algorithm operation and electromechanical system reaction delay may even cause cutting to stop lagging behind. Therefore, to ensure the real-time response of the robot system, the cutting state of the robot needs to be predicted in advance. SUMMARY

[0004] In view of the above defects or deficiencies in the prior art, the present application provides a cutting state prediction method, system and device based on mechanical signals, which has high prediction accuracy, low computational complexity, fast operation speed and strong real-time performance.

[0005] In one aspect of the present application, a cutting state prediction method based on mechanical signals is provided, comprising:

[0006] Preprocessing the sampling data of the cutting force to obtain first time series data with a length of l; concatenating the last l-l p bit data of the first time series data with a length of l p zero sequence to obtain second time series data;

[0007] Inputting the first time series data into a first data embedding layer of a spatio-temporal prior network to obtain a first embedding sequence;

[0008] Inputting the first embedding sequence into an encoder network layer of the spatio-temporal prior network to obtain an encoder output feature sequence;

[0009] inputting the second time sequence data into a second data embedding layer to obtain a second embedding sequence;

[0010] inputting the second embedding sequence and the encoder output feature sequence into a decoder network layer of the spatio-temporal prior network to obtain a decoder output feature sequence;

[0011] inputting the decoder output feature sequence into a full connection layer of the spatio-temporal prior network to obtain an output sequence data, and taking the last l p elements in the output sequence data as a final prediction result.

[0012] Further, the first data embedding layer or the second data embedding layer comprises: an element embedding module, configured to realize expansion of a feature channel number through one-dimensional convolution and output an element embedding sequence; and a position embedding module, configured to add position embedding information on the basis of the element embedding sequence to obtain context information of the sequence and output a position embedding sequence; and the element embedding sequence and the position embedding sequence are added to obtain the first embedding sequence or the second embedding sequence.

[0013] Further, the encoder network layer comprises a plurality of encoder basic layers with down-sampling and one encoder basic layer without down-sampling connected in sequence; the encoder basic layer with down-sampling comprises a plurality of Swin-transformer modules stacked and one token merging module; and the encoder basic layer without down-sampling comprises a plurality of Swin-transformer modules stacked.

[0014] Further, the decoder network layer comprises a plurality of decoder basic layers, each of which comprises a plurality of Swin-transformer modules stacked and one single Swin-transformer module; the plurality of Swin-transformer modules stacked are configured to process the second embedding sequence data and input a processed result into the single Swin-transformer module; and the single Swin-transformer module receives the processing result of the plurality of Swin-transformer modules stacked and the encoder output feature sequence, and performs fusion processing on the received data to output the decoder output feature sequence.

[0015] Further, the step of pre-processing the sampling data of the cutting force comprises Kalman filtering and normalization processing on the sampling data.

[0016] In another aspect of the present application, a cutting state prediction system based on mechanical signals is provided, comprising:

[0017] a preprocessing module configured to preprocess the sampling data of the cutting force to obtain first time series data with a length of l; concatenate the last l-l p bit data of the first time series data with a full zero sequence with a length of l p to obtain second time series data;

[0018] a first embedding module configured to input the first time series data to a first data embedding layer of a spatio-temporal prior network to obtain a first embedding sequence;

[0019] an encoder module configured to input the first embedding sequence to an encoder network layer of the spatio-temporal prior network to obtain an encoder output feature sequence;

[0020] a second embedding module configured to input the second time series data to a second data embedding layer of the spatio-temporal prior network to obtain a second embedding sequence;

[0021] a decoder module configured to input the second embedding sequence and the encoder output feature sequence to a decoder network layer of the spatio-temporal prior network to obtain a decoder output feature sequence;

[0022] a result output module configured to input the decoder output feature sequence to a fully connected layer of the spatio-temporal prior network to obtain output sequence data, and take the last l p elements in the output sequence data as a final prediction result.

[0023] Further, the first data embedding layer or the second data embedding layer comprises:

[0024] an element embedding module for expanding the number of feature channels through one-dimensional convolution and outputting an element embedding sequence; and

[0025] a position embedding module for adding position embedding information on the basis of the element embedding sequence to obtain context information of the sequence and output a position embedding sequence;

[0026] adding the element embedding sequence and the position embedding sequence to obtain the first embedding sequence or the second embedding sequence.

[0027] Further, the encoder network layer comprises a plurality of encoder basic layers with downsampling and one encoder basic layer without downsampling connected in sequence;

[0028] The encoder basic layer with downsampling comprises a plurality of Swin-transformer modules and one token merging module stacked; and the encoder basic layer without downsampling comprises a plurality of Swin-transformer modules stacked.

[0029] Further, the decoder network layer comprises a plurality of decoder basic layers, each decoder basic layer comprising a plurality of stacked Swin-transformer modules and one single Swin-transformer module;

[0030] The plurality of stacked Swin-transformer modules are configured to process the second embedding sequence data and input the processed result into the one single Swin-transformer module;

[0031] The one single Swin-transformer module receives the processing result of the plurality of stacked Swin-transformer modules and the encoder output feature sequence, and performs fusion processing on the received data to output the decoder output feature sequence.

[0032] In another aspect, the application also provides a cutting state prediction device based on mechanical signals, comprising a preprocessing module, a space-time prior network and a result output module; wherein,

[0033] The preprocessing module is configured to preprocess the sampling data of cutting force to obtain first time sequence data with a length of l; concatenate the last l-l p bit data of the first time sequence data with a zero sequence with a length of l p to obtain second time sequence data;

[0034] The space-time prior network comprises:

[0035] A first data embedding layer is configured to perform embedding processing on the input first time sequence data to obtain a first embedding sequence, and output the first embedding sequence to an encoder network layer;

[0036] An encoder network layer is configured to perform feature encoding on the input first embedding sequence to obtain an encoder output feature sequence, and output the encoder output feature sequence to a decoder network layer;

[0037] A second data embedding layer is configured to perform embedding processing on the input second time sequence data to obtain a second embedding sequence, and output the second embedding sequence to a decoder network layer;

[0038] A decoder network layer is configured to perform decoding processing on the input second embedding sequence and the encoder output feature sequence to obtain a decoder output feature sequence, and output the decoder output feature sequence to a fully connected layer;

[0039] A full connection layer is configured to integrate all features of the decoder output feature sequence to obtain output sequence data.

[0040] The result output module is configured to take the last l p elements in the output sequence data as the final prediction result.

[0041] The application provides a cutting state prediction method, system and device based on mechanical signals, which has the following advantages:

[0042] (1) The application can achieve optimal prediction of the cutting state by predicting time sequence signals of various modes.

[0043] (2) The application has linear calculation complexity and extremely high operation efficiency.

[0044] (3) The application has extremely high accuracy in predicting force signal data of a laminectomy decompression robot system or similar systems. BRIEF DESCRIPTION OF DRAWINGS

[0045] Other features, objects and advantages of the application will become more apparent after reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0046] Figure 1 FIG. 1 is a flowchart of a cutting state prediction method based on mechanical signals according to an embodiment of the application;

[0047] Figure 2 FIG. 2 is a structural diagram of a spatio-temporal prior network used in the cutting state prediction method based on mechanical signals according to an embodiment of the application;

[0048] Figure 3 FIG. 3 is a structural diagram of a data embedding layer of the spatio-temporal prior network used according to an embodiment of the application;

[0049] Figure 4 FIG. 4 is a structural diagram of an encoder of the spatio-temporal prior network used according to an embodiment of the application;

[0050] Figure 5 FIG. 5 is a structural diagram of a decoder of the spatio-temporal prior network used according to an embodiment of the application;

[0051] Figure 6 FIG. 6 is a structural diagram of a cutting state prediction system based on mechanical signals according to another embodiment of the application;

[0052] Figure 7 FIG. 7 is a structural diagram of a cutting state prediction device based on mechanical signals according to another embodiment of the application. DETAILED DESCRIPTION

[0053] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall into the scope of the present application.

[0054] The terms used in the embodiments of the present application are only for the purpose of describing particular embodiments and are not intended to limit the present application. The singular forms "a," "an," and "the" used in the embodiments of the present application and the appended claims are intended to include plural forms as well, unless the context clearly indicates otherwise.

[0055] It should be understood that, although the terms first, second, third, etc. can be used in the embodiments of the present application to describe obtaining modules, these obtaining modules should not be limited to these terms. These terms are only used to distinguish the obtaining modules from each other.

[0056] Depending on the context, the word "if" as used herein can be interpreted as meaning "when" or "while" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if it is determined" or "if (a stated condition or event) is detected" can be interpreted as meaning "when it is determined" or "in response to determining" or "when (a stated condition or event) is detected" or "in response to detecting (a stated condition or event)".

[0057] It should be noted that the terms "upper", "lower", "left", "right", and the like described in the embodiments of the present application are described from the perspective of the angle shown in the drawings, and should not be understood as limiting the embodiments of the present application. In addition, in the context, it should also be understood that when referring to an element being formed "on" or "under" another element, it can not only be directly formed "on" or "under" another element, but also indirectly formed "on" or "under" another element through an intermediate element.

[0058] Surgical robot systems based on force, sound, impedance and other intraoperative signal state perception technology have achieved good results. Some researchers use sound signals to measure the density of the spine, and some develop robot pedicle screw implant systems based on force perception technology. Some researchers have developed multi-modal fusion intraoperative robot state perception technology by integrating sound signals and force signals. However, due to system operation time consumption and mechanical and electrical system delay, combined with the fact that modal signals only change significantly when cutting penetrates, cutting state recognition based solely on known signals may cause the robot cutting to stop lagging behind. Time series prediction can help the robot identify the cutting state in advance, thereby addressing the risks caused by lagging behind due to various factors.

[0059] Common time series prediction algorithms mainly include classic ARIMA, time series algorithms developed based on LSTM (Long Short-Term Memory), and various time series prediction algorithms developed based on Transformer. Currently, LSTM is the most widely used basic algorithm for time series prediction. However, LSTM has a severe limitation on the length of the time series to be predicted, and as the length increases, the reasoning accuracy of the model decreases sharply. Transformer has a better ability to capture global information than LSTM, and its Self-attention mechanism can greatly reduce the propagation length of information in the network, avoiding information loss and having the potential to solve long time series prediction. However, due to the high time and space complexity of the Transformer algorithm, the length of the time series to be predicted is severely limited.

[0060] To overcome the above-mentioned defects, one embodiment of the present application proposes a cutting state prediction method based on mechanical signals, which predicts various modal time series signals to achieve cutting state prediction.

[0061] Referring to Figure 1 , the method takes the cutting state prediction of a laminectomy decompression surgical robot as an example for illustration, which includes the following specific implementation steps:

[0062] Step S101, pre-process the sampling data of the cutting force to obtain first time series data with a length of l; concatenate the last l-l p bit data of the first time series data with a length of l p all-zero sequence to obtain second time series data.

[0063] Specifically, first, the robot cutting force is collected by a force sensor, and data sampling is performed at a certain time interval. Then, Kalman filtering and partial normalization methods are used to pre-process the sampling data.

[0064] In order to ensure the real-time response of the robot system, the prediction algorithm must predict the signal at a certain time interval, and cannot be continuously operated in the background. The frequency of the robot system force sensor signal used in the present application is 400HZ, and in most cases, the robot system can perform 4 feeds within 1s, so 400 known data are used to predict the next 100 numbers, so as to basically realize the prediction of the signal of the next period in advance. The present application continuously intercepts known data S exist and predicted target data S predict from the entire string of collected data through a sliding time window. Take t0 as the node, the data in [t0-399, t0] is intercepted as S exist , and the data in (t0, t0+100] is the corresponding S predict . Since the sensor frequency is high, each adjacent data is very similar, so the window is set to slide with a sampling step step, that is, the next sampling time corresponding to t0 is t0+step.

[0065] Since the data collected when the cutting vertebral bone is being cut is more, and the data corresponding to the sudden change of the signal near the cutting penetration time is less, the window sliding step step1 near the cutting penetration time is set to be less than the window sliding step step2 of the ordinary time, so as to alleviate the problem of uneven data. The mechanism for judging the sampling step is as follows:

[0066]

[0067] Due to hardware and external interference, the original mechanical signal input has serious noise, and these noises seriously interfere with signal prediction. The signal amplitude corresponding to different cutting states has a large difference, and the influence of data with large amplitude on model training is much larger than that of data with small amplitude, which will make the model have poor prediction ability for data with small amplitude. Therefore, the original mechanical signal needs to be preprocessed to ensure the prediction effect of the model on the signal.

[0068] The preprocessing method adopted includes two steps: ① Kalman filtering is performed on the original signal to reduce signal noise; and ② the original signal is partially normalized to limit the signal amplitude.

[0069] Kalman filtering can effectively reduce the noise of the signal, and the genetic algorithm is used to optimize the Kalman filter super parameter in the present application. Based on the variance between the observation value and the predicted value and the filtered value and the predicted value, an adaptability function is constructed, as shown in the following formula.

[0070]

[0071] Among them:

[0072] err m = std(src-pred) (3)

[0073] err k = std(des-pred) (4)

[0074] where src is the observation value, pred is the predicted value, and des is the Kalman filtered value. err m represents the standard deviation of the original data and the predicted data of the Kalman filter, and err k represents the standard deviation of the Kalman filter target data and the predicted data.

[0075] Through 200 rounds of iterative optimization, a Kalman filter with strong robustness is obtained, and its hyperparameters are Q = 8 x 10 -3 , R = 3 x 10 -2 .

[0076] In the cutting process, due to the different hardness of different bone quality, the corresponding force signal size is different. The signal with smaller amplitude has little effect on the optimization of the prediction model compared with the signal with larger amplitude, so that the trained prediction model has low prediction ability and low robustness for data with small force signal. In the network training process, if the data amplitude changes too much, it will seriously interfere with the training and optimization of the network. Therefore, it is necessary to normalize the data and unify the data in different amplitude intervals to [0, 1]. However, in the actual cutting process, we only know the amplitude of S exist , and do not know the amplitude of S predict . Therefore, we use a partial normalization method to realize the approximate unification of the data amplitude interval. The method is as follows:

[0077]

[0078]

[0079] Although the above method cannot completely scale all data to [0, 1], it is sufficient to limit the data in a reasonable interval, greatly reducing the impact of data distribution on network optimization. Since the maximum and minimum values of S predict cannot be known in advance in actual application, this method makes the network training result more consistent with the actual application scenario. After obtaining the prediction result, the prediction result can be mapped to its real distribution interval by min exist and max exist . min exist represents the minimum value of S exist , and max exist represents the maximum value of Sexist The maximum value in the above formula is the maximum value in the range of 0 to 1. The superscript src represents the original value before processing, and the superscript des represents the target value after processing.

[0080] In step S102, the first time series data is input to the first data embedding layer of the spatiotemporal prior network to obtain the first embedding sequence.

[0081] Referring to Figure 2 , the improved spatiotemporal prior network is used to predict the cutting force signal. The spatiotemporal prior network of the present application mainly includes an encoder and a decoder. The Swin Transformer module is used in the encoder and the decoder instead of the standard Transformer block, which greatly reduces the computational complexity. In the encoder part, the token merging operation is used to continuously reduce the feature sequence length. In the decoder, the same mechanism as in the Informer algorithm is used to obtain the output sequence in one operation, and the predicted target sequence is obtained by cutting the output sequence.

[0082] As for step 102, the force signal in the laminectomy process is a typical one-dimensional time series, and there is no other reference information. Therefore, when the spatiotemporal prior network of the present application is used for encoding, it contains two parts, namely token embedding and position embedding, as shown in Figure 3 .

[0083] The token embedding module uses a one-dimensional convolution to expand the number of feature channels, i.e., from the original feature channel number 1 to embed_dim. Therefore, the token embedding module is to encode the input original data, and each data in the original data will be encoded into a feature vector. In order to preserve the context information of the sequence, it is necessary to add position embedding on the basis of token embedding. The fixed position embedding is used in this embodiment to obtain the context information:

[0084] PE(pos,2j)=sin(pos / (2L x ) 2j / embed_dim )

[0085] PE(pos,2j+1)=cos(pos / (2L x ) 2j / embed_dim ) (7)

[0086] Wherein, PE represents position embedding, i.e. position encoding / position embedding. pos is the position of the current data in the input sequence, j represents the jth dimension in embed_dim dimensions, embed_dim represents the length of the vector mapped by one data in the input sequence, L x is a parameter set for generating the absolute position encoding matrix, generally set to 5000.

[0087] Both the token embedding module and the position embedding module will map each element of the sequence to a vector of length embed_dim without changing the length of the sequence. After obtaining the token embedding sequence and the position embedding sequence, they are added to obtain the final data embedding sequence.

[0088] It should be pointed out that before the original sequence is input into the encoder and the decoder, it is first processed by the data embedding layer composed of the token embedding module and the position embedding module.

[0089] Step S103, input the first embedding sequence into the encoder network layer of the spatio-temporal prior network to obtain an encoder output feature sequence.

[0090] Specifically, the encoder network layer (i.e. encoder) in the spatio-temporal prior network is used to encode the input data, and the decoder network layer (i.e. decoder) is used to decode the encoded data. The input of the encoder is the original signal data, and the output is a feature sequence with extracted features, which is sent to the decoder. The decoder combines part of the known sequence to perform decoding, and finally outputs to the full connection layer to obtain the final output sequence.

[0091] As shown in Figure 4 The encoder of the embodiment includes a plurality of down-sampled encoder basic layers and one non-down-sampled encoder basic layer connected in sequence. The down-sampled encoder basic layer includes a plurality of stacked Swin-transformer modules and one token merging operation module. The non-down-sampled encoder basic layer includes a plurality of stacked Swin-transformer modules.

[0092] The stacked multiple Swin Transformer modules and the token merging operation module are used to process the feature sequence to extract sequence features. In order to solve the problem that the elements (tokens) in each window cannot flow information, the application adopts a shifted window partitioning approach, cuts the first shifted_size elements (tokens) of the sequence to the last of the sequence, and uses the same method to evenly divide the newly obtained sequence into multiple non-overlapping windows. By default, the application sets

[0093]

[0094] Wherein, shifted_size represents the moving distance of the moving window, M is the size of the window, and floor means rounding down.

[0095] The application also uses a mask operation to ensure that the relationship between the elements (tokens) in the window can be calculated using the same window division method before and after window movement. It should be noted that the application also alternately uses a window multi-head self attention module (W-MSA) and a shifted window multi-head self attention module (SW-MSA) when stacking Swin Transformer blocks.

[0096] When calculating the shifted window based self-attention module, the application also introduces a relative position bias Therefore, the attention calculation method is as follows:

[0097]

[0098] Wherein, Attention is an attention mechanism in the field of deep learning, SoftMax is an activation function in the field of deep learning. Q, K, and V represent the query, key, and value matrices in the transformer field, respectively, d is the dimension of the data, and B is the relative position bias value. Indicates the dimension.

[0099] Since the relative position index between each element (token) in each window is between [-M+1, M-1], the application sets a learnable relative position bias table and in the calculation process, the corresponding relative position bias value is taken out according to the relative position index in Since the relative position index is between [-M+1, M-1], in order to correctly take out the corresponding value in The present application adds M-1 to the original relative position index basis.

[0100] The present application adds a downsampling operation in the first N1 basic layers in each encoder. Since the pooling operation will lose part of the information, unlike the Informer algorithm, the present application does not use the maximum pooling operation, but develops the token merging operation. The present application performs interval sampling in the feature sequence to obtain two sub-sequences with half the length of the original sequence, and splices in the feature channel dimension, and then uses a LayerNorm module and a fully connected layer to process it, and finally obtains a feature sequence with the same other dimensions as the original sequence, but with half the length.

[0101] Step S104, input the second time sequence data into the second data embedding layer to obtain a second embedding sequence.

[0102] Specifically, the original sequence also needs to be processed by the data embedding layer before being input into the decoder, see step S102.

[0103] Step S105, input the second embedding sequence and the encoder output feature sequence into the decoder network layer of the spatio-temporal prior network to obtain a decoder output feature sequence.

[0104] Specifically, the decoder network layer needs to receive its own input as well as the output of the encoder. Referring to Figure 5 The decoder network layer includes a plurality of decoder basic layers, each decoder basic layer includes a plurality of stacked Swin-transformer modules and one single Swin-transformer module; the plurality of stacked Swin-transformer modules are used to process the second embedding sequence data, and the processed results are input into the one single Swin-transformer module; the one single Swin-transformer module receives the processing results of the plurality of stacked Swin-transformer modules and the encoder output feature sequence, and performs fusion processing on the received data to output the decoder output feature sequence.

[0105] The structure of the decoder network layer of the present application has the following advantages:

[0106] (1) Long sequence output is generated at one time, which is faster;

[0107] (2) It avoids the accumulation of errors in dynamic decoding. Therefore,

[0108] Here, we assume the encoder input sequence length is l and the target sequence length to be predicted is l. p After truncating the input sequence of the encoder, ll p Each element (token) is associated with a string of length l. p The sequence of all zeros is concatenated to obtain the decoder input sequence of length l. Since we performed token merging on the feature sequence in the encoder, the length s of the encoder output sequence is less than l. s is the length of the encoder output tensor.

[0109] In the decoder, to prevent elements (tokens) in the sequence from focusing on elements (tokens) in a future state, a mask needs to be set. f This is used to mask the dot product of future elements (tokens). Similarly, in the Swing Transformer module, setting a mask is also necessary when calculating the attention of a moving window. s This avoids calculating attention between originally disconnected regions. The invention uses a mask... f and mask s The area that needs to be masked is filled with -∞, and the mask is set to... f and mask s Adding them together will give you a mask that satisfies both occlusion requirements. t :

[0110] mask t =mask f +mask s (10)

[0111] Among them, mask f Mask is used to shield information from the future. s Used to mask errors caused by window movement in SwingTransformer.

[0112] After all the basic layers of the decoder have been processed, since no downsampling operation is performed in the decoder, an output sequence with the same length as the input sequence will be obtained.

[0113] Step S106: Input the decoder output feature sequence into the fully connected layer of the spatiotemporal prior network to obtain output sequence data, and then input the last l in the output sequence data. p Each element is used as the final prediction result.

[0114] Specifically, the full connection layer is used to compress the feature channel number embed_dim of the decoder output feature sequence to 1, and output sequence data is obtained. p The last l elements in the output sequence data are taken out as the prediction result of the cutting force.

[0115] The cutting state prediction method based on mechanical signals provided by the embodiment can realize optimal prediction of the cutting state by predicting time sequence signals of various modes. The method has linear calculation complexity and extremely high operation efficiency, and has extremely high accuracy in predicting force signal data of a lamina decompression robot system or a similar system.

[0116] Referring to Figure 6 Another embodiment of the present application also provides a cutting state prediction system 200 based on mechanical signals, which comprises a preprocessing module 201, a first embedding module 202, an encoder module 203, a second embedding module 204, a decoder module 205, and a result output module 206. The cutting state prediction system 200 based on mechanical signals is used to execute each step in the above method embodiment.

[0117] Specifically, the cutting state prediction system 200 based on mechanical signals comprises:

[0118] The preprocessing module 201 is configured to preprocess the sampling data of the cutting force to obtain first time sequence data with a length of l; and concatenate the last l-l p bit data of the first time sequence data with a full zero sequence with a length of l p to obtain second time sequence data.

[0119] The first embedding module 202 is configured to input the first time sequence data into a first data embedding layer of a spatio-temporal prior network to obtain a first embedding sequence.

[0120] The encoder module 203 is configured to input the first embedding sequence into an encoder network layer of the spatio-temporal prior network to obtain an encoder output feature sequence.

[0121] The second embedding module 204 is configured to input the second time sequence data into a second data embedding layer of the spatio-temporal prior network to obtain a second embedding sequence.

[0122] The decoder module 205 is configured to input the second embedding sequence and the encoder output feature sequence into a decoder network layer of the spatio-temporal prior network to obtain a decoder output feature sequence.

[0123] The result output module 206 is configured to input the decoder output feature sequence to a full connection layer of the spatio-temporal prior network to obtain output sequence data, and take the last l p elements in the output sequence data as the final prediction result.

[0124] Further, the first data embedding layer or the second data embedding layer comprises:

[0125] An element embedding module 207 is configured to expand the number of feature channels through one-dimensional convolution and output an element embedding sequence; and

[0126] A position embedding module 208 is configured to add position embedding information on the basis of the element embedding sequence to obtain context information of the sequence and output a position embedding sequence.

[0127] The element embedding sequence and the position embedding sequence are added to obtain the first embedding sequence or the second embedding sequence.

[0128] Further, the encoder network layer comprises a plurality of encoder basic layers with down-sampling and one encoder basic layer without down-sampling connected in sequence; the encoder basic layer with down-sampling comprises a plurality of stacked Swin-transformer modules and one token merging module; and the encoder basic layer without down-sampling comprises a plurality of stacked Swin-transformer modules.

[0129] Further, the decoder network layer comprises a plurality of decoder basic layers, each of which comprises a plurality of stacked Swin-transformer modules and one single Swin-transformer module; the plurality of stacked Swin-transformer modules are configured to process the second embedding sequence data and input the processed result to the single Swin-transformer module; and the single Swin-transformer module receives the processing result of the plurality of stacked Swin-transformer modules and the encoder output feature sequence, and performs fusion processing on the received data to output the decoder output feature sequence.

[0130] It should be noted that the cutting state prediction system 200 based on the mechanical signal provided in the embodiment corresponds to a technical solution that can be used to execute the method embodiment, and has similar implementation principles and technical effects to the method, which will not be described here.

[0131] Referring to Figure 7Another embodiment of the application also provides a cutting state prediction device 300 based on mechanical signals, comprising a preprocessing module 301, a space-time prior network 302 and a result output module 303, the space-time prior network 302 comprising a first data embedding layer 3021, an encoder network layer 3022, a second data embedding layer 3023, a decoder network layer 3024 and a full connection layer 3025. The cutting state prediction device 300 based on mechanical signals is used to execute each step in the above method embodiment.

[0132] Specifically,

[0133] The preprocessing module 301 is configured to preprocess the sampling data of the cutting force to obtain first time sequence data with a length of l; and concatenate the last l-l p bit data of the first time sequence data with a full zero sequence with a length of l p to obtain second time sequence data.

[0134] The space-time prior network 302 comprises:

[0135] The first data embedding layer 3021 is configured to perform embedding processing on the input first time sequence data to obtain a first embedding sequence, and output the first embedding sequence to the encoder network layer;

[0136] The encoder network layer 3022 is configured to perform feature coding on the input first embedding sequence to obtain an encoder output feature sequence, and output the encoder output feature sequence to the decoder network layer;

[0137] The second data embedding layer 3023 is configured to perform embedding processing on the input second time sequence data to obtain a second embedding sequence, and output the second embedding sequence to the decoder network layer;

[0138] The decoder network layer 3024 is configured to perform decoding processing on the input second embedding sequence and the encoder output feature sequence to obtain a decoder output feature sequence, and output the decoder output feature sequence to the full connection layer;

[0139] The full connection layer 3025 is configured to integrate all features of the input decoder output feature sequence to obtain output sequence data;

[0140] The result output module 303 is configured to take the last l p elements in the output sequence data as the final prediction result.

[0141] It should be noted that the cutting state prediction device 300 based on the mechanical signal provided in the embodiment corresponds to the technical solution for executing the method embodiment, and the implementation principle and technical effects are similar to the method, which will not be described here.

[0142] The above description is merely preferred embodiments of the present application. It should be understood by those skilled in the art that the disclosed scope of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or equivalent features without departing from the disclosed concept. For example, the above features are replaced with the technical features disclosed in the present application (but not limited to) having similar functions to form technical solutions.

Claims

1. A cutting state prediction method based on a mechanical signal, characterized by, The method comprises the following steps: The sampling data of the cutting force is preprocessed to obtain first time series data with a length of ; last bits of the first time series data are spliced with all-zero sequences with a length of to obtain second time series data; wherein, is a target sequence length to be predicted. inputting first time series data into a first data embedding layer of a spatiotemporal prior network to obtain a first embedding sequence; inputting the first embedding sequence into an encoder network layer of the spatiotemporal prior network to obtain an encoder output feature sequence; the encoder network layer comprises a plurality of encoder basic layers with down-sampling connected in sequence and one encoder basic layer without down-sampling; the encoder basic layer with down-sampling comprises a plurality of Swin-transformer modules stacked and one token merging module; the encoder basic layer without down-sampling comprises a plurality of Swin-transformer modules stacked; inputting second time series data into a second data embedding layer to obtain a second embedding sequence; inputting the second embedding sequence and the encoder output feature sequence into a decoder network layer of the spatiotemporal prior network to obtain a decoder output feature sequence; the decoder network layer comprises a plurality of decoder basic layers, each of which comprises a plurality of Swin-transformer modules stacked and one single Swin-transformer module; the plurality of Swin-transformer modules stacked are used for processing the second embedding sequence data and inputting the processed result into the single Swin-transformer module; the single Swin-transformer module receives the processing result of the plurality of Swin-transformer modules stacked and the encoder output feature sequence, and performs fusion processing on the received data to output the decoder output feature sequence; The decoder output feature sequence is input to a full connection layer of the spatio-temporal prior network to obtain output sequence data, and the last element in the output sequence data is taken as a final prediction result. The decoder output feature sequence is input to a full connection layer of the spatio-temporal prior network to obtain output sequence data, and the last element in the output sequence data is taken as a final prediction result.

2. The cutting state prediction method based on mechanical signals according to claim 1, characterized in that, the first data embedding layer or the second data embedding layer comprises: an element embedding module for expanding the number of feature channels through one-dimensional convolution and outputting an element embedding sequence; and a position embedding module for adding position embedding information on the basis of the element embedding sequence to obtain context information of the sequence and output a position embedding sequence; the element embedding sequence and the position embedding sequence are added to obtain the first embedding sequence or the second embedding sequence.

3. The cutting state prediction method based on mechanical signals according to claim 1, characterized in that: the step of preprocessing the sampling data of the cutting force comprises Kalman filtering and normalization processing on the sampling data.

4. A cutting state prediction system based on a mechanical signal, characterized by, The method comprises the following steps: The preprocessing module is configured to preprocess the sampling data of the cutting force to obtain first time series data with a length of The last bit data of the first time series data and an all-zero sequence with a length of are spliced to obtain second time series data; wherein, is a target sequence length to be predicted. a first embedding module configured to input first time series data into a first data embedding layer of a spatiotemporal prior network to obtain a first embedding sequence; The encoder module is configured to input the first embedding sequence to an encoder network layer of the spatiotemporal prior network to obtain an encoder output feature sequence; the encoder network layer comprises a plurality of encoder basic layers with down-sampling and one encoder basic layer without down-sampling connected in sequence; the encoder basic layer with down-sampling comprises a plurality of Swin-transformer modules stacked and one token merging module; the encoder basic layer without down-sampling comprises a plurality of Swin-transformer modules stacked; The second embedding module is configured to input the second time sequence data to a second data embedding layer of the spatiotemporal prior network to obtain a second embedding sequence; The decoder module is configured to input the second embedding sequence and the encoder output feature sequence to a decoder network layer of the spatiotemporal prior network to obtain a decoder output feature sequence; the decoder network layer comprises a plurality of decoder basic layers, each of which comprises a plurality of Swin-transformer modules stacked and one single Swin-transformer module; the plurality of Swin-transformer modules stacked are used to process the second embedding sequence data and input the processed result to the single Swin-transformer module; the single Swin-transformer module receives the processing result of the plurality of Swin-transformer modules stacked and the encoder output feature sequence, and performs fusion processing on the received data to output the decoder output feature sequence; The result output module is configured to input the decoder output feature sequence to a full connection layer of the spatio-temporal prior network to obtain output sequence data, and take the last element in the output sequence data as the final prediction result. The result output module is configured to input the decoder output feature sequence to a full connection layer of the spatio-temporal prior network to obtain output sequence data, and take the last element in the output sequence data as the final prediction result.

5. The cutting state prediction system based on mechanical signals according to claim 4, characterized in that, The first data embedding layer or the second data embedding layer comprises: An element embedding module for expanding the number of feature channels through one-dimensional convolution and outputting an element embedding sequence; and A position embedding module for adding position embedding information on the basis of the element embedding sequence to obtain context information of the sequence and output a position embedding sequence; The element embedding sequence and the position embedding sequence are added to obtain the first embedding sequence or the second embedding sequence.

6. A cutting state prediction device based on a mechanical signal, characterized by, The system comprises a preprocessing module, a spatiotemporal prior network and a result output module; wherein The preprocessing module is used for preprocessing the sampling data of the cutting force to obtain first time series data with a length of The last bit data of the first time series data and an all-zero sequence with a length of are spliced to obtain second time series data; wherein, is a target sequence length to be predicted. The spatiotemporal prior network comprises: A first data embedding layer for embedding processing of the input first time sequence data to obtain a first embedding sequence and outputting the first embedding sequence to an encoder network layer; The second embedding module is configured to input the second time sequence data to a second data embedding layer of the spatiotemporal prior network to obtain a second embedding sequence; The decoder module is configured to input the second embedding sequence and the encoder output feature sequence to a decoder network layer of the spatiotemporal prior network to obtain a decoder output feature sequence; the decoder network layer comprises a plurality of decoder basic layers, each of which comprises a plurality of Swin-transformer modules stacked and one single Swin-transformer module; the plurality of Swin-transformer modules stacked are used to process the second embedding sequence data and input the processed result to the single Swin-transformer module; the single Swin-transformer module receives the processing result of the plurality of Swin-transformer modules stacked and the encoder output feature sequence, and performs fusion processing on the received data to output the decoder output feature sequence; The first data embedding layer or the second data embedding layer comprises: An element embedding module for expanding the number of feature channels through one-dimensional convolution and outputting an element embedding sequence; and A position embedding module for adding position embedding information on the basis of the element embedding sequence to obtain context information of the sequence and output a position embedding sequence; The element embedding sequence and the position embedding sequence are added to obtain the first embedding sequence or the second embedding sequence. The encoder network layer is configured to encode the input first embedding sequence to obtain an encoder output feature sequence, and output the encoder output feature sequence to the decoder network layer; the encoder network layer comprises a plurality of encoder basic layers with down-sampling and one encoder basic layer without down-sampling connected in sequence; the encoder basic layer with down-sampling comprises a plurality of stacked Swin-transformer modules and one token merging module; the encoder basic layer without down-sampling comprises a plurality of stacked Swin-transformer modules; The second data embedding layer is configured to embed the input second time sequence data to obtain a second embedding sequence, and output the second embedding sequence to the decoder network layer; The decoder network layer is configured to decode the input second embedding sequence and the encoder output feature sequence to obtain a decoder output feature sequence, and output the decoder output feature sequence to the full connection layer; the decoder network layer comprises a plurality of decoder basic layers, each decoder basic layer comprises a plurality of stacked Swin-transformer modules and one single Swin-transformer module; the plurality of stacked Swin-transformer modules are configured to process the second embedding sequence data, and input the processed result to the single Swin-transformer module; the single Swin-transformer module receives the processing result of the plurality of stacked Swin-transformer modules and the encoder output feature sequence, and fuses the received data to output the decoder output feature sequence; The full connection layer is configured to integrate all features of the input decoder output feature sequence to obtain output sequence data. The result output module is used to output the last data in the output sequence. Each element is used as the final prediction result.

Images