Buckle clamping identification method and device, equipment and storage medium

By introducing residual long and short-term memory neural network and data set amplification technology in snap-on state recognition, the poor applicability of snap-on state recognition method to different assembly objects and the efficiency of long-term data processing in the existing technology is solved. The problem of low results in stronger generalization capabilities and recognition performance improvements.

CN119066576APending Publication Date: 2024-12-03SHENZHEN INST OF ADVANCED TECH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202411122865.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In the prior art, the snap-and-engage state recognition method is difficult to deal with different assembly objects, and the statistical-based method significantly degrades the effect of long-term data processing, resulting in the problem that the weight update becomes smaller during training and use.

Method used

Using the residual long and short-term memory neural network (LSTM) model, the long and short-term memory neural network units connected in sequence are improved by combining data set amplification technology to improve gradient flow, alleviate the problems of gradient disappearance and gradient explosion, and improve long-term series data processing capability.

Benefits of technology

It improves the generalization ability and applicability of snap-and-fit state recognition, enhances the memory and backpropagation efficiency under long-term series, alleviates the overfitting problem in network training, and improves the recognition performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066576A_ABST
    Figure CN119066576A_ABST
Patent Text Reader

Abstract

The invention discloses a buckle clamping identification method and device, equipment and a storage medium, and belongs to the technical field of robots. According to the method, for the problem of performance degradation of a traditional recurrent neural network (RNN) during processing of a long-time-sequence buckle detection force sequence, a residual error long-short-term memory (LSTM) neural network model used for identifying a buckle clamping sequence is introduced, the model links residual errors to a long-short-term memory network, the memory and back propagation efficiency under a long-time sequence is enhanced, and the detection accuracy of the buckle clamping sequence is improved. A genetic mechanism of information in long and short-term memory cells is improved by using residual links, the problem of gradient dissipation during processing of a long-time sequence is effectively prevented, training is performed by amplifying sample data in the training process of the residual long and short-term memory neural network model, the problem of overfitting in the network training process can be relieved, and the training efficiency is improved. And the recognition performance after network training convergence is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of robot technology, and in particular relates to a buckle engagement recognition method, device, equipment and storage medium. Background Art

[0002] The application of snap-on connection is widely present in various flexible assembly scenarios. The snap-on connection method works together through the preset or separated positioning structure and locking mechanism on the components to form specific position restrictions on the components. The locking mechanism is deformed during the assembly process and then returns to its original state to achieve locking and provide holding force. This connection method is not only easy and fast to operate, but also low-cost. It is an efficient and economical product assembly method.

[0003] Snap-fit ​​connection is a common industrial assembly method advocated according to the principle of "design for assembly" to replace the traditional threaded assembly method. The snap-fit ​​assembly is connected only by pushing. Considering that a large thrust needs to be applied to break through the resistance of the elastic structure, the force sensed by the finger is often used to obtain the state during manual assembly. Nowadays, industrial robots are gradually replacing manual labor to achieve automated and flexible production. Although the widely used threshold method can successfully guide the robot to identify whether the assembly is complete, the structural characterization process requires that the entire state of the snap-fit ​​cannot be detected using only the threshold. In addition, applying excessive pressure in the presence of assembly failure may cause component damage. Therefore, accurate identification of snap-fit ​​using industrial robots is a necessary condition for automated production to meet safety requirements.

[0004] Most previous studies on robot assembly believe that assembly is a quasi-static process because the overall force during the assembly process does not change much. However, this is not the case with the snap-fit ​​assembly process. During the snap-fit ​​assembly process, the energy is released very quickly at the moment of breaking through the resistance, resulting in significant changes in the assembly force in a short period of time. The current snap-fit ​​state recognition methods are mainly divided into two categories. The first category is based on analysis, and the second category is based on statistics.

[0005] Rusli et al. investigated the tactile feedback of snap-on fasteners during manual assembly, constructed a linear model to identify the snap-on engagement state, experimentally analyzed the effects of the force and hardness of the preloaded sealing element on the tactile feedback, and proposed a new quantitative indicator, “locking signal to retention force ratio”, to predict the confidence of snap-on assembly.

[0006] Doltsinis et al. proposed a machine learning-based framework for real-time identification of successful snap-fit ​​states. They first defined a set of features that can characterize different snap-fit ​​assembly types, including signal energy, Hjorth complexity, Shannon entropy, and the mean absolute value slope of the signal. Based on feature extraction, they used support vector machine (SVM) as a classifier, especially using a nonlinear radial basis function (RBF) kernel. The SVM classifier learns an optimal hyperplane through training data to distinguish between successful and unsuccessful snap-fit ​​assemblies.

[0007] Cui et al. proposed a method for quickly identifying the locking state of buckles, combining a buckle dataset generation strategy with automatic label acquisition and a multi-layer recurrent neural network that can process sequence data and introduce residual links in the time dimension and network layer dimension. The recurrent neural network units are supplemented by residual links in the time dimension and network layer dimension to solve the gradient diffusion problem of traditional RNN when processing long sequence data.

[0008] Although the above methods all propose methods for effectively identifying the snap-fit ​​state, some of them rely heavily on manually constructed features. Although these methods can often achieve excellent results in specific scenarios, due to the limitations of the scenarios, it is difficult for the same model to handle different assembly objects, or even different speeds and sampling frequencies in the same assembly scenario; therefore, if they are used in different scenarios, the model needs to be rebuilt. For other methods, in statistical-based methods, many studies collect force data and mark samples, and then use machine learning or deep learning models to identify snap-fits. For example, recurrent neural networks, although they have gotten rid of the limitations of designing manual features, face the problem of significant degradation of effects when processing long time series data. Since the training sequence data of recurrent neural networks is extended, when they are calculated, the longer time span will lead to a decrease in the residual index, thereby making the weight update smaller, which limits the ability to reflect the long-term memory effect of recurrent neural networks. Although there are methods to introduce residual links into RNNs, they are still limited by the inherent defects of RNN network design. Summary of the invention

[0009] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and to provide a buckle engagement recognition method, device, equipment and storage medium to solve the problem that in the prior art buckle engagement state recognition method, the same model is difficult to handle different assembly objects, and the long time series data effect is seriously degraded in the statistics-based buckle engagement state recognition method, resulting in a smaller weight update during the training and use of the model.

[0010] In order to achieve the above object, the present invention adopts the following technical solutions:

[0011] A first aspect of the present invention provides a buckle engagement identification method, comprising the following steps:

[0012] S1, collecting assembly sequence signals of multiple buckles during the engagement process at set time intervals;

[0013] S2, based on the assembly sequence signal, the residual long short-term memory neural network model determines whether the buckle is successfully engaged;

[0014] The residual long short-term memory neural network model includes multiple layers of long short-term memory neural network units connected in sequence, the input of each layer of long short-term memory neural network units is the sum of the input signal of the previous layer of long short-term memory neural network units and the residual, and the residual is the result of the calculation output of the previous layer of long short-term memory neural network units;

[0015] The long short-term memory neural network is obtained by training and testing a data set, and the data set is obtained by amplifying the original data set;

[0016] S3, output the judgment result.

[0017] A further improvement of the present invention is:

[0018] Preferably, in S2, the original data set expansion process is:

[0019] (1) Obtain the shortest engagement sequence for each successful engagement data in the original data set;

[0020] (2) sampling each successfully engaged data with a sliding window of a fixed preset length T, and when the sliding window contains the respective shortest engaged sequence, marking the sample sequence corresponding to the sliding window as a successfully engaged sample amplification sequence;

[0021] (3) After pooling all sample amplification sequences, an amplified data set is obtained;

[0022] The augmented dataset is trained using a focal loss function or a binary classification loss function.

[0023] Preferably, in step (1), the method for obtaining the shortest engagement sequence from any one piece of successful engagement data is:

[0024] a. Calculate the feature θ for any successful engagement data using the following formula t ,

[0025]

[0026] Where t represents the time step in the card and sequence data, f t Indicates the contact force signal size at this time step, f t-τRepresents the contact force signal size at the time step t-τ, where τ is a constant;

[0027] b. Take the maximum θ t The corresponding moment is the time point when the buckle assembly touches the bottom of the buckle base. b , take the maximum θ t The sequence is at t b The most recent negative value θ t , and obtain a negative value of θ t The time point t at which the buckle engagement threshold force is broken m , time point t m and time point t b The engagement sequence between is the shortest engagement sequence.

[0028] Preferably, the assembly sequence signal is an assembly force sequence signal.

[0029] Preferably, the original data set is divided into cantilever buckle assembly data and annular buckle assembly data according to buckle types;

[0030] The original data set is divided into manually collected data and machine collected data according to the collection method;

[0031] The original data set is divided into successful engagement data and failed assembly data according to whether the engagement is successful.

[0032] Preferably, the input of each layer of long short-term memory neural network unit is the sum of the input signal of the previous layer of long short-term memory neural network unit and the residual, specifically:

[0033] In the current long short-term memory neural network unit, the cell state vector is obtained by adding the input cell state vector of the previous layer of long short-term memory neural network unit to the current cell state vector residual, and the current hidden state vector is obtained by adding the input hidden state vector of the previous layer of long short-term memory neural network unit to the current hidden state vector residual;

[0034] The current cell state vector and the current hidden state vector residual are both obtained by calculating the current long short-term memory neural network unit.

[0035] Preferably, the residual long short-term memory neural network model is

[0036]

[0037] Among them, f t is the activation vector of the forget gate, i t is the activation vector of the input gate, is the cell candidate vector, C t is the cell state vector, o t is the activation vector of the output gate, ht is the current hidden state vector, σ represents the Sigmoid function, tanh represents the hyperbolic tangent function, W f is the weight matrix of the forget gate, h t-1 is the hidden state vector at the previous moment, x t is the input vector at the current moment, b f is the bias vector of the forget gate, W i is the weight matrix of the input gate, b i is the bias vector of the input gate, W C is the weight matrix of the candidate cell state, b c is the bias vector of the candidate cell state, C t-1 is the cell state vector at the previous moment, W o is the weight matrix of the output gate, b o is the bias vector for the output gate.

[0038] A second aspect of the present invention provides a buckle engagement identification device, comprising:

[0039] A collection unit, used to collect assembly sequence signals of the buckle during the locking process at a set time interval;

[0040] A judgment unit, used to judge whether the buckle is successfully engaged based on the assembly sequence signal and the residual long short-term memory neural network model;

[0041] The residual long short-term memory neural network model includes multiple layers of long short-term memory neural network units connected in sequence, the input of each layer of long short-term memory neural network units is the sum of the input signal of the previous layer of long short-term memory neural network units and the residual, and the residual is the result of the calculation output of the previous layer of long short-term memory neural network units;

[0042] The long short-term memory neural network is obtained by training and testing a data set, and the data set is obtained by amplifying the original data set;

[0043] The output unit is used to output the judgment result.

[0044] A third aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned methods for identifying buckle engagement when executing the computer program.

[0045] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the buckle engagement identification method described in any one of the above items is implemented.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The present invention discloses a buckle snap-fit ​​recognition method, which introduces a buckle snap-fit ​​sequence recognition model of a residual long short-term memory neural network (LSTM) for the performance degradation problem of a traditional recurrent neural network (RNN) when processing a long time sequence buckle detection force sequence. LSTM is a special recurrent neural network (RNN). Compared with the residual RNN, it can effectively learn and save long-term information through its internal gating mechanism (such as forget gate, input gate and output gate), selectively retain or discard information, so as to more effectively capture long-term dependencies, and then more deeply memorize long-sequence context information, especially when processing long sequence data, it can greatly improve the learning efficiency and network performance, and enhance the memory and back propagation efficiency under long time sequences. The present invention innovatively adds residual links to LSTM, improves gradient flow, alleviates the problems of gradient vanishing and gradient explosion, and allows the network to directly transmit information between multiple time steps, thereby reducing the loss of information in the propagation process. Therefore, the residual links are used to improve the genetic mechanism of information in long short-term memory cells, effectively preventing the problem of gradient dissipation when processing long time sequences. The method proposed in the present invention is a statistical method. Compared with the method of identifying the engagement sequence based on manually constructed features, this method is not affected by the specific circumstances of the assembled parts. It can be applied to the assembly force signal data set of the assembly process of any snap-fitting parts with almost no modification. It has strong generalization and wide applicability.

[0048] Furthermore, the present invention proposes a feature extraction method for identifying the shortest engagement sequence in a successful engagement sequence for sample amplification. This sample amplification method helps to amplify a large amount of valid data in a limited engagement data set for network training, thereby enhancing the training effect on a limited data set, thereby alleviating the overfitting problem in the network training process and improving the recognition performance after the network training converges.

[0049] Furthermore, the present invention can use a serial signal sensor that generates necessary information about the snap-fitting process during the snap-fitting process while keeping the framework basically unchanged, such as identifying the snap-fitting state based on serial signal data of assembly force, thereby making the residual long short-term memory neural network model more universal. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flow chart of a buckle engagement identification method of the present invention;

[0051] Figure 2 A training diagram for constructing a residual long short-term memory neural network model according to the present invention;

[0052] Figure 3 Amplify flow charts for cards and data samples;

[0053] Figure 4 is the shortest card and sequence schematic diagram of the present invention; wherein, t b Indicates the time point when the engaging part touches the bottom after completing the engaging process, f b Indicates the magnitude of the engagement contact force at that time point, f m Indicates the magnitude of the first peak force before the engaging parts bottom out, t m Indicates the point in time when this force is reached.

[0054] Figure 5 It is a schematic diagram of the residual LSTM network unit of the present invention;

[0055] Figure 6 It is the engagement state recognition network based on residual LSTM of the present invention. DETAILED DESCRIPTION

[0056] In the following, the terms "first", "second", "third", and "fourth" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first", "second", "third", and "fourth" may explicitly or implicitly include one or more of the features.

[0057] The co-shooting method provided in the embodiment of the present invention can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiment of the present application does not impose any restrictions on the specific type of the terminal device.

[0058] It should be noted that the terms "first", "second", etc. in the specification and drawings of the present invention are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0059] See also Figure 1 The first aspect of the present invention discloses a buckle engagement identification method, comprising the following steps:

[0060] S1, collecting assembly information sequence signals at set time intervals to obtain assembly information sequence signals of the buckle during the locking process, wherein the sequence signal is composed of a plurality of assembly signal units arranged in time sequence;

[0061] S2, inputting the assembly information sequence signal into the residual long short-term memory neural network model to obtain the result of whether the buckle is successfully engaged;

[0062] The residual long short-term memory neural network model includes a plurality of long short-term memory neural network units connected in sequence, the input of each layer of long short-term memory neural network units is the sum of the input signal of the previous layer of long short-term memory neural network units and the residual, and the residual is the result of the calculation output of the previous layer of long short-term memory neural network units;

[0063] The long short-term memory neural network is obtained by training and testing a data set, and the data set is obtained by amplifying the original data set.

[0064] S3, output the judgment result.

[0065] In some implementation schemes of the present invention, before S1, the assembly information sequence signal is collected at a set time interval, and the assembly information sequence signal is sampled through a sliding window of length T, and the length of the sliding window is the sliding window length based on the residual long short-term memory neural network model during training.

[0066] In some embodiments of the present invention, in S1, the assembly information sequence signal is an assembly sequence signal that can reflect the necessary information of the snap-fitting process during the snap-fitting process, and is obtained by a sequence signal sensor that can collect the assembly information generated during the snap-fitting process and contains the assembly information of the snap-fitting process. The data used in the verification process of the present invention is a snap-fitting data set based on the assembly force sequence signal, and the memory neural network judges whether the snap-fitting is successful through the torque based on the length of the residual. However, if sequence signals of other assembly information are used, it can also be applied with appropriate adjustments.

[0067] In some embodiments of the present invention, the input of each layer of long short-term memory neural network unit is the sum of the input signal and the residual of the previous layer of long short-term memory neural network unit, and the cell state vector and the hidden state vector are obtained.

[0068] In the current long short-term memory neural network unit, the cell state vector is obtained by adding the input cell vector of the previous layer of long short-term memory neural network unit and the current cell state vector residual, and the current hidden state vector is obtained by adding the input hidden state vector of the previous layer of long short-term memory neural network unit and the current hidden state vector residual.

[0069] More specifically, the residual LSTM unit sets the input size, implicit feature size, number of LSTM layers, number of neurons in the fully connected layer, and number of output neurons of the entire LSTM model. The forward transfer formula of the long short-term memory neural network is as follows:

[0070]

[0071] Among them, f t is the activation vector of the forget gate, which controls which information needs to be ignored. t is the activation vector of the input gate, which controls what new information needs to be stored in the cell state. is a cell candidate vector, containing new information that may be added to the cell state. t is the cell state vector, which is the "memory" part of the LSTM network. t is the activation vector of the output gate, which controls what information will be output. t is the current hidden state vector. W and b represent the weight matrix and bias vector. σ represents the Sigmoid function, which maps the weight input to the (0,1) interval. tanh represents the hyperbolic tangent function, which maps the weight input to the (-1,1) interval. W f , the weight matrix of the forget gate, used to calculate the activation vector f of the forget gate t ;h t-1 : The hidden state vector of the previous moment, used to calculate various gating signals and candidate cell states at the current moment; xt : The input vector at the current moment, used to calculate various gating signals and candidate cell states at the current moment; b f : The bias vector of the forget gate, used to calculate the activation vector f of the forget gate t ; W i : The weight matrix of the input gate, used to calculate the activation vector i of the input gate t ; b i : The bias vector of the input gate, used to calculate the activation vector i of the input gate t ; W C : The weight matrix of the cell candidate state, used to calculate the cell candidate state vector b c : Bias vector of the cell candidate state, used to calculate the cell candidate state vector C t-1 : The cell state vector at the previous moment, used to calculate the cell state vector C at the current moment t ; W o : The weight matrix of the output gate, used to calculate the activation vector o of the output gate t ; b o : Bias vector of the output gate. Used to calculate the activation vector o of the output gate t .

[0072] Combining formula (2), we can see that during the calculation process, the cell state vector C at the current moment is t The cell state vector C at the previous moment is introduced in the calculation process. t-1 , and the current hidden state vector h t , the cell state vector at the previous moment is introduced in the calculation process.

[0073] See also Figure 2 In S2 of the above process, the process of acquiring the long short-term memory neural network model includes the following steps:

[0074] S21, obtaining the original dataset of robot buckle assembly by autonomous collection or online downloading of public datasets, wherein the original dataset includes assembly sequence signals of the buckle during the buckling process.

[0075] S22, perform sample amplification on the data set and divide it into a training set and a test set.

[0076] S23, combining the residual link mechanism to build a long short-term memory neural network model, which is a model for identifying whether a long sequence of buckles is successfully engaged, and inputting the training set data into the neural network for training.

[0077] S24, verifying the obtained buckle engagement recognition network using a test set.

[0078] In some embodiments of the present invention, in S21, the collected raw data set is as rich in the types of collection as possible. For one data set, it is divided into cantilever type buckle assembly data and annular buckle assembly data according to the buckle type; the raw data set is divided into manual collection data and robot collection data according to the collection method; the raw data set is divided into successful engagement data and failed assembly data according to whether the engagement is successful; data collection experiments are conducted on two different buckle devices, including cantilever and annular buckle assemblies. Multiple types of data are set in the same data set, so that the training accuracy of the model is improved during subsequent data training.

[0079] See also Figure 3 In some embodiments of the present invention, in S22, the process of amplifying the data set is:

[0080] (1) Obtain the shortest engagement sequence for each successful engagement data in the original data set;

[0081] The whole process of the snap fastening is divided into four stages: approach, press, engagement and bottoming out. In the approach stage, the engagement components have not yet contacted, and the force signal maintains a constant initial value. In the pressing stage, the contact force gradually increases with the progress of pressing until it reaches a peak value. In the engagement stage, the contact force drops sharply, indicating a smooth engagement. In the bottoming out stage, the contact force continues to increase with the progress of pressing. Therefore, first find the shortest engagement sequence for each successful engagement data, which is the engagement sequence between the peak value of the pressing stage and the bottoming out. When amplifying the data subsequently, only the shortest engagement sequence is contained in the sliding window of the preset length T, which is a successfully engaged data.

[0082] In order to find the shortest snap sequence s for a successful snap sample sequence, the snap snap sequence is analyzed as Figure 2 As shown in the figure, there must be a shortest engagement sequence s in the successful engagement sequence data, that is, m to b This sequence of data shows that the buckle continues to advance after the contact force reaches the peak, thus breaking through the clamping force threshold, entering the rapid force release stage, and finally contacting the bottom of the clamping base. During the process, because the clamping part touches the bottom, the force signal will immediately have a turning point with a large slope change, that is, f b point, which can be used as the key feature point for identifying and locating the shortest engagement sequence s, and the first peak force f before this feature point m , which means the critical point of this engagement. In order to identify t b , calculate the feature θ for any sample of the card force sequence t as follows:

[0083]

[0084] Where t represents the time step in the card and sequence data, f t Indicates the contact force signal size at this time step, f t-τ represents the contact force signal size at the time step t-τ, τ is a constant, then when θ t When >0, it means f t At the trough of the curve, when θ t <0 indicates f t At the peak of the curve. Calculate the corresponding θ for a certain successful engagement sample sequence t Sequence, take the global maximum value, and its corresponding t represents the time point t when the buckle component of the sequence touches the bottom of the buckle base b , take θ t The sequence is at t b Previous distance t b The nearest negative value of θ t The corresponding time point t is the time point t at which the buckle engagement threshold force is broken. m , so the shortest engagement sequence can be determined, and then according to Figure 3 Perform sample augmentation on the dataset samples.

[0085] (2) Then, a sliding window with a fixed preset length of t is used to set a fixed step size to perform sliding window sampling on the data sequence. When the sliding window contains the shortest engagement sequence, the sliding window amplified sample sequence is marked as a successfully engaged sample amplification sequence. Otherwise, it is a failed engaged sample amplification sequence. The flow chart for amplifying a single original engaged data sample is as follows: Figure 1 shown.

[0086] (3) After all sample amplification sequences are pooled, an amplified data set is obtained, and the amplified data set is divided into a training set and a test set according to a set ratio.

[0087] As a preferred solution, the expanded data set is shuffled before being divided to ensure the balance of data in the data set.

[0088] As a preferred solution, considering that the enhanced data set often has a serious data imbalance problem, the enhanced data set is processed by a loss function, and the loss function can be a focal loss function or a binary classification loss function, the binary classification cross entropy loss function (BCE Loss), wherein the focal loss function is calculated as follows:

[0089]

[0090] Among them, α and γ are hyper parameters, p c is the predicted probability.

[0091] In some embodiments of the present invention, in S23, the specific process of constructing a residual long short-term memory neural network is as follows: Long short-term memory neural network (LSTM) is a special recursive neural network (RNN), which can effectively learn and save long-term information through its internal gating mechanism (such as forget gate, input gate and output gate). By introducing LSTM units, it is intended to improve the problem of performance degradation of the existing snap-fit ​​sequence RNN recognition network in processing long sequence snap-fit ​​data. On the other hand, residual links are introduced, and the network directly accesses the output of the previous layer at each layer, which helps the gradient flow more smoothly during training, thereby solving the problem of gradient disappearance or gradient explosion in deep networks.

[0092] After the construction is completed, the training set data is obtained through S22 to train the long short-term memory neural network to obtain the trained long short-term memory neural network.

[0093] The present invention is further described below in conjunction with specific embodiments.

[0094] Example 1

[0095] S1, Data Collection

[0096] The present invention uses the Doltsinis public dataset, which was proposed by Stefanos Doltsinis of the Institute of Information Technology, the Greek Center for Research and Technology. The dataset uses a KUKA LWR4+ robot arm, and a three-finger gripper Barret BH-8 is installed on it. The mounting accessories are fixed to the end of the gripper, and the socket is fixed to the edge of the desktop to realize data collection during the assembly process. The dataset conducts data collection experiments on two different snap-on devices, including cantilever and annular snap-on assemblies. The assembly force between the snap-on components is measured by the force sensor provided by the KUKA robot arm, and no external force sensor is used.

[0097] The data set can be divided into four parts: DS1\DS2\DS3\DS4. DS1 and DS2 are collected in a human-machine collaborative way. The robot is controlled by high stiffness impedance and acts as an intelligent sensor to capture force signals during the assembly process. Each data set contains 60 assembly force sequence data, of which DS1 is the ring buckle installation experimental data and DS2 is the cantilever buckle assembly data. Each data set contains 31 failed assembly data and 29 successful assembly data. DS3 and DS4 are collected in an autonomous assembly way by KUKA robotic arms. The robot is controlled by high stiffness impedance and acts as an intelligent sensor to capture force signals during the assembly process. Each data set contains 50 assembly force sequence data, of which DS3 is the ring buckle installation experimental data and DS4 is the cantilever buckle assembly data. Each data set contains 25 failed assembly data and 25 successful assembly data.

[0098] S2, augment the data set.

[0099] In order to achieve sample amplification, first find the shortest engagement sequence of each successfully engaged data, and then perform sliding window sampling on the original data sequence with a sliding window of a fixed preset length of T. When the sliding window contains the shortest engagement sequence, the sliding window amplified sample sequence is marked as a successfully engaged sample amplification sequence, otherwise, it is a failed engaged sample amplification sequence. After completing sample amplification, the data set is shuffled and divided into a training set and a test set in a ratio of 4:1.

[0100] S3, build a residual LSTM-based engagement state recognition network

[0101] The network overview diagram of building a residual long short-term memory neural network is as follows Figure 4 As shown, the input size of the residual long short-term memory neural network unit is set to 10, the hidden feature size is set to 10, and the number of layers is set to 2. For the fully connected layer, the number of input neurons is set to 10, and the number of output neurons is set to 2. After passing through a Softmax layer, the output identifies whether the sequence is a unique code that is successfully engaged.

[0102] During the training of the card recognition network, the amplified data set was divided into a training set and a test set according to a ratio of 4:1, and the initial learning rate of the network was set to 1×10 -4 , and the Adam optimizer is used, the training batch size is 1000, the training cycle is 200, the gradient descent network training of the focal loss is performed until convergence, and the model is obtained. Finally, the test set is used to test the performance of the model.

[0103] Compared with the shortest successful engagement sequence identification method formula (2) proposed by Cui et al., the shortest successful engagement sequence identification method formula (1) proposed in this paper avoids the problem of f in some cases. m θ is calculated at the point m′ $ratioθ b′ In formula (1), f m and θ calculated from the f_b position t The signs must be opposite, but in formula (2) the signs are the same, which will cause f b The problem of inaccurate positioning.

[0104]

[0105] In addition, in the buckle snap-fit ​​recognition network, compared with the residual RNN used in the method of Cui et al., this paper introduces residual links in the LSTM network unit. Compared with residual RNN, residual LSTM is better because it can use residual links to memorize information more deeply, especially when processing long sequence data, which can greatly improve learning efficiency and network performance. Due to the success of ResNet and the excellent performance of LSTM on sequence data, residual LSTM can better maintain information flow during training, so it can train deeper models, obtain richer features and improve accuracy.

[0106] The network structure proposed in this invention has been well verified on the Doltsinis public dataset, and its prediction performance has been improved compared with the original model, proving that this method is feasible.

[0107] Table 1 Comparison of test results of different models

[0108]

[0109] The present invention introduces residual links in LSTM to bring the following advantages:

[0110] 1. Shortened information transmission path: Residual links provide a direct path from input to output for the network, allowing information to be quickly transmitted between different layers without having to go through layer-by-layer calculations. This reduces the attenuation of information during the transmission process and helps alleviate the gradient vanishing problem.

[0111] 2. Gradient propagation is more stable: Due to the existence of residual links, the gradient can be passed directly through this path during backpropagation, without having to rely entirely on layer-by-layer propagation. This makes gradient propagation more stable and avoids the problem that the gradient may become very small or very large when processing long sequences.

[0112] 3. Easier to optimize: Residual links make the network easier to optimize. Even when the network depth increases, the training remains stable and efficient. This means we can build deeper networks to capture more complex patterns.

[0113] 4. Enhanced memory capacity: In LSTM, the cell state is the key to memorizing long-term information. Residual links can help the cell state remain stable over a longer period of time, thereby enhancing the memory capacity of LSTM.

[0114] A second aspect of the present invention discloses a buckle engagement identification device, comprising:

[0115] A collection unit, used to collect assembly sequence signals of the buckle during the locking process at a set time interval;

[0116] A judgment unit, used to judge whether the buckle is successfully engaged based on the assembly sequence signal and the residual long short-term memory neural network model;

[0117] The residual long short-term memory neural network model includes a plurality of long short-term memory neural network units connected in sequence, the input of each layer of long short-term memory neural network units is the sum of the input signal of the previous layer of long short-term memory neural network units and the residual, and the residual is the result of the calculation output of the previous layer of long short-term memory neural network units;

[0118] The long short-term memory neural network is obtained by training and testing a data set, and the data set is obtained by amplifying the original data set;

[0119] The output unit is used to output the judgment result.

[0120] The third aspect of the present invention discloses a computer device, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow. Or corresponding functions; the processor described in this embodiment can be used to implement a buckle engagement recognition method, including the following steps: S1, collecting assembly sequence signals of multiple buckles in the engagement process at set time intervals; S2, based on the assembly sequence signals, the residual long short-term memory neural network model determines whether the buckle engagement is successful; the residual long short-term memory neural network model includes a plurality of long short-term memory neural network units connected in sequence, the input of each layer of long short-term memory neural network units is the sum of the input signal of the previous layer of long short-term memory neural network units and the residual, and the residual is the result of the calculation output of the previous layer of long short-term memory neural network units; the long short-term memory neural network is obtained after training and testing through a data set, and the data set is obtained after amplifying the original data set; S3, outputting the judgment result.

[0121] The fourth aspect of the present invention discloses a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement a buckle engagement identification method in the above embodiment, including the following steps: S1, collecting assembly sequence signals of multiple buckles in the engagement process at set time intervals; S2, based on the assembly sequence signals, a residual long short-term memory neural network model determines whether the buckle engagement is successful; the residual long short-term memory neural network model includes multiple long short-term memory neural network units connected in sequence, and the input of each layer of long short-term memory neural network units is the sum of the input signal of the previous layer of long short-term memory neural network units and the residual, and the residual is the result of the calculation output of the previous layer of long short-term memory neural network units; the long short-term memory neural network is obtained after training and testing through a data set, and the data set is obtained after amplifying the original data set; S3, outputting the judgment result.

[0122] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A buckle engagement recognition method, characterized in that: The following steps are involved: S1, collecting assembly sequence signals of multiple buckles during the engagement process at set time intervals; S2, based on the assembly sequence signal, the residual long short-term memory neural network model determines whether the buckle is successfully engaged; The residual long short-term memory neural network model includes multiple layers of long short-term memory neural network units connected in sequence, the input of each layer of long short-term memory neural network units is the sum of the input signal of the previous layer of long short-term memory neural network units and the residual, and the residual is the result of the calculation output of the previous layer of long short-term memory neural network units; The long short-term memory neural network is obtained by training and testing a data set, and the data set is obtained by amplifying the original data set; S3, output the judgment result.

2. A buckle engagement identification method according to claim 1, characterized in that: In S2, the original data set expansion process is: (1) Obtain the shortest engagement sequence for each successful engagement data in the original data set; (2) sampling each successfully engaged data with a sliding window of a fixed preset length T, and when the sliding window contains the respective shortest engaged sequence, marking the sample sequence corresponding to the sliding window as a successfully engaged sample amplification sequence; (3) After pooling all sample amplification sequences, an amplified data set is obtained; The augmented dataset is trained using a focal loss function or a binary classification loss function.

3. A buckle engagement identification method according to claim 2, characterized in that: In step (1), the method for obtaining the shortest engagement sequence for any successful engagement data is: a. Calculate the feature θ for any successful engagement data using the following formula t , Where t represents the time step in the card and sequence data, f t Indicates the contact force signal size at this time step, f t-τ Represents the contact force signal size at the time step t-τ, where τ is a constant; b. Take the maximum θ t The corresponding moment is the time point when the buckle assembly touches the bottom of the buckle base. b , take the maximum θ t Sequence at t b The most recent negative value θ t , and obtain a negative value of θ t The time point t at which the buckle engagement threshold force is broken m , time point t m and time point t b The engagement sequence between is the shortest engagement sequence.

4. A buckle engagement identification method according to claim 1, characterized in that: The assembly sequence signal is an assembly force sequence signal.

5. A buckle engagement identification method according to claim 1, characterized in that: The original data set is divided into cantilever buckle assembly data and annular buckle assembly data according to buckle types; The original data set is divided into manually collected data and machine collected data according to the collection method; The original data set is divided into successful engagement data and failed assembly data according to whether the engagement is successful.

6. A buckle engagement identification method according to claim 1, characterized in that: The input of each layer of LSTM neural network unit is the sum of the input signal and the residual of the previous layer of LSTM neural network unit, specifically: In the current long short-term memory neural network unit, the cell state vector is obtained by adding the input cell state vector of the previous layer of long short-term memory neural network unit to the current cell state vector residual, and the current hidden state vector is obtained by adding the input hidden state vector of the previous layer of long short-term memory neural network unit to the current hidden state vector residual; The current cell state vector and the current hidden state vector residual are both obtained by calculating the current long short-term memory neural network unit.

7. A buckle engagement identification method according to claim 6, characterized in that: The residual long short-term memory neural network model is: Among them, f t is the activation vector of the forget gate, i t is the activation vector of the input gate, is the cell candidate vector, C t is the cell state vector, o t is the activation vector of the output gate, h t is the current hidden state vector, σ represents the Sigmoid function, tanh represents the hyperbolic tangent function, W f is the weight matrix of the forget gate, h t-1 is the hidden state vector at the previous moment, x t is the input vector at the current moment, b f is the bias vector of the forget gate, W i is the weight matrix of the input gate, b i is the bias vector of the input gate, W C is the weight matrix of the candidate cell state, b c is the bias vector of the candidate cell state, C t-1 is the cell state vector at the previous moment, W o is the weight matrix of the output gate, b o is the bias vector for the output gate.

8. A buckle engagement recognition device, characterized in that: include: A collection unit, used to collect assembly sequence signals of the buckle during the locking process at a set time interval; A judgment unit, used to judge whether the buckle is successfully engaged based on the assembly sequence signal and the residual long short-term memory neural network model; The residual long short-term memory neural network model includes multiple layers of long short-term memory neural network units connected in sequence, the input of each layer of long short-term memory neural network units is the sum of the input signal of the previous layer of long short-term memory neural network units and the residual, and the residual is the result of the calculation output of the previous layer of long short-term memory neural network units; The long short-term memory neural network is obtained by training and testing a data set, and the data set is obtained by amplifying the original data set; The output unit is used to output the judgment result.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the buckle engagement identification method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the buckle engagement recognition method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • A health public opinion analysis method based on a deep multi-word embedded Bi-LSTM residual network

    CN109670164A