Snap fastener engagement recognition method and apparatus, device, and storage medium

Abstract

The present invention relates to the technical field of robots. Disclosed are a snap fastener engagement recognition method and apparatus, a device, and a storage medium. The method is directed to the problem of performance degradation when a conventional recurrent neural network (RNN) processes a long time series detection force sequence of snap fasteners, and introduces a residual long short-term memory neural network (LSTM) model for recognizing an engagement sequence of snap fasteners. The model links residual to a long short-term memory network, enhances memory and back propagation efficiency in a long time series, and improves a genetic mechanism for information in long short-term memory cells by using residual linking, thereby effectively preventing the problem of gradient dispersion occurring when a long time sequence is processed. During the training process of the residual long short-term memory neural network model, training is performed through augmentation of sample data, thereby alleviating the problem of over-fitting in the network training process, and improving recognition performance after convergence of training of the network.

Description

Buckle clamping identification method, device and equipment and storage medium TECHNICAL FIELD

[0001] The application belongs to the technical field of robots, and particularly relates to a buckle clamping identification method, device, equipment and storage medium. BACKGROUND

[0002] The application of buckle connection widely exists in various flexible assembly scenes. The buckle connection mode plays a role in combination of a positioning structure and a locking mechanism pre-installed or separated on a component, so that the component forms a specific position limitation. The locking mechanism is deformed in the assembly process and then returns to the original state to realize locking and provide a holding force. This connection mode is not only simple and fast to operate, but also low in cost, and is a high-efficiency and economical product assembly method.

[0003] The buckle connection is a common industrial assembly method according to the principle of "assembly-oriented design" to replace the traditional threaded assembly method. The buckle assembly is only pushed to be connected. Considering that a large pushing force needs to be applied to break through the resistance of the elastic structure, therefore, in the manual assembly process, the force perceived by the fingers is often used to obtain the state. Nowadays, industrial robots are gradually replacing manual work to realize automatic and flexible production. Although the widely used threshold method can successfully guide the robot to identify whether the assembly is complete, the structure representation process requires that only the threshold value cannot be used to detect the entire state of the clamping. In addition, the application of excessive pressure in the presence of assembly failure may cause damage to the components. Therefore, the precise identification of the clamping by the industrial robot is a necessary condition for the automatic production to meet the safety requirements.

[0004] Most of the previous researches on robot assembly consider that the assembly is a quasi-static process because the force in the assembly process does not change much as a whole. However, the assembly process of the buckle connection is not like this. In the assembly process of the buckle connection, the energy is released very quickly at the moment of breaking through the resistance, which causes the assembly force to change significantly in a short time. The current buckle clamping state identification method is mainly divided into two categories. The first category is the analysis-based method, and the second category is the statistics-based method.

[0005] Rusli et al. investigated the tactile feedback of the buckle fastener in the manual assembly process, constructed a linear model to identify the buckle clamping state, analyzed the influence of the force size and hardness of the preloaded sealing element on the tactile feedback in the experiment, and put forward a new quantitative index of "lock signal to holding force ratio" to predict the confidence degree of the buckle assembly.

[0006] Doltsinis et al. proposed a machine learning-based framework for real-time identification of successful snap fit conditions. They first defined a set of features that can characterize different snap fit types, including signal energy, Hjorth complexity, Shannon entropy, mean absolute value slope of the signal, etc. On the basis of feature extraction, they used support vector machine (SVM) as the classifier, especially using the nonlinear radial basis function (RBF) kernel. The SVM classifier learns an optimal hyperplane to distinguish between successful and unsuccessful snap fit assemblies through training data.

[0007] Cui et al. proposed a method for quickly identifying the snap fit condition, combining an automatic label acquisition snap fit dataset generation strategy and a multi-layer recurrent neural network that can process sequence data and introduce residual links in the time dimension and network layer dimension. The recurrent neural network unit is supplemented by residual links in the time dimension and network layer dimension to solve the gradient dispersion problem of traditional RNN when processing long sequence data.

[0008] Although the above methods all propose effective identification methods for the snap fit condition, some methods rely heavily on manually constructed features. Although these methods often achieve excellent results in specific scenarios, the same model is difficult to handle different assembly objects, even different speeds and sampling frequencies in the same assembly scenario. Therefore, if they are used in different scenarios, the model needs to be rebuilt. In some methods, in the statistical-based method, many studies collect force data and label samples, and then use machine learning or deep learning models to identify the snap fit, such as recurrent neural networks. Although they get rid of the limitation of designing manual features, they face the problem of significant degradation in processing long sequence data. Since the training sequence data of recurrent neural networks is extended, the time span becomes longer during calculation, which leads to a decrease in residual index, thus making the weight update smaller, which limits the ability to reflect the long-term memory effect of recurrent neural networks. Although some methods introduce residual links into RNN, they are still limited by the inherent defects of RNN network design. SUMMARY

[0009] The purpose of the present application is to overcome the above-mentioned shortcomings of the prior art and provide a snap fit identification method, device, equipment and storage medium to solve the problem that the same model is difficult to handle different assembly objects in the prior art snap fit condition identification method, and the long sequence data effect is severely degraded in the statistical-based snap fit condition identification method, making the weight update smaller during model training and use.

[0010] To achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows:

[0011] The first aspect of the present application provides a buckle clamping identification method, comprising the following steps:

[0012] S1, collecting a plurality of assembly sequence signals of the buckle in the clamping process according to a set time interval;

[0013] S2, judging whether the buckle clamping is successful or not based on the assembly sequence signals by a residual long short-term memory neural network model;

[0014] The residual long short-term memory neural network model comprises a plurality of long short-term memory neural network units connected in sequence, the input of each long short-term memory neural network unit is the sum of the input signal of the previous long short-term memory neural network unit and the residual, and the residual is the result of the calculation output of the previous long short-term memory neural network unit;

[0015] The long short-term memory neural network is obtained after being trained and tested by a data set, and the data set is obtained after the original data set is expanded;

[0016] S3, outputting the judgment result.

[0017] Further improvement of the present application is:

[0018] Preferably, in S2, the original data set expansion process is:

[0019] (1) obtaining the respective shortest clamping sequence of each successful clamping data in the original data set;

[0020] (2) sampling each successful clamping data with a fixed preset length T sliding window, when the sliding window contains the respective shortest clamping sequence, marking the sample sequence corresponding to the sliding window as a successful clamping sample expansion sequence;

[0021] (3) after collecting all sample expansion sequences, obtaining the expanded data set;

[0022] The expanded data set is trained and processed by a focal loss function or a binary classification loss function.

[0023] Preferably, in step (1), the method for obtaining the shortest clamping sequence of any one successful clamping data is:

[0024] a. calculating the feature of any one successful clamping data by the following formula As follows:

[0025]

[0026] Wherein, t represents the time step in the clamping sequence data, represents the contact force signal size at the time step, represents the t- The contact force signal size of the time step t, is a constant;

[0027] b. Take the maximum The corresponding time is the time point when the buckle assembly touches the bottom of the buckle base , take the maximum The sequence before The latest negative value , obtain the negative value The time point when the buckle exceeds the buckle threshold force , the time point And the buckle sequence between the time point The shortest buckle sequence.

[0028] Preferably, the assembly sequence signal is the assembly force sequence signal.

[0029] Preferably, the original data set is divided into cantilever buckle assembly data and ring buckle assembly data according to the buckle type;

[0030] The original data set is divided into artificial collection data and machine collection data according to the collection method;

[0031] The original data set is divided into successful buckle data and failed assembly data according to whether the buckle is successful.

[0032] Preferably, the input of each layer of long and short time memory neural network unit is the sum of the input signal and the residual of the previous layer of long and short time memory neural network unit, and specifically:

[0033] In the current long and short time memory neural network unit, the cell state vector is obtained by adding the input cell state vector of the previous layer of long and short time memory neural network unit and the residual of the current cell state vector, and the current hidden state vector is obtained by adding the input hidden state vector of the previous layer of long and short time memory neural network unit and the residual of the current hidden state vector;

[0034] The residual of the current cell state vector and the current hidden state vector is obtained by calculating the current long and short time memory neural network unit.

[0035] Preferably, the residual long and short time memory neural network model is:

[0036]

[0037] Wherein, is the activation vector of the forget gate, is the activation vector of the input gate, is the cell candidate vector, is the cell state vector, is an activation vector of the output gate, is a current hidden state vector, denotes a sigmoid function, tanh denotes a hyperbolic tangent function, W f is a weight matrix of the forget gate, h t-1 is a hidden state vector of the previous time, x t is an input vector of the current time, b f is a bias vector of the forget gate, W i is a weight matrix of the input gate, b i is a bias vector of the input gate, W C is a weight matrix of the cell candidate state, b c is a bias vector of the cell candidate state, C t-1 is a cell state vector of the previous time, W o is a weight matrix of the output gate, b o is a bias vector of the output gate.

[0038] The second aspect of the present application provides a buckle clamping recognition device, comprising:

[0039] The acquisition unit is configured to acquire an assembly sequence signal of the buckle in a clamping process at a set time interval.

[0040] The judgment unit is configured to determine whether the buckle clamping is successful based on the assembly sequence signal and a residual long short-term memory neural network model.

[0041] The residual long short-term memory neural network model comprises a plurality of long short-term memory neural network units connected in sequence, an input of each long short-term memory neural network unit being a sum of an input signal of a previous long short-term memory neural network unit and a residual, the residual being a result of a calculation output of the previous long short-term memory neural network unit.

[0042] The long short-term memory neural network is obtained after being trained and tested by a data set, and the data set is obtained after being expanded from an original data set.

[0043] The output unit is configured to output a judgment result.

[0044] The third aspect of the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the buckle clamping recognition method of any one of the above aspects when executing the computer program.

[0045] The fourth aspect of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the buckle clamping recognition method of any one of the above aspects.

[0046] Compared with the prior art, the buckle fitting recognition method has the following beneficial effects:

[0047] The buckle fitting recognition method discloses a buckle fitting sequence recognition model of a residual long short-term memory neural network (LSTM) introduced for the performance degradation problem of a traditional recurrent neural network (RNN) in processing a long-time sequence buckle detection force sequence. The LSTM is a special recurrent neural network (RNN), and compared with a residual RNN, can effectively learn and save long-term information through an internal gating mechanism (such as a forgetting gate, an input gate and an output gate), selectively retain or discard information, thereby more effectively capture long-time dependence, and further more deeply remember long sequence context information, especially in processing long sequence data, can greatly improve learning efficiency and network performance, and enhance memory and back propagation efficiency under a long time sequence. The residual link is innovatively added in the LSTM, the gradient flow is improved, the problems of gradient disappearance and gradient explosion are relieved, the residual link allows the network to directly transmit information between multiple time steps, thereby reducing the loss of information in the propagation process, therefore the residual link is used to improve the genetic mechanism of information in the long short-term memory cell, and the problem of gradient dissipation in processing a long time sequence is effectively prevented. The method proposed in the buckle fitting recognition method is a statistical method, compared with a method of recognizing a fitting sequence based on a manually constructed feature, the method is not affected by specific conditions of assembled parts, and for an assembled force signal data set of an arbitrary buckle fitting part assembly process, the method can be applied without modification, has strong generalization and wide applicability.

[0048] Further, the buckle fitting recognition method discloses a feature extraction method for identifying a shortest fitting sequence in a successful fitting sequence for sample amplification. The sample amplification method is helpful to amplify a large amount of effective data from a limited buckle fitting data set for network training, enhances the training effect on the limited data set, thereby relieving the overfitting problem in the network training process, and improves the recognition performance after network training convergence.

[0049] Further, the buckle fitting recognition method can use a sequence signal sensor containing necessary information of a reaction fitting process in a buckle fitting process, such as a sequence signal data based on an assembled force, to recognize a buckle fitting state, so that the residual long short-term memory neural network model has stronger universality. BRIEF DESCRIPTION OF DRAWINGS

[0050] FIG. 1 is a flowchart of the buckle fitting recognition method;

[0051] FIG. 2 is a construction and training diagram of the residual long short-term memory neural network model;

[0052] Fig. 3 is a flow chart of card and data sample amplification;

[0053] Fig. 4 is a shortest card and sequence diagram of the present application; wherein, t b represents the time point when the card engaging component touches the bottom after completing the card engaging process, f b represents the size of the card engaging contact force at this time point, f m represents the size of the first peak force before the card engaging component touches the bottom, t m represents the time point when the force is reached.

[0054] Fig. 5 is a residual LSTM network unit diagram of the present application;

[0055] Fig. 6 is a card engaging state recognition network based on residual LSTM of the present application. Embodiments of the present application

[0056] Hereinafter, the terms "first", "second", "third", "fourth" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second", "third", "fourth" can explicitly or implicitly include one or more of the features.

[0057] The method of matching provided by the embodiments of the present application can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, notebook computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The specific type of terminal device is not limited by the embodiments of the present application.

[0058] It should be noted that the terms "first", "second", etc. in the specification and drawings of the present application are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0059] Referring to FIG. 1, the first aspect of the present application discloses a buckle clamping identification method, comprising the following steps:

[0060] S1, collecting assembly information sequence signals according to a set time interval to obtain assembly information sequence signals of the buckle in the clamping process, wherein the sequence signals are composed of a plurality of assembly signal units arranged in time sequence;

[0061] S2, inputting the assembly information sequence signals into a residual long short-term memory neural network model to obtain a result of whether the buckle is clamped successfully;

[0062] The residual long short-term memory neural network model comprises a plurality of long short-term memory neural network units connected in sequence, the input of each layer of long short-term memory neural network unit is the sum of the input signal of the previous layer of long short-term memory neural network unit and the residual, and the residual is the calculation output result of the previous layer of long short-term memory neural network unit.

[0063] The long short-term memory neural network is obtained after training and testing by a data set, and the data set is obtained after expanding an original data set.

[0064] S3, outputting the judgment result.

[0065] In some embodiments of the present application, before S1, the assembly information sequence signals are collected according to a set time interval, and the assembly information sequence signals are sampled by a sliding window with a length of T, and the length of the sliding window is based on the sliding window length of the residual long short-term memory neural network model during training.

[0066] In S1 of some embodiments of the present application, the assembly information sequence signals are assembly sequence signals capable of reflecting necessary information of the buckle clamping process, which are obtained by a sensor capable of collecting sequence signals containing assembly information of the buckle clamping process. In the verification process of the present application, data is a buckle clamping data set based on the assembly force sequence signal, so that the residual long short-term memory neural network judges whether the clamping is successful by torque. However, if other assembly information sequence signals are used, they can also be applied under appropriate adjustment.

[0067] In some embodiments of the present application, the input of each layer of long short-term memory neural network unit is the sum of the input signal of the previous layer of long short-term memory neural network unit and the residual, and the cell state vector and the hidden state vector are prepared.

[0068] In the current LSTM unit, the cell state vector is obtained by adding the residual error of the input cell vector of the previous layer LSTM unit and the current cell state vector, and the hidden state vector is obtained by adding the residual error of the input hidden state vector of the previous layer LSTM unit and the current hidden state vector.

[0069] More specifically, the residual LSTM unit sets the input size, hidden feature size, number of LSTM layers, number of neurons in the fully connected layer, and number of output neurons of the entire LSTM model, and the forward propagation formula of the LSTM is as follows:

[0070] (1)

[0071] wherein, is the activation vector of the forget gate, which controls which information needs to be ignored, is the activation vector of the input gate, which controls which new information needs to be stored in the cell state, is the cell candidate vector, which contains new information that may be added to the cell state, is the cell state vector, which is the "memory" part of the LSTM network, is the activation vector of the output gate, which controls which information will be output, is the current hidden state vector, and W and b represent the weight matrix and bias vector, represents the Sigmoid function, which maps the weight input to the interval, represents the hyperbolic tangent function, which maps the weight input to the interval, W f : weight matrix of the forget gate, used to calculate the activation vector of the forget gate h t-1 : hidden state vector at the previous time, used to calculate various gate signals and candidate cell states at the current time, x t : input vector at the current time, used to calculate various gate signals and candidate cell states at the current time, b f : bias vector of the forget gate, used to calculate the activation vector of the forget gate W i : weight matrix of the input gate, used to calculate the activation vector of the input gate b i : bias vector of the input gate, used to calculate the activation vector of the input gate ; W C : weight matrix of the cell candidate state, used to calculate the cell candidate state vector b c: the bias vector of the cell candidate state, used for calculating the cell candidate state vector t-1 : the cell state vector of the previous moment, used for calculating the cell state vector of the current moment o : the weight matrix of the output gate, used for calculating the activation vector of the output gate o : the bias vector of the output gate, used for calculating the activation vector of the output gate .

[0072] As can be seen in combination with formula (2), in the calculation process, the cell state vector of the current moment C the cell state vector of the previous moment C t-1 is introduced in the calculation process, and the current hidden state vector H the cell state vector of the previous moment is introduced in the calculation process.

[0073] Referring to FIG. 2, in S2 of the above process, the obtaining process of the long short-term memory neural network model includes the following steps:

[0074] S21, a robot is used to buckle and assemble original data sets by autonomously collecting or online downloading public data sets, and the original data sets include assembly sequence signals in the buckling and assembly process.

[0075] S22, sample expansion is performed on the data set data, and the data set data is divided into a training set and a test set.

[0076] S23, a long short-term memory neural network model is constructed in combination with a residual link mechanism, and the training set data is input into the neural network for training.

[0077] S24, the obtained buckling and assembly recognition network is verified using the test set.

[0078] In some embodiments of the present application, in S21, the original data set collected is as rich as possible in the type of collection, and for a data set, the data set is divided into cantilever buckle assembly data and ring buckle assembly data according to the type of buckle; the original data set is divided into artificial collection data and robot collection data according to the collection method; the original data set is divided into successful assembly data and failed assembly data according to whether the assembly is successful; data collection experiments are performed on two different buckling devices, including cantilever buckle assembly and ring buckle assembly. Multiple types of data are set in the same data set, so that the training precision of the model is improved during subsequent data training.

[0079] Referring to FIG. 3, in some embodiments of the present application, in S22, the expansion process of the data set is as follows:​​​

[0080] (1) Obtain the shortest snap sequence of each successful snap data in the original data set;

[0081] The whole snap process of the buckle is divided into four stages: approaching, pressing, snapping and bottoming. In the approaching stage, the snap assembly has not yet been contacted, and the force signal remains constant at the initial value. In the pressing stage, the contact force gradually increases with the pressing until it reaches the peak value. In the snapping stage, the contact force sharply decreases, indicating a successful snap. In the bottoming stage, the contact force continues to increase with the pressing. Therefore, first find the shortest snap sequence of each successful snap data, which is the snap sequence between the peak value of the pressing stage and the bottoming time. In the subsequent expansion data, only the sliding window of the preset length T contains the shortest snap sequence, which is a successful snap data.

[0082] In order to find the shortest snap sequence s of the successful snap sample sequence, analyze the buckle snap sequence as shown in FIG. 2. In the successful snap sequence data, there must be a shortest snap sequence s, that is, the sequence data from to , which represents the process of the buckle breaking through the snap force threshold and entering the rapid force relief stage after the contact force reaches the peak value, and finally contacting the bottom of the base. In the process, the force signal will immediately have a turning point with a large slope change, i.e. , which can be used as a key feature point to identify and locate the shortest snap sequence s. The first peak force before the feature point indicates the breakthrough critical point of this snap. In order to identify , calculate the feature as follows for any snap force sequence sample:

[0083] (2)

[0084] where t represents the time step in the buckle sequence data, represents the contact force signal size at the time step, represents the contact force signal size at the time step t- , is a constant, then when , it means is at the trough position of the curve, and when <0, it means is at the peak position of the curve. Calculate the corresponding sequence for a certain successful snap sample sequence, and take the global maximum value, which corresponds to the time point t when the buckle assembly of the sequence snaps the base. , take sequence in previous distance the nearest negative value , the corresponding time point is the time point of breaking the buckle engagement threshold force , so the shortest engagement sequence can be determined, and the sample expansion of the data set sample is performed according to FIG. 3.

[0085] (2) Then a fixed preset length of sliding window is taken, a fixed step size is set, and the data sequence is sampled by sliding window, when the shortest engagement sequence is contained in the sliding window, the sample sequence of the sliding window expansion is marked as a successful engagement sample expansion sequence, otherwise, it is a failed engagement sample expansion sequence. The single original engagement data sample expansion flow chart is shown in FIG. 1.

[0086] (3) After all sample expansion sequences are collected, the expanded data set is obtained, and the expanded data set is divided into training set and test set according to the set proportion.

[0087] As a preferred scheme, the expanded data set is shuffled before division to ensure the balance of the data in the data set.

[0088] As a preferred scheme, considering that the enhanced data set often has a serious data imbalance problem, the enhanced data set is processed by a loss function, the loss function can be a focal loss function or a binary classification loss function, the binary cross-entropy loss function (BCE Loss), wherein the focal loss function has the following formula:

[0089] (3)

[0090] wherein, and are hyperparameters, is the prediction probability.

[0091] In some embodiments of the application, in S23, the specific process of constructing a residual long short-term memory neural network is as follows: the long short-term memory neural network (LSTM) is a special recurrent neural network (RNN) that can effectively learn and save long-term information through its internal gating mechanism (such as the forget gate, the input gate and the output gate). By introducing the LSTM unit, it aims to improve the performance degradation problem of the existing buckle engagement sequence RNN recognition network in processing long sequence buckle engagement data. On the other hand, the residual link is introduced, and the network directly accesses the output of the previous layer at each layer, which helps the gradient to flow more smoothly during training, thereby solving the problem of gradient vanishing or gradient explosion in deep network.

[0092] After the construction is completed, the training set data is obtained through S22 to train the long short-term memory neural network, and the trained long short-term memory neural network is obtained.

[0093] The present invention will be further described below with reference to specific embodiments.

[0094] Example 1

[0095] S1, Data Acquisition

[0096] This invention utilizes the Doltsinis public dataset, proposed by Stefanos Doltsinis of the Institute of Information Technology at the Hellenic Research and Technology Center. The dataset employs a KUKA LWR4+ robotic arm with a three-finger Barret BH-8 gripper mounted on it. Assembly components are secured to the gripper's end, and the connector is fixed to the edge of the tabletop to collect data during the assembly process. This dataset includes data acquisition experiments on two different snap-fit ​​devices: cantilever and ring-type snap-fit ​​assemblies. The assembly force between the snap-fit ​​components is measured using a force sensor integrated into the KUKA robotic arm; no external force sensors are used.

[0097] The datasets are divided into four parts: DS1, DS2, DS3, and DS4. DS1 and DS2 were acquired through a human-robot collaborative process, with the robot, under high-stiffness impedance control, simultaneously acting as an intelligent sensor to capture force signals during assembly. Each dataset contains 60 assembly force sequence data points, with DS1 containing data from ring-type snap-fit ​​installation experiments and DS2 containing data from cantilever snap-fit ​​assembly. Each dataset includes 31 failed assembly data points and 29 successful assembly data points. DS3 and DS4 were acquired through autonomous assembly using a KUKA robotic arm, with the robot, under high-stiffness impedance control, simultaneously acting as an intelligent sensor to capture force signals during assembly. Each dataset contains 50 assembly force sequence data points, with DS3 containing data from ring-type snap-fit ​​installation experiments and DS4 containing data from cantilever snap-fit ​​assembly. Each dataset includes 25 failed assembly data points and 25 successful assembly data points.

[0098] S2, performs augmentation processing on the dataset.

[0099] To achieve sample amplification, the shortest chi-string sequence for each successful chi-string data point is first identified, and then a fixed preset length is used. A sliding window is used to sample the original data sequence. When the sliding window contains the shortest snap-fit ​​sequence, the amplified sample sequence is marked as a successfully snap-fitted sample amplification sequence; otherwise, it is marked as a failed snap-fitted sample amplification sequence. After sample amplification is completed, the dataset is shuffled and divided into training and test sets in a 4:1 ratio.

[0100] S3, constructing an engagement state recognition network based on a residual LSTM

[0101] The network overview diagram based on the residual long short-term memory neural network is shown in FIG. 4, wherein the input size in the residual long short-term memory neural network unit is set to 10, the hidden feature size is set to 10, and the number of layers is set to 2. For the full connection layer therein, the number of input neurons is set to 10, and the number of output neurons is set to 2. After passing through a Softmax layer, a unique code for recognizing whether the sequence is a successful engagement is output.

[0102] During the training of the engagement recognition network, the augmented data set is divided into a training set and a test set according to a ratio of 4:1, the initial learning rate of the network is set to , the Adam optimizer is used, the training batch size is 1000, the training period is 200, the Focal loss is subjected to gradient descent network training until convergence, and then a model is obtained. Finally, the performance of the model is tested using the test set.

[0103] Compared with the shortest successful engagement sequence recognition method formula (2) proposed by Cui et al., the shortest successful engagement sequence recognition method formula (1) proposed in this paper avoids the case that the point is calculated as is smaller than . In formula (1), and the positions are calculated as the signs are necessarily opposite, but in formula (2), the signs are the same, which will cause inaccurate positioning for individual samples.

[0104] (4)

[0105] In addition, in the buckle engagement recognition network, compared with the residual RNN used in the method of Cui et al., this paper introduces a residual link in the LSTM network unit. Compared with the residual RNN, the residual LSTM can better remember information by using the residual link, especially when processing long sequence data, which can greatly improve the learning efficiency and the performance of the network. Due to the success of ResNet and the excellent performance of LSTM on sequence data, the residual LSTM can better maintain the information flow during the training process, so that a deeper model can be trained to obtain richer features and improve the accuracy.

[0106] The network structure proposed in this paper has been well verified on the Doltsinis public data set, and the prediction performance has been improved compared with the original model, proving that the method is feasible.

[0107] Table 1 Comparison of test results of different models

[0108]

[0109] The introduction of residual connections in LSTM brings several advantages:

[0110] 1. Shorter information transmission path: Residual connections provide a direct path from input to output for the network, allowing information to be quickly transmitted between different layers without going through layer-by-layer calculations. This reduces the attenuation of information during transmission and helps alleviate the gradient vanishing problem.

[0111] 2. More stable gradient propagation: With the presence of residual connections, gradients can be directly transmitted through this path during backpropagation, rather than relying entirely on layer-by-layer propagation. This makes gradient propagation more stable and avoids the problem of gradients becoming very small or very large when dealing with long sequences.

[0112] 3. Easier optimization: Residual connections make it easier to optimize the network. Even as the network depth increases, training can remain stable and efficient. This means we can build deeper networks to capture more complex patterns.

[0113] 4. Enhanced memory capacity: In LSTM, cell state is key to long-term information memory. Residual connections can help cell state remain stable over a longer time range, enhancing the memory capacity of LSTM.

[0114] The second aspect of the present application discloses a buckle clamping recognition device, comprising:

[0115] The acquisition unit is configured to collect assembly sequence signals of the buckle during the clamping process at a set time interval.

[0116] The judgment unit is configured to determine whether the buckle clamping is successful based on the assembly sequence signals and a residual long short-term memory neural network model.

[0117] The residual long short-term memory neural network model comprises a plurality of sequentially connected long short-term memory neural network units. The input of each layer of long short-term memory neural network unit is the sum of the input signal of the previous layer of long short-term memory neural network unit and the residual. The residual is the result of the calculation output of the previous layer of long short-term memory neural network unit.

[0118] The long short-term memory neural network is obtained after training and testing by a data set, and the data set is obtained by expanding an original data set.

[0119] The output unit is configured to output the judgment result.

[0120] The third aspect of the present application discloses a computer terminal device, which comprises a processor and a memory for storing a computer program comprising program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, and are particularly suitable for loading and executing one or more instructions to implement a corresponding method flow or a corresponding function; the processor of the embodiment can be used to implement a buckle clamping recognition method, which comprises the following steps: collecting assembly sequence signals of a plurality of buckles in a clamping process at a set time interval; determining whether the buckle clamping is successful based on the assembly sequence signals by using a residual long short-term memory neural network model; the residual long short-term memory neural network model comprises a plurality of long short-term memory neural network units connected in sequence, the input of each layer of long short-term memory neural network unit is the sum of the input signal of the previous layer of long short-term memory neural network unit and the residual, and the residual is the calculation result of the output of the previous layer of long short-term memory neural network unit; the long short-term memory neural network is obtained after training and testing by using a data set, the data set is obtained after expanding an original data set; and a determination result is output.

[0121] The fourth aspect of the present application discloses a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in a terminal device, used for storing programs and data. It can be understood that the computer readable storage medium here can include the built-in storage medium in the terminal device, and of course can also include the expansion storage medium supported by the terminal device. The computer readable storage medium provides a storage space which stores the operating system of the terminal. Moreover, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium here can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory. One or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the buckle card recognition method in the above embodiment, including the following steps: collecting assembly sequence signals of a plurality of buckle cards in a carding process according to a set time interval; determining whether the buckle card is successfully carded based on the assembly sequence signals by a residual long short-term memory neural network model; the residual long short-term memory neural network model includes a plurality of long short-term memory neural network units connected in turn, the input of each layer of long short-term memory neural network unit is the sum of the input signal of the previous layer of long short-term memory neural network unit and the residual, and the residual is the calculation output result of the previous layer of long short-term memory neural network unit; the long short-term memory neural network is obtained after training and testing by a data set, the data set is obtained after expanding an original data set; and outputting the determination result.

[0122] The above only describes the preferred embodiments of the present application and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A buckle engagement recognition method characterized by comprising: The method comprises the following steps: S1, collecting assembly sequence signals of buckles in a clamping process at a set time interval; S2, judging whether the buckle clamping is successful based on the assembly sequence signals by a residual long short-term memory neural network model; The residual long short-term memory neural network model comprises a plurality of long short-term memory neural network units connected in sequence, and the input of each layer of the long short-term memory neural network unit is the sum of the input signal of the previous layer of the long short-term memory neural network unit and a residual error, and the residual error is the result of the calculation output of the previous layer of the long short-term memory neural network unit; The long short-term memory neural network is obtained after training and testing by a data set, and the data set is obtained after expansion of an original data set; S3, outputting the judgment result.

2. The buckle engagement recognition method according to claim 1, characterized in that, In S2, the expansion process of the original data set is as follows: (1) obtaining the shortest clamping sequence of each successful clamping data in the original data set; (2) sampling each successful clamping data by a fixed preset length T sliding window, and marking the sample sequence corresponding to the sliding window as a successful clamping sample expansion sequence when the sliding window contains the shortest clamping sequence; (3) collecting all sample expansion sequences to obtain an expanded data set; The expanded data set is trained and processed by a focal loss function or a binary classification loss function.

3. The buckle engagement recognition method according to claim 2, wherein In step (1), the method for obtaining the shortest clamping sequence of any successful clamping data is as follows: a. For any one successful snap data, the feature is calculated by the following formula is a constant; where t denotes the time step in the card and sequence data, a contact force signal size at the time step, represents t- contact force signal size at time step t, The clamping sequence between and is the shortest clamping sequence. b. max The corresponding time is the time point when the buckle assembly touches the bottom of the buckle base , take the maximum The sequence is at the most recent negative value , obtaining a negative value Time point of breaking the snap-fit threshold force , time point and time point The assembly sequence signal is an assembly force sequence signal.

4. The buckle engagement recognition method according to claim 1, wherein The original data set is divided into cantilever buckle assembly data and ring buckle assembly data according to the buckle type; 5. The buckle engagement recognition method of claim 1, wherein, The original data set is divided into artificial collection data and machine collection data according to the collection method; The original data set is divided into successful clamping data and failed assembly data according to whether the clamping is successful. The input of each layer of the long short-term memory neural network unit is the sum of the input signal of the previous layer of the long short-term memory neural network unit and the residual error, and the input of each layer of the long short-term memory neural network unit is the sum of the input signal of the previous layer of the long short-term memory neural network unit and the residual error.

6. The buckle engagement recognition method of claim 1, wherein, In the current long short-term memory neural network unit, the cell state vector is obtained by adding the input cell state vector of the previous layer of the long short-term memory neural network unit and the residual error of the current cell state vector, and the current hidden state vector is obtained by adding the input hidden state vector of the previous layer of the long short-term memory neural network unit and the residual error of the current hidden state vector; The residual error of the current cell state vector and the current hidden state vector is obtained by calculating the current long short-term memory neural network unit. The residual long short-term memory neural network model is:

7. The buckle engagement recognition method according to claim 6, wherein It comprises: wherein is an activation vector for the forget gate, is an activation vector for the input gate, is a cell candidate vector, is the cell state vector, is the activation vector of the output gate, is the current hidden state vector, denotes a sigmoid function, tanh denotes a hyperbolic tangent function, W f is a weight matrix of the forget gate, h t-1 is a hidden state vector of the previous time, x t is an input vector of the current time, b f is a bias vector of the forget gate, W i is a weight matrix of the input gate, b i is a bias vector of the input gate, W C is a weight matrix of the cell candidate state, b c is a bias vector of the cell candidate state, C t-1 is a cell state vector of the previous time, W o is a weight matrix of the output gate, b o is a bias vector of the output gate.

8. A snap-fit recognition device characterized by comprising: a collection unit for collecting assembly sequence signals of buckles in a clamping process at a set time interval; a judgment unit for judging whether the buckle clamping is successful based on the assembly sequence signals by a residual long short-term memory neural network model; The residual long short-term memory neural network model comprises a plurality of long short-term memory neural network units connected in sequence, and the input of each layer of the long short-term memory neural network unit is the sum of the input signal of the previous layer of the long short-term memory neural network unit and a residual error, and the residual error is the result of the calculation output of the previous layer of the long short-term memory neural network unit; ​ The long short-term memory neural network is obtained after being trained and tested by a data set, and the data set is obtained after an original data set is expanded. An output unit is configured to output the determination result.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the buckle clamping identification method of any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the buckle clamping identification method of any one of claims 1-7.

Images