A method, apparatus, device, and storage medium for identifying snap-fit / locking mechanisms.
By using a residual long short-term memory neural network (LSTM) model and a dataset augmentation method, the problems of poor model applicability and long-term data recognition performance in existing technologies are solved. This enables the recognition of the snap-fit status of different assembly objects, improving recognition accuracy and efficiency.
Patent Information
- Application Number
- CN202411122865.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-08-15
AI Technical Summary
In existing technologies, the same model has difficulty in handling different assembly objects. In statistical methods for recognizing snap-fit states, the performance of long-term data degrades significantly, resulting in smaller weight updates during model training and use, which affects the recognition effect.
A residual long short-term memory neural network (LSTM) model is adopted, combined with residual links, and the features of the latch engagement state are extracted by the dataset augmentation method. Multi-layer sequentially connected LSTM neural network units are constructed to identify the latch engagement state.
It improves the model's generalization ability and applicability, effectively identifies the snap-fit states of different assembly objects, alleviates the gradient vanishing problem in long-term data processing, and enhances the network's learning efficiency and recognition performance.
Smart Images

Figure CN119066576B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of robotics technology, specifically relating to a method, apparatus, device, and storage medium for identifying latches and snaps. Background Technology
[0002] Snap-fit connections are widely used in various flexible assembly scenarios. Snap-fit connections work together by using pre-installed or separate positioning structures and locking mechanisms on components to create specific positional constraints on the components. The locking mechanism deforms during assembly and then returns to its original state to achieve locking and provide holding force. This connection method is not only simple and quick to operate, but also inexpensive, making it an efficient and economical product assembly method.
[0003] Snap-fit connections, a common industrial assembly method advocated by the "design for assembly" principle, replace traditional threaded assembly methods. Snap-fit components connect by pushing. Considering the need to apply significant force to overcome the resistance of elastic structures, manual assembly often relies on the force perceived by fingers to determine the engagement state. Today, industrial robots are gradually replacing manual labor, enabling automated and flexible production. While widely used thresholding methods can successfully guide robots to identify assembly integrity, structural characterization processes require that thresholding alone cannot detect the entire engagement state. Furthermore, applying excessive pressure in the presence of assembly defects can damage components. Therefore, utilizing industrial robots for accurate engagement identification is essential for meeting safety requirements in automated production.
[0004] Previous research on robot assembly has largely considered assembly to be a quasi-static process because the forces involved don't change significantly overall. However, this is not the case for snap-fit assembly. During snap-fit assembly, energy is released very rapidly at the moment of overcoming resistance, causing a significant change in assembly force within a short period. Current snap-fit engagement state recognition methods are mainly divided into two categories: the first is analysis-based methods, and the second is statistical methods.
[0005] Rusli et al. investigated the tactile feedback of snap fasteners during manual assembly, constructed a linear model to identify the snap engagement state, experimentally analyzed the influence of the force and hardness of the preloaded sealing element on the tactile feedback, and proposed a new quantitative index, "locking signal to holding force ratio," to predict the confidence level of snap assembly.
[0006] Doltsinis et al. proposed a machine learning-based framework for real-time identification of successful snap-fit states. They first defined a set of features characterizing different snap-fit types, including signal energy, Hjorth complexity, Shannon entropy, and the slope of the signal's mean, absolute value, and signal strength. Based on feature extraction, they employed a Support Vector Machine (SVM) as the classifier, specifically using a nonlinear radial basis function (RBF) kernel. The SVM classifier learns an optimal hyperplane from the training data to distinguish between successful and unsuccessful snap-fits.
[0007] Cui et al. proposed a method for rapidly identifying the engagement / disengagement state of snap fasteners. This method combines a snap fastener dataset generation strategy that automatically acquires labels with a multi-layer recurrent neural network (RNN). This network can process sequential data and introduces residual connections in both the temporal and network layer dimensions. The RNN units are supplemented by residual connections in both the temporal and network layer dimensions to address the gradient vanishing problem of traditional RNNs when processing long sequence data.
[0008] While the methods described above all offer effective ways to identify latching states, some heavily rely on manually constructed features. Although these methods often achieve excellent results in specific scenarios, due to the limitations of these scenarios, the same model struggles to handle different assembly objects, or even different speeds and sampling frequencies within the same assembly scenario. Therefore, if they are used in different scenarios, the model needs to be rebuilt. Other methods, particularly statistical methods, often involve collecting force data and labeling samples, then using machine learning or deep learning models to identify latching states, such as recurrent neural networks (RNNs). While these methods overcome the limitations of manually designed features, they face the problem of significant performance degradation when processing long-term data. Because the training sequence data for RNNs is extended, the longer time span during computation leads to a decrease in the residual exponent, resulting in smaller weight updates. This limits the ability to reflect the long-term memory effect of RNNs. Although methods have introduced residual connections into RNNs, they are still limited by the inherent design flaws of RNNs. Summary of the Invention
[0009] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, apparatus, device and storage medium for identifying snap-fit states, so as to solve the problems that in the prior art snap-fit state identification methods, the same model is difficult to handle different assembly objects, and the long-term data performance of the statistical snap-fit state identification method degrades severely, resulting in smaller weight updates during model training and use.
[0010] To achieve the above objectives, the present invention employs the following technical solution:
[0011] The first aspect of the present invention provides a method for recognizing snap-fit engagement, comprising the following steps:
[0012] S1, according to the set time interval, collect the assembly sequence signals of multiple buckles during the engagement process;
[0013] S2, based on the assembly sequence signal, the residual long short-time memory neural network model determines whether the buckle engagement is successful;
[0014] The residual long short-term memory neural network model includes multiple layers of sequentially connected long short-term memory neural network units. The input of each layer of long short-term memory neural network unit is the sum of the input signal of the previous layer of long short-term memory neural network unit and the residual. The residual is the result calculated and output by the previous layer of long short-term memory neural network unit.
[0015] The long short-term memory neural network is obtained by training and testing on a dataset, which is obtained by augmenting the original dataset.
[0016] S3, output the judgment result.
[0017] A further improvement of the present invention is that:
[0018] Preferably, in S2, the original dataset augmentation process is as follows:
[0019] (1) Obtain the shortest card combination sequence for each successful card combination in the original dataset;
[0020] (2) Sample each successful carding data with a sliding window of a fixed preset length T. When the sliding window contains the shortest carding sequence of each, mark the sample sequence corresponding to the sliding window as the sample amplification sequence of successful carding.
[0021] (3) After collecting all the amplified sequences of the samples, the amplified dataset is obtained;
[0022] The augmented dataset is trained using either a focus loss function or a binary classification loss function.
[0023] Preferably, in step (1), the method for obtaining the shortest card combination sequence from any successful card combination data is as follows:
[0024] a. Calculate the characteristic θ for any successful card combination using the following formula. t ,
[0025]
[0026] Where t represents the time step in the card and sequence data, f t f represents the magnitude of the contact force signal at that time step. t-τThis represents the magnitude of the contact force signal at time step t-τ, where τ is a constant;
[0027] b. Take the maximum θ t The corresponding time is the time point t when the buckle assembly touches the bottom of the buckle base. b Take the maximum θ t Sequence in t b The most recent negative value θ t To obtain a negative value θ t The time point t when the buckle engagement threshold force is exceeded m Time point t m and time point t b The sequence of card combinations between them is the shortest card combination sequence.
[0028] Preferably, the assembly sequence signal is an assembly force sequence signal.
[0029] Preferably, the original dataset is divided into cantilever buckle assembly data and ring buckle assembly data according to the buckle type;
[0030] The original dataset is divided into manually collected data and machine-collected data according to the collection method;
[0031] The original dataset is divided into successful assembly data and failed assembly data according to whether the card connection was successful.
[0032] Preferably, the input to each layer of the Long Short-Term Memory (LSTM) neural network unit is the sum of the input signal of the previous layer of the LSM neural network unit and the residual, specifically as follows:
[0033] In the current Long Short-Term Memory (LSTM) neural network unit, the cell state vector is obtained by adding the residual of the input cell state vector of the previous LSM neural network unit to the current cell state vector, and the current hidden state vector is obtained by adding the residual of the input hidden state vector of the previous LSM neural network unit to the current hidden state vector.
[0034] The current cell state vector and the current hidden state vector residual are both obtained by calculation through the current long short-term memory neural network unit.
[0035] Preferably, the residual long short-term memory neural network model is
[0036]
[0037] Among them, f t It is the activation vector of the forget gate, i t It is the activation vector of the input gate. It is a candidate vector for cells, C t It is the cell state vector, o t h is the activation vector of the output gate.t This is the current hidden state vector, where σ represents the Sigmoid function, tanh represents the hyperbolic tangent function, and W... f It is the weight matrix of the forget gate, h t-1 It is the hidden state vector from the previous time step, x t It is the input vector at the current moment, b f W is the bias vector of the forget gate. i It is the weight matrix of the input gate, b i W is the bias vector of the input gate. C It is the weight matrix of the candidate cell states, b c C is the bias vector of the candidate cell states. t-1 W is the cell state vector from the previous moment. o It is the weight matrix of the output gate, b o It is the bias vector of the output gate.
[0038] A second aspect of the present invention provides a snap-fit identification device, comprising:
[0039] The acquisition unit is used to acquire the assembly sequence signal of the buckle during the engagement process at set time intervals;
[0040] The judgment unit is used to determine whether the snap-fit is successfully engaged based on the assembly sequence signal and the residual short-time memory neural network model.
[0041] The residual long short-term memory neural network model includes multiple layers of sequentially connected long short-term memory neural network units. The input of each layer of long short-term memory neural network unit is the sum of the input signal of the previous layer of long short-term memory neural network unit and the residual. The residual is the result calculated and output by the previous layer of long short-term memory neural network unit.
[0042] The long short-term memory neural network is obtained by training and testing on a dataset, which is obtained by augmenting the original dataset.
[0043] The output unit is used to output the judgment result.
[0044] A third aspect of the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the latch engagement recognition method described in any of the preceding claims.
[0045] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the latch engagement and locking method described in any of the preceding claims.
[0046] Compared with the prior art, the present invention has the following beneficial effects:
[0047] This invention discloses a method for identifying snap-fit sequences. Addressing the performance degradation of traditional recurrent neural networks (RNNs) when processing long-term snap-fit detection force sequences, this method introduces a snap-fit sequence recognition model using a residual long short-term memory neural network (LSTM). LSTM is a special type of RNN that, compared to residual RNNs, can effectively learn and store long-term information through its internal gating mechanisms (such as forget gates, input gates, and output gates), selectively retaining or discarding information. This allows for more effective capture of long-term dependencies and deeper memorization of long-term sequence context information. Especially when processing long-term data, it can significantly improve learning efficiency and network performance, enhancing memory and backpropagation efficiency for long-term sequences. This invention innovatively incorporates residual links into the LSTM, improving gradient flow and mitigating the problems of gradient vanishing and exploding. Residual links allow the network to directly transfer information between multiple time steps, reducing information loss during propagation. Therefore, utilizing residual links improves the genetic mechanism of information in long short-term memory cells, effectively preventing gradient vanishing when processing long-term sequences. The method proposed in this invention is based on statistics. Compared with methods that rely on manually constructed features for identification of snap-fit sequences, this method is not affected by the specific circumstances of the assembled parts. It can be applied to the assembly force signal dataset of any snap-fit parts during the assembly process with almost no modification, demonstrating strong generalization and wide applicability.
[0048] Furthermore, this invention proposes a feature extraction method for identifying the shortest k-k sequence in successful k-k sequences for sample amplification. This sample amplification method helps to amplify a large amount of effective data in a limited k-k sequence dataset for network training, enhancing the training effect on a limited dataset, thereby alleviating the overfitting problem in the network training process and improving the recognition performance after the network training converges.
[0049] Furthermore, this invention can utilize a sequence signal sensor that generates necessary information about the engagement process during the snap-fit process, such as sequence signal data based on assembly force, to identify the snap-fit status while keeping the framework essentially unchanged. This makes the residual long short-term memory neural network model more universal. Attached Figure Description
[0050] Figure 1 This is a flowchart of a snap-fit identification method according to the present invention;
[0051] Figure 2 This is a training diagram for constructing the residual long short-term memory neural network model of the present invention;
[0052] Figure 3 Flowchart for card and data sample amplification;
[0053] Figure 4 This is a schematic diagram of the shortest card and sequence of the present invention; wherein, t b f indicates the time point at which the engaging component touches the bottom after completing the engaging process. b f represents the magnitude of the contact force at that point in time. m t represents the magnitude of the first peak force before the engaging components bottom out. m This indicates the point in time when the force is reached.
[0054] Figure 5 This is a schematic diagram of the residual LSTM network unit of the present invention;
[0055] Figure 6 This invention relates to a card engagement / disengagement state recognition network based on residual LSTM. Detailed Implementation
[0056] Hereinafter, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first," "second," "third," or "fourth" may explicitly or implicitly include one or more of that feature.
[0057] The method for simultaneous shooting provided in this invention can be applied to terminal devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of terminal device.
[0058] It should be noted that the terms "first," "second," etc., used in the specification and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0059] See Figure 1 The first aspect of this invention discloses a method for recognizing snap-fit engagement, comprising the following steps:
[0060] S1, Acquire assembly information sequence signals at set time intervals to obtain assembly information sequence signals during the engagement process of the buckle, wherein the sequence signals are composed of multiple assembly signal units arranged in chronological order;
[0061] S2, input the assembly information sequence signal into the residual long short-term memory neural network model to obtain the result of whether the buckle is successfully engaged;
[0062] The residual long short-term memory neural network model includes multiple long short-term memory neural network units connected in sequence. The input of each long short-term memory neural network unit is the sum of the input signal of the previous long short-term memory neural network unit and the residual. The residual is the result calculated and output by the previous long short-term memory neural network unit.
[0063] The long short-term memory neural network is obtained by training and testing on a dataset, which is obtained by augmenting the original dataset.
[0064] S3, output the judgment result.
[0065] In some embodiments of the present invention, before S1, assembly information sequence signals are collected at set time intervals, and the assembly information sequence signals are sampled through a sliding window of length T. The length of the sliding window is the length of the sliding window during training of the residual long short-term memory neural network model.
[0066] In some embodiments of the present invention, in S1, the assembly information sequence signal is an assembly sequence signal that can reflect the necessary information of the snap-fit process. It is obtained by a sensor that can collect the sequence signal containing the assembly information of the snap-fit process generated during the snap-fit process. In the verification process, the present invention uses a snap-fit dataset based on the assembly force sequence signal. The residual length is used by the memory neural network to judge whether the snap-fit is successful through torque. However, other assembly information sequence signals can also be used with appropriate adjustments.
[0067] In some embodiments of the present invention, the input to each layer of the Long Short-Term Memory (LSTM) neural network unit is the sum of the input signal of the previous LSM unit and the residual, resulting in a cell state vector and a hidden state vector.
[0068] In the current Long Short-Term Memory (LSTM) neural network unit, the cell state vector is obtained by adding the residual of the input cell vector of the previous LSM neural network unit to the current cell state vector, and the current hidden state vector is obtained by adding the residual of the input hidden state vector of the previous LSM neural network unit to the current hidden state vector.
[0069] More specifically, for the residual LSTM unit, the input size, hidden feature size, number of LSTM layers, number of neurons in the fully connected layers, and number of output neurons of the entire LSTM model are set. The forward propagation formula for the Long Short-Term Memory neural network is as follows:
[0070]
[0071] Among them, f t It is the activation vector of the forget gate, which controls which information needs to be ignored. t It is the activation vector of the input gate, which controls which new information needs to be stored in the cell state. It is a cell candidate vector, containing new information that may be added to the cell state. C t It is the cell state vector, the "memory" part of the LSTM network. t It is the activation vector of the output gate, which controls which information will be output. t This is the current hidden state vector. W and b represent the weight matrix and bias vector, respectively. σ represents the sigmoid function, which maps the weight inputs to the interval (0,1). tanh represents the hyperbolic tangent function, which maps the weight inputs to the interval (-1,1). W f The weight matrix of the forget gate is used to calculate the activation vector f of the forget gate. t h t-1 The hidden state vector from the previous time step, used to calculate various gating signals and candidate cell states at the current time step; xt : The input vector at the current time step, used to calculate various gating signals and candidate cell states at the current time step; b f The bias vector of the forget gate, used to calculate the activation vector f of the forget gate. t W i The weight matrix of the input gate, used to calculate the activation vector i of the input gate. t b i The bias vector of the input gate, used to compute the activation vector i of the input gate. t W C The weight matrix of the cell candidate states, used to calculate the cell candidate state vector. b c : The bias vector of the candidate cell state, used to compute the candidate cell state vector. C t-1 The cell state vector from the previous time step, used to calculate the cell state vector C at the current time step. t W o The weight matrix of the output gate, used to calculate the activation vector o of the output gate. t b o The bias vector of the output gate. Used to calculate the activation vector o of the output gate. t .
[0072] Combining equation (2), it can be seen that during the calculation process, the cell state vector C at the current moment... t The cell state vector C from the previous time step is introduced into the calculation process. t-1 Meanwhile, the current hidden state vector h t The cell state vector from the previous moment is introduced during the calculation process.
[0073] See Figure 2 In step S2 of the above process, the acquisition process of the Long Short-Term Memory Neural Network Model includes the following steps:
[0074] S21, obtain the original dataset of robot buckle engagement assembly by autonomously collecting or downloading public datasets online. The original dataset includes the assembly sequence signals of the buckle during the engagement process.
[0075] S22, perform sample augmentation on the dataset and divide it into training set and test set.
[0076] S23, Combine the residual linking mechanism to construct a long short-term memory neural network model, which is used to identify whether the long sequence of snapping and locking is successful. The training set data is input into the neural network for training.
[0077] S24, the obtained card-locking recognition network is verified using a test set.
[0078] In some embodiments of the present invention, in step S21, the collected raw dataset is made as diverse as possible in terms of the types of data collected. For a single dataset, it is categorized by snap-fit type into cantilever snap-fit assembly data and ring snap-fit assembly data; the raw dataset is also categorized by collection method into manually collected data and robot-collected data; and by whether the snap-fit was successful into successfully snap-fit data and unsuccessfully assembled data. Data collection experiments were conducted on two different snap-fit devices, including cantilever and ring snap-fit assemblies. Setting multiple types of data within the same dataset improves the training accuracy of the model during subsequent data training.
[0079] See Figure 3 In some embodiments of the present invention, in S22, the data set augmentation process is as follows:
[0080] (1) Obtain the shortest card combination sequence for each successful card combination in the original dataset;
[0081] The entire engagement process of the latch is divided into four stages: approach, pressing, engagement, and bottoming out. In the approach stage, the engagement components have not yet made contact, and the force signal remains constant at its initial value. In the pressing stage, the contact force gradually increases until it reaches its peak. In the engagement stage, the contact force drops sharply, indicating successful engagement. In the bottoming out stage, the contact force continues to increase with pressing. Therefore, the shortest engagement sequence for each successful engagement data point is first identified; this sequence is the engagement sequence between the peak value of the pressing stage and the bottoming out point. During subsequent data amplification, only when this shortest engagement sequence is contained within a sliding window of a preset length T is it considered a successful engagement data point.
[0082] To find the shortest snap-fit sequence s of a successful snap-fit sample sequence, analyze the snap-fit sequence as follows: Figure 2 As shown, in the successful card combination sequence data, there must exist a shortest card combination sequence s, that is, the sequence from t in the figure. m to t b This sequence of data represents the process by which the latch continues to advance after the contact force reaches its peak, thus exceeding the engagement force threshold, entering a rapid release phase, and finally contacting the bottom of the engagement base. During this process, as the engagement component touches the bottom, the force signal immediately exhibits a sharp inflection point with a drastically different slope, namely f. b This point can serve as a key feature point for identifying and locating the shortest kinematic sequence s, and the first peak force f preceding this feature point is... m This indicates the critical point at which the card breaks through. To identify t... b Calculate the characteristic θ for any sample of the resultant force sequence. t as follows:
[0083]
[0084] Where t represents the time step in the card and sequence data, f t f represents the magnitude of the contact force signal at that time step. t-τ Let represent the magnitude of the contact force signal at time step t-τ, where τ is a constant. Then, when θ t When >0, it means f t At the trough of the curve, when θ t When <0, it means f t At the peak of the curve. Calculate the corresponding θ for a given sequence of successfully matched samples. t The sequence is taken as the global maximum value, and the corresponding t represents the time point t when the buckle component of the sequence touches the bottom of the buckle base. b Take θ t Sequence in t b Previously, the distance from t b The most recent negative value θ t The corresponding time point t is the time point t that breaks through the threshold force of the latch engagement. m Therefore, the shortest card combination sequence can be determined, and thus, based on Figure 3 Perform sample augmentation on the dataset samples.
[0085] (2) Then, using a sliding window with a fixed preset length of t and a fixed step size, the data sequence is sampled using a sliding window. When the sliding window contains the shortest kink sequence, the amplified sample sequence is marked as a successfully kink-bound sample amplification sequence; otherwise, it is a failed kink-bound sample amplification sequence. The flowchart for amplifying a single original kink-bound data sample is as follows: Figure 1 As shown.
[0086] (3) After collecting all the amplified sequences of the samples, the amplified dataset is obtained. The amplified dataset is then divided into training set and test set according to a set ratio.
[0087] As a preferred approach, the expanded dataset is shuffled before being partitioned to ensure the balance of data within the dataset.
[0088] As a preferred approach, considering that augmented datasets often suffer from severe data imbalance, the augmented dataset is processed using a loss function. This loss function can be either Focal loss or a binary classification loss function. The binary classification cross-entropy loss function (BCE Loss) is used, and the calculation formula for Focal loss is as follows:
[0089]
[0090] Where α and γ are hyperparameters, p c To predict probabilities.
[0091] In some embodiments of the present invention, in S23, the specific process of constructing a residual long short-term memory neural network is as follows: A long short-term memory neural network (LSTM) is a special type of recurrent neural network (RNN) that can effectively learn and store long-term information through its internal gating mechanisms (such as forget gates, input gates, and output gates). By introducing LSTM units, the aim is to improve the performance degradation problem of existing snap-fit sequence RNN recognition networks in handling long-sequence snap-fit data. On the other hand, residual connections are introduced, allowing the network to directly access the output of previous layers at each layer. This helps gradients flow more smoothly during training, thereby solving the problems of vanishing or exploding gradients in deep networks.
[0092] After the construction is completed, the training set data is obtained through S22 to train the long short-term memory neural network, and the trained long short-term memory neural network is obtained.
[0093] The present invention will be further described below with reference to specific embodiments.
[0094] Example 1
[0095] S1, Data Acquisition
[0096] This invention utilizes the Doltsinis public dataset, proposed by Stefanos Doltsinis of the Institute of Information Technology at the Hellenic Research and Technology Center. The dataset employs a KUKA LWR4+ robotic arm with a three-finger Barret BH-8 gripper mounted on it. Assembly components are secured to the gripper's end, and the connector is fixed to the edge of the tabletop to collect data during the assembly process. This dataset includes data acquisition experiments on two different snap-fit devices: cantilever and ring-type snap-fit assemblies. The assembly force between the snap-fit components is measured using a force sensor integrated into the KUKA robotic arm; no external force sensors are used.
[0097] The datasets are divided into four parts: DS1, DS2, DS3, and DS4. DS1 and DS2 were acquired through a human-robot collaborative process, with the robot, under high-stiffness impedance control, simultaneously acting as an intelligent sensor to capture force signals during assembly. Each dataset contains 60 assembly force sequence data points, with DS1 containing data from ring-type snap-fit installation experiments and DS2 containing data from cantilever snap-fit assembly. Each dataset includes 31 failed assembly data points and 29 successful assembly data points. DS3 and DS4 were acquired through autonomous assembly using a KUKA robotic arm, with the robot, under high-stiffness impedance control, simultaneously acting as an intelligent sensor to capture force signals during assembly. Each dataset contains 50 assembly force sequence data points, with DS3 containing data from ring-type snap-fit installation experiments and DS4 containing data from cantilever snap-fit assembly. Each dataset includes 25 failed assembly data points and 25 successful assembly data points.
[0098] S2, performs augmentation processing on the dataset.
[0099] To achieve sample augmentation, the shortest snap-fit sequence for each successfully snapped data point is first identified. Then, a sliding window of fixed preset length T is used to sample the original data sequence. If the sliding window contains the shortest snap-fit sequence, the augmented sample sequence is marked as a successfully snapped sample augmentation sequence; otherwise, it is marked as a failed snap-fit sample augmentation sequence. After sample augmentation, the dataset is shuffled and divided into training and test sets in a 4:1 ratio.
[0100] S3, Construct a card-locking state recognition network based on residual LSTM.
[0101] A network overview diagram based on residual long short-term memory neural networks is shown below. Figure 4 As shown, the input size of the residual long short-term memory neural network unit is set to 10, the hidden feature size is set to 10, and the number of layers is set to 2. For the fully connected layer, the number of input neurons is set to 10, and the number of output neurons is set to 2. After passing through a softmax layer, the output is a unique code that identifies whether the sequence segment is successfully matched.
[0102] During the training of the card recognition network, the augmented dataset was divided into training and test sets in a 4:1 ratio, and the initial learning rate of the network was set to 1×10⁻⁶. -4 The Adam optimizer was used, with a training batch size of 1000 and a training cycle of 200. The Focal loss was trained using a gradient descent network until convergence, and the model was obtained. Finally, the performance of the model was tested using a test set.
[0103] Compared with the shortest successful card combination sequence identification method formula (2) proposed by Cui et al., the shortest successful card combination sequence identification method formula (1) proposed in this paper avoids the problem of f in some cases. m θ is calculated from the point. m′ $ratio to θ b′ In smaller cases, in formula (1), f m θ calculated from the position of f_b t The signs must be opposite, but in formula (2) the signs are the same, which will cause f for individual samples. b The problem of inaccurate positioning.
[0104]
[0105] Furthermore, in the card-locking recognition network, compared to the residual RNN used by Cui et al., this paper introduces residual connections into the LSTM network units. The reason residual LSTM is better than residual RNN is that it can utilize residual connections to remember information more deeply, especially when processing long sequence data. This can greatly improve learning efficiency and network performance. Due to the success of ResNet and the excellent performance of LSTM on sequence data, residual LSTM can better maintain information flow during training, thus enabling the training of deeper models, acquiring richer features, and improving accuracy.
[0106] The network structure proposed in this invention has been well validated on the Doltsinis public dataset, and its prediction performance has been improved compared with the original model, proving that the method is feasible.
[0107] Table 1 Comparison of test results for different models
[0108]
[0109] This invention introduces residual links into LSTM, which brings the following advantages:
[0110] 1. Shortened Information Transmission Path: Residual links provide a direct path from input to output in the network, allowing information to be quickly transmitted between different layers without having to go through layers of computation. This reduces information attenuation during propagation and helps alleviate the vanishing gradient problem.
[0111] 2. More stable gradient propagation: Due to the existence of residual links, gradients can be directly propagated through this path during backpropagation, without having to rely entirely on layer-by-layer propagation. This makes gradient propagation more stable and avoids the problem of gradients becoming very small or very large when dealing with long sequences.
[0112] 3. Easier to optimize: Residual connections make the network easier to optimize. Training remains stable and efficient even as network depth increases. This means we can build deeper networks to capture more complex patterns.
[0113] 4. Enhanced Memory Capacity: In LSTM, cell state is key to remembering long-term information. Residual links can help cell state remain stable over a longer timeframe, thereby enhancing the memory capacity of LSTM.
[0114] A second aspect of the present invention discloses a snap-fit identification device, comprising:
[0115] The acquisition unit is used to acquire the assembly sequence signal of the buckle during the engagement process at set time intervals;
[0116] The judgment unit is used to determine whether the snap-fit is successfully engaged based on the assembly sequence signal and the residual short-time memory neural network model.
[0117] The residual long short-term memory neural network model includes multiple long short-term memory neural network units connected in sequence. The input of each long short-term memory neural network unit is the sum of the input signal of the previous long short-term memory neural network unit and the residual. The residual is the result calculated and output by the previous long short-term memory neural network unit.
[0118] The long short-term memory neural network is obtained by training and testing on a dataset, which is obtained by augmenting the original dataset.
[0119] The output unit is used to output the judgment result.
[0120] A third aspect of this invention discloses a computer device, which includes a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to implement a corresponding method flow. Or corresponding functions; the processor described in this embodiment can be used to implement a snap-fit identification method, including the following steps: S1, according to a set time interval, collect assembly sequence signals of multiple snaps during the snap-fit process; S2, based on the assembly sequence signals, a residual long short-term memory neural network model determines whether the snap-fit is successful; the residual long short-term memory neural network model includes multiple sequentially connected long short-term memory neural network units, the input of each layer of long short-term memory neural network unit is the sum of the input signal of the previous layer of long short-term memory neural network unit and the residual, the residual is the result calculated and output by the previous layer of long short-term memory neural network unit; the long short-term memory neural network is obtained after training and testing with a dataset, the dataset is obtained after augmenting the original dataset; S3, output the judgment result.
[0121] A fourth aspect of this invention discloses a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor, which can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement a snap-fit identification method in the above embodiments, including the following steps: S1, collecting assembly sequence signals of multiple snaps during the snap-fit process at set time intervals; S2, judging whether the snap-fit is successful based on the assembly sequence signals using a residual long short-term memory neural network model; the residual long short-term memory neural network model includes multiple sequentially connected long short-term memory neural network units, the input of each layer of long short-term memory neural network unit is the sum of the input signal of the previous layer of long short-term memory neural network unit and the residual, the residual being the result calculated and output by the previous layer of long short-term memory neural network unit; the long short-term memory neural network is obtained after training and testing with a dataset, the dataset being obtained after augmenting the original dataset; S3, outputting the judgment result.
[0122] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A buckle engagement recognition method characterized by comprising: Includes the following steps: S1, according to a set time interval, collect assembly sequence signals of multiple buckles during the engagement process; the assembly sequence signal is an assembly force sequence signal; S2, based on the assembly sequence signal, uses a residual long short-time memory neural network model to determine whether the snap-fit is successful; The residual long short-term memory neural network model includes multiple layers of sequentially connected long short-term memory neural network units. The input of each layer of long short-term memory neural network unit is the sum of the input signal of the previous layer of long short-term memory neural network unit and the residual. The residual is the result calculated and output by the previous layer of long short-term memory neural network unit. The residual long short-term memory neural network model is as follows: in, It is the activation vector of the forget gate. It is the activation vector of the input gate. It is a cell candidate vector. It is a cell state vector. It is the activation vector of the output gate. It is the current hidden state vector. This represents the Sigmoid function. tanh Represents the hyperbolic tangent function. W f It is the weight matrix of the forget gate. h t-1 It is the hidden state vector from the previous time step. x t It is the input vector at the current moment. b f It is the bias vector of the forget gate. W i It is the weight matrix of the input gate. b i It is the bias vector of the input gate. W C It is the weight matrix of the candidate cell states. b c It is the bias vector of the candidate cell state. C t-1 It is the cell state vector from the previous moment. W o It is the weight matrix of the output gate. b o It is the bias vector of the output gate; The residual long short-term memory neural network model is obtained by training and testing on a dataset, which is obtained by augmenting the original dataset. The process of augmenting the original dataset is as follows: (1) Obtain the shortest card combination sequence for each successful card combination in the original dataset; In step (1), the method for obtaining the shortest card combination sequence for any successful card combination data is as follows: a. Calculate the characteristics of any successful card combination data using the following formula. , in, t This represents the time step in the sequence data. This indicates the magnitude of the contact force signal at that time step. express t- The magnitude of the contact force signal at the time step. express t + The magnitude of the contact force signal at the time step. It is a constant; b. Take the maximum The corresponding moment is the point at which the latching component touches the bottom of the latching base. Take the largest Sequence in The most recent negative value , obtain negative value The time point when the buckle engagement threshold force is exceeded Time point and time point The sequence of card combinations between them is the shortest card combination sequence; (2) Sample each successful card-connection data using a sliding window with a fixed preset length of T. When the sliding window contains the shortest card-connection sequence of each, mark the sample sequence corresponding to the sliding window as the sample amplification sequence of the successful card-connection. (3) After collecting all the amplified sequences of the samples, the amplified dataset is obtained; The augmented dataset is trained using a focus loss function or a binary classification loss function; the input to each layer of the long short-term memory neural network unit is the sum of the input signal of the previous layer of the long short-term memory neural network unit and the residual, specifically: In the current Long Short-Term Memory (LSTM) neural network unit, the cell state vector is obtained by adding the residual of the input cell state vector of the previous LSM neural network unit to the current cell state vector, and the current hidden state vector is obtained by adding the residual of the input hidden state vector of the previous LSM neural network unit to the current hidden state vector. The current cell state vector and the current hidden state vector residual are both obtained by calculation through the current long short-term memory neural network unit; S3, output the judgment result.
2. The method for identifying snap-fit engagement according to claim 1, characterized in that, The original dataset is divided into cantilever buckle assembly data and ring buckle assembly data according to the buckle type. The original dataset is divided into manually collected data and machine-collected data according to the collection method; The original dataset is divided into successful assembly data and failed assembly data according to whether the card connection was successful.
3. A buckle engagement recognition device for implementing the buckle engagement recognition method of claim 1, characterized in that, include: The acquisition unit is used to acquire the assembly sequence signal of the buckle during the engagement process at set time intervals; The judgment unit is used to determine whether the snap-fit is successfully engaged based on the assembly sequence signal and the residual long short-time memory neural network model. The residual long short-term memory neural network model includes multiple layers of sequentially connected long short-term memory neural network units. The input of each layer of long short-term memory neural network unit is the sum of the input signal of the previous layer of long short-term memory neural network unit and the residual. The residual is the result calculated and output by the previous layer of long short-term memory neural network unit. The residual long short-term memory neural network model is obtained by training and testing on a dataset, which is obtained by augmenting the original dataset. The output unit is used to output the judgment result.
4. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the buckle engagement recognition method as described in any one of claims 1 to 2.
5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the buckle engagement recognition method as described in any one of claims 1 to 2.
