A model training method, a joint safety identification method of a humanoid robot and related equipment

Abstract

The application discloses a model training method, a joint safety identification method of a humanoid robot and related equipment. The method trains an unsupervised reward model through a first-stage training set, trains a difficult example classification model through a second-stage training set, fine-tunes the difficult example classification model through a key data set, obtains a sparse key degree prediction model, realizes the construction of a joint safety identification model of a humanoid robot based on a three-stage training framework, can solve the data sparsity and imbalance problem, realizes high-precision joint fault probability prediction, strengthens the classification ability of the model, identifies the current sensor state data of the humanoid robot through the joint safety identification model after three-stage training, outputs the corresponding joint safety identification result, and improves the joint safety identification efficiency and accuracy of the humanoid robot. The application can be widely applied in the technical field of robots.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a model training method, a joint safety recognition method for humanoid robots, and related equipment. Background Technology

[0002] With the widespread application of humanoid robots in various fields, the safety of their joint operation is of paramount importance. Joint failures (such as jamming or wear) can lead to serious accidents. However, predicting safety-critical events faces significant technical challenges: First, failure events occur very infrequently, resulting in a severe imbalance between positive and negative samples in the training data. Traditional models are prone to majority class bias, neglecting crucial information. Second, joint sensor data has high-dimensional and nonlinear characteristics, making it difficult for conventional feature extraction methods to capture subtle signs of failure. Third, application scenarios require predictive models to possess both high accuracy and low latency.

[0003] In related technologies, methods based on fixed thresholds struggle to adapt to complex dynamic working conditions, resulting in insufficient prediction accuracy. While deep learning-based methods can handle high-dimensional data, they are prone to overfitting and exhibit poor generalization ability on sparse and imbalanced datasets, leading to persistently high false negative rates for critical events. Therefore, existing technologies cannot effectively meet the urgent need for humanoid robots to perform high-precision, real-time predictions of joint safety status.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0005] The main objective of this application is to propose a model training method, a joint safety recognition method for humanoid robots, and related equipment.

[0006] To achieve the above objectives, one aspect of this application proposes a model training method, which includes the following steps:

[0007] Obtain the first-stage training set; the first-stage training set includes an initial positive sample set and an initial negative sample set; the initial positive sample set includes sensor state data of multiple humanoid robots when joint safety events occur; the initial negative sample set includes sensor state data of multiple humanoid robots when no joint safety events occur;

[0008] Based on the training set from the first stage, an unsupervised reward model is trained using the ranking loss function.

[0009] The unsupervised reward model is trained, and multiple difficult samples are selected from the initial negative sample set. Based on each difficult sample and the initial positive sample set, a second-stage training set is constructed.

[0010] A second neural network model is constructed. Using the second-stage training set and with classification loss as the objective function, the second neural network model is trained to obtain the corresponding hard example classification model.

[0011] Based on the aforementioned difficult example classification model, samples that were misclassified are selected from the negative samples in the second-stage training set and combined with the positive samples in the second-stage training set to form a key dataset.

[0012] Based on dense reinforcement learning, the difficult example classification model is fine-tuned using the key dataset to obtain the corresponding sparse keyness prediction model.

[0013] In some embodiments, the step of selecting multiple difficult samples from the initial negative sample set using the trained unsupervised reward model, and constructing a second-stage training set based on each of the difficult samples and the initial positive sample set, includes:

[0014] The initial negative sample set is input into the trained unsupervised reward model, each negative sample in the initial negative sample set is scored, and the feature score corresponding to each negative sample is output.

[0015] Multiple negative samples with feature scores higher than a preset score threshold are selected from the initial negative sample set to construct a false positive sample set;

[0016] The false positive sample set is used as the second-stage negative sample set, and the initial positive sample set is used as the second-stage positive sample set. The second-stage training set is constructed based on the second-stage negative sample set and the second-stage positive sample set.

[0017] In some embodiments, constructing the second neural network model involves training the second neural network model using the second-stage training set and employing classification loss as the objective function to obtain a corresponding hard-example classification model, including:

[0018] Construct the second neural network model, which is a supervised classification model based on an improved dual-branch network structure, the improved dual-branch network structure including an upper branch and a lower branch;

[0019] The second-stage training set is input into the improved dual-branch network structure, wherein the positive sample set of the second stage is processed through the upper branch, and the negative sample set of the second stage is processed through the lower branch;

[0020] Positive sample features are extracted through the concatenated structure of the upper branch convolutional neural network and the long short-term memory network, and negative sample features are extracted through the lower branch convolutional neural network.

[0021] The feature fusion classification layer of the second neural network model concatenates the positive sample features output by the upper branch and the negative sample features output by the lower branch to output the classification probability vector corresponding to each second-stage sample.

[0022] The classification loss is calculated based on the classification probability vector and the true label of each sample in the second stage. The classification loss includes cross-entropy loss and focus loss.

[0023] Based on the classification loss corresponding to each second-stage sample, the model parameters of the second neural network model are optimized through backpropagation to obtain the hard-case classification model.

[0024] In some embodiments, the step of selecting misclassified samples from the negative samples of the second-stage training set based on the difficult example classification model and combining them with the positive samples of the second-stage training set to form a key dataset includes:

[0025] The second-stage negative sample set is input into the hard example classification model, and the target classification category corresponding to each second-stage negative sample is output.

[0026] One or more negative samples of the second stage that are positive samples in the target classification category are selected from the negative samples of the second stage, and combined with the positive sample set of the second stage to form the key dataset.

[0027] To achieve the above objectives, another aspect of this application proposes a joint safety recognition method for a humanoid robot, the method comprising the following steps:

[0028] Acquire the current sensor status data of the humanoid robot;

[0029] The current sensor state data is input into the joint safety recognition model, and the corresponding joint safety recognition result is output. The joint safety recognition model is trained using the model training method described above. The joint safety recognition result includes the target joint safety category and category prediction probability corresponding to each joint in the humanoid robot.

[0030] In some embodiments, the method further includes:

[0031] Obtain the joint simulation model corresponding to the humanoid robot;

[0032] Based on the joint safety detection results and pre-configured annotation rules, the joint simulation model is visually annotated.

[0033] In some embodiments, the step of visually annotating the joint simulation model based on the joint safety detection results and pre-configured annotation rules includes:

[0034] Obtain the annotation rules, which include multiple preset joint safety categories and the corresponding safety level mapping relationship and visual perception feature mapping relationship for each joint safety category;

[0035] Based on the target joint safety category corresponding to each joint and the predicted probability of the category, the target joint safety level corresponding to each joint is determined from the safety level mapping relationship;

[0036] Based on the target joint safety level corresponding to each joint, determine the target visual perception features that match the target joint safety level from the visual perception feature mapping relationship;

[0037] Determine the joint region of each joint on the joint simulation model, and associate and bind the target visual perception features with the joint region;

[0038] The joint simulation model with associated features is dynamically rendered and displayed on the human-computer interaction interface.

[0039] To achieve the above objectives, another aspect of this application provides a joint safety recognition device for a humanoid robot, the device comprising:

[0040] The first module is used to acquire the current sensor status data of the humanoid robot;

[0041] The second module is used to input the current sensor state data into the joint safety recognition model and output the corresponding joint safety recognition result; the joint safety recognition model is trained by the model training method described above; the joint safety recognition result includes the target joint safety category and category prediction probability corresponding to each joint in the humanoid robot.

[0042] To achieve the above objectives, another aspect of this application proposes an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned model training method or the joint safety recognition method for humanoid robots.

[0043] To achieve the above objectives, another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned model training method or the joint safety recognition method for humanoid robots.

[0044] The embodiments of this application include at least the following beneficial effects: This application provides a model training method, a joint safety recognition method for humanoid robots, and related equipment. This scheme trains an unsupervised reward model using a first-stage training set, trains a hard example classification model using a second-stage training set, and fine-tunes the hard example classification model using a key dataset to obtain a sparse keyness prediction model. This achieves the construction of a joint safety recognition model for humanoid robots based on a three-stage training framework, which can solve the problems of data sparsity and imbalance, achieve high-precision joint failure probability prediction, enhance the model's classification ability, and identify the current sensor state data of the humanoid robot using the joint safety recognition model trained in the three stages, outputting the corresponding joint safety recognition results, thereby improving the efficiency and accuracy of joint safety recognition for humanoid robots. Attached Figure Description

[0045] Figure 1 This is a flowchart of the model training method provided in the embodiments of this application;

[0046] Figure 2 This is a schematic diagram of the structure of the second neural network model in the embodiments of this application;

[0047] Figure 3 This is a flowchart of the joint safety recognition method for a humanoid robot provided in an embodiment of this application;

[0048] Figure 4 This is a schematic diagram of the joint safety recognition device for a humanoid robot provided in an embodiment of this application;

[0049] Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0051] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0052] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0054] The model training method or the joint safety recognition method for humanoid robots provided in this application relates to the field of information technology. The model training method or the joint safety recognition method for humanoid robots provided in this application can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It can also be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the model training method or the joint safety recognition method for humanoid robots, but is not limited to the above forms.

[0055] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0056] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0057] Figure 1 This is an optional flowchart of the model training method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S106.

[0058] Step S101: Obtain the first-stage training set; the first-stage training set includes an initial positive sample set and an initial negative sample set; the initial positive sample set includes sensor state data of multiple humanoid robots when joint safety events occur; the initial negative sample set includes sensor state data of multiple humanoid robots when no joint safety events occur.

[0059] Step S102: Based on the first-stage training set, train an unsupervised reward model using the ranking loss function;

[0060] Step S103: Using the trained unsupervised reward model, select multiple difficult samples from the initial negative sample set, and construct the second-stage training set based on each difficult sample and the initial positive sample set;

[0061] Step S104: Construct a second neural network model. Using the second-stage training set and the classification loss as the objective function, train the second neural network model to obtain the corresponding hard example classification model.

[0062] Step S105: Based on the difficult example classification model, the misclassified samples are selected from the negative samples in the second stage training set and combined with the positive samples in the second stage training set to form the key dataset.

[0063] Step S106: Based on dense reinforcement learning, the difficult example classification model is fine-tuned using a key dataset to obtain the corresponding sparse keyness prediction model.

[0064] Steps S101 to S106 as shown in the embodiments of this application construct a joint safety recognition model for humanoid robots through a three-stage training framework, enhance the model's classification ability, improve the model's accuracy in recognizing joint faults, and effectively solve the problems of data sparsity and imbalance.

[0065] In some embodiments, the first-stage training set includes an initial positive sample set P1 composed of sensor state data Xi when a joint safety event occurs in the humanoid robot and an initial negative sample set N1 composed of sensor state data Xj when no joint safety event occurs. The sensors mounted on the humanoid robot include joint angle sensors, torque sensors, temperature sensors, and vibration sensors, etc. Joint safety events may include, but are not limited to, joint jamming, abnormal wear, overheating failure, and abnormal torque fluctuations.

[0066] Optionally, sensor status data when joint safety events such as joint jamming or abnormal wear occur can be collected as initial positive samples. The sensor status data includes:

[0067] 1) Joint angle sensor data: sampling frequency 100Hz, recording joint position deviation;

[0068] 2) Torque sensor data: Sampling frequency 500Hz, recording abnormal fluctuations in drive torque;

[0069] 3) Temperature sensor data: Sampling frequency 20Hz, recording data when the bearing temperature exceeds the threshold (e.g., 80℃);

[0070] 4) Vibration sensor data: Sampling frequency 1000Hz, record abnormal vibration spectrum data.

[0071] Sensor status data collected during normal joint operation are used as negative samples, with the sample size being 1000 times that of the positive samples (initial sparsity IR = 1000).

[0072] Using the initial positive sample set P1 and the initial negative sample set N1 as input data, the first stage of unsupervised reward model training based on ranking loss is performed. The ranking loss function adopts the contrastive loss function, which is as follows:

[0073] L rank =max(0,margin+f(xn)-f(xp));

[0074] Where f(xp) is the positive sample feature output, representing the feature score given by the model to the positive sample, f(xn) is the negative sample feature output, representing the feature score given by the model to the negative sample, and margin is a preset safety boundary threshold (e.g., 0.5), representing the expected minimum difference in feature scores between the positive and negative samples, that is, the feature score of the positive sample must be at least higher than the feature score of the negative sample by margin (e.g., 0.5).

[0075] Optionally, the unsupervised reward model can be trained using the first-stage training set, as follows:

[0076] Sample pair construction: For each positive sample xp, select 3 negative samples xn from the initial negative sample set N1 according to the Euclidean distance principle, and construct positive and negative sample pairs (xp, xn) respectively.

[0077] Reward model structure: A multilayer perceptron (MLP) is used (structure: a 3-layer MLP with hidden layer dimensions of 256 and 128 respectively, ReLU activation function, and a 1-dimensional scalar output layer). The input is a concatenated feature of (xp, xn), and the output is a scalar value representing the similarity of sample pairs.

[0078] Reward function design: R = 1 - ||f(xp) - f(xn)||2, where f is the MLP feature extraction function, the larger R is, the more similar the sample pairs are, and ||……||2 is the L2 norm (L2 Norm), i.e. Euclidean distance;

[0079] Iterative screening: Set the reward threshold Rth = 0.7. When R < Rth, xn is determined to be a simple negative sample and is removed. Iterative training continues until IR ≤ 50 (e.g., after 5 iterations, IR drops from 1000 to 35).

[0080] In some embodiments, step S103 may include, but is not limited to, steps S201 to S203.

[0081] Step S201: Input the initial negative sample set into the trained unsupervised reward model, score each negative sample in the initial negative sample set, and output the feature score corresponding to each negative sample.

[0082] Step S202: Select multiple negative samples with feature scores higher than a preset score threshold from the initial negative sample set to construct a false positive sample set;

[0083] Step S203: Use the false positive sample set as the second-stage negative sample set, use the initial positive sample set as the second-stage positive sample set, and construct the second-stage training set based on the second-stage negative sample set and the second-stage positive sample set.

[0084] In some embodiments, the unsupervised reward model trained is used to score all negative samples in the initial negative sample set N1, output the feature score corresponding to each negative sample, and identify multiple negative samples with feature scores higher than a preset score threshold as false positive samples, and construct the corresponding false positive sample set. The false positive sample set is used as the second-stage negative sample set N2, and the initial positive sample set P1 is directly used as the second-stage positive sample set P2. The second-stage negative sample set N2 and the second-stage positive sample set P2 constitute the second-stage training set.

[0085] In some embodiments, step S104 may include, but is not limited to, steps S301 to S306.

[0086] Step S301: Construct a second neural network model. The second neural network model is a supervised classification model based on an improved dual-branch network structure, which includes an upper branch and a lower branch.

[0087] Step S302: Input the second-stage training set into the improved dual-branch network structure, wherein the positive sample set of the second stage is processed through the upper branch and the negative sample set of the second stage is processed through the lower branch;

[0088] Step S303: Extract positive sample features through the concatenated structure of the upper branch convolutional neural network and the long short-term memory network, and extract negative sample features through the lower branch convolutional neural network.

[0089] Step S304: Through the feature fusion classification layer of the second neural network model, the positive sample features output by the upper branch and the negative sample features output by the lower branch are concatenated to output the classification probability vector corresponding to each second-stage sample.

[0090] Step S305: Calculate the classification loss based on the classification probability vector and the true label of each second-stage sample. The classification loss includes cross-entropy loss and focus loss.

[0091] Step S306: Based on the classification loss corresponding to each second-stage sample, optimize the model parameters of the second neural network model through backpropagation to obtain the hard-case classification model.

[0092] In some embodiments, the second neural network model is trained using the second-stage negative sample set N2 and the second-stage positive sample set P2 as input data for the second stage to obtain a difficult example classification model.

[0093] Specifically, refer to Figure 2 The second neural network model includes an input layer, an upper branch, a lower branch, and a feature fusion classification layer. The upper branch employs a balanced sampling strategy, extracting positive sample features through a concatenated structure of a convolutional neural network and a long short-term memory network. The lower branch employs a uniform sampling strategy, extracting negative sample features through a convolutional neural network. The feature fusion classification layer concatenates the positive sample features output from the upper branch and the negative sample features output from the lower branch to obtain a feature vector fused from the positive and negative sample features. Based on the feature vector, the regularized linear classifier in the feature fusion classification layer uses the Softmax activation function to output the corresponding classification probability vector. Based on the classification probability vector, the category with the highest probability is determined, and this category is identified as the sample prediction label.

[0094] Based on the predicted and true labels of the samples, the focus loss and cross-entropy loss are calculated using the classification loss function. Through backpropagation, the model parameters are optimized based on the focus loss and cross-entropy loss. The classification loss function includes the focus loss function and the cross-entropy loss function, which are detailed below:

[0095] L = La(p,ya) + Lb(p,yb);

[0096] Where L is the classification loss, La is the focus loss function, Lb is the cross-entropy loss function, p is the predicted label of the sample, ya is the true label of the sample in the task corresponding to the focus loss, and yb is the true label of the sample in the task corresponding to the cross-entropy loss.

[0097] The second neural network model is trained iteratively until the classification loss L is less than or equal to a preset threshold (e.g., 0.3).

[0098] In some embodiments, exemplarily, the upper branch employs a balanced sampling approach, extracting angle-torque joint features through CNN+LSTM, focusing on fault features dominated by positive samples (such as sudden torque changes and angle deviation trends). The CNN uses two convolutional layers (3×3 kernels, stride 1), and the LSTM has one layer (64 hidden dimensions). DQN parameters: empirical replay pool size is 10000, learning rate is 0.001, and the target network is updated every 100 steps. The lower branch employs uniform sampling, extracting temperature-vibration spectrum features through CNN, focusing on normal features dominated by negative samples (such as stable temperature and regular vibration spectrum).

[0099] In step S105 of some embodiments, the second-stage negative sample set is input into the hard example classification model, and the target classification category corresponding to each second-stage negative sample is output; then, one or more second-stage negative samples with the target classification category as positive samples are selected from the second-stage negative samples and combined with the second-stage positive sample set to form a key dataset.

[0100] Optionally, the second-stage negative sample set N2 is input into the hard example classification model, and the classification probability of each second-stage negative sample is output. The class with the highest probability is taken as the target classification class. Then, one or more second-stage negative samples with the target classification class as positive samples are selected from the second-stage negative samples to construct the third-stage negative sample set N3. The second-stage positive sample set P2 is taken as the third-stage positive sample set P3. The third-stage positive sample set P3 and the third-stage negative sample set N3 constitute the third-stage training set, which is the key dataset mentioned above.

[0101] In step S106 of some embodiments, optionally, in the third-stage fine-tuning, the hard example classification model is converted into an offline deep Q network, with key data (s,a,r,s') as input, where s is the current state of the joint, a is the action, r is the reward (sample r=1 in P3, sample r=0 in N3), and s' is the next state. The regularized linear classifier head and softmax layer parameters of the hard example classification model are updated through the Q loss function. For example, the DQN algorithm is used to construct an experience replay pool with the key dataset, and the Q network parameters are iteratively updated until the Q loss is less than or equal to a preset threshold (e.g., 0.1), thus obtaining the final sparse keyness prediction model.

[0102] Optionally, the key dataset can be used as the training data for the third stage to fine-tune the difficult example classification model based on dense reinforcement learning, thereby obtaining the corresponding sparse keyness prediction model. The specific implementation method is as follows:

[0103] Figure 3 This is an optional flowchart of the joint safety recognition method for humanoid robots provided in the embodiments of this application. Figure 3 The method may include, but is not limited to, steps A101 to A102.

[0104] Step A101: Obtain the current sensor status data of the humanoid robot;

[0105] Step A102: Input the current sensor status data into the joint safety recognition model and output the corresponding joint safety recognition result; the joint safety recognition model is trained using the model training method described above; the joint safety recognition result includes the target joint safety category and category prediction probability corresponding to each joint in the humanoid robot.

[0106] In some embodiments, joint safety categories may include, but are not limited to, joint jamming, abnormal wear, overheating failure, abnormal torque fluctuations, and normal joint operation.

[0107] In some embodiments, the joint safety identification method described above may further include steps A201 to A202.

[0108] Step A201: Obtain the joint simulation model corresponding to the humanoid robot;

[0109] Step A202: Visualize and annotate the joint simulation model based on the joint safety test results and pre-configured annotation rules.

[0110] In some embodiments, the joint simulation model is optionally a 3D model based on multibody dynamics, including the structure and spatial position of humanoid robot joints 1 to N.

[0111] In some embodiments, step A202 may include, but is not limited to, steps A301 to A305.

[0112] Step A301: Obtain annotation rules. The annotation rules include multiple preset joint safety categories and the mapping relationship between the safety level and visual perception features corresponding to each joint safety category.

[0113] Step A302: Determine the target joint safety level corresponding to each joint from the safety level mapping relationship based on the target joint safety category and category prediction probability corresponding to each joint.

[0114] Step A303: Based on the target joint safety level corresponding to each joint, determine the target visual perception features that match the target joint safety level from the visual perception feature mapping relationship;

[0115] Step A304: Determine the joint region of each joint on the joint simulation model, and associate and bind the target visual perception features with the joint region;

[0116] Step A305: Dynamically render the joint simulation model after associating features and display it in the human-computer interaction interface.

[0117] In some embodiments, visual perception features include mapped color, brightness, and flicker frequency. By using annotation rules, the target visual perception features that match the target joint safety level of each joint are determined. Based on the target visual perception features of each joint, the joint regions associated with each joint on the joint simulation model are visualized and dynamically rendered.

[0118] For example, assuming that the target visual perception features of any joint i among joints 1 to N include red, brightness X and flashing frequency Y, the joint region i associated with joint i is located on the joint simulation model. According to the target visual perception features, the joint region i is dynamically rendered so that the joint region i is red, the brightness of which is X and the flashing frequency is Y, and so on.

[0119] Please see Figure 4 This application also provides a joint safety recognition device for a humanoid robot, which can implement the above-described method. The device includes:

[0120] The first module is used to acquire the current sensor status data of the humanoid robot;

[0121] The second module is used to input the current sensor status data into the joint safety recognition model and output the corresponding joint safety recognition result. The joint safety recognition model is trained using the model training method described above. The joint safety recognition result includes the target joint safety category and category prediction probability corresponding to each joint in the humanoid robot.

[0122] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0123] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0124] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0125] Please see Figure 5 , Figure 5 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0126] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0127] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the methods described in the embodiments of this application.

[0128] The input / output interface 903 is used to implement information input and output;

[0129] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0130] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0131] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0132] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0133] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0134] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0135] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0136] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0137] The humanoid robot joint safety recognition method, device, electronic device, storage medium, and program product provided in this application embodiment train an unsupervised reward model through a first-stage training set, train a hard example classification model through a second-stage training set, and fine-tune the hard example classification model through a key dataset to obtain a sparse keyness prediction model. This achieves the construction of a humanoid robot joint safety recognition model based on a three-stage training framework, which can solve the problems of data sparsity and imbalance, achieve high-precision joint failure probability prediction, enhance the model's classification ability, and identify the current sensor state data of the humanoid robot through the joint safety recognition model trained in the three stages, outputting the corresponding joint safety recognition results, thereby improving the efficiency and accuracy of joint safety recognition of the humanoid robot.

[0138] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0139] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0140] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0141] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0142] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0143] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0144] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0145] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0147] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A model training method, characterized in that, The method includes the following steps: Obtain the first-stage training set; the first-stage training set includes an initial positive sample set and an initial negative sample set; the initial positive sample set includes sensor state data of multiple humanoid robots when joint safety events occur; the initial negative sample set includes sensor state data of multiple humanoid robots when no joint safety events occur; Based on the first-stage training set, an unsupervised reward model is trained using a ranking loss function, specifically a contrastive loss function. The unsupervised reward model is based on a three-layer multilayer perceptron architecture, with hidden layer dimensions of 256 and 128 respectively, ReLU activation functions, and a one-dimensional scalar output layer. During training, for each positive sample, three negative samples are selected according to the Euclidean distance principle to construct a positive-negative sample pair. The concatenated features of these positive-negative sample pairs are then input into the unsupervised reward model. The reward function is designed as follows: ,in, This represents the feature output of the unsupervised reward model after extracting positive samples. This represents the feature output of the unsupervised reward model after extracting negative samples, with a reward threshold Rth=0.7 set. Simple negative samples are iteratively filtered and removed until the sample sparsity is reached. 50; The initial negative sample set is input into the trained unsupervised reward model, and each negative sample in the initial negative sample set is scored, outputting the feature score corresponding to each negative sample. Multiple negative samples with feature scores higher than a preset score threshold are selected from the initial negative sample set to construct a false positive sample set. The false positive sample set is used as the second-stage negative sample set, and the initial positive sample set is used as the second-stage positive sample set. Based on the second-stage negative sample set and the second-stage positive sample set, a second-stage training set is constructed. A second neural network model is constructed, which is a supervised classification model based on an improved dual-branch network structure, comprising an upper branch and a lower branch. The second-stage training set is input into the improved dual-branch network structure, wherein the upper branch processes the positive sample set of the second stage, and the lower branch processes the negative sample set of the second stage. Positive sample features are extracted through a concatenated structure of a convolutional neural network and a long short-term memory network in the upper branch. The convolutional neural network in the upper branch comprises two layers of convolutions with 3×3 kernels and a stride of 1, and the long short-term memory network comprises one layer with a hidden dimension of 64. DQN training is performed, with an experience replay pool size of 10000, a learning rate of 0.001, and a replay interval of 100 steps. The new target network extracts negative sample features through the convolutional neural network of the lower branch; through the feature fusion classification layer of the second neural network model, it concatenates the positive sample features output from the upper branch and the negative sample features output from the lower branch to output the classification probability vector corresponding to each second-stage sample; the upper branch handles the balance problem through a class-balanced sampling strategy and extracts angle-torque joint features during training; the lower branch handles the balance problem through a uniform sampling strategy and extracts temperature-vibration spectrum features during training; based on the classification probability vector corresponding to each second-stage sample and the true label of the sample, the classification loss is calculated, which includes cross-entropy loss and focus loss, and the classification loss function formula is: Where La represents the focus loss function, Lb represents the cross-entropy loss function, p represents the predicted label of the sample, ya represents the true label of the sample in the task corresponding to the focus loss, and yb represents the true label of the sample in the task corresponding to the cross-entropy loss; based on the classification loss corresponding to each sample in the second stage, the model parameters of the second neural network model are optimized through backpropagation, and iterative training is performed until the classification loss L is reached. Stop at 0.3, obtaining the difficult example classification model; The second-stage negative sample set is input into the hard example classification model, and the target classification category corresponding to each second-stage negative sample is output; one or more second-stage negative samples whose target classification category is positive are selected from the second-stage negative samples and combined with the second-stage positive sample set to form a key dataset; Based on dense reinforcement learning, the difficult example classification model is fine-tuned using the key dataset. The difficult example classification model is converted into an offline deep Q-network for fine-tuning. The training objective of fine-tuning is to update the regular linear classifier head and softmax layer parameters of the model to obtain the corresponding sparse keyness prediction model.

2. A method for joint safety recognition of a humanoid robot, characterized in that, The method includes the following steps: Acquire the current sensor status data of the humanoid robot; the sensor status data includes joint angle sensor data, torque sensor data, temperature sensor data, and vibration sensor data, wherein the joint angle sensor data is sampled at a frequency of 100Hz and records joint position deviation, the torque sensor data is sampled at a frequency of 500Hz and records abnormal fluctuations in drive torque, the temperature sensor data is sampled at a frequency of 20Hz and records data on bearing temperature exceeding the threshold, and the vibration sensor data is sampled at a frequency of 1000Hz and records abnormal vibration spectrum data; The current sensor status data is input into the joint safety recognition model, and the corresponding joint safety recognition result is output. The joint safety recognition model is trained by the model training method described in claim 1. The joint safety recognition result includes the target joint safety category and category prediction probability corresponding to each joint in the humanoid robot. The joint safety category includes joint jamming, abnormal wear, overheating failure, abnormal torque fluctuation, and normal joint operation. Obtain the joint simulation model corresponding to the humanoid robot; the joint simulation model is a 3D model constructed based on multibody dynamics, which includes the structure and spatial position of each joint of the humanoid robot; Based on the joint safety detection results and pre-configured annotation rules, the joint simulation model is visually annotated. The specific visual perception features of the visual annotation include mapped color, brightness, and flicker frequency. The visual annotation includes locating the joint regions corresponding to each joint on the joint simulation model, associating and binding the target visual perception features of each joint with the corresponding joint regions, and dynamically rendering the joint simulation model after associating the features.

3. The method according to claim 2, characterized in that, The step of visually annotating the joint simulation model based on the joint safety detection results and pre-configured annotation rules includes: Obtain the annotation rules, which include multiple preset joint safety categories and the corresponding safety level mapping relationship and visual perception feature mapping relationship for each joint safety category; Based on the target joint safety category corresponding to each joint and the predicted probability of the category, the target joint safety level corresponding to each joint is determined from the safety level mapping relationship; Based on the target joint safety level corresponding to each joint, determine the target visual perception features that match the target joint safety level from the visual perception feature mapping relationship; Determine the joint region of each joint on the joint simulation model, and associate and bind the target visual perception features with the joint region; The joint simulation model with associated features is dynamically rendered and displayed on the human-computer interaction interface.

4. A joint safety recognition device for a humanoid robot, characterized in that, The joint safety recognition device of the humanoid robot is used to perform the joint safety recognition method of the humanoid robot as described in claim 2, the device comprising: The first module is used to acquire the current sensor status data of the humanoid robot; The second module is used to input the current sensor state data into the joint safety recognition model and output the corresponding joint safety recognition result; the joint safety recognition model is trained by the model training method as described in claim 1; the joint safety recognition result includes the target joint safety category and category prediction probability corresponding to each joint in the humanoid robot.

5. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 3.

6. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 3.

Images