Model training method, joint safety identification method of humanoid robot and related equipment

Abstract

The invention discloses a model training method, a humanoid robot joint safety identification method and related equipment, and the method comprises the steps: training an unsupervised reward model through a first-stage training set, training a difficult case classification model through a second-stage training set, and carrying out the fine adjustment of the difficult case classification model through a key data set, the joint safety identification model of the humanoid robot is constructed based on a three-stage training framework, the problems of data sparsity and imbalance can be solved, high-precision joint fault probability prediction is realized, the classification capability of the model is enhanced, and the joint safety identification efficiency is improved. The current sensor state data of the humanoid robot is identified through the joint safety identification model after three-stage training, a corresponding joint safety identification result is output, and the joint safety identification efficiency and accuracy of the humanoid robot are improved. The robot can be widely applied to the technical field of robots.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, and in particular to a model training method, a joint safety identification method of a humanoid robot, and related equipment. BACKGROUND

[0002] With the wide application of humanoid robots in various fields, the joint operation safety of the humanoid robots is crucial. Joint failures (such as jamming and wear) can cause serious accidents, however, the prediction of safety critical events faces significant technical challenges: first, the frequency of failure events is extremely low, resulting in extremely unbalanced positive and negative samples of training data, and traditional models are prone to majority class bias, ignoring critical information; second, joint sensor data has high-dimensional and nonlinear characteristics, and conventional feature extraction methods are difficult to capture weak failure signs; third, the application scenario requires the prediction model to have both high precision and low delay.

[0003] In related technologies, the method based on a fixed threshold is difficult to adapt to complex dynamic working conditions, and the prediction accuracy is insufficient; while the method based on deep learning can process high-dimensional data, but is prone to overfitting and has poor generalization ability on sparse and unbalanced data sets, resulting in a high false negative rate for critical events. Therefore, the existing technology cannot effectively meet the urgent needs of humanoid robots for high-precision and real-time prediction of joint safety states.

[0004] In summary, the technical problems in related technologies need to be improved. SUMMARY

[0005] The main purpose of the embodiments of the present application is to provide a model training method, a joint safety identification method of a humanoid robot, and related equipment.

[0006] To achieve the above-mentioned purpose, one aspect of the embodiments of the present application provides a model training method, which comprises the following steps:

[0007] obtaining a first stage training set; the first stage training set comprises an initial positive sample set and an initial negative sample set; the initial positive sample set comprises sensor state data of multiple humanoid robots when a joint safety event occurs; the initial negative sample set comprises sensor state data of multiple humanoid robots when a joint safety event does not occur;

[0008] training an unsupervised reward model based on the first stage training set using a ranking loss function;

[0009] filtering a plurality of difficult samples from the initial negative sample set through the trained unsupervised reward model, and constructing a second stage training set according to each of the difficult samples and the initial positive sample set;

[0010] constructing a second neural network model, training the second neural network model by the second stage training set, taking a classification loss as a target function, and obtaining a corresponding difficult example classification model;

[0011] filtering misjudged samples from the negative samples in the second stage training set based on the difficult example classification model, and combining the positive samples in the second stage training set into a key data set;

[0012] based on dense reinforcement learning, fine-tuning the difficult example classification model through the key data set, and obtaining a corresponding sparse key degree prediction model.

[0013] In some embodiments, the trained unsupervised reward model is used to filter difficult samples from the initial negative sample set, and a second stage training set is constructed according to each difficult sample and the initial positive sample set, including:

[0014] inputting the initial negative sample set into the trained unsupervised reward model, scoring each negative sample in the initial negative sample set, and outputting the feature score corresponding to each negative sample;

[0015] filtering a plurality of negative samples with a feature score higher than a preset score threshold from the initial negative sample set to construct a false positive sample set;

[0016] The false positive sample set is used as a second stage negative sample set, and the initial positive sample set is used as a second stage positive sample set. The second stage negative sample set and the second stage positive sample set are used to construct the second stage training set.

[0017] In some embodiments, the second neural network model is constructed, the second stage training set is used to train the second neural network model, a classification loss is taken as a target function, and a corresponding difficult example classification model is obtained, including:

[0018] The second neural network model is constructed, and the second neural network model is a supervised classification model based on an improved double-branch network structure. The improved double-branch network structure includes an upper branch and a lower branch.

[0019] The second stage training set is input into the improved double-branch network structure. The second stage positive sample set is processed through the upper branch, and the second stage negative sample set is processed through the lower branch.

[0020] The upper branch is extracted through the convolutional neural network and the long short-term memory network in series, and the negative sample feature is extracted through the convolutional neural network of the lower branch.

[0021] The feature fusion classification layer of the second neural network model is used for performing feature splicing on the positive sample features output by the upper branch and the negative sample features output by the lower branch, and outputting a classification probability vector corresponding to each second-stage sample;

[0022] According to the classification probability vector corresponding to each second-stage sample and a sample true label, a classification loss is calculated, the classification loss including a cross-entropy loss and a focal loss;

[0023] According to the classification loss corresponding to each second-stage sample, model parameters of the second neural network model are optimized through back propagation, and the difficult example classification model is obtained.

[0024] In some embodiments, the difficult example classification model is used to screen misjudged samples from the negative samples of the second-stage training set, and the misjudged samples are combined with the positive samples in the second-stage training set to form a key data set, which includes:

[0025] The second-stage negative sample set is input into the difficult example classification model, and a target classification category corresponding to each second-stage negative sample is output.

[0026] One or more second-stage negative samples whose target classification category is a positive sample are screened from the second-stage negative samples, and the one or more second-stage negative samples are combined with the second-stage positive sample set to form the key data set.

[0027] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application proposes a joint safety identification method of a humanoid robot, which includes the following steps:

[0028] Obtaining current sensor state data of the humanoid robot;

[0029] Inputting the current sensor state data into a joint safety identification model to output a corresponding joint safety identification result; the joint safety identification model is trained by the model training method as described above; the joint safety identification result includes a target joint safety category and a category prediction probability corresponding to each joint of the humanoid robot.

[0030] In some embodiments, the method further includes:

[0031] Obtaining a joint simulation model corresponding to the humanoid robot;

[0032] According to the joint safety detection result and a pre-configured labeling rule, the joint simulation model is visually labeled.

[0033] In some embodiments, the joint simulation model is visually labeled according to the joint safety detection result and the pre-configured labeling rule, which includes:

[0034] obtain the labeling rule, the labeling rule comprising a plurality of preset joint safety categories, and a safety level mapping relationship and a visual perception feature mapping relationship corresponding to each joint safety category;

[0035] determine a target joint safety level corresponding to each joint from the safety level mapping relationship according to the target joint safety category corresponding to each joint and the category prediction probability;

[0036] determine a target visual perception feature matching the target joint safety level from the visual perception feature mapping relationship according to the target joint safety level corresponding to each joint;

[0037] determine a joint region of each joint on the joint simulation model, and bind the target visual perception feature to the joint region;

[0038] perform dynamic rendering on the joint simulation model after the binding, and display the joint simulation model in a human-computer interaction interface.

[0039] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application proposes a joint safety identification device of a humanoid robot, the device comprising:

[0040] a first module configured to obtain current sensor state data of the humanoid robot;

[0041] a second module configured to input the current sensor state data into a joint safety identification model, and output a corresponding joint safety identification result; the joint safety identification model is trained by the model training method as described above; the joint safety identification result comprises a target joint safety category and a category prediction probability corresponding to each joint of the humanoid robot.

[0042] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application proposes an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the model training method or the joint safety identification method of the humanoid robot as described above when executing the computer program.

[0043] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application proposes a computer program product comprising a computer program, the computer program being executed by a processor to implement the model training method or the joint safety identification method of the humanoid robot as described above.

[0044] The embodiment of the present application at least has the following beneficial effects: the present application provides a model training method, a joint safety identification method of a humanoid robot and related equipment, the scheme trains an unsupervised reward model through a first-stage training set, trains a difficult example classification model through a second-stage training set, fine-tunes the difficult example classification model through a key data set, obtains a sparse key degree prediction model, realizes the construction of a joint safety identification model of a humanoid robot based on a three-stage training framework, can solve the problems of data sparsity and imbalance, realizes high-precision joint fault probability prediction, strengthens the classification ability of the model, and through the joint safety identification model after three-stage training, the current sensor state data of the humanoid robot is identified, the corresponding joint safety identification result is output, and the joint safety identification efficiency and accuracy of the humanoid robot are improved. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a flowchart of the model training method provided by the embodiment of the present application;

[0046] Figure 2 is a structural schematic diagram of the second neural network model in the embodiment of the present application;

[0047] Figure 3 is a flowchart of the joint safety identification method of the humanoid robot provided by the embodiment of the present application;

[0048] Figure 4 is a structural schematic diagram of the joint safety identification device of the humanoid robot provided by the embodiment of the present application;

[0049] Figure 5 is a hardware structural schematic diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the present application clearer, the following will further describe the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. When the following description relates to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementation described in the following exemplary embodiments does not represent all the implementations consistent with the embodiments of the present application, but is only an example of devices and methods consistent with some aspects of the embodiments of the present application as described in the appended claims.

[0051] It can be understood that the terms "first", "second" and the like used in the present application can be used herein to describe various concepts, but unless specifically stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiments of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon determining" or "in response to determining".

[0052] The terms "at least one", "multiple", "each", "any" and the like used in the present application include one, two or more than two, multiple includes two or more than two, each refers to each of the corresponding plurality, and any refers to any one of the plurality.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by those skilled in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0054] The model training method or joint safety identification method of the humanoid robot provided by the embodiments of the present application relates to the field of information technology. The model training method or joint safety identification method of the humanoid robot provided by the embodiments of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, and the like, but is not limited thereto; the server end can be configured as an independent physical server, can also be configured as a server cluster or a distributed system composed of multiple physical servers, can also be configured as a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms, and the server can also be a node server in a blockchain network; the software can be an application that implements the model training method or the joint safety identification method of the humanoid robot, and the like, but is not limited to the above forms.

[0055] The application is operable in a multitude of generic or specific computer system environments or configurations. Examples of well known computing systems, environments, and / or configurations that can be suitable for use with the application include personal computers, server computers, handheld or laptop devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. The application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0056] It should be noted that in each specific embodiment of the present application, when it is necessary to perform relevant processing according to user information, user behavior data, user historical data, and user location information, and other data related to the identity or characteristics of the user, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the separate permission or separate consent of the user will be obtained through a pop-up window or by jumping to a confirmation page, and after obtaining the separate permission or separate consent of the user, the necessary user-related data for enabling the embodiments of the present application to function normally will be obtained.

[0057] Figure 1 is an optional flowchart of the model training method provided by the embodiments of the present application, Figure 1 The method in can include but is not limited to including steps S101 to S106.

[0058] Step S101, obtaining a first-stage training set; the first-stage training set includes an initial positive sample set and an initial negative sample set; the initial positive sample set includes sensor state data when a joint safety event occurs for a plurality of humanoid robots; the initial negative sample set includes sensor state data when a joint safety event does not occur for the plurality of humanoid robots;

[0059] Step S102, training an unsupervised reward model based on the first-stage training set by using a ranking loss function;

[0060] Step S103, filtering a plurality of difficult samples from the initial negative sample set by using the trained unsupervised reward model, and constructing a second-stage training set according to each difficult sample and the initial positive sample set;

[0061] Step S104, a second neural network model is constructed, and the second neural network model is trained by taking a classification loss as a target function through a second-stage training set to obtain a corresponding difficult example classification model;

[0062] Step S105, based on the difficult example classification model, misjudged samples are screened out from negative samples in the second-stage training set, and the misjudged samples are combined with positive samples in the second-stage training set to form a key data set;

[0063] Step S106, based on dense reinforcement learning, the difficult example classification model is fine-tuned through the key data set to obtain a corresponding sparse key degree prediction model.

[0064] The steps S101 to S106 shown in the embodiments of the present application construct a joint safety recognition model of a humanoid robot through a three-stage training framework, strengthen the classification ability of the model, improve the recognition accuracy of the model for joint faults, and effectively solve the problems of data sparsity and imbalance.

[0065] In some embodiments, the first-stage training set includes an initial positive sample set P1 composed of sensor state data Xi when the humanoid robot has a joint safety event and an initial negative sample set N1 composed of sensor state data Xj when the humanoid robot does not have a joint safety event, the sensors carried on the humanoid robot include joint angle sensors, torque sensors, temperature sensors, vibration sensors, and the like, and the joint safety event can include but is not limited to joint jamming, abnormal wear, overheating failure, torque abnormal fluctuation, and the like.

[0066] Optionally, the sensor state data when the joint has a joint safety event such as joint jamming and abnormal wear is collected as the initial positive sample, and the sensor state data includes:

[0067] 1) Joint angle sensor data: sampling frequency 100Hz, record joint position deviation;

[0068] 2) Torque sensor data: sampling frequency 500Hz, record driving torque abnormal fluctuation;

[0069] 3) Temperature sensor data: sampling frequency 20Hz, record data when bearing temperature exceeds threshold value (such as 80℃);

[0070] 4) Vibration sensor data: sampling frequency 1000Hz, record abnormal vibration spectrum data.

[0071] The sensor state data when the joint is in normal operation is collected as the negative sample, and the sample amount is 1000 times (initial sparsity IR=1000) of the positive sample.

[0072] With the initial positive sample set P1 and the initial negative sample set N1 as input data, the first stage unsupervised reward model training based on the ranking loss is performed. The ranking loss function adopts a contrast loss function, and the contrast loss function is specifically as follows:

[0073] L rank = max(0, margin + f(xn) - f(xp));

[0074] Wherein, f(xp) is a positive sample feature output, indicating the feature score given by the model to the positive sample, f(xn) is a negative sample feature output, indicating the feature score given by the model to the negative sample, margin is a preset safety boundary threshold (such as 0.5), indicating the minimum difference between the feature scores of the positive sample and the negative sample, that is, the feature score of the positive sample is at least margin (such as 0.5) higher than the feature score of the negative sample.

[0075] Optionally, the unsupervised reward model is trained by the first stage training set, and the specific implementation manner is as follows:

[0076] Sample pair construction: for each positive sample xp, select 3 negative samples xn in the initial negative sample set N1 according to the principle of the nearest Euclidean distance, and construct a positive-negative sample pair (xp, xn).

[0077] Reward model structure: adopt a multi-layer perceptron (MLP-structure: adopt a 3-layer MLP, the hidden layer dimensions are 256 and 128 respectively, the activation function is ReLU, and the output layer is a 1-dimensional scalar"), the input is the spliced features of (xp, xn), and the output is a scalar value, indicating the similarity of the sample pair;

[0078] Reward function design: R = 1 - ||f(xp) - f(xn)||2, wherein f is an MLP feature extraction function, R is larger, indicating that the sample pair is more similar, and ||……||2 is an L2 norm (L2 Norm), that is, an Euclidean distance (Euclidean Distance);

[0079] Iterative screening: set a reward threshold Rth = 0.7, when R < Rth, determine xn as a simple negative sample and eliminate; iteratively train until IR ≤ 50 (such as after 5 iterations, IR decreases from 1000 to 35).

[0080] In some embodiments, step S103 can include, but is not limited to, steps S201 to S203.

[0081] Step S201, input the initial negative sample set to the trained unsupervised reward model, score each negative sample in the initial negative sample set, and output the feature score corresponding to each negative sample;

[0082] Step S202, screening a plurality of negative samples with feature scores higher than a preset score threshold from the initial negative sample set to construct a set of false positive samples;

[0083] Step S203, taking the set of false positive samples as a second-stage negative sample set, taking the initial positive sample set as a second-stage positive sample set, and constructing a second-stage training set according to the second-stage negative sample set and the second-stage positive sample set.

[0084] In some embodiments, the feature scores of all negative samples in the initial negative sample set N1 are output by the trained unsupervised reward model, the feature scores corresponding to each negative sample are output, a plurality of negative samples with feature scores higher than a preset score threshold are determined as false positive samples, and a corresponding set of false positive samples is constructed. The set of false positive samples is taken as the second-stage negative sample set N2, and the initial positive sample set P1 is directly taken as the second-stage positive sample set P2, and the second-stage negative sample set N2 and the second-stage positive sample set P2 form the second-stage training set.

[0085] In some embodiments, step S104 can include but is not limited to steps S301 to S306.

[0086] Step S301, constructing a second neural network model, the second neural network model being a supervised classification model based on an improved double-branch network structure, the improved double-branch network structure including an upper branch and a lower branch;

[0087] Step S302, inputting the second-stage training set into the improved double-branch network structure, wherein the second-stage positive sample set is processed through the upper branch, and the second-stage negative sample set is processed through the lower branch;

[0088] Step S303, extracting positive sample features through the series connection structure of the convolutional neural network and the long short-term memory network of the upper branch, and extracting negative sample features through the convolutional neural network of the lower branch;

[0089] Step S304, performing feature splicing on the positive sample features output by the upper branch and the negative sample features output by the lower branch through the feature fusion classification layer of the second neural network model, and outputting a classification probability vector corresponding to each second-stage sample;

[0090] Step S305, calculating a classification loss according to the classification probability vector corresponding to each second-stage sample and the sample true label, the classification loss including a cross-entropy loss and a focal loss;

[0091] Step S306, optimizing the model parameters of the second neural network model through back propagation according to the classification loss corresponding to each second-stage sample to obtain a difficult example classification model.

[0092] In some embodiments, the second neural network model is trained by taking the second stage negative sample set N2 and the second stage positive sample set P2 as input data of the second stage, and a difficult example classification model is obtained.

[0093] Specifically, referring to Figure 2 , the second neural network model includes an input layer, an upper branch, a lower branch, and a feature fusion classification layer. The upper branch adopts a class-balanced sampling strategy, extracts positive sample features through a series structure of a convolutional neural network and a long short-term memory network; the lower branch adopts a uniform sampling strategy, extracts negative sample features through a convolutional neural network; the feature fusion classification layer is used to perform feature splicing on the positive sample features output by the upper branch and the negative sample features output by the lower branch, to obtain a feature vector after fusion of the positive sample features and the negative sample features, and a regularized linear classifier in the feature fusion classification layer is used to output a corresponding classification probability vector according to the feature vector using a Softmax activation function, and a class with the maximum probability is determined as a sample predicted label according to the classification probability vector.

[0094] According to the sample predicted label and the sample real label, a focal loss and a cross-entropy loss are calculated using a classification loss function, and model parameters are optimized according to the focal loss and the cross-entropy loss through back propagation, wherein the classification loss function includes a focal loss function and a cross-entropy loss function, and the classification loss function is specifically as follows:

[0095] L = La(p, ya) + Lb(p, yb);

[0096] Wherein, L is the classification loss, La is the focal loss function, Lb is the cross-entropy loss function, p is the sample predicted label, ya is the sample real label corresponding to the task of the focal loss, and yb is the sample real label corresponding to the task of the cross-entropy loss.

[0097] The second neural network model is iteratively trained until the classification loss L is less than or equal to a preset threshold (such as 0.3).

[0098] In some embodiments, for example, the upper branch adopts class-balanced sampling, extracts angle-torque joint features through CNN+LSTM, and focuses on fault features (such as torque mutation and angle deviation trend) dominated by positive samples, wherein CNN adopts 2 layers of convolution (convolution kernel 3x3, step 1), and LSTM is 1 layer (hidden dimension 64); DQN parameters: experience replay pool size is 10000, learning rate is 0.001, and the target network is updated every 100 steps; the lower branch adopts uniform sampling, extracts temperature-vibration spectrum features through CNN, and focuses on normal features (such as temperature stability and vibration spectrum regularity) dominated by negative samples.

[0099] In step S105 of some embodiments, the second-stage negative sample set is input into the difficult example classification model, and the target classification category corresponding to each second-stage negative sample is output; then, one or more second-stage negative samples whose target classification category is a positive sample are screened out from the second-stage negative samples, and combined with the second-stage positive sample set to form a key data set.

[0100] Optionally, the second-stage negative sample set N2 is input into the difficult example classification model, and the classification probability of each second-stage negative sample is output, and the category with the highest probability is taken as the target classification category; then, one or more second-stage negative samples whose target classification category is a positive sample are screened out from the second-stage negative samples, a third-stage negative sample set N3 is constructed, the second-stage positive sample set P2 is taken as a third-stage positive sample set P3, and the third-stage positive sample set P3 and the third-stage negative sample set N3 form a third-stage training set, i.e., the key data set described above.

[0101] In step S106 of some embodiments, optionally, in the third-stage fine-tuning, the difficult example classification model is converted into an offline deep Q network, and the key data (s, a, r, s') is taken as input, where s is the current state of the joint, a is the action, r is the reward (r=1 in the sample P3 and r=0 in the sample N3), and s' is the next state; the regular linear classification head and the softmax layer parameters of the difficult example classification model are updated through a Q loss function, for example, using the DQN algorithm, an experience replay pool is constructed using the key data set, the Q network parameters are iteratively updated until the Q loss is less than or equal to a preset threshold (such as 0.1), and finally a sparse key degree prediction model is obtained.

[0102] Optionally, the key data set is taken as third-stage training data, and the difficult example classification model is fine-tuned based on dense reinforcement learning to obtain a corresponding sparse key degree prediction model, and the specific implementation manner is as follows:

[0103] Figure 3 is an optional flowchart of the joint safety identification method of the humanoid robot provided in the embodiments of the present application, Figure 3 The method in the above formula (1) can include but is not limited to including steps A101 to A102.

[0104] In step A101, current sensor state data of a humanoid robot is acquired.

[0105] In step A102, the current sensor state data is input into a joint safety identification model, and a corresponding joint safety identification result is output; the joint safety identification model is trained through the model training method described above; the joint safety identification result includes a target joint safety category and a category prediction probability corresponding to each joint in the humanoid robot.

[0106] In some embodiments, the joint safety categories can include, but are not limited to, joint jam, abnormal wear, overheating failure, abnormal torque fluctuation, and normal joint operation, etc.

[0107] In some embodiments, the joint safety identification method described above can further include steps A201 to A202.

[0108] Step A201, obtaining a joint simulation model corresponding to the humanoid robot;

[0109] Step A202, according to the joint safety detection result and the preconfigured labeling rule, visualizing labeling the joint simulation model.

[0110] In some embodiments, optionally, the joint simulation model is a 3D model constructed based on multi-body dynamics, including the structure and spatial position of joints 1 to N of the humanoid robot.

[0111] In some embodiments, step A202 can include, but is not limited to, steps A301 to A305.

[0112] Step A301, obtaining a labeling rule, the labeling rule including a plurality of preset joint safety categories and a safety level mapping relationship and a visual perception feature mapping relationship corresponding to each joint safety category;

[0113] Step A302, according to the target joint safety category and the category prediction probability corresponding to each joint, determining the target joint safety level corresponding to each joint from the safety level mapping relationship;

[0114] Step A303, according to the target joint safety level corresponding to each joint, determining the target visual perception feature matching the target joint safety level from the visual perception feature mapping relationship;

[0115] Step A304, determining the joint region of each joint on the joint simulation model, and associating and binding the target visual perception feature with the joint region;

[0116] Step A305, dynamically rendering the joint simulation model after the association feature and displaying it in the human-computer interaction interface.

[0117] In some embodiments, the visual perception features include mapping colors, brightness, and flicker frequencies, etc. Through the labeling rule, the target visual perception feature matching the target joint safety level of each joint is determined, and according to the target visual perception feature of each joint, the joint region associated with each joint on the joint simulation model is visualized and dynamically rendered.

[0118] Exemplarily, assuming that the target visual perception feature of any joint i in joints 1 to N includes red, brightness X, and flicker frequency Y, the joint region i associated with joint i is positioned on the joint simulation model, and according to the target visual perception feature, the joint region i is dynamically rendered so that the joint region i appears red, the brightness of the red color is X, and the flicker frequency is Y, and so on.

[0119] Referring to Figure 4 The embodiment of the present application further provides a joint safety identification device of a humanoid robot, which can implement the method.

[0120] The first module is configured to acquire current sensor state data of the humanoid robot.

[0121] The second module is configured to input the current sensor state data into a joint safety identification model and output a corresponding joint safety identification result. The joint safety identification model is trained by the model training method. The joint safety identification result includes target joint safety categories and category prediction probabilities of each joint in the humanoid robot.

[0122] It can be understood that the contents in the method embodiments are applicable to the device embodiments, the device embodiments specifically implement the functions of the method embodiments, and achieve the same beneficial effects as the method embodiments.

[0123] The embodiment of the present application further provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the method when executing the computer program. The electronic device can be any intelligent terminal, such as a tablet computer or a vehicle-mounted computer.

[0124] It can be understood that the contents in the method embodiments are applicable to the device embodiments, the device embodiments specifically implement the functions of the method embodiments, and achieve the same beneficial effects as the method embodiments.

[0125] Referring to Figure 5 , Figure 5 The hardware structure of the electronic device of another embodiment is illustrated, which includes:

[0126] The processor 901 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0127] The memory 902 can be implemented in the form of a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a Random Access Memory (RAM), etc. The memory 902 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 902 and are called and executed by the processor 901 to implement the above-mentioned method of the embodiments of the present application.

[0128] The input / output interface 903 is configured to realize information input and output.

[0129] The communication interface 904 is configured to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0130] The bus 905 is configured to transmit information between various components (for example, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904) of the device.

[0131] The processor 901, the memory 902, the input / output interface 903, and the communication interface 904 are connected to each other through the bus 905 to realize the communication connection between the device.

[0132] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the above-mentioned method.

[0133] It can be understood that the content in the above method embodiments is applicable to the present storage medium embodiments. The present storage medium embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0134] The embodiments of the present application also provide a computer program product, which includes a computer program. The computer program is executed by a processor to implement the above-mentioned method.

[0135] It can be understood that the content in the above method embodiments is applicable to the present program product embodiments. The present program product embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0136] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include a high-speed random access memory and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory that is remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0137] The joint safety identification method and device of the humanoid robot, the electronic device, the storage medium and the program product provided by the embodiments of the present application train an unsupervised reward model through a first-stage training set, train a difficult example classification model through a second-stage training set, fine-tune the difficult example classification model through a key data set, obtain a sparse key degree prediction model, realize the construction of the joint safety identification model of the humanoid robot based on a three-stage training framework, can solve the problems of data sparsity and imbalance, realize high-precision joint fault probability prediction, strengthen the classification ability of the model, identify the current sensor state data of the humanoid robot through the joint safety identification model after three-stage training, output the corresponding joint safety identification result, and improve the joint safety identification efficiency and accuracy of the humanoid robot.

[0138] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0139] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than the figures shown, or combine certain steps, or different steps.

[0140] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0141] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the function modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0142] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of this application, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of orderly or chronological mundane operation, reverse order operation, based on circuitry availability, based on stated preference or the like, and that "default" or other orderings are thus permissible. Further, the terms "comprise", "comprising", "include", "including", and the like, are specifically intended to be open-ended. That is, references to individual steps and the like do not suhstantially exclude the presence of two or more of a given step or its integral presence in the process, method, system, article, or apparatus having been made with a wider scope. The use of notation such as "first", "second", "third", etc. does not generally limit the areas, but is used to connect like elements or to distinguish one claim from another. These terms can be used interchangeably when appropriate. Terms concerning the relative position of elements can be interpreted such that their use adheres to their normal meaning, but they can also be interpreted to mean the opposite according to specific claims.

[0143] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are only A, only B, and A and B at the same time. Where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and back associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c, can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0144] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the above-described device embodiments are only illustrative, for example, the division of the above-mentioned units is only a logical functional division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0145] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the application.

[0146] In addition, each of the functional units in the embodiments of the present application can be integrated in one processing unit, or each unit can exist alone physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0147] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present application, essentially or partially, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause a computer device (such as a personal computer, a server, or a network device) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: various memories (such as a read-only memory, a random access memory, a flash memory, or the like) and a magnetic disk or an optical disk and the like.

[0148] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not intended to limit the scope of the embodiments of the present application. Any modification, equivalent replacement, and improvement made by those skilled in the art without departing from the scope and spirit of the embodiments of the present application shall fall within the scope of the embodiments of the present application.

Claims

1. A model training method, characterized in that, The method comprises the following steps: obtaining a first-stage training set; the first-stage training set comprises an initial positive sample set and an initial negative sample set; the initial positive sample set comprises sensor state data when a joint safety event occurs for a plurality of humanoid robots; the initial negative sample set comprises sensor state data when a joint safety event does not occur for the plurality of humanoid robots; training an unsupervised reward model based on the first-stage training set using a ranking loss function; filtering a plurality of difficult samples from the initial negative sample set through the trained unsupervised reward model, and constructing a second-stage training set according to each of the difficult samples and the initial positive sample set; constructing a second neural network model, training the second neural network model using the second-stage training set with a classification loss as a target function, and obtaining a corresponding difficult example classification model; based on the difficult example classification model, filtering misjudged samples from the negative samples of the second-stage training set, and combining the misjudged samples with the positive samples in the second-stage training set to form a key data set; based on dense reinforcement learning, fine-tuning the difficult example classification model through the key data set to obtain a corresponding sparse key degree prediction model.

2. The method of claim 1, wherein, The method comprises the following steps: inputting the initial negative sample set into the trained unsupervised reward model, scoring each negative sample in the initial negative sample set, and outputting a feature score corresponding to each negative sample; filtering a plurality of negative samples with a feature score higher than a preset score threshold from the initial negative sample set to construct a false positive sample set; using the false positive sample set as a second-stage negative sample set and the initial positive sample set as a second-stage positive sample set, and constructing the second-stage training set according to the second-stage negative sample set and the second-stage positive sample set.

3. The method of claim 2, wherein, The method comprises the following steps: constructing the second neural network model, which is a supervised classification model based on an improved double-branch network structure, the improved double-branch network structure comprising an upper branch and a lower branch; inputting the second-stage training set into the improved double-branch network structure, wherein the second-stage positive sample set is processed through the upper branch and the second-stage negative sample set is processed through the lower branch; extracting positive sample features through a series connection structure of a convolutional neural network and a long short-term memory network of the upper branch, and extracting negative sample features through a convolutional neural network of the lower branch; performing feature splicing on the positive sample features output by the upper branch and the negative sample features output by the lower branch through a feature fusion classification layer of the second neural network model, and outputting a classification probability vector corresponding to each second-stage sample; According to the classification probability vector corresponding to each of the second stage samples and the sample true label, a classification loss is calculated, the classification loss comprising a cross-entropy loss and a focal loss; According to the classification loss corresponding to each of the second stage samples, the model parameters of the second neural network model are optimized through back propagation to obtain the difficult example classification model.

4. The method of claim 1, wherein, The difficult example classification model is used to screen out misjudged samples from the negative samples of the second stage training set, and the misjudged samples are combined with the positive samples in the second stage training set to form a key data set, which comprises: The second stage negative sample set is input into the difficult example classification model, and the target classification category corresponding to each second stage negative sample is output; One or more second stage negative samples with the target classification category being a positive sample are screened out from the second stage negative samples, and the one or more second stage negative samples are combined with the second stage positive sample set to form the key data set.

5. A joint safety recognition method for a humanoid robot, characterized by, The method comprises the following steps: Obtain the current sensor state data of the humanoid robot; Input the current sensor state data into a joint safety identification model to output a corresponding joint safety identification result; the joint safety identification model is trained by the model training method of any one of claims 1 to 4; the joint safety identification result comprises a target joint safety category and a category prediction probability corresponding to each joint in the humanoid robot.

6. The method of claim 5, wherein, The method further comprises: Obtain the joint simulation model corresponding to the humanoid robot; According to the joint safety detection result and the preconfigured labeling rule, the joint simulation model is visually labeled.

7. The method of claim 6, wherein, The joint simulation model is visually labeled according to the joint safety detection result and the preconfigured labeling rule, comprising: Obtain the labeling rule, the labeling rule comprising a plurality of preset joint safety categories and a safety level mapping relationship and a visual perception feature mapping relationship corresponding to each joint safety category; According to the target joint safety category and the category prediction probability corresponding to each joint, determine the target joint safety level corresponding to each joint from the safety level mapping relationship; According to the target joint safety level corresponding to each joint, determine the target visual perception feature matching the target joint safety level from the visual perception feature mapping relationship; Determine the joint region of each joint on the joint simulation model, and bind the target visual perception feature to the joint region; The joint simulation model after the association feature is dynamically rendered and displayed in the human-computer interaction interface.

8. A joint safety recognition device for a humanoid robot, characterized by comprising: a joint safety recognition device according to any one of claims 1 to 7, wherein the joint safety recognition device is provided to a humanoid robot. The device comprises: A first module for obtaining the current sensor state data of the humanoid robot; A second module for inputting the current sensor state data into a joint safety identification model to output a corresponding joint safety identification result; the joint safety identification model is trained by the model training method of any one of claims 1 to 4; the joint safety identification result comprises a target joint safety category and a category prediction probability corresponding to each joint in the humanoid robot.

9. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method in any one of claims 1 to 7 when executing the computer program.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method in any one of claims 1 to 7.

Images