Industrial robot safety collaboration method fusing multi-modal data and reinforcement learning

Abstract

The application provides a method for industrial robot safety cooperation by fusing multi-modal data and reinforcement learning, and belongs to the technical field of industrial robots. The method collects multi-modal data in the working environment of the industrial robot through four sensors, namely a depth camera, an RGB camera, a torque sensor and a pickup device; pre-processes the collected multi-modal data, including window division and step setting of sequence data, denoising, normalization, feature extraction operation, and scaling of image data, so that it is suitable for input into a deep learning model; improves the ViT architecture to build a multi-modal fusion model SaW-ViT-Env, and trains it; the classification encoding vector output by the Transformer Encoder in SaW-ViT-Env is used as the dynamic state, and the probability category in the final output is used as the basis for processing the reward; a Q network is built based on the MLP architecture, and is trained in combination with the safety-first DQN algorithm; and the safety and efficiency of the SaW-ViT-Env model, the Q network and the safety-first DQN algorithm in a complex human-robot cooperation scene are verified through experiments.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial robots, in particular to an industrial robot safety cooperation method fusing multi-modal data and reinforcement learning. BACKGROUND

[0002] Industrial robots play a crucial role in modern manufacturing, especially in high-precision and high-efficiency production processes. With the rapid development of industrial automation and digitization, human-robot collaboration (HRC) applications are increasing. However, industrial robots face complex and dynamic environments when working with humans, such as sudden intrusions, workpiece slips, or accidental contacts, which can lead to serious safety accidents and production interruptions. Therefore, improving the safety and efficiency of industrial robots in HRC scenarios has become a key problem to be solved.

[0003] Currently, the technical implementation of industrial robot safety cooperation control mainly relies on a single sensing method, such as a vision sensor or a force sensor, to detect changes in the surrounding environment. However, these methods have many limitations in complex and dynamic environments. For example, a single vision sensor cannot accurately detect obstacles in low light or severe occlusion conditions, while a force sensor often has a reaction delay when detecting accidental contacts, making it difficult to make effective obstacle avoidance or emergency stop decisions within milliseconds. In addition, existing control algorithms lack effective fusion mechanisms when facing multi-modal data, leading to decision delays and inaccuracies, making it difficult to meet the dual demands of high safety and high efficiency.

[0004] Based on the above problems, the present application aims to propose an innovative industrial robot safety cooperation control method by fusing multi-modal perception data and reinforcement learning algorithms. This method not only can accurately identify a variety of abnormal states in real time, but also can prioritize triggering safety control actions in high-risk scenarios, achieving millisecond-level dynamic obstacle avoidance, adaptive force control, and safety emergency stop. SUMMARY

[0005] The embodiments of the present application provide an industrial robot safety cooperation method fusing multi-modal data and reinforcement learning, which is used to solve the technical problem of how to effectively fuse multi-modal data, improve the adaptability of robots to complex environments, and make industrial robots prioritize triggering safety control actions, thereby achieving dynamic obstacle avoidance, adaptive force control, and safety emergency stop, and improving the safety and efficiency of industrial robots in complex HRC scenarios.

[0006] The embodiments of the present application provide an industrial robot safety cooperation method fusing multi-modal data and reinforcement learning, which includes:

[0007] The multi-modal data in the working environment of the industrial robot is collected through four sensors, namely, a depth camera, an RGB camera, a torque sensor and a pickup;

[0008] The collected multi-modal data is preprocessed, including window division and step setting of sequence data, denoising, normalization, feature extraction operation, and scaling of image data, so as to adapt to input into a deep learning model;

[0009] A multi-modal fusion model SaW-ViT-Env is constructed by improving the ViT architecture, and is trained, wherein the classification encoding vector output by the Transformer Encoder in the SaW-ViT-Env is used as a dynamic state, and the probability category in the final output is used as a basis for processing the reward;

[0010] A Q network is constructed based on the MLP architecture, and is trained in combination with the safety-first DQN algorithm;

[0011] The safety and efficiency of the SaW-ViT-Env model, the Q network and the safety-first DQN algorithm in a complex human-robot collaboration scenario are verified through experiments.

[0012] In an implementation manner of the present application, the multi-modal data includes: a depth map photographed by using an Intel RealSense L515 depth camera, acquiring three-dimensional space information, detecting the positions of personnel, obstacles and workpieces; a visible light image photographed by using a Basler acA1300-20gc industrial camera, providing texture and color information of the scene, assisting in identifying dynamic objects; a voltage time domain signal collected by using a SCHUNK torque sensor, the torque sensor being installed at the elbow and wrist of the double arms of the industrial robot, and being used for detecting the voltage time domain signal and reflecting the change of the joint force / torque of the robot, detecting accidental contact or workpiece falling; and a sound signal in the environment collected by using an Audio-Technica ATR2100-USB stereo condenser microphone, detecting abnormal sound such as collision sound and falling sound, and assisting in judging potential risks.

[0013] In an implementation manner of the present application, the sampling rate of the SCHUNK torque sensor is set to 400 Hz, and the sampling rate of the Audio-Technica ATR2100-USB stereo condenser microphone is set to 44.1 kHz.

[0014] In an implementation manner of the present application, when the multi-modal data is collected, the specific collection strategy includes:

[0015] Synchronous collection: all the sensors synchronously collect data with the same time stamp, ensuring the time consistency of the multi-modal data;

[0016] Scene simulation: simulate the real scene in which the industrial robot works in the laboratory environment, including four abnormal states of sudden intrusion of personnel, accidental contact, workpiece falling and obstacle.

[0017] In an implementation manner of the present application, window division and step setting of sequence data, denoising, normalization, feature extraction operation, and scaling of image data to adapt to input into a deep learning model, the specific operations of pre-processing the sequence data and the image data include:

[0018] The depth map is denoised using a Gaussian filter, and the pixel values of the depth map are normalized to the range [0, 1] to obtain a pre-processed depth map.

[0019] The size of the RGB image is adjusted to 224x224 to meet the model input requirements, and the pixel values of the adjusted RGB image are normalized to the range [0, 1] to obtain a pre-processed RGB image.

[0020] The voltage time domain signal is removed using a low-pass filter to remove high-frequency noise, the window size is set to 1s, the voltage time domain signal is divided into frames, each frame has a length of 400 sampling points, then four voltage signal frames are spliced by rows, the spliced voltage time domain signal contains 1600 sampling points, and a pre-processed voltage time domain signal is obtained.

[0021] The sound signal 7s before the current time is taken, a 7695ms audio is constructed by a 0 filling operation, the window size is set to 25ms, the step is set to 10ms, the 7695ms audio is divided into frames, 768 audio frames are divided, each audio frame has a length of 1102 sampling points, the logarithmic mel spectrum of each audio frame is calculated, and the value of the logarithmic mel spectrum is normalized to the range [0, 1] to obtain a pre-processed logarithmic mel spectrum.

[0022] In an implementation manner of the present application, a multi-modal fusion model SaW-ViT-Env is constructed by improving the ViT architecture, and the network architecture of the SaW-ViT-Env model includes:

[0023] The preprocessed depth map is subjected to standard two-dimensional convolution, group normalization, a rectified linear unit activation function, two cascaded ordinary two-dimensional convolution modules, and a dimension expansion module to obtain a depth map Token embedding matrix. The preprocessed RGB image is subjected to two-dimensional convolution and a dimension expansion module to obtain an RGB image Token embedding matrix. The preprocessed voltage time domain signal is subjected to a linear module and a one-dimensional convolution module to obtain a voltage time domain signal Token embedding matrix. The preprocessed log Mel spectrum is subjected to two cascaded one-dimensional convolution modules, a Gaussian error linear unit activation function, and a one-dimensional convolution module to obtain a log Mel spectrum Token embedding matrix.

[0024] The cross-attention module calculates the cross-attention of the depth map Token embedding matrix and the RGB image Token embedding matrix. The cross-attention of the depth map Token embedding matrix and the voltage time domain signal Token embedding matrix is calculated. The cross-attention of the depth map Token embedding matrix and the log Mel spectrum Token embedding matrix is calculated. The three cross-attention results are subjected to matrix addition calculation to obtain a secondary modality fusion feature matrix.

[0025] The encoder and the classifier input the secondary modality fusion feature matrix into a Transformer Encoder containing twelve layers of EncoderBlock to obtain an output feature matrix. The vector of the classification Token in the output feature matrix is extracted as the final encoding vector of the multi-modal input data. The final encoding vector is input into a multi-layer perceptron classifier to output a five-class probability vector. The class corresponding to the maximum value index in the probability vector is taken as the output of the SaW-ViT-Env model.

[0026] In an implementation manner of the present application, a Q network is constructed based on an MLP architecture, and the architecture of the Q network comprises:

[0027] The Q network is a 7-layer MLP, and the number of input and output layer neurons is 768, 1024, 2048, 1024, 512, 32, and 3 in sequence. The input of the Q network is an environment state, and the output is three types of actions, including: robot joint angle adjustment for dynamic obstacle avoidance; force control parameter adjustment for adaptive force control, such as adjusting clamping force or contact force; and emergency stop instruction triggered when a high-risk state is detected.

[0028] In an implementation manner of the present application, the DQN algorithm combined with safety priority is used for training, and the DQN algorithm process comprises:

[0029] The Q network and the target network are initialized, and the target network is consistent with the Q network in model structure and parameters; an empty experience replay pool is created ; repeat the same Episode.

[0030] The procedure of each of the Episodes includes: resetting the environment, using the output of the Transformer Encoder in the SaW-ViT-Env model as the initial state of the environment . Setting a random number, if the random number is less than epsilon, then taking the state into the Q network, outputting the Q value of the action and the action ; if the random number is greater than the epsilon, then randomly selecting an action from the action pool. The industrial robot performs the action, and again obtains the next state of the environment and the reward obtained by the safety-first reward mechanism. Build experience and store it in the experience replay pool. Determine whether the current step number is greater than 200, if it is greater than 200, then determine whether the current step number is an integer multiple of 5, if the condition is met, then learning of the Q network is performed, otherwise, the next Episode is entered. When the condition is met, determine whether the current step number is an integer multiple of 30, or manually confirm the start, if the condition is met, then the weights of the Q network are copied to the weights of the target network, otherwise, the target network is not updated. Randomly select the experience with a batch size of batch from the experience replay pool, and select the state and the next state as the input of the Q network and the target network respectively. The Q network outputs the Q value ; the target network outputs the Q value of all the actions , and selects the action with the maximum Q value , calculates the target Q value based on the formula . Use the Q value and the target Q value to construct the loss constraint , and update the Q network parameters based on the loss constraint through gradient descent.

[0031] In an implementation manner of the present application, the reward is obtained by the safety-first reward mechanism, and the safety-first reward mechanism is: when detecting sudden intrusion of personnel, accidental contact, workpiece falling or obstacles, outputting a negative reward, r taking a value of -10, forcing the Q network to trigger obstacle avoidance or emergency stop action; when detecting a normal state, outputting a positive reward, r taking a value of 10, indicating successful collision avoidance or smooth handling of abnormal state.

[0032] Compared with the prior art, the present application has the following significant effects:

[0033] By the technical scheme, a multi-modal fusion model named self-adjusting modal weight according to environment state (SaW-ViT-Env) is constructed, the model is based on a visual Transformer (ViT) architecture, can effectively fuse multi-modal data from multiple sensors, and improves the adaptability of the model to complex environments. In addition, the application also constructs a Q network based on the MLP architecture, and introduces a safety priority deep Q network (DQN) algorithm. In the policy evaluation part of the DQN, a safety priority reward mechanism is designed. This reward mechanism ensures that in high-risk scenarios, the robot can preferentially trigger safety control actions, thereby realizing efficient cooperation of dynamic obstacle avoidance, adaptive force control and safety emergency stop. Through the above method, the safety and efficiency of the industrial robot in complex human-robot collaboration scenarios are improved, which has broad prospects in practical applications and can provide strong protection for the safe and efficient cooperation of industrial robots in complex dynamic environments. BRIEF DESCRIPTION OF DRAWINGS

[0034] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0035] Figure 1 The figure is a whole flowchart of the industrial robot safety cooperation method of the present application embodiment which fuses multi-modal data and reinforcement learning;

[0036] Figure 2 The figure is a flowchart of the extraction of Log-Mel Spectrogram in the present application embodiment;

[0037] Figure 3 The figure is a network architecture diagram of SaW-ViT-Env in the present application embodiment;

[0038] Figure 4 The figure is a flowchart of the safety priority DQN algorithm in the present application embodiment. DETAILED DESCRIPTION

[0039] To make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described clearly and completely in combination with the specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0040] Currently, the technical implementation of industrial robot safety collaboration control mainly relies on a single sensing method, such as a visual sensor or a force sensor, to detect changes in the surrounding environment. However, these methods have many limitations in complex and dynamic environments. For example, a single visual sensor is difficult to accurately detect obstacles in insufficient light or severe occlusion conditions, while a force sensor often has a reaction delay when detecting unexpected contact, and cannot make effective obstacle avoidance or emergency stop decisions within milliseconds. In addition, existing control algorithms lack effective fusion mechanisms when facing multi-modal data, resulting in delay and inaccuracy of decisions, making it difficult to meet the dual demands of high safety and high efficiency.

[0041] Based on this, the embodiments of the present application provide an industrial robot safety collaboration method that fuses multi-modal data and reinforcement learning. This method not only can perceive and accurately identify multiple abnormal states in real time, but also can trigger safety control actions in high-risk scenarios, achieving millisecond-level dynamic obstacle avoidance, adaptive force control, and safety emergency stop.

[0042] The various embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0043] The embodiments of the present application provide an industrial robot safety collaboration method that fuses multi-modal data and reinforcement learning, as shown in Figure 1 The method can include steps S101-S105:

[0044] S101, through four sensors of a depth camera, an RGB camera, a torque sensor, and a microphone, multi-modal data in the working environment of an industrial robot is collected.

[0045] In the embodiments of the present application, the data collection devices used include the following:

[0046] Depth camera: an Intel RealSense L515 depth camera is used to shoot a depth map and obtain three-dimensional spatial information to detect the positions of personnel, obstacles, and workpieces.

[0047] RGB camera: a Basler acA1300-20gc industrial camera is used to shoot a visible light image to provide texture and color information of the scene and assist in identifying dynamic objects.

[0048] Torque sensor: a SCHUNK torque sensor is used and installed on the elbows and wrists of the dual arms of the industrial robot to detect voltage time domain signals and reflect changes in the joint force / torque of the robot to detect unexpected contact or workpiece falling.

[0049] Microphone: an Audio-Technica ATR2100-USB stereo condenser microphone is used to collect sound signals in the environment to detect abnormal sounds such as collision sounds and falling sounds and assist in judging potential risks.

[0050] In the embodiments of the present application, the data acquisition strategy adopted includes:

[0051] Synchronous acquisition: all sensors synchronously acquire data with the same timestamp, ensuring the time consistency of multi-modal data.

[0052] Scene simulation: simulate the real scene of industrial robot work in the laboratory environment, including four abnormal states of sudden intrusion of personnel, accidental contact, workpiece falling and obstacle.

[0053] In the embodiments of the present application, the sampling rate of SCHUNK torque sensor is set to 400Hz; the sampling rate of Audio-Technica ATR2100-USB stereo condenser microphone is set to 44.1kHz, and the acquisition frequency can be set by the user, which is not limited here.

[0054] S102, pre-process the acquired multi-modal data, including window division and step setting of sequence data, denoising, normalization, feature extraction operation, and scaling of image data, so as to adapt to input into the deep learning model.

[0055] In the embodiments of the present application, the specific operations of pre-processing sequence data and image data include:

[0056] The depth map is denoised using a Gaussian filter, and the pixel value of the depth map is normalized to the range of [0, 1] to obtain the pre-processed depth map;

[0057] The size of the RGB image is adjusted to 224x224 to meet the model input requirements, and the pixel value of the adjusted RGB image is normalized to the range of [0, 1] to obtain the pre-processed RGB image;

[0058] A low-pass filter is used to remove high-frequency noise of the voltage time domain signal, a window size of 1s is set, the voltage time domain signal is framed, each frame has a length of 400 sampling points, then four voltage signal frames are spliced by row, the spliced voltage time domain signal contains 1600 sampling points, and a pre-processed voltage time domain signal is obtained;

[0059] Take the sound signal 7s before the current time, construct a 7695ms audio through a 0 operation, set a window size of 25ms and a step of 10ms, frame the 7695ms audio, divide it into 768 audio frames, each audio frame has a length of 1102 sampling points, calculate the log mel spectrum of each audio frame, and normalize the value of the log mel spectrum to the range of [0, 1] to obtain the pre-processed log mel spectrum, and the overall processing process is as shown in Figure 2 .

[0060] Further, the mathematical expressions of the pre-processed data are as follows: depth map , RGB image , voltage time domain signal , log-mel spectrogram .

[0061] In S103, a multi-modal fusion model SaW-ViT-Env is constructed by improving the ViT architecture, and the model is trained. The classification encoding vector output by the Transformer Encoder in the SaW-ViT-Env is taken as the dynamic state, and the probability category in the final output is taken as the basis for processing the reward.

[0062] In the embodiments of the present application, a multi-modal fusion model SaW-ViT-Env is constructed by improving the ViT architecture, and the network architecture of the SaW-ViT-Env model is as shown in Figure 3 The SaW-ViT-Env model includes the following components:

[0063] An input module: the pre-processed depth map is subjected to standard two-dimensional convolution, group normalization, a rectified linear unit activation function, two cascaded ordinary two-dimensional convolution modules, and a dimension expansion module to obtain a depth map Token embedding matrix. The pre-processed RGB image is subjected to a two-dimensional convolution and a dimension expansion module to obtain an RGB image Token embedding matrix. The pre-processed voltage time domain signal is subjected to a linear module and a one-dimensional convolution module to obtain a voltage time domain signal Token embedding matrix. The pre-processed log-mel spectrogram is subjected to two cascaded one-dimensional convolution modules, a Gaussian error linear unit activation function, and a one-dimensional convolution module to obtain a log-mel spectrogram Token embedding matrix.

[0064] A cross-attention module: the cross-attention of the depth map Token embedding matrix and the RGB image Token embedding matrix is calculated. The cross-attention of the depth map Token embedding matrix and the voltage time domain signal Token embedding matrix is calculated. The cross-attention of the depth map Token embedding matrix and the log-mel spectrogram Token embedding matrix is calculated. The three cross-attention results are subjected to matrix addition calculation to obtain a secondary modal fusion feature matrix.

[0065] An encoder and a classifier: the secondary modal fusion feature matrix is input into a Transformer Encoder containing twelve Encoder Blocks to obtain an output feature matrix; the vector of the classification Token in the output feature matrix is extracted as the final encoding vector of the multi-modal input data, and the final encoding vector is input into a multi-layer perceptron classifier to output a five-class probability vector. The class corresponding to the maximum value index in the probability vector is taken as the output of the SaW-ViT-Env model.

[0066] Further, the output of the SaW-ViT-Env model is: a 5-class probability vector, including 1 normal state and 4 scene anomalies, including: sudden intrusion of personnel, accidental contact, workpiece falling, and obstacles.

[0067] Further, the calculation process of the input module is: a graph passes through a standard two-dimensional convolution (StdConv2D) module, a group normalization (GroupNorm) module, a rectified linear unit (ReLU) activation function, two concatenated ordinary two-dimensional convolution (Conv2D) modules, and a dimension expansion (Flatten) module in sequence, and the Token embedding matrix of the depth map is calculated ; an RGB image passes through a Conv2D and a Flatten module in sequence, and the Token embedding matrix of the RGB image is calculated ; a voltage time domain signal passes through a linear (Linear) module and a one-dimensional convolution (Conv1D) module in sequence, and the Token embedding matrix of the voltage time domain signal is calculated ; a Log-MelSpectrogram passes through two concatenated Conv1D modules, a Gaussian error linear unit (GELU) activation function, and a Conv1D module in sequence, and the Token embedding matrix of the Log-MelSpectrogram is calculated .

[0068] Further, the calculation process of the cross attention module is:

[0069] Calculate and cross attention (Cross Attention): use and the randomly initialized query embedding matrix to perform matrix multiplication calculation to obtain the query matrix ; use and the randomly initialized key embedding matrix and the value embedding matrix to perform matrix multiplication calculation respectively to obtain the key matrix and the value matrix ; use and to perform matrix multiplication calculation and bring the calculation result into the Softmax function to obtain the cross attention weight ; use and to perform matrix multiplication calculation to obtain the feature matrix of the fused modal .

[0070] calculate and Cross attention: using With randomly initialized query embedding matrix Perform matrix multiplication to obtain the query matrix. ;use With randomly initialized key embedding matrix Sum embedding matrix Perform matrix multiplication separately to obtain the key matrix. Sum matrix ;use and Perform matrix multiplication and substitute the results into the Softmax function to obtain the cross-attention weights. ;use and The feature matrix of the fused mode is obtained by performing matrix multiplication. .

[0071] calculate and Cross attention: using With randomly initialized query embedding matrix Perform matrix multiplication to obtain the query matrix. ;use With randomly initialized key embedding matrix Sum embedding matrix Perform matrix multiplication separately to obtain the key matrix. Sum matrix ;use and Perform matrix multiplication and substitute the results into the Softmax function to obtain the cross-attention weights. ;use and The feature matrix of the fused mode is obtained by performing matrix multiplication. .

[0072] right , and Perform matrix addition to obtain the quadratic modality fusion feature matrix. ;right Place the categorized token matrix in front of the matrix rows. The characteristic matrix is ​​obtained. And by introducing positional encoding embedding, a final multimodal token matrix is ​​obtained. .

[0073] Furthermore, the computational process for the encoder and classifier is as follows: The output feature matrix is ​​obtained after passing through a Transformer Encoder containing 12 Encoder Blocks. ;exist The first row vector of the features after encoding the corresponding category token. As the final encoded vector of multimodal input data; As input to the MLP classifier, the MLP classifier is followed by the Softmax output, which gives a probability vector of 5 classes. The class with the highest probability is taken as the class output of SaW-ViT-Env.

[0074] S104 is a Q-network built on an MLP architecture and trained using a safety-first DQN algorithm.

[0075] In this embodiment of the application, a Q network is constructed based on an MLP architecture. The architecture of the Q network includes:

[0076] The Q-network is a 7-layer MLP, with the number of neurons in the input, output, and layer lengths being 768, 1024, 2048, 1024, 512, 32, and 3, respectively. The input to the Q-network is the environmental state, and the output consists of three types of actions: robot joint angle adjustment for dynamic obstacle avoidance; force control parameter adjustment for adaptive force control, such as adjusting gripping or contact forces; and emergency stop commands triggered when a high-risk state is detected.

[0077] In this embodiment, the security-first DQN algorithm is used for training. The DQN algorithm flow is as follows: Figure 4 As shown, it includes:

[0078] Initialize the Q-network and target network, ensuring the target network is identical to the Q-network in model structure and parameters; create an empty experience replay pool. Repeat the same episode.

[0079] First, initialize the Q network and the target network; second, create an empty experience replay pool. Finally, the same episode is repeated.

[0080] Each episode's process includes: resetting the environment and obtaining the environment's initial state. , Inferenced from the SaW-ViT-Env model; a random number is set, and if the random number is less than epsilon, then... Input into the Q network and output the action. Q value and action If the random number is greater than epsilon, then a random action is selected from the action pool. ; the robot performs an action and acquires the environment state again and a reward , the reward is obtained by a safety-first reward mechanism; experience is constructed and stored in an experience replay pool ; it is determined whether the current step number is greater than 200, if it is greater than 200, it is determined in cascade whether the current step number is an integer multiple of 5, if the condition is met, learning of the Q network is performed, otherwise the next episode is entered; when the condition is met, it is determined whether the current step number is an integer multiple of 30 or manual confirmation of the start is made, if the condition is met, the weights of the Q network are copied to the target network, otherwise the target network is not updated; from the experience replay pool a number of experiences with a batch size are randomly selected , and and are selected as inputs of the Q network and the target network respectively; the Q network outputs Q values ; the target network outputs Q values of all actions and selects the action with the maximum Q value , based on the formula: , the target Q value is calculated; using the Q value and the target Q value , a loss constraint is constructed, the formula is: , the Q network parameters are updated by gradient descent based on the loss constraint.

[0081] Further, in the embodiments of the application, the reward is obtained by a safety-first reward mechanism, and the safety-first reward mechanism is:

[0082] When a sudden intrusion of a person, accidental contact, a workpiece falling or an obstacle is detected, a negative reward is output, r takes a value of -10, forcing the Q network to trigger an obstacle avoidance or emergency stop action; when a normal state is detected, a positive reward is output, r takes a value of 10, indicating successful collision avoidance or smooth handling of an abnormal state.

[0083] In S105, the safety and efficiency of the SaW-ViT-Env model, the Q network and the safety-first DQN algorithm in a complex human-robot collaboration scenario are verified by experiments.

[0084] In the embodiments of the application, the evaluation indexes verified by experiments include: accuracy (Accuracy), that is, the classification accuracy of the model on the normal state and the four abnormal states; recall rate (Recall), that is, the recall rate of the model on the four abnormal states; F1 value (F1 Score), that is, the F1 value of the model on the four abnormal states.

[0085] In the embodiments of the present application, the results of the experimental verification are as follows: the accuracy of the SaW-ViT-Env model is 98.5%, and the recall rates of the four abnormal states are as follows: sudden intrusion of personnel 99.2%, accidental contact 98.8%, workpiece falling 97.5%, and obstacle 98.1%; and the F1 values of the four abnormal states are as follows: sudden intrusion of personnel 99.0%, accidental contact 98.5%, workpiece falling 97.8%, and obstacle 98.0%. In the high-risk scene, the industrial robot can preferentially trigger the obstacle avoidance or emergency stop action by using the DQN algorithm, and realizes the millisecond-level dynamic obstacle avoidance, adaptive force control and safety emergency stop. The experimental results show that the industrial robot safety assistance method of the present application which fuses multi-modal data and reinforcement learning can improve the safety and efficiency of the industrial robot in a complex dynamic environment, and reduce the risk of production accidents.

[0086] The above technical solution realizes the dynamic fusion of depth maps, RGB maps, voltage time domain signals and Log-MelSpectrogram through SaW-ViT-Env, can accurately identify four types of abnormal states such as sudden intrusion of personnel, accidental contact, workpiece falling and obstacle; constructs a Q network, combines the safety-first DQN algorithm, introduces a negative reward mechanism, ensures that the robot preferentially triggers the obstacle avoidance or emergency stop action in a high-risk scene, and realizes the efficient cooperation of dynamic obstacle avoidance, adaptive force control and safety emergency stop. This method significantly improves the safety and efficiency of the industrial robot in a complex human-robot collaboration scene, reduces the risk of production accidents, and provides reliable technical support for industrial digitization and human-robot collaboration.

[0087] Each of the embodiments in the present application is described in a progressive manner, and the same or similar parts of each embodiment can be referred to each other. Each embodiment mainly explains the difference from other embodiments. Especially, for system and device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the related parts can be referred to the part of the method embodiment.

[0088] The system and device provided by the embodiments of the present application are one-to-one corresponding to the method, so the system and device also have similar beneficial technical effects to the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the system and device will not be described here.

[0089] It should also be noted that the terms "comprising," "including," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0090] The above description is merely illustrative of the application, and not restrictive. Various modifications and changes can become apparent to those skilled in the art. Incorporating any modification, equivalent substitution, improvement, etc. within the spirit and principle of the application, shall be included in the scope of the claims of the application.

Claims

1. An industrial robot safety collaboration method fusing multi-modal data and reinforcement learning, characterized in that, The method comprises: Collecting multi-modal data in the working environment of an industrial robot through four sensors: a depth camera, an RGB camera, a torque sensor, and a microphone; Preprocessing the collected multi-modal data, including window division and step setting, denoising, normalization, feature extraction operations of sequence data, and scaling of image data to make it suitable for input into a deep learning model; Improving the ViT architecture to build a multi-modal fusion model SaW-ViT-Env, and training it, wherein the classification encoding vector output by the Transformer Encoder in the SaW-ViT-Env is used as the dynamic state, and the probability category in the final output is used as the basis for processing the reward; Building a Q network based on the MLP architecture and training it in combination with the safety-first DQN algorithm; Verifying the safety and efficiency of the SaW-ViT-Env model, the Q network, and the safety-first DQN algorithm in complex human-robot collaboration scenarios through experiments.

2. The method of claim 1, wherein, The multi-modal data includes: a depth map taken by an Intel RealSense L515 depth camera to obtain three-dimensional spatial information and detect the positions of personnel, obstacles, and workpieces; a visible light image taken by a Basler acA1300-20gc industrial camera to provide texture and color information of the scene and assist in identifying dynamic objects; a voltage time domain signal collected by a SCHUNK torque sensor installed on the elbows and wrists of the dual arms of an industrial robot to detect voltage time domain signals and reflect changes in the joint force / torque of the robot, detect accidental contact or workpiece falling; and a sound signal collected by an Audio-Technica ATR2100-USB stereo condenser microphone in the environment to detect abnormal sounds such as collision sounds and falling sounds and assist in judging potential risks. The sampling rate of the SCHUNK torque sensor is set to 400Hz, and the sampling rate of the Audio-Technica ATR2100-USB stereo condenser microphone is set to 44.1kHz.

3. The method of claim 2, wherein, When collecting the multi-modal data, the specific collection strategies include: Synchronous collection: all the sensors collect data synchronously with the same timestamp to ensure the temporal consistency of the multi-modal data; Scene simulation: simulating the real scene in which the industrial robot works in a laboratory environment, including four abnormal states: sudden intrusion of personnel, accidental contact, workpiece falling, and obstacles.

4. The method of claim 2, wherein, Window division and step setting, denoising, normalization, and feature extraction operations of sequence data, and scaling of image data to make it suitable for input into a deep learning model, the specific operations for preprocessing the sequence data and the image data include: Using a Gaussian filter to denoise the depth map and normalizing the pixel values of the depth map to the range [0, 1] to obtain the preprocessed depth map; Adjusting the size of the RGB image to 224x224 to meet the model input requirements, and normalizing the pixel values of the adjusted RGB image to the range [0, 1] to obtain the preprocessed RGB image; The low-pass filter is used to remove high-frequency noise of the voltage time domain signal, a window size of 1s is set, the voltage time domain signal is framed, each frame has a length of 400 sampling points, then four voltage signal frames are spliced in rows, the spliced voltage time domain signal contains 1600 sampling points, and a preprocessed voltage time domain signal is obtained; The sound signal of 7s before the current time is taken, a 7695ms audio is constructed through a 0 filling operation, a window size of 25ms is set, a step size of 10ms is set, the 7695ms audio is framed, 768 audio frames are divided, each audio frame has a length of 1102 sampling points, a logarithmic mel spectrum is calculated for each audio frame, and the value of the logarithmic mel spectrum is normalized to a range of [0, 1], and a preprocessed logarithmic mel spectrum is obtained.

5. The method of claim 4, wherein, A multi-modal fusion model SaW-ViT-Env is constructed by improving the ViT architecture, and the network architecture of the SaW-ViT-Env model includes: The preprocessed depth map, RGB image, voltage time domain signal and logarithmic mel spectrum are respectively processed to obtain a Token embedding matrix, specifically including: the depth map is processed through two-dimensional convolution, group normalization, rectified linear unit, two ordinary two-dimensional convolution modules and dimension expansion module, the RGB image is processed through two-dimensional convolution and dimension expansion module, the voltage time domain signal is processed through linear module and one-dimensional convolution module, and the logarithmic mel spectrum is processed through two one-dimensional convolution modules, Gaussian error linear unit and one-dimensional convolution module; cross attention of the depth map and RGB image Token embedding matrices is calculated to obtain a result M, and cross attention of the M and the voltage time domain signal and logarithmic mel spectrum Token embedding matrices is calculated respectively, matrix addition is performed on the three results to obtain a secondary modal fusion feature matrix; the secondary modal fusion feature matrix is input into a Transformer Encoder containing 12 layers of Encoder Block, a classification Token vector is extracted, and a five-classification result is output through a multilayer perceptron classifier.

6. The method of claim 1, wherein, A Q network is constructed based on the MLP architecture, and the architecture of the Q network includes: The Q network is a 7-layer MLP, and the number of neurons from input to output layers is 768, 1024, 2048, 1024, 512, 32 and 3 respectively; the input of the Q network is an environment state, and the output is three types of actions, including: robot joint angle adjustment for dynamic obstacle avoidance; force control parameter adjustment for adaptive force control, such as adjusting clamping force or contact force; emergency stop instruction triggered when a high-risk state is detected.

7. The method of claim 1, wherein, The DQN algorithm is combined for training, and the DQN algorithm process includes: The Q network and the target network with consistent model structure and parameters are initialized; an empty experience replay pool is created; the Episode is repeatedly executed, each of which includes the following steps: resetting the environment, using the output of the Transformer Encoder of the SaW-ViT-Env model as the initial state; selecting an action according to an epsilon strategy, if a random number is less than the epsilon, inputting the state into the Q network to output the Q value of the action and the action, otherwise, randomly selecting an action from the action pool; executing the action, obtaining the next state and the reward, the reward being obtained by the safety-first reward mechanism; constructing experience and storing it into the experience replay pool; determining whether the current step number is greater than 200, if yes, further determining whether the step number is an integer multiple of 5, if yes, performing the Q network learning, otherwise, entering the next Episode; if the step number is an integer multiple of 30 or manual confirmation starts, updating the target network weight, and copying the Q network weight to the target network; finally, randomly selecting a batch of experience from the experience replay pool, calculating the Q value and the target Q value, and updating the Q network parameters through gradient descent.

8. The method of claim 7, wherein, The reward is obtained by the safety-first reward mechanism, the safety-first reward mechanism being: when detecting sudden intrusion of personnel, accidental contact, workpiece falling or obstacle, outputting a negative reward, r taking a value of -10, forcing the Q network to trigger an obstacle avoidance or emergency stop action; when detecting a normal state, outputting a positive reward, r taking a value of 10, indicating successful collision avoidance or smooth handling of abnormal state.

Images