Obstacle avoidance method, device and computer equipment of robot

Abstract

Embodiments of the present application disclose a robot obstacle avoidance method and device and a computer device. Image signals of the surrounding environment and contact force signals are acquired, an obstacle avoidance path is planned according to the image signals and the force signals, the image signals, the force signals and the obstacle avoidance path are input into a multi-modal fusion decision model to generate an obstacle avoidance priority, a target obstacle avoidance path is obtained, and an obstacle avoidance operation is performed according to the target obstacle avoidance path. In this way, multi-source data acquisition of the image signals and the force signals breaks through the limitation of a single sensor, and full perception of various obstacles is achieved. The target obstacle avoidance path is optimized based on the priority and is executed through closed-loop control, which ensures that the motion trajectories of the two arms do not collide with each other and dynamically adjusts the strategy for different types of obstacles, so as to finally achieve high-precision and high-safety obstacle avoidance in a complex environment.

Description

Technical Field

[0001] This application relates to the field of automation control technology, and more specifically, to a method, apparatus, and computer device for obstacle avoidance of a robot. Background Technology

[0002] In industrial production and service operations, robots are widely used in complex task scenarios due to their flexibility in collaborative operation of dual robotic arms. However, their obstacle avoidance capabilities in dynamically changing or complex and unknown environments still have significant shortcomings. In existing technologies, robot obstacle avoidance mostly relies on a single vision sensor or force sensor. A single vision sensor is easily affected by factors such as light intensity, obstacle occlusion, and lack of environmental texture, and its recognition accuracy for transparent obstacles and low-contrast obstacles is insufficient. It is also difficult to deal with sudden unknown obstacles. A single force sensor can only detect collisions after the robotic arm comes into contact with an obstacle, which is a form of "passive obstacle avoidance." It cannot make advance predictions and is prone to damage to obstacles or the robotic arm itself, which is especially risky in precision operation scenarios.

[0003] As application scenarios become increasingly complex, robots face diverse environmental obstacles (such as static fixed obstacles, dynamic moving obstacles, and flexible, easily deformable obstacles). Furthermore, when robots work in tandem, collisions between the robotic arms must be avoided. Single-sensor obstacle avoidance methods are no longer sufficient to meet the demands for high-precision and high-safety obstacle avoidance. Therefore, how to integrate multi-source sensor information to achieve accurate perception, early prediction, and dynamic obstacle avoidance in complex environments has become a critical issue that urgently needs to be addressed in the current development of robotics technology. Summary of the Invention

[0004] This application provides a robot obstacle avoidance method, apparatus, and computer device to achieve accurate perception and prediction of obstacles in complex environments, thereby enabling dynamic obstacle avoidance of the robot.

[0005] A first aspect of this application provides an obstacle avoidance method for a robot, which is applied to a robot and includes:

[0006] Acquire image signals of the surrounding environment and detect contact force signals;

[0007] Plan an obstacle avoidance path based on the image signal and the force signal;

[0008] The image signal, the force signal, and the obstacle avoidance path are input into a multimodal fusion decision model to generate an obstacle avoidance priority and obtain the target obstacle avoidance path.

[0009] Perform obstacle avoidance operations according to the target obstacle avoidance path.

[0010] In an optional embodiment of this application, the step of acquiring image information of the surrounding environment includes:

[0011] A high-resolution camera is used to acquire real-time images of the robot's surrounding environment;

[0012] Image processing technology is used to identify the surrounding environment image to determine the position, size, and shape of obstacles in the surrounding environment image, thereby obtaining image information.

[0013] In an optional embodiment of this application, the force signal includes a first force signal and a second force signal, and the step of acquiring the force signal of the detected contact includes:

[0014] Based on the robot's dynamic model, the change in joint torque of the robot is determined by observing the change in joint current. When the change in joint torque exceeds a preset threshold, a first force signal is generated. The first force signal is used to characterize the robot's force perception risk factor.

[0015] The robot detects end-effector contact force using a six-dimensional force sensor. When the six-dimensional force sensor detects a collision, it generates a second force signal, which is used to characterize the collision basis of the robot.

[0016] In an optional embodiment of this application, the step of planning an obstacle avoidance path based on the image signal and the force signal includes:

[0017] Add the task target point to the initial path node queue. The task target point is the initial position of the robot in the pre-constructed three-dimensional coordinate system.

[0018] Calculate the node cost of the task target point based on the image signal and the force signal;

[0019] The second task target point is determined based on the node cost of the aforementioned task target point;

[0020] Add the second task target point to the initial path node queue;

[0021] Replace the task target point with the second task target point, and return to execute the step of calculating the node cost of the task target point based on the image signal and the force signal until the second task target point is the target position;

[0022] Redundant target points are removed from the initial path node queue to obtain an optimized path node queue, wherein the redundant target points are nodes to be eliminated determined based on smooth optimization calculations.

[0023] Based on the optimized path node queue, an obstacle avoidance path is generated.

[0024] In an optional embodiment of this application, the step of calculating the node cost of the task target point based on the image signal and the force signal includes:

[0025] The image signal and the force signal are substituted into a preset node cost calculation formula to determine the node cost of the task target point;

[0026] The formula for calculating the node cost is as follows:

[0027] Among them, K m =K m +h(s last +s start The result term Km is the heuristic distance of the robot at the current position, the parameter term Km is the heuristic distance of the robot at the previous position, Slast is the robot's previous position information, and Sstart is the robot's current position information.

[0028] Wherein, K is the node cost to be calculated, and K1 and K2 are two different key values. The key values ​​are used to determine the priority of the node cost, and the key values ​​are inversely proportional to the priority.

[0029] g(s) is the actual cost between the current node and the destination, and rhs(s) is the minimum sum of the costs of the current node's adjacent parent nodes and the costs of the parent nodes.

[0030] In an optional embodiment of this application, the step of inputting the image signal, the force signal, and the obstacle avoidance path into a multimodal fusion decision model to generate an obstacle avoidance priority and obtain a target obstacle avoidance path further includes:

[0031] Define a multimodal fusion decision model, wherein the multimodal fusion decision model is Θ={O,S}, where O represents that the robot is in an emergency obstacle avoidance state and S represents that the robot is in a safe state;

[0032] The step of inputting the image signal, the force signal, and the obstacle avoidance path into a multimodal fusion decision model to generate obstacle avoidance priorities and obtain the target obstacle avoidance path includes:

[0033] Calculate the basic allocation probabilities corresponding to the image signal, the force signal, and the obstacle avoidance path, respectively;

[0034] The corresponding obstacle avoidance priority is calculated based on the image signal, the force signal, and the basic allocation probability corresponding to the obstacle avoidance path to obtain the corresponding target obstacle avoidance path.

[0035] In an optional embodiment of this application, the step of calculating the basic allocation probabilities corresponding to the image signal, the force signal, and the obstacle avoidance path respectively includes:

[0036] The image signal is substituted into the multimodal fusion decision model, and the corresponding basic allocation probability is determined based on the image signal. The basic allocation probability corresponding to the image signal includes:

[0037] m v (O) = Conf v m v (S) = 1 - Conf v m v ({O,S})=0;

[0038] in, α, β, and γ are the weights corresponding to the target confidence, depth estimation error, and obstacle approach velocity component, respectively, and the sum of α, β, and γ is 1; the P detect For target identification confidence, the σ depth For depth estimation error, the v approach v is the obstacle approach velocity component. max Maximum speed;

[0039] Substituting the force signal into the multimodal fusion decision model, the corresponding basic allocation probability is determined based on the force signal, wherein the basic allocation probability corresponding to the force signal includes:

[0040] m f (O) = Risk f m f (S)=0.5(1-Risk f ), m f ({O,S})=0.5(1-Risk f );

[0041] Among them, Risk f =max(τ) contact ), τ contact =|τ actual -τ model |

[0042] Where max(τ) contact τ represents the maximum contact force on the sensor. contact The actual feedback torque τ of the current actual And the model calculates the torque τ model The absolute value of the difference between them;

[0043] Substituting the obstacle avoidance path into the multimodal fusion decision model, the corresponding basic allocation probability is determined based on the obstacle avoidance path, wherein the basic allocation probability corresponding to the obstacle avoidance path includes:

[0044]

[0045] in, δ is the path deviation, TTC is the collision time with the nearest obstacle, and δ and θ are the dynamic weights corresponding to the path deviation and the collision time with the nearest obstacle, respectively.

[0046] In an optional embodiment of this application, the step of calculating the corresponding obstacle avoidance priority based on the image signal, the force signal, and the basic allocation probability corresponding to the obstacle avoidance path to obtain the corresponding target obstacle avoidance path includes:

[0047] The collision quality is calculated by substituting the basic allocation probabilities corresponding to the image signal, the force signal, and the obstacle avoidance path into a preset collision formula.

[0048] The conflict formula is: Where mv(B) is the basic allocation probability of the image signal, mf(C) is the basic allocation probability of the force signal, and Mp(D) is the basic allocation probability of the obstacle avoidance path.

[0049] Based on the conflict quality, the image signal, the force signal, and the obstacle avoidance path are substituted into a preset fusion formula for weighted fusion to determine the fusion result;

[0050] The fusion formula is:

[0051] Among them, w v w f w p These are the weighting factors corresponding to the image signal, the force signal, and the obstacle avoidance path, respectively.

[0052] The fusion result is input into a preset priority calculation formula to calculate the obstacle avoidance priority and obtain the corresponding target obstacle avoidance path;

[0053] The priority calculation formula is Priority=10×[m(O)+λ·m({O,S})]; where λ is the penalty factor.

[0054] A third aspect of this application provides an obstacle avoidance device for a robot, the obstacle avoidance device comprising:

[0055] The data acquisition module is used to acquire image signals of the surrounding environment and detect contact force signals;

[0056] The path planning module is used to plan an obstacle avoidance path based on the image signal and the force signal;

[0057] The multimodal fusion module is used to input the image signal, the force signal and the obstacle avoidance path into the multimodal fusion decision model to generate obstacle avoidance priority and obtain the target obstacle avoidance path;

[0058] The control module is used to perform obstacle avoidance operations according to the target obstacle avoidance path.

[0059] A third aspect of this application provides a computer device, including: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods. Attached Figure Description

[0060] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0061] Figure 1 A schematic diagram illustrating the architecture of an obstacle avoidance method for a robot provided in one embodiment of this application.

[0062] Figure 2 A flowchart illustrating a robot obstacle avoidance method provided in one embodiment of this application;

[0063] Figure 3 A flowchart illustrating a robot obstacle avoidance method provided in one embodiment of this application;

[0064] Figure 4 This is a schematic diagram of a sub-process of an obstacle avoidance method for a robot provided in one embodiment of this application;

[0065] Figure 5 This is a schematic diagram of a sub-process of an obstacle avoidance method for a robot provided in one embodiment of this application;

[0066] Figure 6 This is a schematic diagram of a sub-process of an obstacle avoidance method for a robot provided in one embodiment of this application;

[0067] Figure 7 A schematic diagram of the obstacle avoidance device for a robot provided in one embodiment of this application;

[0068] Figure 8 This is a schematic diagram of a computer device structure provided in one embodiment of this application. Detailed Implementation

[0069] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0070] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0071] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0072] It should be noted that the sequence number of each step in the embodiments of this application does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0073] The following is a brief description of the application environment of the obstacle avoidance method for robots provided in the embodiments of this application:

[0074] Please see Figure 1 , Figure 1 This is a schematic diagram of the robot obstacle avoidance method provided in this application embodiment, applied in a specific scenario. As an example, the robot can be an intelligent device equipped with a dual-arm structure. The core feature of this robot is its ability to achieve high-precision operation in complex environments through collaborative work between the two arms. Its obstacle avoidance system architecture includes a multimodal perception layer 10, a data fusion layer 20, a decision-making and planning layer 30, and a dual-arm execution layer 40. It should be noted that the unique characteristics of the dual-arm structure require the obstacle avoidance system to not only avoid collisions between the robot body and the external environment but also to handle motion interference between the two arms in real time. Therefore, a dual-arm motion coupling verification unit is specifically added to the architecture design. By calculating the coupling relationship between the joint angles of the two arms and the pose of the end effector in real time, it ensures that the obstacle avoidance decision simultaneously meets the dual constraints of single-arm obstacle avoidance and dual-arm collaboration.

[0075] The following embodiments use the aforementioned robot as the execution subject, applying the obstacle avoidance method of the robot provided in the embodiments of this application to the aforementioned robot, and providing a specific explanation of the obstacle avoidance method of the robot as an example. Please refer to... Figure 2 , Figure 2 This is a flowchart illustrating an obstacle avoidance method for a robot provided in an embodiment of this application, wherein, as... Figure 2 As shown, the obstacle avoidance method of this robot may include the following steps:

[0076] Step 201: Acquire image signals of the surrounding environment and detect contact force signals.

[0077] The aforementioned image information is two-dimensional / three-dimensional image data of the surrounding environment collected by the robot's visual sensors (such as industrial cameras and depth cameras). This image information may include information such as the location, shape, size, and dynamic changes of obstacles.

[0078] The force signal mentioned above is the force feedback data generated when the robot comes into contact with the outside world, detected by force sensors (such as six-dimensional force sensors and tactile sensors). This force signal is used to sense collisions, the magnitude and direction of contact forces.

[0079] When the robot is performing a task, it can initiate obstacle avoidance. The visual sensors acquire environmental images at a frequency of 50-100Hz, while simultaneously triggering depth calculations (such as through structured light or Time-of-Flight technology) to generate point cloud data containing RGB information and depth values. Force sensors synchronously monitor the force state of each joint of the robot's two arms, with a sampling frequency of up to 1kHz, capturing minute changes in contact force in real time (such as 0.5N-level force feedback when touching an obstacle).

[0080] Step 202: Plan the obstacle avoidance path based on the image signal and force signal.

[0081] The obstacle avoidance path is a motion trajectory generated based on images and force signals to avoid known obstacles. This obstacle avoidance path is usually represented as a series of continuous joint angles or Cartesian space coordinate points.

[0082] After determining the image and force signals, feature extraction can be performed on each signal separately. Specifically, the semantic and geometric features of obstacles in the image signal can be extracted using environmental perception and obstacle recognition algorithms; the gradient change rate of the contact force can be calculated from the force signal to identify potential collision risks. Following this, a weighted fusion strategy is used to map the calculated image and force signals to a unified feature space to obtain the obstacle avoidance path.

[0083] It should be noted that in step 202, after determining the image signal and force signal, an initial obstacle avoidance path is planned based on the image signal and force signal detected by the robot in the current environment.

[0084] Step 203: Input the image signal, force signal and obstacle avoidance path into the multimodal fusion decision model to generate obstacle avoidance priority and obtain the target obstacle avoidance path.

[0085] The multimodal fusion decision model integrates deep learning models with image, force, and path information. It automatically assigns weights to each modality through an attention mechanism and outputs the optimal obstacle avoidance strategy. Obstacle avoidance priority is used to quantitatively assess the danger level of different obstacles. The target obstacle avoidance path is the final execution path that comprehensively considers safety and efficiency; it is generated by the model dynamically adjusting the initial path.

[0086] In step 203, a multimodal fusion decision model is constructed to deeply integrate image signals (visual perception), force signals (tactile feedback), and obstacle avoidance paths (motion planning) to achieve multi-dimensional collaborative reasoning. Within the multimodal fusion decision model, weights are dynamically assigned to each modality. Specifically, when the image signal detects a suspected obstacle but the force signal provides no contact feedback, the model reduces the visual weight (e.g., from 0.6 to 0.3) to avoid misjudgments caused by lighting interference; when the force signal detects abnormal contact force (e.g., >5N) but the image does not identify an obstacle, the model increases the force weight (e.g., from 0.2 to 0.7), prioritizing tactile feedback to address the risk of sudden collisions. This adaptive weight allocation mechanism improves the obstacle recognition accuracy of the multimodal fusion decision model in different scenarios.

[0087] Here, the obstacle avoidance priority output by the model achieves refined decision-making by quantifying the degree of danger (e.g., a range of 0-1). For example, the priority of dynamic obstacles (e.g., AGVs with a speed > 1m / s) is increased to 0.9, the priority of static obstacles (e.g., shelves) is decreased to 0.3, and the priority of impassable areas (e.g., cliff edges) is set to 1.0. Based on this priority, the multimodal fusion decision model can perform secondary optimization on the initial obstacle avoidance path, adopting a safety-first strategy (e.g., detour distance ≥ 0.5m) for high-priority obstacles and an efficiency-first strategy (e.g., critical safety distance 0.1m) for low-priority obstacles, ultimately generating the corresponding target obstacle avoidance path.

[0088] It should be noted that after planning the obstacle avoidance path based on the image signal and force signal in step 202, the calculated obstacle avoidance path can be fused with the image signal and force signal in step 203 to obtain the target obstacle avoidance path. In this way, the secondary fusion mechanism not only compensates for the limitations of single-stage planning but also achieves a dynamic balance between safety and efficiency, enabling the dual-arm robot to have higher obstacle avoidance reliability and operational flexibility in complex and dynamic working environments.

[0089] Step 204: Perform obstacle avoidance operations according to the target obstacle avoidance path.

[0090] After determining the target obstacle avoidance path, it can be converted into angle commands in the robot's joint space. An inverse kinematics algorithm is then used to calculate the target position of each joint motor. Furthermore, a PID controller drives the servo motors to perform trajectory tracking at a sampling frequency of 1kHz, adjusting the motor output torque in real time to compensate for load changes, thus enabling the robot's obstacle avoidance operation.

[0091] In particular, to meet the collaborative needs of dual-arm robots, a master-slave synchronous control mechanism can be activated. In this mechanism, the master arm is responsible for performing the core tasks, while the slave arm adjusts its posture in real time through force feedback to maintain the relative positional error with the master arm, thereby ensuring that the dual-arm robot can complete the relevant tasks.

[0092] During execution, the system continuously monitors environmental changes and execution deviations using multimodal sensors. Specifically, the vision sensor compares the actual obstacle position with the expected position planned for the path in real time, the force sensor detects whether the contact force exceeds the safety threshold (e.g., collision force > 10N), and the IMU sensor monitors the robot's posture stability. Once a deviation is detected (e.g., path offset > 5mm or abnormal contact force), the Kalman filter predicts the deviation's development trend, and the PID parameters are dynamically adjusted for compensation. If the deviation continues to increase, the current path is paused, and the process jumps to step 201 to restart the target obstacle avoidance path planning process.

[0093] In one embodiment, the step of controlling the robot to perform obstacle avoidance operations according to the target obstacle avoidance path includes: generating progressive obstacle avoidance instructions from warning to emergency stop based on a multi-level force threshold and visual confidence joint evaluation model; and controlling the robot to perform corresponding obstacle avoidance actions according to the progressive obstacle avoidance instructions.

[0094] Specifically, the multi-level force threshold is a sequence of force sensor triggering thresholds preset according to obstacle type and work scenario, corresponding to slight contact tendency, potential collision risk, and emergency collision, respectively. Visual confidence is the confidence score (0-100%) of obstacle recognition in the image signal, reflecting the reliability of visual perception.

[0095] Therefore, flexible adjustments are adopted for low-risk scenarios to reduce work interruptions, while decisive emergency stops are adopted for high-risk scenarios to ensure safety. Through the dynamic linkage of multi-level thresholds and confidence levels, the robot's obstacle avoidance response accurately matches the risk level while maximizing the preservation of work continuity, making it particularly suitable for scenarios with high requirements for both flexibility and safety, such as human-robot collaboration and precision assembly.

[0096] This application discloses an obstacle avoidance method for robots. The method involves acquiring image signals from the surrounding environment and detecting contact force signals; planning an obstacle avoidance path based on the image and force signals; inputting the image, force, and obstacle avoidance paths into a multimodal fusion decision model to generate obstacle avoidance priorities and obtain a target obstacle avoidance path; and executing obstacle avoidance operations according to the target path. By employing multi-source data acquisition of image and force signals, the limitations of a single sensor are overcome. This approach achieves comprehensive perception of diverse obstacles, optimizes the target obstacle avoidance path based on priorities, and executes it through closed-loop control. This ensures that the movement trajectories of the two arms do not collide with each other and dynamically adjusts the strategy for different types of obstacles. Ultimately, it achieves high-precision and high-safety obstacle avoidance in complex environments, overcoming the technical shortcomings of single sensors in perceiving diverse obstacles and dynamically coordinating obstacle avoidance.

[0097] Please see Figure 3 , Figure 3 The embodiments of this application are based on Figure 2 A schematic diagram of a sub-process of an obstacle avoidance method for a robot is provided, wherein, as shown in the diagram... Figure 3 As shown, acquiring image information of the surrounding environment includes the following steps:

[0098] Step 301: Use a high-resolution camera to acquire real-time images of the robot's surrounding environment.

[0099] In step 301, the high-resolution camera, as a visual sensor, can capture detailed images of the robot's surrounding environment in real time, including static and dynamic obstacles, providing a high-definition raw data basis for subsequent identification and ensuring that even tiny obstacles can be captured.

[0100] Step 302: Use image processing technology to identify the surrounding environment image to determine the position, size and shape of obstacles in the surrounding environment image and obtain image information.

[0101] In step 302, the acquired image is analyzed using image processing techniques. Specifically, obstacles and background can be distinguished from the image first, then the coordinates, physical size, and geometry of the obstacles can be calculated. Finally, this information is integrated into structured image information, providing specific visual basis for subsequent path planning and obstacle avoidance decisions.

[0102] Please see Figure 4 , Figure 4 The embodiments of this application are based on Figure 2 A sub-process diagram of an obstacle avoidance method for a robot is provided, wherein the force signal may include a first force signal and a second force signal, such as... Figure 4 As shown, acquiring the force signal from the detected contact may include the following steps:

[0103] Step 401: Based on the robot's dynamic model, determine the changes in the robot's joint torque by observing the changes in joint current. When the changes in joint torque exceed a preset threshold, generate a first force signal.

[0104] The first force signal is used to characterize the robot's force-sensing hazard coefficient, that is, the first force signal is used to indicate whether the robot is about to collide.

[0105] In this application, the robot can infer changes in joint torque by monitoring changes in joint current based on a dynamic model: when the change in torque exceeds a preset threshold, a first force signal is generated, which quantifies the force perception risk coefficient and is used to provide early warning of potential force perception abnormalities.

[0106] Step 402: Detect the end contact force using the end six-dimensional force sensor. When the end six-dimensional force sensor detects a collision, it generates a second force signal.

[0107] The aforementioned second force signal is used to characterize the robot's collision basis; that is, the second force signal is used to indicate whether the robot has already collided.

[0108] In step 402, the contact force between the end of the robotic arm and the outside world is directly detected by the six-dimensional force sensor at the end. When a significant collision is detected, a second force signal is generated. This signal serves as a clear basis for collision and is used to trigger a more urgent obstacle avoidance response.

[0109] It should be noted that the force control module constructs a collision detection system for the dual-arm robot through a dual-modal architecture of joint current monitoring and end-effector force sensing. Based on a precise dynamic model, the mapping relationship between joint current and torque is analyzed, and combined with direct measurements from a six-dimensional force sensor at the end-effector, a comprehensive force sensing protection system is formed from the joints to the end effector. Here, the dynamic model equations can be expressed as:

[0110]

[0111] Where M(q) is the positive definite inertia matrix, Let G(q) be the centrifugal force and Coriolis force matrix, and G(q) be the gravity vector. These are the joint angle vector, joint angular velocity vector, and joint angular acceleration vector, respectively, τ. f Let τ be the joint friction torque vector, and τ be the joint driving torque vector.

[0112] Since the equation has a linear parameter relationship, it can be reconstructed as:

[0113]

[0114] After obtaining the parameter set vector P through the system identification method, the model torque τmodel can be established. When a sudden change in joint current causes the deviation Δτ = |τactual-τmodel| from the actual torque τactual to exceed the threshold ∈, a collision warning is triggered. The force perception hazard factor R is defined as:

[0115] R = Δτ / ∈ (when R ≥ 1, it is considered a collision risk)

[0116] In addition, the end-effector six-dimensional force sensor measures three-dimensional forces F = [Fx, Fy, Fz] in Cartesian space. T And the three-dimensional torque M = [Mx, My, Mz] T This provides higher precision collision detection. When the resultant force F = Fx² + Fy² + Fz² or the resultant torque M = Mx² + My² + Mz² detected by the sensor exceeds the safety threshold (such as Fthres, Mthres), a collision signal S is generated.

[0117] S = 1 or 0, where 1 represents F ≥ Fthres or M ≥ Mthres; and 0 represents other cases.

[0118] This signal complements the joint current monitoring, which is good at capturing collisions at the proximal end of the robotic arm, while the six-dimensional force sensor is more sensitive to collisions during fine end-effector manipulation. The combination of the two improves collision detection sensitivity and shortens response time.

[0119] It should be noted that the dual-modal force control system achieves a graded response to collision risk by fusing the force risk coefficient R from joint current monitoring and the collision signal S from a six-dimensional force sensor. The comprehensive risk index I is defined as:

[0120] I = w1·R + w2·S (w1 + w2 = 1)

[0121] Here, w1 and w2 are weighting coefficients. When I ≥ 0.8, an emergency stop is triggered; when 0.5 ≤ I < 0.8, deceleration and obstacle avoidance are performed; and when I < 0.5, normal operation continues. This hierarchical mechanism enables the robot to avoid collision damage and maintain operational continuity in human-robot collaborative scenarios, adapting to various needs.

[0122] Please see Figure 5 , Figure 5 The embodiments of this application are based on Figure 2 A schematic diagram of a sub-process of an obstacle avoidance method for a robot is provided, wherein, as shown in the diagram... Figure 5 As shown, the obstacle avoidance method of this robot may include the following steps:

[0123] Step 501: Add the task target point to the initial path node queue. The task target point is the robot's initial position in the pre-built three-dimensional coordinate system.

[0124] Step 502: Calculate the node cost of the task target point based on the image signal and force signal;

[0125] Step 503: Determine the second task objective point based on the node cost of the task objective point;

[0126] Step 504: Add the second task target point to the initial path node queue;

[0127] Step 505: Replace the task target point with the second task target point, and return to execution. Calculate the node cost of the task target point based on the image signal and force signal until the second task target point is the target position.

[0128] Step 506: Remove redundant target points from the initial path node queue to obtain the optimized path node queue. The redundant target points are nodes to be eliminated, determined based on smooth optimization calculations.

[0129] Step 507: Generate an obstacle avoidance path based on the optimized path node queue.

[0130] Here, after determining the image signal and force signal, the obstacle avoidance path without redundancy is finally generated by dynamically calculating the node cost and iteratively expanding the path nodes.

[0131] In step 501, the task target point is added to the initial path node queue. This task target point is the robot's initial position in the pre-built three-dimensional coordinate system. It should be noted that, firstly, in the pre-established three-dimensional coordinate system (such as the X, Y, and Z axis coordinate system with the robot base as the origin), the robot's initial position is defined as the task target point, and the task target point is added to the preset initial path node queue as the starting point for path planning.

[0132] In step 502, the step of calculating the nodal cost of the task target point based on the image signal and the force signal may include:

[0133] The image signal and force signal are substituted into the preset node cost calculation formula to determine the node cost of the task target point;

[0134] The formula for calculating node cost is as follows:

[0135] Among them, K m =K m +h(s last +s start The result term Km is the heuristic distance of the robot at the current position, the parameter term Km is the heuristic distance of the robot at the previous position, Slast is the robot's previous position information, and Sstart is the robot's current position information.

[0136] Where K is the node cost to be calculated, K1 and K2 are two different key values, which are used to determine the priority of the node cost, and the key values ​​are inversely proportional to the priority.

[0137] g(s) is the actual cost between the current node and the destination, and rhs(s) is the minimum sum of the costs of the current node's adjacent parent nodes and the costs of the parent nodes.

[0138] This step integrates visual and force information to quantitatively evaluate the merits of the current task target point, providing a basis for path planning decisions. Specifically, this application calculates the node cost of the robot's current position by computing image and force signals. After determining the node cost, the merits of the robot's current position relative to the task target point can be judged based on this cost. Specifically, the node cost is an indicator parameter used in path planning to quantify the cost and risk of a single path node (or position point), primarily providing a basis for path search decisions. The node cost integrates multiple dimensions of factors, including the distance from the node to obstacles (the closer the distance, the higher the cost), kinematic constraints (such as increased energy consumption due to excessive turning angles), and environmental dynamics (such as temporary risk increases in densely populated areas), reflecting the merits of the node numerically. During path optimization, the algorithm prioritizes node combinations with lower total costs, thereby shortening path distances, reducing sharp turns to improve stability, and avoiding temporary obstacles to enhance path adaptability while ensuring safety, thus guiding the robot to generate a better motion trajectory.

[0139] It should be noted that the priority of a node is determined by two key values ​​(K1 and K2). The smaller the key value, the higher the priority. First, K1 is checked, and if K1 = K2, then K2 is checked.

[0140] Here, Km is initialized to 0, Slast represents the previous starting point, and Sstart is the robot's current position. Whenever the robot detects a change in the map, it calculates the heuristic distance between the two points (ignoring obstacles) and sets the current point as the new starting point, i.e., updates the starting point's position.

[0141] Furthermore, K1 consists of three terms. Specifically, the first term is the actual distance to the destination, the second term is the estimated distance to the starting point. If Km equals 0 before the robot moves, the algorithm is essentially an A* algorithm that searches in reverse from the destination towards the starting point. K2 is the minimum of the g and rhs values. Its significance lies in the fact that when the first key values ​​of two points are equal, the algorithm will prioritize the point closer to the destination.

[0142] When the robot detects a change in obstacles, it replans its path. The actual starting point at this point should be the robot's current position. Since the starting point has changed, the h value for each point will also change accordingly, and the key value will also change. Therefore, this application introduces Km to ensure key value consistency to a certain extent and reduce computational load.

[0143] It should be noted that this application calculates the node cost of the task target point based on image and force signals by quantifying the comprehensive cost of the current node, providing a basis for determining node priority in path planning. Specifically, the node cost is calculated using the formula mentioned above, where the key values ​​K1 and K2 are inversely proportional to the priority. For example, the smaller the key value, the higher the node priority, and the calculation of key values ​​K1 and K2 is associated with the heuristic distance Km. The cost is the minimum sum of the actual cost from the current node to the destination and the costs of the current node's adjacent parent nodes. This process integrates image and force signals into the cost calculation, and the final node cost can comprehensively reflect the safety, efficiency, and feasibility of the path, guiding the algorithm to prioritize nodes with lower costs (higher priority), ensuring that the robot moves in a better direction in path planning, and balancing obstacle avoidance safety and task execution efficiency.

[0144] In step 503, a second task target point is determined based on the node cost of the current task target point. A better next node (second task target point) is dynamically generated by evaluating the cost of the current node (task target point). The comprehensive cost of the current task target point is calculated based on image signals (such as obstacle position and size) and force signals (such as collision risk). For example, if the point is close to an obstacle (image signal) or has a collision risk (force signal), the cost is high; otherwise, it is low. Multiple candidate nodes are determined among the neighboring nodes of the current task target point, each representing a possible next node. The cost of all candidate nodes is calculated, and the node with the lowest cost is selected as the second task target point. For example, if candidate node A is far from an obstacle and has a low force signal risk, its cost is low and it may be selected; if candidate node B is close to an obstacle or has a collision risk, its cost is high and it will be excluded.

[0145] In step 504, after determining the second task target point, it can be added to the initial path node queue. The initial path node queue is the set of nodes in the path calculation process used to determine the initial path. Starting from the initial task target point (starting point), each time a better second task target point is determined, it is added to the queue, gradually building the node set from the starting point to the final target position (i.e., the initial path node queue). For example, if the initial queue only contains the starting point (0,0,0), after calculating the second task target point (1,0,0), the queue is updated to [(0,0,0),(1,0,0)]. New nodes will continue to be added until the queue contains a complete sequence of nodes leading to the target position, providing the foundation for the final obstacle avoidance path generation.

[0146] In step 505, replacing the task target point with the second task target point and returning to the execution step of calculating the node cost of the task target point based on the image signal and force signal until the second task target point is the target position is the iterative expansion mechanism in path planning. By continuously replacing the current node with newly generated nodes, it gradually moves towards the target position until the destination is reached.

[0147] For example, the currently calculated second task target point (e.g., coordinates (1,0,0)) is replaced with a new task target point, i.e., the current position is updated to (1,0,0). Then, return to step 502, recalculate the node cost based on this point, and generate a new second task target point (e.g., (2,0,0)), repeating this process continuously. The iteration continues until a generated second task target point coincides with (or is close enough to) the preset final target position. For example, if the target position is (10,0,0), when the second task target point generated in a certain iteration is (9.9,0,0), the termination condition is met, and the iteration stops. During the iteration process, each newly generated second task target point is added to the initial path node queue (e.g., from [(0,0,0)]→[(0,0,0),(1,0,0)]→[(0,0,0),(1,0,0),(2,0,0)]), eventually forming a complete node chain from the starting point to the ending point, providing a foundation for subsequent path optimization.

[0148] In step 506, redundant target points are removed from the initial path node queue to obtain an optimized path node queue. The redundant target points are nodes to be eliminated based on smoothness optimization calculations. This step involves simplifying and optimizing the initially generated path node queue by removing redundant nodes to improve path smoothness.

[0149] Specifically, the initial path node queue contains all iteratively generated nodes from the starting point to the ending point. However, some nodes may become redundant due to reasons such as being too close or having the same direction (for example, three consecutive nodes are almost on the same straight line, and the intermediate nodes have no substantial impact on the path direction). Through smoothing optimization calculations (such as Bézier curve fitting, curvature analysis, etc.), these removable redundant target points are identified and removed, ultimately resulting in the optimized path node queue.

[0150] For example, before calculation, the node sequence in the initial queue corresponding to the initial path determined by the robot can be [(0,0,0),(0.5,0,0),(1,0,0),(1.5,0,0),(2,0,0)]. After smoothing calculation, the middle nodes (0.5,0,0) and (1.5,0,0) are determined to be redundant nodes. After removal, the optimized queue [(0,0,0),(1,0,0),(2,0,0)] is obtained, making the subsequently generated path simpler and smoother, reducing unnecessary turning in robot movement, and improving obstacle avoidance efficiency.

[0151] After determining the initial path, it can be optimized. After path generation, Bézier curves can be used for smoothing optimization. For the sequence of control points P0, P1, ..., Pn on the path, the parameterized curve is represented as:

[0152] B(t)=∑i=0nPi·C(n,i)·ti·(1-t)ni,t∈[0,1]

[0153] Where C(n,i) = i! (ni)! n! is the combination number. The control point weights are adjusted by introducing a contact risk coefficient λ(s) from force sensor feedback, ensuring the optimized path maintains curvature continuity while avoiding obstacles. For example, increasing the control point spacing in high-risk areas reduces path curvature, ensuring smooth robot joint movements. The final generated target path satisfies:

[0154] min∫01[∥B′(t)∥2+γ·λ(B(t))]dt

[0155] Here, γ is the risk weighting coefficient, balancing path smoothness and safety. This reduces joint angular velocity fluctuations, significantly improving robot motion efficiency and stability.

[0156] In step 507, an obstacle avoidance path is generated based on the optimized path node queue. This is achieved by converting the optimized node queue into a continuous motion trajectory that the robot can execute.

[0157] Thus, steps 501 to 507 generate a corresponding obstacle avoidance path. This obstacle avoidance path is a continuous, smooth trajectory without collision risk, which can be directly converted into robot joint control commands to guide the robot to move safely from the starting point to the target position, while meeting the requirements of task efficiency and safety.

[0158] Please see Figure 6 , Figure 6 The embodiments of this application are based on Figure 2 A schematic diagram of a sub-process of an obstacle avoidance method for a robot is provided, wherein, as shown in the diagram... Figure 6 As shown, the obstacle avoidance method of this robot may include the following steps:

[0159] Step 601: Calculate the basic allocation probabilities corresponding to the image signal, force signal, and obstacle avoidance path, respectively.

[0160] Step 602: Calculate the corresponding obstacle avoidance priority based on the image signal, force signal and the basic allocation probability of the obstacle avoidance path to obtain the corresponding target obstacle avoidance path.

[0161] In an optional embodiment of this application, before steps 601 and 602, the obstacle avoidance method for the robot provided in this application further includes: defining a multimodal fusion decision model, wherein the multimodal fusion decision model is Θ = {O, S}, O represents that the robot is in an emergency obstacle avoidance state, and S represents that the robot is in a safe state.

[0162] Here, the definition of the multimodal fusion decision model is introduced before steps 601 and 602 to establish a logical framework for subsequent data fusion and priority judgment. This model maps multi-source information from vision, force perception, and path planning into a unified state space by explicitly identifying the framework Θ = {O, S} (where O represents the emergency obstacle avoidance state and S represents the safe state).

[0163] Specifically, heterogeneous data such as obstacle locations detected by force sensors, collision risks reported by force sensors, and safety levels assessed by path planning are transformed into support for O and S, thus solving the problem of inconsistent dimensions of data from different sensors.

[0164] In step 601, the steps of calculating the basic allocation probabilities corresponding to the image signal, force signal, and obstacle avoidance path respectively include the following steps:

[0165] The image signal is substituted into the multimodal fusion decision model, and the corresponding basic allocation probability is determined based on the image signal. The basic allocation probability corresponding to the image signal includes:

[0166] m v (O) = Conf v m v(S) = 1 - Conf v m v ({0, S}) = 0;

[0167] Here, when determining the basic allocation probability corresponding to the image signal, the image signal can directly support O or S at high confidence.

[0168] in, α, β, and γ are the weights corresponding to the target confidence, depth estimation error, and obstacle approach velocity component, respectively, and the sum of α, β, and γ is 1; the P detect For target identification confidence, the σ depth For depth estimation error, the v approach v is the obstacle approach velocity component. max Maximum speed;

[0169] Substituting the force signal into the multimodal fusion decision model, the corresponding basic allocation probability is determined based on the force signal, wherein the basic allocation probability corresponding to the force signal includes:

[0170] m f (O) = Risk f m f (S)=0.5(1-Risk f ), m f ({O, S}) = 0.5(1-Risk) f );

[0171] Here, when determining the basic allocation probability corresponding to the force signal, the force signal can directly support O or S at a high confidence level;

[0172] Among them, Risk f =max(τ) contact ), τ contact =|τ actual -τ model |

[0173] Where max(τ) contact τ represents the maximum contact force on the sensor. contact The actual feedback torque τ of the current actual And the model calculates the torque τ model The absolute value of the difference between them;

[0174] Substituting the obstacle avoidance path into the multimodal fusion decision model, the corresponding basic allocation probability is determined based on the obstacle avoidance path, wherein the basic allocation probability corresponding to the obstacle avoidance path includes:

[0175]

[0176] Here, when determining the basic allocation probability of obstacle avoidance paths, if the urgency of the obstacle avoidance path is high, it can be directly supported as 0, and 10% uncertainty is retained for subsequent calculations.

[0177] in, δ is the path deviation, TTC is the collision time with the nearest obstacle, and δ and θ are the dynamic weights corresponding to the path deviation and the collision time with the nearest obstacle, respectively.

[0178] It should be noted that this application transforms heterogeneous data from vision, force sensing, and path planning into unified evidence support by fusing and calculating the basic allocation probabilities of multimodal data, providing a quantitative basis for subsequent fusion decision-making. Specifically, visual evidence is calculated from image signals, normalized by combining target confidence, depth error, and obstacle approach speed, and the support of vision for emergency obstacle avoidance, safe states, and uncertainties is comprehensively evaluated through weights; force sensing evidence is calculated from force signals, quantifying the support of force sensing for two states based on the ratio of the sensor's maximum external force to torque deviation; path planning evidence is calculated from obstacle avoidance paths, combining path deviation and collision time-to-cruise (TTC), allocating support according to dynamic weights, and retaining the corresponding uncertainties. In this way, multi-source sensor data is transformed into fusionable probability values, achieving a standardized transformation from raw signals to decision evidence, providing allocation probabilities in different dimensions of image signals, force signals, and path planning for the fusion calculation of DS evidence theory, and improving the accuracy and robustness of the robot's obstacle avoidance state judgment.

[0179] In step 602, the step of calculating the corresponding obstacle avoidance priority based on the image signal, force signal, and the basic allocation probability corresponding to the obstacle avoidance path, and obtaining the corresponding target obstacle avoidance path, may include:

[0180] The collision quality is calculated by substituting the basic allocation probabilities corresponding to the image signal, the force signal, and the obstacle avoidance path into a preset collision formula.

[0181] The conflict formula is: Where mv(B) is the basic allocation probability of the image signal, mf(C) is the basic allocation probability of the force signal, and Mp(D) is the basic allocation probability of the obstacle avoidance path.

[0182] Based on the conflict quality, the image signal, the force signal, and the obstacle avoidance path are substituted into a preset fusion formula for weighted fusion to determine the fusion result;

[0183] The fusion formula is:

[0184] Among them, w v w fw p These are the weighting factors corresponding to the image signal, the force signal, and the obstacle avoidance path, respectively.

[0185] The fusion result is input into a preset priority calculation formula to calculate the obstacle avoidance priority and obtain the corresponding target obstacle avoidance path;

[0186] The priority calculation formula is Priority=10×[m(O)+λ·m({O,S})]; where λ is the penalty factor.

[0187] This application improves the reliability and accuracy of decision-making in complex environments by quantifying the conflict level of multi-source information and performing weighted fusion to accurately determine obstacle avoidance priorities and ultimately output the optimal target obstacle avoidance path. Specifically, firstly, based on the basic allocation probabilities of image signals, force signals, and obstacle avoidance paths, the conflict quality *k* among the three is calculated using a conflict formula to quantify the inconsistency between different modalities (e.g., when visual detection is safe but force feedback is risky, the conflict quality increases). Then, based on the conflict quality, the three types of signals are weighted and fused using a fusion formula. During the weighted fusion process, the influence weights of each signal are adjusted using weighting factors to obtain a comprehensive fusion result that balances the impact of conflict information. Finally, the fusion result is substituted into a priority calculation formula containing a penalty factor to output the obstacle avoidance priority. This priority directly corresponds to the urgent obstacle avoidance needs of the path (e.g., higher priority corresponds to higher risk and paths that need to be avoided first), ultimately selecting the target obstacle avoidance path that matches the current priority. This solves the potential contradictions in multimodal data and, through weighted fusion and priority quantification, transforms abstract sensor information into clear path selection criteria, ensuring that the robot can select the safest and most efficient motion path based on the real-time risk level in dynamic environments.

[0188] Therefore, the priority quantification rule resolves the contradiction between fuzzy judgment and over-response in traditional obstacle avoidance decision-making by transforming multimodal fusion results into directly executable hierarchical instructions. On the one hand, the quantification output based on DS evidence theory clearly corresponds to high priority through rules, avoiding subjective errors from manually setting thresholds and improving decision consistency. On the other hand, the hierarchical response mechanism achieves a dynamic balance between safety and efficiency, avoiding efficiency losses caused by excessive obstacle avoidance.

[0189] It should be understood that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order constraint on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the diagram may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0190] Please see Figure 7 One embodiment of this application provides an obstacle avoidance device 70 for a robot, which includes:

[0191] The data acquisition module 701 is used to acquire image signals of the surrounding environment and detect contact force signals when the robot is performing a task.

[0192] Path planning module 702 is used to plan an obstacle avoidance path based on the image signal and the force signal;

[0193] The multimodal fusion module 703 is used to construct a multimodal fusion decision model based on the image signal, the force signal and the obstacle avoidance path to generate obstacle avoidance priority and obtain the target obstacle avoidance path;

[0194] The control module 704 is used to control the robot to perform obstacle avoidance operations according to the target obstacle avoidance path.

[0195] For specific limitations regarding the obstacle avoidance device of the robot, please refer to the limitations on the obstacle avoidance device method of the robot mentioned above, which will not be repeated here. Each module in the obstacle avoidance device of the robot can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0196] In one embodiment, a computer device is provided, the internal structure of which can be as follows: Figure 8As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data. The network interface communicates with external terminals via a network connection. When the processor executes the computer program, it implements the obstacle avoidance method for a robot described above. It includes: a memory and a processor; the memory stores a computer program; and the processor executes the computer program to implement any step of the obstacle avoidance method for the robot described above.

[0197] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0198] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0199] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0200] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0201] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0202] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An obstacle avoidance method for a robot, characterized in that, The obstacle avoidance method for the robot is applied to the robot, and the method includes: Acquire image signals of the surrounding environment and detect contact force signals; Plan an obstacle avoidance path based on the image signal and the force signal; The image signal, the force signal, and the obstacle avoidance path are input into a multimodal fusion decision model to generate an obstacle avoidance priority and obtain the target obstacle avoidance path. Perform obstacle avoidance operations according to the target obstacle avoidance path; The step of inputting the image signal, the force signal, and the obstacle avoidance path into a multimodal fusion decision model to generate an obstacle avoidance priority and obtain the target obstacle avoidance path further includes: Define a multimodal fusion decision model, wherein the multimodal fusion decision model is as follows: O indicates that the robot is in an emergency obstacle avoidance state, and S indicates that the robot is in a safe state; Calculate the basic allocation probabilities corresponding to the image signal, the force signal, and the obstacle avoidance path, respectively; The corresponding obstacle avoidance priority is calculated based on the image signal, the force signal and the basic allocation probability corresponding to the obstacle avoidance path, and the corresponding target obstacle avoidance path is obtained. The steps of calculating the basic allocation probabilities corresponding to the image signal, the force signal, and the obstacle avoidance path respectively include: The image signal is substituted into the multimodal fusion decision model, and the corresponding basic allocation probability is determined based on the image signal. The basic allocation probability corresponding to the image signal includes: 、 、 ； in, , , , These are the weights corresponding to the target confidence level, depth estimation error, and obstacle approach velocity component, respectively. , , The sum of P is 1; detect For the target confidence level, the For depth estimation error, the v approach v is the obstacle approach velocity component. max Maximum speed; Substituting the force signal into the multimodal fusion decision model, the corresponding basic allocation probability is determined based on the force signal, wherein the basic allocation probability corresponding to the force signal includes: 、 、 ； in, , , The maximum contact force on the sensor, Actual feedback torque of the current and model calculation torque The absolute value of the difference between them; Substituting the obstacle avoidance path into the multimodal fusion decision model, the corresponding basic allocation probability is determined based on the obstacle avoidance path, wherein the basic allocation probability corresponding to the obstacle avoidance path includes: 、 、 ； in, , It is path deviation, and TTC is the time to collision with the nearest obstacle. , These are the dynamic weights corresponding to path deviation and the collision time of the nearest collider, respectively. The step of calculating the corresponding obstacle avoidance priority based on the image signal, the force signal, and the basic allocation probability corresponding to the obstacle avoidance path to obtain the corresponding target obstacle avoidance path includes: The collision quality is calculated by substituting the basic allocation probabilities corresponding to the image signal, the force signal, and the obstacle avoidance path into a preset collision formula. The conflict formula is: ; where m v (B) represents the basic assignment probability of the image signal, m f (C) represents the basic assignment probability of the force signal, m p (D) represents the basic allocation probability of the obstacle avoidance path; Based on the conflict quality, the image signal, the force signal, and the obstacle avoidance path are substituted into a preset fusion formula for weighted fusion to determine the fusion result; The fusion formula is: ; Among them, w v w f w p These are the weighting factors corresponding to the image signal, the force signal, and the obstacle avoidance path, respectively. The fusion result is input into a preset priority calculation formula to calculate the obstacle avoidance priority and obtain the corresponding target obstacle avoidance path; The priority calculation formula is as follows: ;in, This is a penalty factor.

2. The obstacle avoidance method for a robot according to claim 1, characterized in that, The step of acquiring image signals of the surrounding environment includes: A high-resolution camera is used to acquire real-time images of the robot's surrounding environment; Image processing technology is used to identify the surrounding environment image to determine the position, size, and shape of obstacles in the surrounding environment image, thereby obtaining image information.

3. The obstacle avoidance method for a robot according to claim 1, characterized in that, The force signal includes a first force signal and a second force signal. The step of acquiring the force signal of the detected contact includes: Based on the robot's dynamic model, the change in joint torque of the robot is determined by observing the change in joint current. When the change in joint torque exceeds a preset threshold, a first force signal is generated. The first force signal is used to characterize the robot's force perception risk factor. The robot detects end-effector contact force using a six-dimensional force sensor. When the six-dimensional force sensor detects a collision, it generates a second force signal, which is used to characterize the collision basis of the robot.

4. The obstacle avoidance method for a robot according to claim 1, characterized in that, The step of planning an obstacle avoidance path based on the image signal and the force signal includes: Add the task target point to the initial path node queue. The task target point is the initial position of the robot in the pre-constructed three-dimensional coordinate system. Calculate the node cost of the task target point based on the image signal and the force signal; The second task target point is determined based on the node cost of the aforementioned task target point; Add the second task target point to the initial path node queue; Replace the task target point with the second task target point, and return to execute the step of calculating the node cost of the task target point based on the image signal and the force signal until the second task target point is the target position; Redundant target points are removed from the initial path node queue to obtain an optimized path node queue, wherein the redundant target points are nodes to be eliminated determined based on smooth optimization calculations. Based on the optimized path node queue, an obstacle avoidance path is generated.

5. The obstacle avoidance method for a robot according to claim 4, characterized in that, The step of calculating the node cost of the task target point based on the image signal and the force signal includes: The image signal and the force signal are substituted into a preset node cost calculation formula to determine the node cost of the task target point; The formula for calculating the node cost is as follows: ; in, Result item K m The heuristic distance of the robot at its current position is given by parameter K. m S represents the heuristic distance of the robot at the previous position. last For the robot's previous position information, S start This refers to the robot's current location information; Wherein, K is the node cost to be calculated, and K1 and K2 are two different key values. The key values ​​are used to determine the priority of the node cost, and the key values ​​are inversely proportional to the priority. g(s) is the actual cost between the current node and the destination, and rhs(s) is the minimum sum of the costs of the current node's adjacent parent nodes and the costs of the parent nodes.

6. An obstacle avoidance device for a robot, used to implement the method according to any one of claims 1 to 5, characterized in that, The obstacle avoidance device of the robot includes: The data acquisition module is used to acquire image signals of the surrounding environment and detect contact force signals; The path planning module is used to plan an obstacle avoidance path based on the image signal and the force signal; The multimodal fusion module is used to input the image signal, the force signal and the obstacle avoidance path into the multimodal fusion decision model to generate obstacle avoidance priority and obtain the target obstacle avoidance path; The control module is used to perform obstacle avoidance operations according to the target obstacle avoidance path.

7. A computer device, comprising: The method includes a memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

Images