A vision-based method for robot blind spot risk prediction

Abstract

The application discloses a vision-based robot blind area risk prediction method for reducing the risk of contact or impact caused by blind area attacks on robots. The method comprises the following steps: acquiring visual information of the robot, determining whether a target part is in a blind area state according to a visibility state and a duration of the visibility state; if yes, inputting historical motion information, current attitude information, a risk area and a preset motion constraint condition into a pre-trained target prediction model to obtain a plurality of predicted motion trajectories output by the target prediction model and a trajectory weight corresponding to each predicted motion trajectory; performing spatial expansion on each predicted motion trajectory to obtain an accessible attack space corresponding to each predicted motion trajectory; determining an overlap amount between each accessible attack space and the risk area to perform weighted calculation and obtain an attack risk of the target part; and comparing the attack risk with a preset risk condition to determine a defensive operation according to a comparison result.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a vision-based method for predicting robot blind spot risks. Background Technology

[0002] With the development of robot motion control and visual perception technologies, robots are increasingly being applied to scenarios requiring offensive and defensive interactions, such as adversarial training and competitive demonstrations. During adversarial training, robots can use visual sensors to capture images of their opponents and determine the opponent's attack actions based on the position and movement of limbs in the images.

[0003] However, when the two sides are close together or their limbs are rapidly intersecting, the opponent's fist can easily be obstructed by the torso, upper limbs, or the robot's own limbs, or may even move out of the effective field of view of the visual sensors. This prevents the robot from continuously acquiring the position and movement of the opponent's fist. Once the fist becomes invisible, the robot struggles to promptly judge subsequent changes in the opponent's attack, leading to delayed defensive responses or a mismatch between defensive actions and the actual attack direction, thus affecting the robot's defensive safety during combat. Summary of the Invention

[0004] This application provides a vision-based method for predicting robot blind spot risks, which can reduce the risk of blind spot attacks causing contact or impact to the robot.

[0005] The first aspect of this application provides a vision-based method for predicting blind spot risks in robots, including:

[0006] The robot acquires visual information, determines the visibility status of the target part based on the visual information, and identifies the robot's risk area. Based on the visibility status and the duration of the visibility status, determine whether the target area is in a blind spot. If so, then obtain the historical motion information of the target part before entering the blind zone state and the current posture information of the other party, input the historical motion information, the current posture information, the risk area and the preset motion constraints into the pre-trained target prediction model, and obtain multiple predicted motion trajectories output by the target prediction model and the trajectory weights corresponding to each predicted motion trajectory; Based on the spatial dimensions of the target location and the prediction error range that varies with the prediction time interval, the spatial expansion of each predicted motion trajectory is performed to obtain the reachable attack space corresponding to each predicted motion trajectory; The overlap between each of the reachable attack spaces and the risk areas is determined, and the corresponding overlap is weighted according to the trajectory weight to obtain the attack risk of the target location. The attack risk is compared with preset risk conditions, defensive actions are determined based on the comparison results, and the robot is controlled to execute the defensive actions.

[0007] A second aspect of this application provides a storage medium storing a program that, when executed on a computer, performs the first aspect and any optional method of vision-based robot blind spot risk prediction.

[0008] As can be seen from the above technical solutions, this application has the following advantages: By using the robot's visual information to determine the visibility state of the target part and the robot's risk area, and combining the duration of the visibility state to identify when the target part enters the blind zone, the robot can make timely blind zone risk predictions when real-time observation information of the target part is lacking. By inputting the historical motion information of the target part before entering the blind zone, the current posture information of the target, the risk area, and preset motion constraints into a pre-trained target prediction model, multiple predicted motion trajectories and corresponding trajectory weights are generated. This can cover multiple possible motion directions of the target part in the blind zone state, and the feasibility of the predicted motion trajectory is improved by using motion constraints. Furthermore, by combining the spatial size of the target part and the prediction error range that changes with the prediction time interval, the spatial expansion of each predicted motion trajectory is performed, converting the predicted position of the target part into an accessible attack space that includes the entity size and prediction uncertainty. Then, the overlap between each accessible attack space and the robot's risk area is calculated, and weighted according to the corresponding trajectory weights, which can comprehensively evaluate the attack risk posed to the robot by different possible motion trajectories. Finally, by comparing the attack risk with the preset risk conditions, defensive operations are determined, enabling the robot to perform defensive operations in advance that are appropriate for the attack risk while the target part is continuously invisible. This allows the robot to protect risk areas such as the head in a timely manner even when visual information is incomplete, reducing the risk of contact or impact to the robot caused by blind spot attacks. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic diagram of the overall structure of a humanoid robot provided as an example of a robot in this application; Figure 2 A schematic flowchart of an embodiment of the vision-based robot blind spot risk prediction method provided in this application; Figure 3 A schematic diagram of the component architecture of the target prediction model provided in this application; Figure 4 A schematic flowchart of an embodiment of the training method for the target prediction model provided in this application; Figure 5 A schematic diagram of an embodiment for determining the reachable attack space provided in this application; Figure 6 A schematic flowchart of an embodiment of the determination of defensive operations provided in this application; Figure 7 This is a schematic flowchart of another embodiment of the defensive operation provided in this application. Detailed Implementation

[0011] This application provides a vision-based method for predicting robot blind spot risks, which can reduce the risk of blind spot attacks causing contact or impact to the robot.

[0012] The robot provided in this application includes a robot body, a head, a binocular camera, at least one processor, and at least one storage medium. The robot body may include a torso and at least one movable limb connected to the torso. The head is connected to the torso, and the binocular camera is mounted on the head. The binocular camera can move with the head to change the image acquisition direction and range. The movable limb may include a robotic arm, legs, or other motion mechanisms capable of performing defensive maneuvers. The robot may be a humanoid robot, a legged robot, a wheeled robot, or other robots with offensive and defensive capabilities. For ease of description, the following description primarily uses a robot with a humanoid structure as an example. Please refer to... Figure 1 The head 1 is connected to the torso 2, which includes an upper torso 21 and hips 22. The upper torso 21 and hips 22 are rotatably connected via a power module. Two upper limbs 3 are connected to the upper torso 21, and two lower limbs 4 are connected to the hips 22. A binocular camera is located inside the head 1. It should be noted that in the blind spot risk prediction and defense control process of this embodiment, the binocular camera is the only external sensing device for the robot to obtain the opponent's motion state, and the sensing information provided by the site camera, external motion capture device, or other external detection device is not considered.

[0013] Please see Figure 2 , Figure 2 An embodiment of the vision-based robot blind spot risk prediction method provided in this application includes: 201. Obtain the robot's visual information, determine the visibility status of the target part based on the visual information, and identify the robot's risk area.

[0014] During the robot's combat mission, visual information is continuously acquired by binocular cameras mounted on the robot's head. This visual information includes images simultaneously captured by the left and right cameras, or image sequences composed of multiple consecutive frames of binocular images. By acquiring continuous visual information, the robot can determine the movement of the opponent's target body part. The target body part refers to a body part or moving component that the opponent can approach, contact, or impact the robot, potentially causing it to be attacked. For example, in a boxing scenario, the target body part could be the opponent's left or right fist, or the hand area including the palm and gloves; in a combat scenario involving kicking motions, the target body part could also be the foot, knee, or the end of the leg; in this embodiment, the opponent's fist is primarily used as an example for illustration.

[0015] The robot can detect and spatially locate target parts in visual information, and determine the visibility status of the target part based on whether it can be effectively detected in the visual information and whether a reliable spatial position can be obtained. When the target part can be identified from the binocular images and a reliable spatial position can be obtained through the correspondence between the left and right eye images, the target part is determined to be in a visible state; when the target part is occluded by the other party's body, the robot's own limbs, or other objects, moves out of the image acquisition range of the binocular camera, or a reliable spatial position cannot be obtained, the target part is determined to be in an invisible state.

[0016] The risk zone of a robot refers to the spatial area on the robot that may be attacked by a target part or needs to be prioritized for protection, such as the robot's head area. The risk zone can be determined based on the robot's structural dimensions, current pose, and the location of the part to be protected, and can be updated as the robot's posture changes. Specifically, a base region corresponding to the part to be protected can be pre-set in the robot's coordinate system. Based on the robot's current joint state and the connection relationships between various components, the base region is transformed to its current spatial position, thus obtaining the risk zone. The risk zone and the spatial position of the target part obtained from visual information use the same coordinate reference to facilitate subsequent determination of the spatial relationship between the possible range of motion of the target part and the risk zone.

[0017] 202. Determine whether the target area is in a blind spot based on the visibility status and the duration of the visibility status.

[0018] After acquiring the visibility status of a target area, the robot continuously records this visibility status and determines the duration of its visibility. Since image jitter, lighting changes, or single-frame detection anomalies can cause a target area to become temporarily invisible, combining the duration of the visibility status with the determination of its visibility reduces the probability of triggering blind zone risk prediction due to occasional missed detections of the target area. Specifically, when a target area is invisible and the duration of this invisibility meets a preset blind zone condition, the target area is determined to be in a blind zone. The preset blind zone condition can be set based on the image acquisition frequency of the binocular camera, the movement speed of the target area, the response time required for the robot to perform defensive operations, and the actual combat scenario. When the target area remains visible, or when the duration of the target area's invisibility has not yet met the preset blind zone condition, the robot can continue to directly detect and track the target area based on the visual information acquired by the binocular camera.

[0019] Once the target part enters the blind zone, the robot can no longer obtain a reliable spatial location of the target part from the current visual information, causing the continuous movement trajectory of the target part to be interrupted. During the period of invisibility, the target part may continue its original movement direction, change its movement direction, move towards the robot's risk area, or launch another attack after being retrieved. As the invisibility time increases, the deviation between the actual position of the target part and the last reliable observation position may gradually increase, making it impossible for the robot to accurately determine the attack direction and arrival position of the target part based on the last observation result. If the robot continues to wait for the target part to become visible again, the target part may have already approached or entered the robot's risk area, thereby shortening the time available for the robot to perform defensive operations and increasing the risk of the robot being contacted or impacted. Based on this, when it is determined that the target part is in the blind zone, step 203 is executed, using the historical movement information of the target part before entering the blind zone, the current posture information of the target, and preset movement constraints to predict the possible movement trajectory of the target part in the blind zone. When it is determined that the target part is not in the blind zone, the target part continues to be detected and tracked based on visual information, and the newly obtained movement information is used to update the historical movement information.

[0020] 203. Obtain the historical motion information of the target part before it enters the blind zone and the current posture information of the target. Input the historical motion information, current posture information, risk area and preset motion constraints into the pre-trained target prediction model to obtain multiple predicted motion trajectories and the trajectory weights corresponding to each predicted motion trajectory output by the target prediction model.

[0021] After determining that the target area is in a blind zone, the robot retrieves historical motion information of the target area from the saved target area tracking results before it entered the blind zone. This historical motion information reflects the positional changes and movement trends of the target area before it lost effective observation, enabling the target prediction model to extract information related to subsequent movements from the movement already occurring at the target area. Historical motion information can originate from continuous observations of the target area while it was visible, and is input into the target prediction model as motion data arranged in chronological order. Since the target area may not be directly detectable after entering the blind zone, the robot also retrieves the still-visible body parts of the target area based on current visual information and determines the target area's current posture information based on the spatial position and relative relationships of these body parts. This current posture information reflects the motion correlation between the target area and other body parts of the target area, allowing the target prediction model to combine the overall posture of the target area to determine the movements the target area may continue to perform. Therefore, even if the target area itself is temporarily invisible, the robot can still predict the target area using information provided by the visible body parts of the target area. The risk area is used to provide the relative positional relationship between the target area and the space the robot needs to protect. Inputting the risk area into the target prediction model allows the model to consider different possible movements of the target when generating the predicted trajectory, such as continuing along the original direction of movement, approaching the risk area, or moving away from the risk area. Preset motion constraints limit the spatial range and range of motion changes that the target can reach, thereby reducing the probability of predicted positions in the trajectory that exceed the target's actual movement capabilities.

[0022] The target prediction model is a pre-trained neural network model deployed in the robot. During step 203, the target prediction model infers the motion state of the target part in the subsequent prediction time domain based on the correlation between historical motion information, current posture information, risk areas, and preset motion constraints, outputting multiple predicted motion trajectories. These multiple predicted motion trajectories correspond to different potential motion paths of the target part in the blind zone state. Each predicted motion trajectory includes the predicted position of the target part at multiple prediction times, representing a possible motion process of the target part in the blind zone state. Multiple predicted motion trajectories can cover different motion paths of the target part, such as maintaining its current motion trend, changing its motion direction, approaching the risk area, or moving away from the risk area, thereby reducing the possibility of missing potential attack paths when only a single prediction result is used. For example, during a robot's combat task, the motion of the target part after entering the blind zone state is usually highly uncertain. Since the opponent can adjust their attack actions according to the current confrontation state—for example, maintaining the original punching direction, changing the attack direction through shoulder rotation and elbow joint adjustment, or forming a new attack path through action switching—the subsequent motion of the target part may correspond to multiple different motion results. If only a single predicted trajectory is output based on historical movement trends, potential attack paths may be missed when the actual movement of the target deviates from the predicted trajectory, preventing the robot from timely assessing the attack risk during the blind zone. Therefore, by outputting multiple predicted trajectories through a target prediction model, each predicted trajectory corresponds to a different possible movement of the target part in the blind zone, thus utilizing multiple prediction results to collectively cover the uncertainty of the target part's future movement. By retaining multiple possible future movements, even if the actual movement path of the target part deviates from one of the predicted trajectories, other predicted trajectories can still provide corresponding risk information, thereby reducing the possibility of missed attack risks due to a single prediction result.

[0023] Furthermore, the trajectory weights output by the target prediction model can further reflect the degree of matching between different predicted motion trajectories and the current adversarial state, enabling subsequent attack risk calculations to comprehensively assess the reliability of the prediction results. Predicted motion trajectories that better match historical motion information, current posture information, and motion constraints can be assigned higher trajectory weights, giving them a greater impact on risk assessment results; predicted motion trajectories with lower reliability but still possessing potential attack possibilities retain their corresponding risk contribution. Thus, in complex situations such as close-range occlusion, rapid changes of direction, and shifts in attack strategies during combat, the robot can assess multiple potential attack directions in advance without relying on a single future motion assumption, improving risk perception capabilities and the reliability of defensive decisions in blind spots.

[0024] 204. Based on the spatial dimensions of the target location and the prediction error range that varies with the prediction time interval, each predicted motion trajectory is spatially expanded to obtain the reachable attack space corresponding to each predicted motion trajectory.

[0025] The predicted motion trajectory output by the target prediction model describes the predicted positions of the target part at multiple prediction times. These predicted positions can be considered as reference positions for the target part. Since the target part has actual spatial dimensions, even if the reference position of the target part does not enter the robot's risk area, the edge of the target part may still come into contact with the risk area. Therefore, the predicted motion trajectory can be spatially expanded based on the spatial dimensions of the target part, taking into account the space actually occupied by the target part around the reference position. The spatial dimensions of the target part can be determined based on its structural parameters, pre-set dimensional parameters, or the dimensions obtained from visual information. For example, when the target part is a fist, the spatial dimensions can correspond to the space occupied by the fist and its external protective structure. By incorporating the spatial dimensions of the target part, the spatial expansion result can cover the physical space that the target part may occupy at each predicted position.

[0026] Predicting motion trajectories also suffers from prediction biases caused by visual measurement errors, historical motion information errors, and attitude information errors, and these biases typically increase with the prediction time interval. Therefore, for the predicted positions at different prediction times within the predicted motion trajectory, a spatial expansion is performed using a prediction error range corresponding to the respective prediction time interval. This results in smaller expansion ranges for predicted positions closer to the current time and larger expansion ranges for predicted positions farther from the current time. In other words, when the prediction time interval is short, the predicted position of the target is usually more accurate, so a smaller prediction error range is used to avoid excessive expansion of the reachable attack space; when the prediction time interval is long, the motion changes of the target have greater uncertainty, so a larger prediction error range is used to cover positions where the target may deviate from the predicted motion trajectory.

[0027] For each predicted motion trajectory, the robot expands the space around the predicted trajectory that may be occupied by the target part, based on the spatial dimensions of the target part and the prediction error range at each prediction time. The spatial expansion results formed by the same predicted motion trajectory within the prediction time domain are then combined into a corresponding reachable attack space. The reachable attack space represents the spatial range that the target part can reach or cover when moving along the corresponding predicted motion trajectory, considering the target part's physical dimensions and prediction deviations. Different predicted motion trajectories correspond to different reachable attack spaces, thus preserving the correspondence between the possible predicted motions.

[0028] 205. Determine the overlap between each reachable attack space and the risk area, and calculate the corresponding overlap based on the trajectory weight to obtain the attack risk of the target location.

[0029] After obtaining multiple reachable attack spaces, each reachable attack space and the robot's risk area are transformed to the same coordinate system, and the spatial overlap between each reachable attack space and the risk area is determined. Since each reachable attack space corresponds to a predicted motion trajectory of the target part, the overlap between each reachable attack space and the risk area can be used to reflect the degree to which the target part may come into contact with the robot's risk area when moving according to the corresponding predicted motion trajectory. For each reachable attack space, the spatial range jointly covered by the reachable attack space and the risk area is determined, and the corresponding overlap amount is determined based on the jointly covered spatial range. When the reachable attack space does not cover the risk area, the corresponding overlap amount can be zero; when the reachable attack space covers part of the space in the risk area, the overlap amount is determined to be greater than zero based on the coverage range; the larger the coverage range of the reachable attack space over the risk area, the larger the corresponding overlap amount. Through the overlap amount, the spatial relationship between the reachable attack space and the risk area can be converted into risk data that can be compared and calculated.

[0030] Each reachable attack space corresponds to a predicted motion trajectory with a specific trajectory weight. The trajectory weight reflects the reliability of the predicted trajectory given the current motion and attitude information. Therefore, when determining attack risk, the overlap amount corresponding to each reachable attack space is associated with its corresponding trajectory weight. This ensures that the overlap amount of predicted motion trajectories with higher reliability has a greater impact on attack risk, while retaining the potential risks associated with predicted motion trajectories with lower reliability. Specifically, the attack risk of the target location can be obtained by calculating the product of each trajectory weight and its corresponding overlap amount, and then summing these products.

[0031] 206. Compare the attack risk with the preset risk conditions, determine the defensive operation based on the comparison result, and control the robot to execute the defensive operation.

[0032] Preset risk conditions are used to establish the correspondence between attack risks and robot defensive responses. These conditions can be preset based on the robot's structural dimensions, motion capabilities, control response time, and the safety requirements of the adversarial task, and stored in the robot's storage medium. After the processor obtains the attack risk determined in step 205, it compares the attack risk with the preset risk conditions to determine the degree of threat posed by the target part to the robot in the current blind zone state, and selects an appropriate defensive operation based on the degree of threat.

[0033] Defensive maneuvers are used to reduce the likelihood of contact or impact between the target area and the robot, or to mitigate the impact of contact when the target area does come into contact with the robot. When the attack risk is low, the robot can maintain its current motion or make minor adjustments to its current action. As the attack risk increases, the robot can enhance the responsiveness of its defensive maneuvers to increase the distance between the robot's risk area and the reachable attack space, or use movable limbs to protect the risk area. After determining a defensive maneuver, the processor generates corresponding motion control commands based on the maneuver, causing the robot body or movable limbs to reach the motion state corresponding to the defensive maneuver, thereby completing the defensive operation.

[0034] During defensive maneuvers, the robot continues to acquire visual information and updates the visibility status of the target area, historical motion information, and the robot's current risk zone based on the newly acquired visual information. When the target area remains in a blind spot, the robot can re-execute predicted trajectory generation, reachable attack space construction, and attack risk calculation based on the updated information to adjust defensive operations. When the target area becomes visible again, the robot can continue tracking the target area based on the newly acquired motion information. Thus, the robot can continuously update risk assessment and defensive control results while the target area is in a blind spot.

[0035] In this embodiment, the visibility state of the target part and the robot's risk area are determined by the robot's visual information. The duration of the visibility state is used to identify when the target part enters the blind zone, enabling the robot to predict blind zone risks in a timely manner when real-time observation information of the target part is lacking. By inputting the historical motion information of the target part before entering the blind zone, the current posture information of the target, the risk area, and preset motion constraints into a pre-trained target prediction model, multiple predicted motion trajectories and corresponding trajectory weights are generated. This can cover multiple possible motion directions of the target part in the blind zone state, and the feasibility of predicting motion trajectories is improved by using motion constraints. Furthermore, the spatial dimensions of the target part and the prediction error range that changes with the prediction time interval are combined to spatially expand each predicted motion trajectory, converting the predicted position of the target part into an accessible attack space that includes the entity size and prediction uncertainty. The overlap between each accessible attack space and the robot's risk area is then calculated and weighted according to the corresponding trajectory weights, which can comprehensively evaluate the attack risks posed to the robot by different possible motion trajectories. Finally, by comparing the attack risk with the preset risk conditions, defensive operations are determined, enabling the robot to perform defensive operations in advance that are appropriate for the attack risk while the target part is continuously invisible. This allows the robot to protect risk areas such as the head in a timely manner even when visual information is incomplete, reducing the risk of contact or impact to the robot caused by blind spot attacks.

[0036] The composition and training process of the target prediction model provided in this application are described below.

[0037] Please see Figure 3 , Figure 3 This is a schematic diagram of the component architecture of the target prediction model provided in this application. In some embodiments, the target prediction model includes a historical motion encoding module 301, a posture encoding module 302, a trajectory generation module 303, and a trajectory weight determination module 304. The historical motion information includes the three-dimensional position sequence, motion speed, and motion direction of the target part before it enters the blind zone state. The current posture information includes the spatial position of the body part currently visible to the target. The preset motion constraints include at least one of limb link length, joint range of motion, and maximum motion speed. The historical motion encoding module 301 is used to generate historical motion features based on the historical motion information, and the posture encoding module 302 is used to generate posture features based on the current posture information. The trajectory generation module 303 is used to determine the kinematic reachability of the target part based on the historical motion features, posture features, risk area, and preset motion constraints, and to generate multiple predicted motion trajectories within the kinematic reachability. The trajectory weight determination module 304 is used to determine the trajectory weight corresponding to each predicted motion trajectory based on the degree of consistency between each predicted motion trajectory and the historical motion information, current posture information, and preset motion constraints.

[0038] In this embodiment, for historical motion information, the last reliable 3D position of the target part before entering the blind zone is used as the reference position. Each 3D position in the historical 3D position sequence is converted into a relative position relative to the reference position, and the relative position, motion speed, and motion direction are arranged according to the acquisition time to obtain the historical motion input sequence. By using relative positions, the impact of changes in the absolute position of the robot and the opponent in the field on the trajectory prediction results can be reduced. For current posture information, the spatial position of currently visible body parts can be converted to the same coordinate system as the historical 3D position sequence, and organized into posture input data according to the connection relationships between body parts. For temporarily invisible body parts, corresponding availability status flags can be set in the posture input data, enabling the target prediction model to distinguish between effective and missing spatial positions.

[0039] The historical motion encoding module 301 can use a recurrent neural network, a temporal convolutional network, or an attention network to process the historical motion input sequence to obtain the historical motion features h. hist The pose encoding module 302 can use a multilayer perceptron, graph neural network, or other networks capable of handling spatial connections between body parts to process the pose input data and obtain pose features h. poseRisk areas can be described by their center location, spatial extent, or boundary sampling points. Preset motion constraints can be converted into constraint parameters corresponding to limb size and motor ability. Historical motion features h... hist Posture characteristics h pose Information about the risk areas (z) risk The constraint information z corresponding to the preset motion constraint conditions con By combining these features, we can obtain the fusion feature h used for trajectory prediction. The fusion feature h can be expressed as: h = F(h hist h pose , z risk , z con ); where F represents feature concatenation, weighted fusion, or feature fusion processing implemented by a neural network.

[0040] Assuming the prediction time domain includes N prediction times, the trajectory generation module 303 generates K predicted motion trajectories P1, P2, ..., P3 based on the fusion feature h. k The k-th predicted trajectory can be represented as P k ={p k，0 p k，1 , ..., p k，N}, where p k，0 p is the last reliable 3D position of the target area before it enters the blind zone. k，n This is the predicted position of the k-th predicted trajectory at the nth prediction time. The trajectory generation module 303 can output the predicted velocity at each prediction time and gradually determine the predicted position based on the predicted velocity. For example, the initial predicted position at the nth prediction time can be represented as: p k，n =p k，n-1 +Δt·v k，n ; Where Δt is the time interval between adjacent prediction times, v k，n The initial predicted velocity is output by the trajectory generation module 303.

[0041] To ensure that the predicted motion trajectory meets the preset motion constraints, the kinematic reachable set Q corresponding to each prediction moment can be determined based on the limb link length, joint range of motion, and maximum motion velocity. n The initial predicted position is then projected onto the kinematically reachable set Q. n ,get: p k，n =Π Qn (p k，n ); Among them, Π Qn (·) indicates that the initial predicted position is mapped to the kinematically reachable set Q. n Projection operation within the kinematically reachable set Q. When the initial predicted position is located within the kinematically reachable set Q. n When the initial predicted position is within the range, it can be retained; when the initial predicted position exceeds the kinematically reachable set Q... n At that time, it can be along the kinematically reachable set Q n The initial predicted position is corrected by adjusting the boundary direction. For initial predicted velocities exceeding the maximum motion velocity, scaling can also be applied based on the maximum motion velocity.

[0042] The trajectory weight determination module 304 determines the consistency error between each predicted motion trajectory and the motion trend of the target part before entering the blind zone, the current posture of the target, and the preset motion constraints. The consistency error e corresponding to the k-th predicted motion trajectory is... k It can be represented as: e k =αe k，motion +βe k，pose +γe k，constraint ; Among them, e k，motion Let e ​​be the motion error between the predicted trajectory of the k-th line and the historical motion trend. k，pose Let e ​​be the attitude error between the k-th predicted trajectory and the current attitude information. k，constraint Let α be the constraint error of the k-th predicted motion trajectory relative to the preset motion constraints, and let α, β, and γ be the coefficients of the corresponding error terms.

[0043] Based on the consistency error corresponding to each predicted motion trajectory, the trajectory weights corresponding to each predicted motion trajectory are obtained through normalization calculation. The trajectory weight corresponding to the k-th predicted motion trajectory is... It can be represented as: ; in, The consistency error represents the kth predicted motion trajectory. This consistency error is used to indicate the degree of matching between the predicted motion trajectory and the historical motion trend, current posture information and kinematic constraints of the target part. The parameter represents the smoothness of the trajectory weight distribution; K represents the number of predicted motion trajectories; exp(·) represents the exponential function. Through the above calculation, the predicted motion trajectory with smaller consistency error is given a larger trajectory weight, and the sum of all trajectory weights is made 1.

[0044] In some embodiments, the target prediction model is trained using training samples, which are constructed in the following manner: determining the blind zone start time from continuous motion data of the target part; acquiring historical motion information of samples before the blind zone start time, sample posture information corresponding to the blind zone start time, and sample risk areas, and using the historical motion information, sample posture information, and sample risk areas as sample inputs; acquiring the actual motion trajectory after the blind zone start time, and using the actual motion trajectory as the supervision data corresponding to the sample inputs.

[0045] In this embodiment, for the processing of training samples, the blind zone start time t0 is selected from a continuous motion data segment containing the complete motion process of the target part. Using t0 as the boundary, data within a preset historical time period before t0 is taken as the observation data obtainable by the model, while the target part motion data within a preset prediction time period after t0 is taken as the actual motion trajectory. The blind zone start time t0 can correspond to the moment when the target part is occluded or leaves the image acquisition range during a real confrontation, or it can be obtained by setting a simulated occlusion moment in the complete motion data. Using this construction method to train samples ensures that the sample input is consistent with the information obtainable when the robot actually enters the blind zone state. Simultaneously, the actual motion results of the target part after the blind zone start time are used to supervise the target prediction model in learning the motion change patterns during the invisible period. Selecting historical motion information of samples before the blind zone start time provides the target prediction model with the motion trend of the target part before entering the blind zone state; selecting the posture information of samples corresponding to the blind zone start time provides the target prediction model with the posture correlation between the target part and other body parts of the opponent; selecting risk areas of samples provides the target prediction model with the positional relationship between the target part and the space to be protected by the robot. Using the actual motion trajectory after the blind zone starts as supervision data, we can compare the differences between the multiple sample predicted motion trajectories output by the target prediction model and the actual motion results of the target part. This allows us to train the target prediction model to generate multiple prediction results that can cover the actual future motion when we only have information before the blind zone starts.

[0046] It should be noted that training samples can be derived from one or more of the following: synthetic data, 3D adversarial scenario synthetic data, and real-machine replay data. Synthetic data can be generated by setting the initial position, speed, direction, and mode of the target part, and is used to supplement samples such as lateral attacks, sudden changes in direction, and high-speed approach. 3D adversarial scenario synthetic data can generate different blind-zone scenes by setting the poses of both sides, camera parameters, and occlusion relationships, and simultaneously output the complete 3D motion trajectory of the target part as supervision data. Real-machine replay data can be derived from binocular images and motion records during actual adversarial processes, and the actual motion trajectory of the target part during occlusion can be obtained through multi-view cameras or motion capture devices. Multi-view cameras and motion capture devices are only used to generate supervision data during the training phase; the target prediction model still uses information obtained from the robot's head binocular camera as the external perception input for the opponent's motion state during the deployment phase.

[0047] Furthermore, when constructing training samples, the length of historical time periods, the length of predicted time periods, and the duration of blind spots can be varied. Perturbations within permissible limits can also be applied to the spatial position, movement speed, and opponent's posture of the target area to improve the training samples' coverage of visual measurement errors and motion changes. By jointly using synthetic data, 3D adversarial scenario synthetic data, and real-machine playback data, the training samples can retain the visual and motion characteristics of real robot adversarial processes, thereby improving the target prediction model's adaptability to generate multiple predicted motion trajectories under different occlusion states, adversarial distances, and movement speeds.

[0048] In some embodiments, when training the target prediction model, at least four sample predicted motion trajectories are generated based on the same initial motion state. The initial motion state includes the last reliable three-dimensional position, motion velocity, and sample posture information of the target part before the start of the blind zone. The at least four sample predicted motion trajectories include: a first sample predicted motion trajectory extending along the original motion direction of the target part, a second sample predicted motion trajectory extending towards the risk area of ​​the sample head, a third sample predicted motion trajectory extending towards the risk area of ​​the sample torso, and a fourth sample predicted motion trajectory extending along the direction of the opponent's limb contraction. When generating each sample predicted motion trajectory, kinematic propagation is performed on the predicted position corresponding to each prediction time according to preset motion constraints, and the predicted position is constrained to the kinematic reachability range of the target part.

[0049] In this embodiment, generating multiple sample predicted motion trajectories for the same initial motion state allows the target prediction model to learn different subsequent movements that the target part may exhibit under the same historical observation conditions. Since the actual motion result is uncertain after the target part enters a blind zone, the same initial position, movement speed, and opponent posture may correspond to various motion processes such as continuing the attack, changing the attack target, or retrieving the limb. By setting sample predicted motion trajectories with different motion meanings, the situation where multiple prediction results converge on similar motion paths can be reduced, and the coverage of multiple prediction results for different possible future movements can be improved.

[0050] The first sample predicted trajectory extends along the original direction of motion of the target part before it enters the blind zone, corresponding to the situation where the target part maintains its original motion trend. When the target part enters the blind zone at a high speed or when the blind zone lasts for a short time, the target part usually has a certain degree of motion inertia. Therefore, the prediction result that continues the original direction of motion can serve as the basis for the short-term motion prediction of the target part. The first sample predicted trajectory can continue to extend based on the motion speed of the target part before it enters the blind zone, or it can gradually decrease the motion speed as the prediction time increases, to correspond to the motion process of the target part continuing to extend and then decelerating.

[0051] The second sample's predicted motion trajectory extends towards the head risk area, corresponding to situations where the target part changes direction and approaches the robot's head after entering the blind zone. This trajectory can cover straight-line attacks, curved attacks, or movements turning towards the head under occlusion, enabling the target prediction model to retain predictions of potential threats to the head, thereby reducing the probability of missing head attack risks due to the target part changing direction within the blind zone.

[0052] The third-sample predicted trajectory extends towards the sample torso risk area, corresponding to situations where the target part moves towards the robot's torso after entering the blind zone. The target part may change its attack height while invisible, or change from moving towards the robot's upper region to moving towards the torso region. Setting a third-sample predicted trajectory allows the target prediction model to distinguish movement patterns towards different risk areas and provides candidate movement results for subsequent determination of the main threatened areas.

[0053] The fourth sample predicts the motion trajectory along the direction of the opponent's limb contraction, corresponding to the situation where the target part ends its current extension action and retracts towards the opponent's body. After the target part enters the blind zone, it may have already ended its attack action, or it may be preparing for the next movement through the retraction action. Incorporating the limb retraction trajectory into multiple sample predicted motion trajectories allows the target prediction model to learn both attack and non-attack retraction movements simultaneously. This reduces the probability that the target part will be continuously judged as approaching the robot's risk area once it enters the blind zone, thereby reducing the likelihood of the robot frequently performing high-intensity defensive operations due to the prediction results being overly concentrated on the attack path.

[0054] In addition to the four types of sample predicted motion trajectories mentioned above, other sample predicted motion trajectories can be added based on the movement patterns of the target parts in the adversarial task. For example, a lateral sweeping trajectory extending laterally along the robot's risk area can be set to correspond to a punch, a lateral swing, or a lateral swing of the robotic arm; an ascending attack trajectory extending from a lower position to a higher position can be set to correspond to an uppercut or the target part moving from near the torso towards the upper part of the robot; a descending attack trajectory extending from a higher position to a lower position can be set to correspond to a downward attack or the target part changing the attack height; sample predicted motion trajectories can also be set with gradually increasing speed, gradually decreasing speed, a brief pause followed by continued movement, a movement from one risk area to another, or a change of direction before approaching a risk area, to cover movement scenarios such as accelerated attacks, decelerated probing, pauses, feints, and mid-course changes of direction.

[0055] Different sample predicted motion trajectories share the same initial motion state, concentrating the differences between trajectories on the motion changes after the target body enters the blind zone. During the generation of each sample predicted motion trajectory, preset motion constraints limit the positional changes of the target body between adjacent prediction times, and adjust predicted positions exceeding kinematic reach to within the allowable range of motion. This avoids sample predicted motion trajectories including motion results exceeding limb length, joint range of motion, or motion speed. Therefore, the generated multiple sample predicted motion trajectories can cover the possibilities of the target body continuing its original motion, approaching different risk areas, retracting the limb, and performing other directional movements while maintaining motion feasibility, providing a training foundation for the target prediction model to learn multiple blind zone motion patterns.

[0056] Please see Figure 4 , Figure 4 A schematic flowchart of an embodiment of the training method for the target prediction model provided in this application; in this embodiment, the training method for the target prediction model includes: 401. Input the sample's historical motion information, sample's posture information, sample's risk area, and sample's motion constraints into the target prediction model to obtain multiple sample predicted motion trajectories and the sample trajectory weights corresponding to each sample predicted motion trajectory.

[0057] Before inputting the data into the target prediction model, coordinate unification and numerical normalization can be performed on various data in the training samples. Specifically, the last reliable 3D position of the target part of the sample before the start of the blind zone can be used as the reference position. The historical motion information, posture information, and risk area of ​​the sample can be transformed into the same coordinate system, and the motion speed, limb structure parameters, and motion ability parameters can be normalized. The historical motion information of the sample is arranged into a historical motion sequence according to the acquisition time; the posture information of the sample is organized according to the connection relationship between body parts, and a status label is set to indicate whether the body parts are visible. The target prediction model extracts features from the historical motion information and posture information of the sample, and fuses the extracted features with the risk area and motion constraints of the sample to obtain fused features that include historical motion trends, current posture relationships, risk area positions, and allowed motion ranges. The target prediction model generates multiple trajectory branches based on the fused features. Each trajectory branch shares the last reliable 3D position and outputs a sample prediction motion trajectory that includes multiple sample prediction positions.

[0058] When generating sample prediction locations, the target prediction model determines the kinematic reachability of the target body part based on the sample motion constraints. It then adjusts the predicted locations of samples that exceed the kinematic reachability to ensure that the predicted motion trajectories conform to the limb structure and motor abilities of the sample object. Furthermore, the target prediction model determines the initial weight scores for each predicted motion trajectory based on the fusion features and the trajectory characteristics of each sample, and performs normalization to obtain multiple sample trajectory weights that sum to 1. These weights indicate the relative reliability of each predicted motion trajectory under the current training sample conditions.

[0059] 402. Obtain the actual motion trajectory corresponding to the predicted motion trajectory of multiple samples, determine the trajectory error between the predicted motion trajectory and the actual motion trajectory of each sample, and determine the trajectory coverage loss based on the minimum trajectory error among the trajectory errors.

[0060] The actual motion trajectory is the actual motion trajectory of the target part of the sample after the start time of the blind zone in the training samples. It is obtained from complete continuous motion data, a three-dimensional simulation environment, or an external trajectory acquisition device used during the training phase. The actual motion trajectory includes the actual position of the target part of the sample at multiple actual times. In order to make the actual motion trajectory comparable to the sample predicted motion trajectory output in step 401, the actual motion trajectory can be sampled or interpolated according to the prediction time of each sample predicted motion trajectory to obtain the actual position corresponding to each prediction time, and the actual position and the sample predicted position can be transformed to the same coordinate system.

[0061] For the predicted trajectory of the k-th sample, the positional error between the predicted and actual positions of the sample at each prediction time can be calculated. Based on these multiple positional errors within the prediction time domain, the trajectory error corresponding to the predicted trajectory of the k-th sample can be determined. For example, the trajectory error e corresponding to the predicted trajectory of the k-th sample... k It can be represented as: e k = ; Where N is the number of prediction times. The predicted position of the sample at the nth prediction time corresponds to the predicted trajectory of the k-th sample. Let be the actual position of the actual trajectory at the nth predicted time, and let ‖·‖2 represent the Euclidean distance.

[0062] In some implementations, time weights can be set based on the importance of risk assessment at different prediction times, giving larger error weights to predicted positions closer to the current time or closer to the robot's risk area. Alternatively, the trajectory error can be determined by combining the average position error of the sample predicted trajectory and the position error at the end of the prediction time domain, thus simultaneously reflecting the degree of deviation of the sample predicted trajectory throughout the entire prediction time domain and the degree of deviation of the final position.

[0063] After obtaining the trajectory errors corresponding to the predicted motion trajectories of each sample, the minimum trajectory error is determined from multiple trajectory errors, and the trajectory coverage loss is determined based on the minimum trajectory error. The trajectory coverage loss L... cover It can be represented as: L cover = ; Where K is the number of samples predicted for motion trajectories. This represents the trajectory error corresponding to the predicted motion trajectory of the k-th sample.

[0064] Each training sample typically corresponds to one actual motion trajectory, while the target prediction model needs to output multiple sample predicted motion trajectories with different motion directions. Using the minimum trajectory error to determine the trajectory coverage loss allows the sample predicted motion trajectory closest to the actual motion trajectory to be directly supervised, and allows other sample predicted motion trajectories to continue learning other possible motion modes under the same initial motion state, thereby reducing the probability of multiple trajectory branches generating the same motion trajectory.

[0065] 403. Determine the target trajectory weight corresponding to the predicted motion trajectory of each sample based on the trajectory error, and determine the weight loss based on the target trajectory weight and the corresponding sample trajectory weight.

[0066] The trajectory error obtained in step 402 reflects the degree of closeness between the predicted motion trajectory and the actual motion trajectory of each sample. The smaller the trajectory error, the closer the predicted motion trajectory of the corresponding sample is to the actual future motion of the target part of the sample. Therefore, a larger target trajectory weight can be set for the predicted motion trajectory of the sample. The larger the trajectory error, the greater the deviation between the predicted motion trajectory and the actual motion trajectory of the corresponding sample. Therefore, a smaller target trajectory weight can be set for the predicted motion trajectory of the sample.

[0067] Specifically, the negative values ​​of each trajectory error can be normalized to obtain the target trajectory weights corresponding to the predicted motion trajectories of each sample. The target trajectory weights corresponding to the predicted motion trajectory of the k-th sample are shown below. It can be represented as:

[0068] in, The trajectory error corresponding to the predicted motion trajectory of the k-th sample is represented by the trajectory error, which indicates the degree of deviation between the predicted motion trajectory and the actual motion trajectory; K represents the number of predicted motion trajectories. The temperature parameter is used to adjust the smoothness of the target trajectory weight distribution. Through the above calculations, samples with smaller trajectory errors are given larger target trajectory weights in their predicted motion trajectories, and the sum of the target trajectory weights corresponding to the predicted motion trajectories of all samples is equal to 1. Specifically, when the temperature parameter... When the temperature parameter is small, the target trajectory weights are more sensitive to differences in trajectory error; samples with smaller trajectory errors have more concentrated target trajectory weights corresponding to the predicted motion trajectories. When the value is large, the difference between the target trajectory weights corresponding to the predicted motion trajectories of each sample decreases, so that the predicted motion trajectories of multiple samples maintain a relatively balanced weight distribution.

[0069] In some implementations, the target trajectory weight of the sample predicted motion trajectory corresponding to the minimum trajectory error can be set to 1, while the target trajectory weights of the remaining sample predicted motion trajectories can be set to 0, thus forming a supervision result for a single target trajectory branch. When there are multiple sample predicted motion trajectories that are close to the actual motion trajectory, continuously distributed target trajectory weights can also be generated based on each trajectory error, so that multiple similar sample predicted motion trajectories all receive corresponding weight supervision.

[0070] In step 401, the target prediction model has already output the sample trajectory weights corresponding to the predicted motion trajectories of each sample. The sample trajectory weights are weights predicted by the target prediction model based on the sample's historical motion information, sample posture information, sample risk region, and sample motion constraints; the target trajectory weights are supervised weights determined based on the trajectory error between the predicted and actual motion trajectories. By comparing the sample trajectory weights and the corresponding target trajectory weights, the difference between the weight distribution output by the target prediction model and the actual accuracy of each sample's predicted motion trajectory can be determined. The weight loss can be determined using cross-entropy loss, relative entropy loss, or mean squared error loss; no specific limitation is made here.

[0071] 404. Determine the motion constraint loss based on the degree to which the predicted motion trajectory of each sample satisfies the motion constraint conditions of the sample, and determine the trajectory diversity loss based on the trajectory differences between the predicted motion trajectories of different samples.

[0072] Motion constraints for samples can include the spatial reachability corresponding to the length of the limb link, the range of motion of the joints, and the maximum motion velocity. For each sample predicted motion trajectory, the motion constraint loss is determined based on the distance by which the predicted position exceeds the kinematic reachability, the degree to which the range of motion of the joints is exceeded, and the degree to which the predicted velocity exceeds the maximum motion velocity. When correcting the predicted position using a projection method, the motion constraint loss can also be determined based on the difference in the predicted position before and after projection processing, so that the trajectory generation module can directly generate a predicted position that conforms to the motion constraints.

[0073] Trajectory differences can be determined based on the differences in position distance, direction of motion, final position distance, or trajectory shape between different samples predicting motion trajectories at the corresponding prediction time. When the trajectory differences between different samples predicting motion trajectories are less than a preset difference condition, the trajectory diversity loss is increased to encourage different trajectory branches to generate prediction results with different motion directions, target positions, or motion patterns. Thus, multiple sample predicted motion trajectories can cover different future motion possibilities within the range of satisfying motion constraints.

[0074] 405. Train the target prediction model based on trajectory coverage loss, weight loss, motion constraint loss, and trajectory diversity loss.

[0075] After obtaining the various losses, the trajectory coverage loss, weight loss, motion constraint loss, and trajectory diversity loss can be combined according to preset loss coefficients to obtain the total training loss corresponding to the target prediction model. The network parameters of the historical motion encoding module 301, posture encoding module 302, trajectory generation module 303, and trajectory weight determination module 304 are then adjusted based on the total training loss. Each loss coefficient is used to coordinate the degree to which candidate trajectories cover the actual motion trajectory, the accuracy of sample trajectory weights, the motion feasibility of predicted motion trajectories, and the degree of difference between different trajectory branches. Each loss coefficient can be set according to the model output during the training phase. The loss coefficient corresponding to the trajectory coverage loss controls how closely the sample predicted motion trajectory approximates the actual motion trajectory; the loss coefficient corresponding to the weight loss controls the consistency between sample trajectory weights and target trajectory weights; the loss coefficient corresponding to the motion constraint loss controls the degree to which the sample predicted motion trajectory satisfies the sample motion constraint conditions; and the loss coefficient corresponding to the trajectory diversity loss controls the degree of difference between different trajectory branches. By adjusting each loss coefficient, the relationship between trajectory prediction accuracy, trajectory weight reliability, motion feasibility, and candidate trajectory diversity can be coordinated.

[0076] During training, multiple training samples can be input into the target prediction model in batches. Steps 401 to 404 yield the total training loss for each batch of training samples, and the model parameters are adjusted based on this total training loss until the model evaluation results meet application requirements. Model parameters can include network parameters from the historical motion encoding module 301, posture encoding module 302, trajectory generation module 303, and trajectory weight determination module 304. Through multiple training iterations, the trajectory coverage loss is gradually reduced, ensuring that at least one sample predicted motion trajectory output by the target prediction model closely approximates the actual motion trajectory; the weight loss is gradually reduced, enabling the sample trajectory weights to reflect the closeness between each sample's predicted motion trajectory and the actual motion trajectory; the motion constraint loss is kept within an acceptable range, ensuring that the sample predicted motion trajectory meets the motion capabilities of the sample object; and the effective differences between different trajectory branches are maintained through trajectory diversity loss. After training, the target prediction model is deployed on the robot, enabling it to output multiple predicted motion trajectories and corresponding trajectory weights that conform to motion constraints, based on the historical motion information of the target part before entering the blind zone, the current posture information of the target, the robot's risk area, and preset motion constraints.

[0077] Please see Figure 5 , Figure 5A schematic flowchart of an embodiment for determining the reachable attack space provided in this application; in some embodiments, the prediction error range increases with the increase of the prediction time interval; based on the spatial size of the target location and the prediction error range, each predicted motion trajectory is spatially expanded to obtain the reachable attack space corresponding to each predicted motion trajectory, including: 501. Sample each predicted motion trajectory according to the preset time step to obtain the predicted position corresponding to each predicted time.

[0078] The predicted motion trajectories output by the target prediction model can include the continuously changing motion path of the target part within the prediction time domain, or a discrete position sequence generated by the target prediction model according to its own output frequency. To facilitate subsequent determination of the prediction error range based on the prediction time interval and to adopt a unified spatial expansion method for different predicted motion trajectories, each predicted motion trajectory can be sampled at the same preset time step, ensuring that each predicted motion trajectory has a corresponding prediction time. Specifically, the moment when the target part enters the blind zone can be used as the prediction start time, and multiple prediction times can be sequentially set within the prediction time domain according to the preset time step. The prediction time domain can be determined based on the time required for the robot to complete risk calculation and defense operations, for example, 0.3 seconds to 0.6 seconds; the preset time step can be determined based on the output frequency of the target prediction model, the robot's control cycle, and computing power, for example, 20 milliseconds to 40 milliseconds. A longer prediction time domain covers a longer future motion time; a smaller preset time step results in a shorter time interval between adjacent predicted positions, leading to more detailed predicted motion trajectories, but also increases the number of sampling points and subsequent spatial computation.

[0079] When the discrete position sequence output by the target prediction model corresponds to a preset time step, each position in the discrete position sequence can be directly used as the predicted position for the corresponding prediction time. When the output time of the target prediction model is inconsistent with the prediction time set according to the preset time step, interpolation can be performed based on adjacent output positions and their corresponding times to determine the predicted position corresponding to each prediction time. Interpolation methods can include linear interpolation, spline interpolation, or position estimation based on the target part's movement velocity, ensuring that the sampled predicted positions are continuously arranged along the corresponding predicted motion trajectory. For the k-th predicted motion trajectory, the sequence of predicted position points obtained after sampling can be denoted as p. k，0 p k，1 ... p k，N , where p k，0 The target location corresponding to the prediction start time, p k，NThe predicted position corresponds to the nth prediction time, where N is the number of sampling intervals within the prediction time domain. Different predicted motion trajectories with the same index correspond to the same prediction time interval at their predicted positions, enabling subsequent processing to expand each predicted motion trajectory using the prediction error range corresponding to the same prediction time interval.

[0080] 502. Determine the size radius based on the spatial dimensions of the target location, and determine the expansion radius based on the size radius and the prediction error radius corresponding to each prediction position.

[0081] The spatial dimensions of the target part are used to determine the actual spatial range occupied by the target part around the predicted location. These spatial dimensions can be determined based on preset structural parameters of the target part, or based on binocular images and 3D detection results when the target part is visible. For example, when the target part is a fist, the spatial dimensions can be determined based on the overall space occupied by the fist and the gloves. When approximating the target part with a sphere, the radius that can cover the target part can be determined based on the dimensions of the target part in three spatial directions. For example, half the length of the spatial diagonal of the target part's 3D bounding box can be used as the radius, or the maximum value of the half-dimensions of the target part in each spatial direction can be used. Using a radius that can cover the outer contour of the target part ensures that the spatial range constructed centered on the predicted location includes the physical space that the target part may occupy at that predicted location.

[0082] For the predicted position corresponding to the nth prediction time, the prediction time interval is determined based on the time interval between this prediction time and the prediction start time, and the corresponding prediction error radius is determined based on the prediction time interval. The prediction error radius is used to describe the possible offset range of the actual position of the target part relative to the predicted position. When the prediction time interval is short, the predicted position of the target part is usually more accurate, and a smaller prediction error radius can be used; when the prediction time interval is long, the direction and speed of the target part's movement may change more, and a larger prediction error radius can be used to cover the position that the target part may reach due to deviation from the predicted movement trajectory.

[0083] Adding the size radius to the prediction error radius yields the expansion radius at the corresponding predicted position. The expansion radius *r* corresponding to the *n*-th predicted position in the *k*-th predicted trajectory is... k，n It can be represented as: r k，n =r size +r err (t n ) Where, r size r is the radius determined based on the spatial dimensions of the target area. err (t n ) represents the predicted time interval t nThe corresponding prediction error radius.

[0084] The size radius is used to cover the space occupied by the target part itself, while the prediction error radius is used to cover the possible offset of the actual position of the target part relative to the predicted position. Combining the two to obtain the expansion radius allows the subsequent expansion body built around the predicted position to simultaneously include the solid size of the target part and the uncertainty range of the predicted position.

[0085] In some specific implementations, the prediction error radius can be determined by using model evaluation data with actual motion trajectories after the target prediction model is trained, and then statistically analyzing the trajectory position errors of the target prediction model at different prediction time intervals. For each evaluation sample, the position error between the predicted position and the actual position of each evaluated predicted motion trajectory at the same prediction time is calculated, and the minimum position error is determined as the trajectory position error at the corresponding prediction time interval. The trajectory position errors of multiple evaluation samples at the same prediction time interval are summarized, and the error corresponding to a preset quantile value is determined as the prediction error radius corresponding to that prediction time interval. For example, the error corresponding to the 90th or 95th percentile can be used. The higher the error quantile value, the larger the coverage of the actual motion position in the attack space; the lower the error quantile value, the smaller the space expansion range and the smaller the computational load. The prediction error radii corresponding to different prediction time intervals are determined sequentially, and a preset correspondence between prediction time intervals and prediction error radii is established. The preset correspondence can be stored using a lookup table, piecewise function, or fitted curve, and the prediction error radius corresponding to unrecorded prediction time intervals can be determined by interpolation. Furthermore, the prediction error radii can be smoothed to ensure they remain constant as the prediction time interval increases. After the target prediction model is deployed, the preset correspondence is queried based on the prediction time interval corresponding to each prediction position to obtain the corresponding prediction error radius. This allows the spatial expansion range to adapt to the prediction accuracy of the target prediction model at different prediction time intervals, reducing the probability of the actual movement position exceeding the reachable attack space and controlling the extent of expansion of the reachable attack space.

[0086] 503. Using each predicted position as the center, construct a three-dimensional extended body according to the corresponding extended radius, and connect the three-dimensional extended bodies corresponding to adjacent predicted positions in the same predicted motion trajectory to obtain the sweep body between adjacent predicted positions.

[0087] For the nth predicted position p in the kth predicted motion trajectory k，n The predicted position can be used as the center of the sphere, with the corresponding expansion radius r. k，n Construct a spherical extension body with a radius. The spherical extension body includes spatial points whose distance from the predicted position is less than or equal to the extension radius, covering the spatial range that the target part may occupy at the corresponding predicted time. The three-dimensional extension body B corresponds to the nth predicted position in the k-th predicted motion trajectory.k，n It can be represented as: B k，n ={x|‖xp k，n ||2≤r k，n}; Where x is a point in space, p k，n To predict the location, r k，n This represents the corresponding expansion radius.

[0088] The adjacent predicted positions obtained by sampling according to a preset time step correspond to two discrete prediction times, while the target part moves continuously between the two prediction times. To cover the space that the target part may occupy during its movement from the previous prediction position to the next prediction position, the corresponding three-dimensional extension body can be continuously moved along the connection path between the adjacent prediction positions, and the space covered during the movement is defined as the sweep body. When the extension radii corresponding to the adjacent prediction positions are the same, the sweep body can be constructed as a capsule with the line segment between the two prediction positions as the central axis. The capsule includes a cylindrical central region connecting the two prediction positions and spherical regions located at both ends of the central axis. When the extension radii corresponding to the adjacent prediction positions are different, the extension radius can be continuously varied between the two prediction positions to form sweep bodies with different sizes at both ends, so that the sweep body is continuously connected to the two adjacent three-dimensional extension bodies. Specifically, multiple interpolation positions can be set between adjacent prediction positions, and the corresponding interpolation extension radius can be determined according to the relative proportion of the interpolation position between the two prediction positions. Interpolation extension bodies are constructed with each interpolation position as the center and according to the corresponding interpolation extension radius. The space covered by each interpolation extension body is merged to obtain the sweep body between the adjacent prediction positions. The number of interpolation locations can be determined based on the distance between adjacent predicted locations, the degree of variation in the expansion radius, and available computing resources.

[0089] When the predicted trajectory exhibits a significant curvature between adjacent predicted positions, the connection path between these positions can be determined based on the velocity or trajectory curve output by the target prediction model, and a sweep volume can be constructed along this path. This reduces the path approximation error that occurs when connecting adjacent predicted positions with straight lines. By constructing this three-dimensional extended volume, the space that the target may occupy at each discrete prediction time can be covered; by constructing the sweep volume, the space that the target may traverse during continuous movement between adjacent prediction times can be covered, thus avoiding the omission of potential attack ranges between adjacent predicted positions due to the use of discrete positions to describe the predicted trajectory.

[0090] 504. Merge the three-dimensional extended volumes and sweep volumes corresponding to the same predicted motion trajectory to obtain the reachable attack space corresponding to the predicted motion trajectory.

[0091] For the k-th predicted motion trajectory, a spatial union operation is performed on the 3D extended bodies corresponding to each predicted position in the predicted motion trajectory, as well as the sweep bodies between adjacent predicted positions, to merge the space jointly covered by the extended bodies and sweep bodies. During the spatial union operation, overlapping spaces between different 3D extended bodies, between different sweep bodies, and between 3D extended bodies and sweep bodies can be eliminated, resulting in a continuously extending spatial region along the corresponding predicted motion trajectory. This spatial region covers the space that the target part may occupy at each predicted time and the space that the target part may traverse when moving between adjacent predicted times. The reachable attack space can be recorded using a set of 3D geometric bodies, a 3D mesh, a set of spatial sampling points, or a distance field. When using a set of 3D geometric bodies, the spherical extended bodies and capsule-shaped sweep bodies constituting the reachable attack space can be saved, and geometric intersection can be directly determined in subsequent calculations. When using a set of spatial sampling points, sampling points can be set inside each 3D extended body and sweep body, and the spatial overlap can be determined based on whether the sampling points are located within the robot's risk area.

[0092] In some implementations, the predicted time or predicted time period corresponding to each spatial region can be retained when merging the 3D extended volume and the swept volume. For example, the 3D extended volume can be associated with a corresponding predicted time, and the swept volume can be associated with the time period between two adjacent predicted times. Thus, when subsequently determining the overlap between the reachable attack space and the risk area, the time when the target part may reach the overlapping area can be further determined.

[0093] Each predicted motion trajectory generates a corresponding reachable attack space, and each reachable attack space maintains a correspondence with the corresponding predicted motion trajectory and trajectory weight. Through step 504, the spatial expansion results consisting of discrete predicted positions, target part spatial dimensions, and prediction error range can be integrated into a continuous reachable attack space. This reachable attack space completely covers the space that the target part may reach when moving along the corresponding predicted motion trajectory, and provides a geometric calculation object for subsequent calculation of the attack risk formed by each reachable attack space on the robot's risk area according to the trajectory weight.

[0094] In some embodiments, the risk area includes a head risk area and a torso risk area; the overlap between each reachable attack space and the risk area is determined, and the corresponding overlap is weighted according to the trajectory weight to obtain the attack risk of the target part, including: for each reachable attack space, determining the head overlap between the reachable attack space and the head risk area, and the torso overlap between the reachable attack space and the torso risk area; weighting the overlap according to the trajectory weight and the corresponding head overlap to obtain the head attack risk; weighting the overlap according to the trajectory weight and the corresponding torso overlap to obtain the torso attack risk; and determining the attack risk of the target part based on the head attack risk and the torso attack risk.

[0095] In this embodiment, a head risk region can be determined in the robot coordinate system based on the robot's head structure and current head pose; a torso risk region can be determined in the robot coordinate system based on the robot's torso structure and current torso pose. The head and torso risk regions can respectively cover the outer contours of their corresponding parts, with a preset safety distance added to the outer contours of the corresponding parts. This ensures that the risk region simultaneously includes the space occupied by the robot entity and the safety space required to trigger defensive operations before the target part approaches the robot entity. When the robot's posture changes, the corresponding risk regions can be updated based on the current spatial positions of the head and torso.

[0096] The reachable attack space A corresponding to the k-th predicted motion trajectory k Calculate the reachable attack space A separately. k With head risk area Z head The spatial intersection between them, and the reachable attack space A k Risk area Z of the torso body The spatial intersection between them. The spatial intersection can be determined through the intersection operation of three-dimensional geometry or the hit judgment of spatial sampling points. When the reachable attack space is recorded as a set of spherical extensions and capsule-shaped sweeps, it can be determined whether each extension and sweep intersects with the head risk area or the torso risk area, and the intersecting parts can be merged.

[0097] To reduce the impact of volume differences between the head and torso risk regions on the overlap amount, the overlap amount can be determined based on the proportion of the spatial intersection to the corresponding risk region. The head overlap amount O corresponding to the k-th predicted motion trajectory is... (k,head) Overlap with torso O (k,body) They can be represented as follows: O (k,head) = ; O (k,body) = ; Where Vol represents the volume of the spatial region. The values ​​of head overlap and torso overlap can be between 0 and 1. When the overlap is 0, it means that the corresponding reachable attack space does not cover the corresponding risk area; when the overlap increases, it means that the proportion of the corresponding reachable attack space covering the corresponding risk area increases.

[0098] After obtaining the head overlap amount corresponding to each reachable attack space, each head overlap amount is multiplied by the trajectory weight of the corresponding predicted motion trajectory, and the calculation results are summed to obtain the head attack risk R. head The risk of headshot attacks can be represented as: ; Accordingly, the torso attack risk R is obtained by weighting each trajectory based on its weight and the corresponding torso overlap. body : ; Where K is the number of predicted motion trajectories. Let be the trajectory weight corresponding to the k-th predicted motion trajectory. Through weighted calculation, predicted motion trajectories that are highly consistent with current historical motion and attitude information can have a greater impact on the corresponding attack risk, while retaining the potential attack risks corresponding to other predicted motion trajectories.

[0099] After obtaining the head attack risk and torso attack risk, the attack risk of the target body part can be determined based on the larger of the two attack risks. Alternatively, regional risk coefficients can be set according to the protection priorities corresponding to the head and torso, and the head attack risk and torso attack risk can be combined. For example, the attack risk R of the target body part can be expressed as: R=η head R head +η body R body ; Where, η head η represents the head risk coefficient. body This represents the torso risk coefficient. For robots that require priority protection of the head, the head risk coefficient can be set higher than the torso risk coefficient, ensuring a higher attack risk when the accessible attack space approaches the head risk area.

[0100] By calculating the head attack risk and torso attack risk separately, the overall attack risk of the target area can be determined, while identifying the robot parts that pose the primary threat to the target area. Subsequent defensive control can then determine the level of defensive response based on the attack risk and identify areas requiring priority protection based on the relationship between head and torso attack risks, improving the targeted nature of defensive actions taken by the robot when the target area is in a blind spot.

[0101] In some specific implementations, the calculations of the reachable attack space, overlap, and attack risk can be integrated into the target prediction model, meaning the target prediction model directly outputs the attack risk. Specifically, the target prediction model may further include a spatial expansion module, an overlap calculation module, and a risk aggregation module. The spatial expansion module receives multiple predicted motion trajectories output by the trajectory generation module 303 and, based on the spatial dimensions of the target location and the prediction error range corresponding to each prediction time interval, spatially expands each predicted motion trajectory to obtain the reachable attack space corresponding to each predicted motion trajectory. The overlap calculation module receives each reachable attack space and the robot's risk region, determining the overlap between each reachable attack space and the risk region. The risk aggregation module receives each overlap and the corresponding trajectory weight output by the trajectory weight determination module 304, performs weighted calculation on each overlap, and obtains the attack risk of the target location. The spatial expansion module, overlap calculation module, and risk aggregation module can be implemented using preset calculation rules or using a differentiable computation layer capable of participating in model training. For example, the reachable attack space can be converted into voxel occupancy data or spatial sampling data. An expanded spatial occupancy result can be generated through differentiable dilation operations. The overlap between the reachable attack space and the risk region can be determined through differentiable intersection operations, and risk convergence can be completed based on trajectory weights. When training the target prediction model, the actual attack result can be determined based on the actual future motion trajectory of the target part. The risk loss can be determined based on the difference between the attack risk output by the model and the actual attack result. This risk loss can be transmitted to the trajectory generation module 303 and the trajectory weight determination module 304 through the risk convergence module, overlap calculation module, and spatial expansion module, thereby jointly training the target prediction model.

[0102] When directly outputting attack risks using a target prediction model, the attack results can be used to jointly optimize the front-end trajectory prediction and trajectory weights. This allows the target prediction model to not only focus on trajectory position errors but also to further consider whether the predicted motion trajectory covers the robot's risk area. This approach reduces data conversion and calling overhead between different processing modules and enables the sharing of intermediate features and computing resources. This helps reduce attack risks and computational latency, and improves the robot's real-time response capability after the target area enters a blind zone.

[0103] Please see Figure 6 , Figure 6 This is a schematic flowchart illustrating an embodiment of determining a defensive action provided in this application. In some embodiments, the preset risk conditions include a first risk threshold and a second risk threshold greater than the first risk threshold; comparing the attack risk with the preset risk conditions and determining the defensive action based on the comparison result includes: 601. When the attack risk is less than the first risk threshold, control the robot to maintain its current motion state.

[0104] When the attack risk is less than the first risk threshold, it indicates that the overall overlap between the reachable attack space corresponding to each predicted motion trajectory and the robot's risk area is low, and the probability of the target part contacting the robot's risk area within the current prediction time domain is small. In this case, the processor can maintain the robot's current motion control commands, allowing the robot to continue executing its current standing, moving, observing, or adversarial actions, and to continue acquiring new visual information to update the visibility state of the target part and the attack risk. By maintaining the current motion state under low-risk conditions, the frequent interruptions of the robot's current actions due to the target part being temporarily invisible can be reduced, allowing the robot to maintain normal motion continuity and adversarial capabilities.

[0105] 602. When the attack risk is greater than or equal to the first risk threshold and less than the second risk threshold, determine the corresponding defensive action based on the head attack risk and the torso attack risk.

[0106] When the attack risk falls between the first and second risk thresholds, it indicates that the target part poses a potential threat to the robot's risk area that requires intervention. The processor can compare the head attack risk and the torso attack risk, and determine the robot parts that need priority protection based on the risk area with higher attack risk. When the head attack risk is high, the robot's movable limbs can be controlled to move towards the head to reduce the likelihood of the target part contacting the head risk area; when the torso attack risk is high, the robot's movable limbs can be controlled to move towards the torso, or the torso posture and robot position can be adjusted to reduce the likelihood of the target part contacting the torso risk area.

[0107] When the risks of head and torso attacks are similar, the robot can be controlled to perform defensive maneuvers that simultaneously cover both the head and torso risk areas, or the defensive maneuver can be determined based on the distribution of accessible attack space corresponding to the two risk areas. When dealing with moderate risks, the speed or amplitude of the robot's current offensive actions can be reduced to allow for sufficient movement space and response time for movable limbs to switch to a defensive posture. Therefore, appropriate defensive maneuvers can be selected based on the primary threatened areas, enabling the robot to maintain a certain level of combat capability while providing targeted protection to areas potentially vulnerable to attack.

[0108] 603. When the attack risk is greater than or equal to the second risk threshold, control the robot to perform preset defensive operations and restrict the robot from performing offensive operations.

[0109] When the attack risk reaches or exceeds the second risk threshold, it indicates that at least one reachable attack space with a high trajectory weight significantly overlaps with the robot's risk area, and the target location may pose a high threat to the robot in the current prediction time domain. The processor can invoke pre-stored defensive operations and generate corresponding motion control commands based on these operations. Pre-set defensive operations may include controlling movable limbs to cover the robot's risk area, controlling the robot to retreat, adjusting the robot's posture, or putting the robot into a preset safe posture. While executing preset defensive operations, the processor can pause, cancel, or reduce the execution priority of the control commands corresponding to the current offensive operation, restricting the robot from continuing to execute offensive operations that may increase the exposure of the risk area, and prioritizing the robot's motion capabilities for defensive operations. By restricting offensive operations under high-risk conditions, control conflicts between offensive and defensive actions can be reduced, shortening the time required for the robot to enter a safe motion state.

[0110] In some implementations, a state maintenance time or risk hysteresis range can be set when the attack risk is near a risk threshold. This allows the robot to switch to a defensive state after the attack risk continuously meets the corresponding conditions, and to exit the defensive state when the attack risk decreases to the corresponding recovery condition. This reduces the frequency of robot state switching caused by fluctuations in attack risk near the risk threshold, improving the stability of the hierarchical defense control process.

[0111] Please see Figure 7 , Figure 7 A flowchart illustrating another embodiment of determining a defensive action provided in this application. In some embodiments, the preset risk conditions include defensive triggering conditions; comparing the attack risk with the preset risk conditions and determining the defensive action based on the comparison result includes: 701. When the comparison result shows that the attack risk meets the defense triggering condition, the risk contribution value corresponding to each predicted motion trajectory is determined according to the weight of each trajectory and the corresponding head overlap and torso overlap, and the predicted motion trajectory corresponding to the maximum risk contribution value is determined as the dominant predicted motion trajectory. Defense trigger conditions are used to determine whether the robot needs to perform directional defensive operations based on the dominant predicted motion trajectory. Defense trigger conditions can be: the attack risk reaching a preset trigger threshold, the attack risk entering a preset risk range, or the attack risk continuously meeting preset conditions for a preset time. When the comparison result indicates that the attack risk meets the defense trigger conditions, for the k-th predicted motion trajectory, the trajectory overlap result can be determined based on the head overlap and torso overlap corresponding to that predicted motion trajectory. The trajectory overlap result is then multiplied by the corresponding trajectory weight to obtain the risk contribution value corresponding to the k-th predicted motion trajectory. This risk contribution value reflects both the reliability of the predicted motion trajectory and its coverage of the robot's risk area. Predicted motion trajectories with larger trajectory weights and a higher degree of overlap between the reachable attack space and the risk area have larger risk contribution values. By comparing the risk contribution values ​​corresponding to each predicted motion trajectory, the predicted motion trajectory with the largest risk contribution value is determined as the dominant predicted motion trajectory. This allows the robot to identify the current motion outcome that poses the main threat from multiple possible future motion outcomes.

[0112] When multiple predicted motion trajectories correspond to the same risk contribution value or have a difference less than a preset difference, the dominant predicted motion trajectory can be further determined based on the earliest overlap time, the maximum overlap amount, or the trajectory weight of each predicted motion trajectory. For example, the predicted motion trajectory that overlaps with the risk area earliest can be selected first, or the predicted motion trajectory with the larger maximum overlap amount can be selected, so that the dominant predicted motion trajectory corresponds to the attack situation that requires the robot to respond first.

[0113] 702. Determine the defensive direction based on the direction of movement when the predicted movement trajectory overlaps with the head risk area or the torso risk area; Based on the predicted time, it is sequentially determined whether the reachable attack space corresponding to the dominant predicted movement trajectory overlaps with the head or torso risk area, and the predicted time of the first overlap is determined as the target overlap time. Based on the predicted position corresponding to the target overlap time and the predicted position corresponding to the previous predicted time, the movement direction of the target body part when entering the corresponding risk area can be determined. Furthermore, the average movement direction can be determined based on multiple predicted positions of the dominant predicted movement trajectory near the target overlap time to reduce the impact of fluctuations in a single predicted position on the determination of the movement direction.

[0114] After determining the direction of movement of the target part, the defensive direction can be determined based on the direction of movement, the location of the corresponding risk area, and the robot's current posture. When using movable limbs for blocking, the defensive direction can be directed towards the side where the dominant predicted motion trajectory enters the risk area, allowing the movable limbs to move towards the target part along a path approaching the risk area. When using the robot's own body for avoidance, the defensive direction can be determined along a direction away from the dominant predicted motion trajectory or along a lateral direction with a small degree of intersection with the dominant predicted motion trajectory. By determining the defensive direction based on the direction of movement of the dominant predicted motion trajectory, the robot's defensive actions can be aligned with the path that the target part may take to reach the risk area.

[0115] 703. Determine the main threatened areas based on the risk of head and torso attacks, and determine defensive actions based on the main threatened areas and defensive direction, including head protection, blocking, retreating, and lateral movement.

[0116] By comparing the risks of head attacks and torso attacks, the head-risk area is identified as the primary threatened area when the head-attack risk is greater than the torso attack risk; the torso-risk area is identified as the primary threatened area when the torso attack risk is greater than the head-attack risk; and the head-risk area and the torso-risk area can be jointly identified as the primary threatened area when the difference between the head-attack risk and the torso attack risk is less than a preset risk difference.

[0117] When the head is the primary threatened area, the movable limbs on the corresponding side can be controlled to move towards the head to perform head protection, depending on the defensive direction; alternatively, the movable limbs can be controlled to move between the predicted trajectory and the head threatened area to perform a blocking maneuver. When the torso is the primary threatened area, the movable limbs can be controlled to cover the torso threatened area, depending on the defensive direction, or the robot can be controlled to retreat or move laterally to increase the distance between the torso threatened area and the reachable attack space corresponding to the predicted trajectory.

[0118] When both the head and torso risk areas are identified as the primary threatened areas, the robot can be controlled to perform a retreat maneuver that simultaneously reduces the probability of attack on both risk areas. Alternatively, it can perform a lateral movement maneuver based on the direction of the dominant predicted motion trajectory. The robot can also combine two or more of the following maneuvers: head protection, blocking, retreat, and lateral movement. For example, it can retreat while protecting its head, or laterally move while blocking. The selected defensive maneuver should meet the motion conditions corresponding to the robot's current joint position, motion speed, and posture stability to ensure that the robot can complete the corresponding action before the target body reaches the risk area.

[0119] Through steps 701 to 703, when the attack risk meets the defense triggering conditions, the dominant predicted motion trajectory that poses the main threat to the robot can be determined from multiple predicted motion trajectories, and the risk area and motion direction that the target part may approach can be further determined, so that the robot can select targeted defensive operations according to the attack position and attack direction.

[0120] This application also relates to a storage medium on which a program is stored, which, when run on a computer, causes the computer to perform any of the above-described vision-based methods for predicting robot blind spot risks.

[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0123] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0124] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0125] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A vision-based method for predicting blind spot risks in robots, characterized in that, The method includes: The robot acquires visual information, determines the visibility status of the target part based on the visual information, and identifies the robot's risk area. Based on the visibility status and the duration of the visibility status, determine whether the target area is in a blind zone. If so, then the historical motion information of the target part before entering the blind zone state and the current posture information of the other party are obtained. The historical motion information, the current posture information, the risk area and the preset motion constraints are input into the pre-trained target prediction model to obtain multiple predicted motion trajectories output by the target prediction model and the trajectory weights corresponding to each predicted motion trajectory; wherein, the multiple predicted motion trajectories correspond to different potential motion paths of the target part in the blind zone state. Based on the spatial dimensions of the target location and the prediction error range that varies with the prediction time interval, the spatial expansion of each predicted motion trajectory is performed to obtain the reachable attack space corresponding to each predicted motion trajectory. The overlap between each of the reachable attack spaces and the risk areas is determined, and the corresponding overlap is weighted according to the trajectory weight to obtain the attack risk of the target location. The attack risk is compared with preset risk conditions, defensive actions are determined based on the comparison results, and the robot is controlled to execute the defensive actions.

2. The method according to claim 1, characterized in that, The target prediction model includes a historical motion encoding module, an attitude encoding module, a trajectory generation module, and a trajectory weight determination module; The historical motion information includes the three-dimensional position sequence, motion speed and motion direction of the target part before it enters the blind zone state; the current posture information includes the spatial position of the body part that is currently visible to the other party; and the preset motion constraints include at least one of limb link length, joint range of motion and maximum motion speed. The historical motion encoding module is used to generate historical motion features based on the historical motion information, and the posture encoding module is used to generate posture features based on the current posture information. The trajectory generation module is used to determine the kinematic reachability of the target part based on the historical motion characteristics, the posture characteristics, the risk area and the preset motion constraints, and generate the multiple predicted motion trajectories within the kinematic reachability range; The trajectory weight determination module is used to determine the trajectory weight corresponding to each predicted motion trajectory based on the degree of consistency between each predicted motion trajectory and the historical motion information, the current posture information, and the preset motion constraints.

3. The method according to claim 1, characterized in that, The target prediction model is trained using training samples, which are constructed in the following way: The blind zone initiation time is determined from the continuous motion data of the target area; The sample historical motion information before the start time of the blind zone, the sample posture information corresponding to the start time of the blind zone, and the sample risk area are obtained, and the sample historical motion information, the sample posture information, and the sample risk area are used as sample inputs; Obtain the actual motion trajectory after the start time of the blind zone, and use the actual motion trajectory as the supervision data corresponding to the sample input.

4. The method according to claim 2, characterized in that, When training the target prediction model, at least four sample predicted motion trajectories are generated based on the same initial motion state. The initial motion state includes the last reliable three-dimensional position, motion speed and sample posture information of the target part before the start of the blind zone. The not less than four sample predicted motion trajectories include: a first sample predicted motion trajectory extending along the original motion direction of the target part, a second sample predicted motion trajectory extending toward the risk area of ​​the sample head, a third sample predicted motion trajectory extending toward the risk area of ​​the sample torso, and a fourth sample predicted motion trajectory extending along the direction of the other party's limb contraction. When generating the predicted motion trajectory of each sample, kinematic propagation is performed on the predicted position corresponding to each prediction time according to the preset motion constraints, and the predicted position is constrained to the kinematic reachable range of the target part.

5. The method according to claim 2, characterized in that, The training method for the target prediction model includes: Input the sample historical motion information, sample posture information, sample risk area and sample motion constraints into the target prediction model to obtain multiple sample predicted motion trajectories and sample trajectory weights corresponding to each sample predicted motion trajectory. Obtain the actual motion trajectory corresponding to the predicted motion trajectory of the multiple samples, determine the trajectory error between each of the predicted motion trajectories and the actual motion trajectory, and determine the trajectory coverage loss based on the minimum trajectory error among the trajectory errors. The target trajectory weight corresponding to each sample predicted motion trajectory is determined based on the trajectory error, and the weight loss is determined based on the target trajectory weight and the corresponding sample trajectory weight. The motion constraint loss is determined based on the degree to which the predicted motion trajectory of each sample satisfies the motion constraint conditions of the sample, and the trajectory diversity loss is determined based on the trajectory differences between the predicted motion trajectories of different samples. The target prediction model is trained based on the trajectory coverage loss, the weight loss, the motion constraint loss, and the trajectory diversity loss.

6. The method according to claim 1, characterized in that, The prediction error range increases with the increase of the prediction time interval; based on the spatial dimensions of the target location and the prediction error range, the predicted motion trajectories are spatially expanded to obtain the reachable attack space corresponding to each predicted motion trajectory, including: The predicted motion trajectories are sampled according to a preset time step to obtain the predicted position corresponding to each prediction time. The size radius is determined based on the spatial dimensions of the target location, and the expansion radius is determined based on the size radius and the prediction error radius corresponding to each prediction position. With each predicted position as the center, a three-dimensional extended body is constructed according to the corresponding extended radius, and the three-dimensional extended bodies corresponding to adjacent predicted positions in the same predicted motion trajectory are connected to obtain the sweep body between adjacent predicted positions. By merging the three-dimensional extended volumes and sweep volumes corresponding to the same predicted motion trajectory, the reachable attack space corresponding to the predicted motion trajectory is obtained.

7. The method according to claim 6, characterized in that, The prediction error range includes the prediction error radius corresponding to each prediction time interval; The prediction error radius is determined according to a preset correspondence between the prediction time interval and the trajectory position error quantile. The preset correspondence is established by statistically analyzing the trajectory position error of the target prediction model under different prediction time intervals.

8. The method according to claim 1, characterized in that, The risk areas include the head risk area and the torso risk area; The overlap between each of the reachable attack spaces and the risk areas is determined, and the corresponding overlaps are weighted according to the trajectory weights to obtain the attack risk of the target location, including: For each of the reachable attack spaces, the head overlap between the reachable attack space and the head risk area, and the torso overlap between the reachable attack space and the torso risk area are determined respectively. The head attack risk is obtained by weighting each trajectory weight and the corresponding head overlap amount; The risk of a torso attack is obtained by weighting each trajectory weight and the corresponding torso overlap. The attack risk of the target body part is determined based on the head attack risk and the torso attack risk.

9. The method according to claim 8, characterized in that, The preset risk conditions include a first risk threshold and a second risk threshold that is greater than the first risk threshold; The attack risk is compared with preset risk conditions, and defensive actions are determined based on the comparison results, including: When the attack risk is less than the first risk threshold, the robot is controlled to maintain its current motion state. When the attack risk is greater than or equal to the first risk threshold and less than the second risk threshold, the corresponding defensive action is determined based on the head attack risk and the torso attack risk. When the attack risk is greater than or equal to the second risk threshold, the robot is controlled to perform a preset defensive operation, and the robot is restricted from performing offensive operations.

10. The method according to claim 8, characterized in that, The preset risk conditions include defense trigger conditions; The attack risk is compared with the preset risk conditions, and defensive actions are determined based on the comparison results, including: When the comparison result indicates that the attack risk meets the defense triggering condition, the risk contribution value corresponding to each predicted motion trajectory is determined according to the trajectory weights and the corresponding head overlap and torso overlap, and the predicted motion trajectory corresponding to the maximum risk contribution value is determined as the dominant predicted motion trajectory. The defensive direction is determined based on the direction of movement when the dominant predicted movement trajectory overlaps with the head risk area or the torso risk area; The primary threatened area is determined based on the head attack risk and the torso attack risk, and defensive actions are determined from head protection, blocking, retreating, and lateral movement based on the primary threatened area and the defensive direction.

Images