Data processing method, storage medium, electronic device, and program product

Abstract

The application provides a data processing method, a storage medium, an electronic device and a program product, and relates to the technical field of robot control. The data processing method comprises the following steps: performing safety inspection on a target control instruction of a target robot, wherein the safety inspection refers to checking whether the target robot is safe if the target robot is controlled based on the target control instruction; in the case that the target control instruction passes the safety inspection, determining a movable structure of the target robot involved in the target control instruction and a cross-structure coordination restriction condition corresponding to the movable structure; and in the case that the target control instruction does not satisfy the cross-structure coordination restriction condition, adjusting the target control instruction through a soft constraint strategy.

Description

Technical Field

[0001] This application relates to the field of robot control technology, specifically to a data processing method, storage medium, electronic device, and program product. Background Technology

[0002] During operation, the control commands received by a robot (such as joint angles, chassis speed, head posture, etc.) may pose safety hazards. If unverified control commands are executed directly, it may cause mechanical structure to exceed limits, impact, or endanger environmental safety, such as causing robot joint overload, end effector collision with obstacles, or system dynamic instability. Summary of the Invention

[0003] In view of the above, embodiments of this application provide a data processing method, a storage medium, an electronic device, and a program product.

[0004] In a first aspect, one embodiment of this application provides a data processing method, comprising: performing a safety check on a target control command for a target robot, wherein the safety check refers to checking whether the target robot is safe if controlled based on the target control command; if the target control command passes the safety check, determining the movable structure of the target robot involved in the target control command, and the cross-structure cooperative constraint conditions corresponding to the movable structure; if the target control command does not meet the cross-structure cooperative constraint conditions, adjusting the target control command through a soft constraint strategy.

[0005] In conjunction with the first aspect, in some implementations of the first aspect, the target control command includes at least one of joint angle data, joint angular velocity data, and chassis speed data. A safety check is performed on the target control command of the target robot, including at least one of the following: checking whether the joint angle data falls within a first safe angle range; checking whether the difference between joint angular velocity data at adjacent times is less than the target safe angular velocity difference; and checking whether the chassis speed data is less than the target safe speed.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: if the joint angle data exceeds the first safe angle range, then using a soft saturation strategy to process the joint angle data so as to control the joint angle data to be smoothly attenuated within the target buffer of the first safe angle range.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, a soft saturation strategy is used to process the joint angle data, including: determining a first target buffer located on the lower limit side and a second target buffer located on the upper limit side in the first safe angle interval based on the boundary values ​​on both sides of the first safe angle interval and a preset buffer width; determining the lower limit value of the first target buffer and the upper limit value of the second target buffer; within the first target buffer, adjusting the joint angle data to the sum of the lower limit value and a first correction amount, where the first correction amount is used to characterize the value of smooth transition adjustment of the joint angle data within the first target buffer; within the second target buffer, adjusting the joint angle data to the difference between the upper limit value and the second correction amount, where the second correction amount is used to characterize the value of smooth transition adjustment of the joint angle data within the second target buffer.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining a first difference by subtracting a lower limit value and a preset buffer width from the joint angle data; determining the ratio of the first difference to the steepness control parameter as the input value of the soft saturation function to obtain a first transition value; adding the first transition value to a first constant and multiplying it by the preset buffer width and a first coefficient to obtain a first correction amount; obtaining a second difference by subtracting the preset buffer width and the joint angle data from the upper limit value; determining the ratio of the second difference to the steepness control parameter as the input value of the soft saturation function to obtain a second transition value; adding the second transition value to a second constant and multiplying it by the preset buffer width and a second coefficient to obtain a second correction amount.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: if the difference between joint angular velocity data at adjacent time points is greater than or equal to the target safe angular velocity difference, then determining acceleration limit data; adjusting the joint angular velocity data at adjacent time points so that the difference between joint angular velocity data at adjacent time points is less than or equal to the product of the acceleration limit data and the time difference between adjacent time points.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, if the movable structure includes a chassis and a robotic arm joint, then the cross-structure coordination constraint includes: when the chassis speed data is greater than the motion stability threshold, the joint angle data falls within the second safe angle range.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, if the movable structure includes a chassis and a robotic arm, the target control command is adjusted through a soft constraint strategy, including: determining the target joint angle data and preset safety angle data of the robotic arm; determining a first weighting function and a second weighting function based on the chassis speed data; determining the first product result of the first weighting function and the target joint angle data, and the second product result of the second weighting function and the safety angle data; and adjusting the joint angle data of the robotic arm based on the sum of the first product result and the second product result.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, before performing a safety check on the target control command of the target robot, the method further includes: receiving first control data sent by the virtual reality terminal; determining the delay time of the first control data if the first control data is not outdated; determining second control data of the target robot if the delay time is less than a target time threshold; and obtaining the target control command of the target robot based on the second control data.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, determining the second control data of the target robot includes: determining whether the historical control data sent by the received virtual reality terminal meets the prediction conditions; if the historical control data meets the prediction conditions, predicting the second control data of the target robot based on the historical control data; if the historical control data does not meet the prediction conditions, determining the first control data as the second control data.

[0014] In conjunction with the first aspect, in certain implementations of the first aspect, the second control data of the target robot is predicted based on historical control data, including at least one of the following: processing historical control data using an extended Kalman filter algorithm to generate head posture data of the target robot, the head posture data including pitch angle and yaw angle; processing historical control data using a long short-term memory neural network model to generate joint angle data of the target robot's robotic arm; processing historical control data using a differential drive-based dynamics model and a Kalman filter algorithm to obtain chassis speed data of the target robot; and processing historical control data using a sliding window averaging method and / or a uniform acceleration model to obtain waist state data of the target robot, the waist state data including height data and pitch angle.

[0015] In conjunction with the first aspect, in certain implementations of the first aspect, the sliding window averaging method and / or uniform acceleration model are used to process historical control data to obtain waist state data of the target robot, including: processing historical control data using a uniform acceleration model to obtain first waist state data; processing historical control data using a sliding window averaging method to obtain second waist state data; if the first state data meets the prediction accuracy condition, then the first state data is determined as waist state data; if the first state data does not meet the prediction accuracy condition, then the first weight corresponding to the first state data and the second weight corresponding to the second state data are determined respectively; the sum of the product of the first state data and the first weight and the product of the second state data and the second weight is determined, and the sum is determined as waist state data.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the first control data includes a transmission timestamp. Determining the delay time of the first control data includes: determining the original delay time of the first control data based on the reception time and transmission timestamp of the first control data; determining the system time offset data between the virtual reality terminal and the target robot; and determining the difference between the original delay time and the system time offset data as the delay time of the first control data.

[0017] Secondly, one embodiment of this application provides a data processing apparatus, including: a checking module, used to perform a safety check on a target control command of a target robot, wherein the safety check refers to checking whether the target robot is safe if controlled based on the target control command; a determining module, used to determine the movable structure of the target robot involved in the target control command and the cross-structure cooperative constraint conditions corresponding to the movable structure if the target control command passes the safety check; and an adjusting module, used to adjust the target control command through a soft constraint strategy if the target control command does not meet the cross-structure cooperative constraint conditions.

[0018] Thirdly, one embodiment of this application provides a computer-readable storage medium storing a computer program for performing the data processing method described in the first aspect.

[0019] Fourthly, one embodiment of this application provides an electronic device, the electronic device comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to perform the data processing method described in the first aspect.

[0020] Fifthly, one embodiment of this application provides a computer program product including instructions that, when executed on an electronic device, cause the electronic device to implement the data processing method described in the first aspect.

[0021] In this application, a safety check is first performed on the target control commands to accurately identify potential safety risks and prevent robot malfunctions or safety accidents caused by the direct execution of unsafe control commands. Furthermore, to ensure the structural stability and operational safety of the target robot during high-speed movement or large-amplitude actions, this embodiment introduces cross-structure cooperative constraints on movable structures. These constraints dynamically adjust the controllable space and range of motion through inter-structure state coupling, enabling automatic avoidance of unsafe postures and ensuring that each structure works collaboratively in a reasonable manner. Simultaneously, when commands do not meet the cooperative constraints, a soft constraint strategy is used for adjustment. This ensures that the commands meet the requirements while preserving the original intent to the greatest extent possible, avoiding task deviations caused by rigid modifications, reducing mechanical shocks caused by sudden command changes, extending the robot's lifespan, and ensuring reliable and efficient robot operation to adapt to complex task requirements. Attached Figure Description

[0022] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0023] Figure 1 The diagram shown is a flowchart of a data processing method provided in an embodiment of this application.

[0024] Figure 2 The diagram shown is a schematic diagram of the division of the first safety angle range according to an embodiment of this application.

[0025] Figure 3 The diagram shown is a structural schematic of a data processing apparatus provided in an embodiment of this application.

[0026] Figure 4 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Figure 1 The diagram shown is a schematic flowchart of a data processing method provided in an embodiment of this application. For example, as shown... Figure 1As shown, the method includes the following steps.

[0029] Step S110: Perform a safety check on the target control commands of the target robot.

[0030] Target control instructions are commands used to instruct a target robot to perform specific actions or behaviors. For example, target control instructions include specific values ​​or requirements for parameters such as joint angle data, end effector posture, and movement speed. Safety checks refer to assessing the safety of the target robot if controlled based on the target control instructions. That is, evaluating and verifying the feasibility, reliability, and safety of the target control instructions to ensure that the target robot will not exceed its physical limits, suffer mechanical damage, or collide with its surroundings when executing the instructions, thereby ensuring the safety of the target robot itself, as well as surrounding personnel, equipment, and the environment.

[0031] Optionally, the parameter range of the target control command is determined, including safety thresholds for joint angle data, chassis speed data, head posture, etc. Then, each parameter in the target control command is compared with the preset safety thresholds one by one to determine whether it exceeds the range. For example, if the joint angle data required by the target control command exceeds the maximum range of motion of the joint, the target control command is determined to be an unsafe command.

[0032] Step S120: If the target control command passes the safety check, determine the movable structure of the target robot involved in the target control command, and the cross-structure cooperative constraints corresponding to the movable structure.

[0033] The movable structure of a target robot refers to the parts of the robot that can move or change position, such as the joints of the robotic arm, chassis, head, waist, etc. These movable structures have motion interdependence in different functional scenarios. For example, when the chassis moves at high speed, the robotic arm should be kept in the retracted or safe range to reduce the impact of centrifugal force on the structure; the pitch posture of the waist will affect the visual range and posture limit of the head; the large extension of the robotic arm may obstruct the head's perception and also needs to be restricted by linkage.

[0034] Correspondingly, cross-structure cooperative constraints refer to the specific conditions that need to be met to ensure the overall stability and safety of the robot, as well as the coordination and cooperation between the various structures, when multiple movable structures of a target robot work together. Specifically, cross-structure cooperative constraints typically involve the requirements and limitations of different movable structures in terms of velocity, acceleration, positional relationships, and force transmission.

[0035] Optionally, the target control instructions are parsed to identify the specific movable structures involved. For example, if the target control instruction is to move the end effector of the target robot's robotic arm to a specific position and perform a grasping action, then the movable structures involved include multiple joints of the robotic arm, such as the shoulder joint, elbow joint, and wrist joint. Next, for the identified movable structures, the constraints related to their cooperative operation are extracted from the robot's design specifications, kinematic and dynamic models, as well as past operational experience and safety standards.

[0036] Step S130: If the target control command does not meet the cross-structure coordination constraint conditions, the target control command is adjusted through a soft constraint strategy.

[0037] Soft constraint strategy is a strategy that flexibly adjusts target control instructions. Unlike hard constraints, which directly restrict certain parameters of instructions to a fixed range, soft constraint strategy adjusts instructions that do not meet cross-structure coordination constraints by optimizing algorithms and compensation mechanisms while preserving the original intent of the instructions as much as possible. This ensures that the target control instructions meet coordination requirements and achieve the expected task objectives to the greatest extent.

[0038] Optionally, the target control commands are compared and analyzed with the target robot's dynamics, kinematics model, and cross-structure cooperative constraints to accurately identify factors that do not meet the conditions, such as speed differences. Next, a control adjustment model is constructed that encompasses the physical characteristics of the target robot's movable structures and the cross-structure cooperative constraints. Based on the model, a suitable optimization algorithm, such as gradient descent, is selected to calculate the control command parameters that satisfy the cross-structure cooperative constraints. Then, the target control commands are flexibly adjusted, appropriately modifying parameters such as velocity, acceleration, and angle of each movable structure to meet the cross-structure cooperative constraints while preserving the original command's task intent as much as possible. Finally, the adjusted commands are verified through simulation or operation in a robot virtual environment. If they do not meet the requirements, repeated optimization and adjustments are made until the cross-structure cooperative constraints are satisfied.

[0039] In this embodiment, a safety check is first performed on the target control commands to accurately identify potential safety risks and prevent robot malfunctions or safety accidents caused by the direct execution of unsafe control commands. Furthermore, to ensure the structural stability and operational safety of the target robot during high-speed movement or large-amplitude actions, this embodiment introduces cross-structure cooperative constraints on movable structures. These constraints dynamically adjust the controllable space and range of motion through inter-structure state coupling, enabling automatic avoidance of unsafe postures and ensuring that each structure works collaboratively in a reasonable manner. Simultaneously, when commands do not meet the cooperative constraints, a soft constraint strategy is used for adjustment. This ensures that the commands meet the requirements while preserving the original intent to the greatest extent possible, avoiding task deviations caused by rigid modifications, reducing mechanical shocks caused by sudden command changes, extending the robot's lifespan, and ensuring reliable and efficient robot operation to meet complex task requirements.

[0040] exist Figure 1 Based on the embodiments shown, this application also provides an optional embodiment, which can improve the security of the target robot when executing control commands by constructing a multi-dimensional dynamic security verification mechanism, the specific implementation of which is described below.

[0041] The target control command includes at least one of joint angle data, joint angular velocity data, and chassis speed data. A safety check is performed on the target control command of the target robot, including at least one of the following: checking whether the joint angle data falls within the first safe angle range; checking whether the difference between joint angular velocity data at adjacent times is less than the target safe angular velocity difference; and checking whether the chassis speed data is less than the target safe speed.

[0042] Specifically, joint angle data is used to describe the angular positions that the robot joints need to reach. Correspondingly, the first safe angle range is a safe range set for the joint angles of the target robot. Optionally, this range is determined based on factors such as the robot's physical structure, mechanical strength, and kinematic characteristics to ensure that when the target robot moves within this range, the joints will not experience excessive bending, stretching, or collisions with other components, which could pose a danger. Optionally, for the joint angle data θ... t Compare it with the pre-set first safety angle range [θ] min ,θ max Compare the results to check if they exceed the specified range.

[0043] The target safe angular velocity difference is the maximum allowable change in joint angular velocity data between adjacent moments, designed to prevent sudden changes in joint angular velocity. Specifically, if the difference is less than the target safe angular velocity difference, it indicates that the change in joint angular velocity between adjacent moments is within a safe range, avoiding mechanical shocks, vibrations, and other problems caused by sudden changes in angular velocity, thereby ensuring the stability and safety of the target robot's joints during movement.

[0044] Chassis speed data refers to the speed at which the target robot's chassis moves, including linear velocity v and angular velocity ω. Correspondingly, the target safe speed is a safe upper limit for the target robot's chassis speed, including v... max and ω max The parameters are set comprehensively based on factors such as the target robot's chassis load-bearing capacity, motor performance, and motion environment. When |v|≤v max ,|ω|≤ω max Only when the target robot chassis moves safely and stably can it avoid problems such as loss of control, slippage, or impact on the robot structure caused by excessive speed.

[0045] In some embodiments, the target control command also includes head posture data and waist posture data. Similarly, the head posture data and waist posture data each correspond to a safe range interval, and a safety check is performed on the target control command, that is, to check whether the head posture data and waist posture data are within this safe range interval.

[0046] This embodiment uses a multi-dimensional dynamic safety verification mechanism to comprehensively check the joint angle data, joint angular velocity data, and chassis speed data in the target control commands. It effectively determines whether the target control commands exceed the safety range or have the risk of sudden speed changes, thereby timely discovering and avoiding potential safety hazards, significantly improving the safety and stability of the target robot when executing commands, and ensuring its reliable operation.

[0047] Furthermore, based on the above checks of the target control commands, if the difference in joint angular velocity data between adjacent moments is found to be greater than or equal to the target safe angular velocity difference, this application also provides a corresponding processing strategy to determine that the joints of the target robot will not experience sudden changes or loss of control within a unit of time. Specifically, this includes: determining acceleration limit data; adjusting the joint angular velocity data between adjacent moments so that the difference in joint angular velocity data between adjacent moments is less than or equal to the product of the acceleration limit data and the time difference between adjacent moments.

[0048] Specifically, when the difference in joint angular velocity between adjacent moments is detected... ≥ Target safety angular velocity difference θ max First, the acceleration limit data a must be determined. max Optionally, it is a threshold set based on the maximum acceleration that the target robot's joints can withstand and the stability requirements during movement. Then, the joint angular velocity data at adjacent time points are adjusted, with the goal of making... ≤Acceleration limit data a maxThe product of the time difference Δt between adjacent moments. The reason is that by limiting the rate of change of the joint angular velocity, sudden changes in angular velocity of the joint in a short period of time can be avoided, preventing the impact, vibration or even damage to the joint's mechanical structure caused by excessively rapid changes in angular velocity. At the same time, it ensures the smoothness and controllability of the target robot's motion, ensuring that the joint changes speed in a safe and reasonable manner, so that the target robot can execute control commands stably and reliably.

[0049] Similarly, if the joint angle data is found to be outside the safe range, this application also provides a corresponding processing strategy to ensure that the joint angle data is always in a safe and controllable state. Specifically, if the joint angle data exceeds the first safe angle range, a soft saturation strategy is used to process the joint angle data so as to control the joint angle data to be smoothly attenuated within the target buffer of the first safe angle range.

[0050] Soft saturation refers to a process where, as joint angle data approaches its limit (saturation), the output is not abruptly restricted, but rather gradually approaches saturation in a smooth and gradual manner. This avoids control oscillations or discontinuous jumps caused by hard truncation, effectively limiting joint angle data while preventing abrupt changes from impacting the robot's control system. This contrasts with hard saturation, where the output is immediately limited to a fixed value after reaching a certain threshold.

[0051] The target buffer is a specific transition area set within the first safe angle range. It serves as a buffer zone for data attenuation adjustment when the joint angle data exceeds the safe angle range, ensuring that the joint angle data decreases smoothly and preventing mechanical shocks or control instability caused by drastic data changes. Optionally, the target buffer is located on both sides of the first safe angle range.

[0052] When the joint angle data exceeds the first safe angle range, the joint angle data located in the target buffer is smoothly decayed, gradually bringing the joint angle data back to the first safe angle range. This prevents the joints from going out of control due to abnormal angle data, and the smooth decay avoids sudden data changes, ensuring the smoothness and controllability of the target robot's movement, and improving the safety and reliability of the target robot when executing control commands.

[0053] To process joint angle data that exceeds the first safe angle range more accurately, this application further details the processing flow to ensure that the joint angle data transitions smoothly within the safe range. The specific implementation is as follows.

[0054] The joint angle data is processed using a soft saturation strategy, including: determining a first target buffer located on the lower limit side and a second target buffer located on the upper limit side within the first safe angle interval based on the boundary values ​​on both sides of the first safe angle interval and the preset buffer width; determining the lower limit value of the first target buffer and the upper limit value of the second target buffer; adjusting the joint angle data within the first target buffer to the sum of the lower limit value and a first correction amount; and adjusting the joint angle data within the second target buffer to the difference between the upper limit value and a second correction amount.

[0055] The preset buffer width refers to the width range of the buffer area pre-set within the first safe angle range. It is used to define the transition range where the joint angle data can be adjusted, so as to avoid the joint angle data directly touching the hard boundary of the safe range and causing sudden changes or impacts.

[0056] Figure 2 The diagram shown is a schematic representation of the division of a first safety angle range according to an embodiment of this application. Figure 2 As shown, the two endpoints of the first safety angle interval are the lower limit and the upper limit, respectively. The first target buffer is located in the buffer area on the lower limit side of the first safety angle interval. It is a range formed by extending the lower limit of the first safety angle interval towards larger joint angle data based on a preset buffer width. Correspondingly, the second target buffer is located in the buffer area on the upper limit side of the first safety angle interval. It is a range formed by extending the upper limit of the safety angle interval towards smaller joint angle data based on a preset buffer width.

[0057] Specifically, when the joint angle data is within the first target buffer, it is adjusted to the lower limit plus an appropriate first correction amount. The first correction amount characterizes the value used for smooth transition adjustment of the joint angle data within the first target buffer. The first correction amount varies as the joint angle data approaches the lower limit. Optionally, the closer to the lower limit, the smaller the first correction amount, so that the joint angle data is stabilized within the first safe angle range.

[0058] Within the second target buffer, when the joint angle data is located within the second target buffer, it is adjusted to the upper limit value minus an appropriate second correction amount. The second correction amount characterizes the value used for smooth transition adjustment of the joint angle data within the second target buffer. The second correction amount varies as the joint angle data approaches the upper limit value; optionally, the closer to the upper limit value, the smaller the second correction amount, so that the joint angle data adjusts into the safe angle range more quickly.

[0059] In this embodiment, a target buffer zone is determined based on a first safety angle range and a preset buffer zone width, and its lower and upper limits are clearly defined. Subsequently, within the corresponding buffer zones, the joint angle data is adjusted to the sum of the lower limit and the first correction amount, and the difference between the upper limit and the second correction amount, respectively, achieving smooth adjustment of the joint angle data. This not only avoids mechanical shocks and vibrations caused by sudden changes in joint angle data, but also ensures the stability and controllability of the target robot's joint movements, improving the stability and reliability of the target robot in complex tasks.

[0060] Optionally, a first difference is obtained by subtracting the lower limit value and the preset buffer width from the joint angle data. The ratio of the first difference to the steepness control parameter is determined as the input value of the soft saturation function to obtain a first transition value. The first transition value is added to a first constant and then multiplied by the preset buffer width and a first coefficient to obtain a first correction amount.

[0061] The second difference is obtained by subtracting the preset buffer width and joint angle data from the upper limit value. The ratio of the second difference to the steepness control parameter is determined as the input value of the soft saturation function to obtain the second transition value. The second transition value is added to the second constant and then multiplied by the preset buffer width and the second coefficient to obtain the second correction amount.

[0062] The kurtosis control parameter controls the steepness of the soft saturation function curve, adjusting the rate at which the function output changes with the input value. The first and second constants are fixed values ​​used to calculate the first and second transition values, respectively, adjusting the magnitude and range of the transition values. The first and second coefficients adjust the magnitude of the correction to suit different joint and control requirements. The goal of the soft saturation function is to maintain the original joint angle data between the first and second target buffers; and to smoothly compress the joint angle data within the first and second target buffers.

[0063] For example, the lower limit value is denoted as θ. min The upper limit is denoted as θ max Let the preset buffer width be denoted as δ, the soft saturation function as f(x), the kurtosis control parameter as ε, the first constant as b1, the second constant as b2, the first coefficient as a1, and the second coefficient as a2. Then,

[0064] For example, the kurtosis control parameter is denoted as ε∈[0.01,0.1], and the soft saturation function is the hyperbolic tangent function tanh. In this case, the first constant b1 and the second constant b2 can be determined to be 1, and the first coefficient a1 and the second coefficient a2 can be determined to be... The reason is that the tanh function can smoothly transition from [-∞, +∞] to [-1, +1]. That is, the numerical range of the first and second transition values ​​is [-1, +1]. After adding the constant 1, the value is converted to [0, 2]. Multiplying by... After that, the numerical range of the first and second correction values ​​is [0, δ].

[0065] In this embodiment, the first difference obtained based on the joint angle data, the lower limit value, and the preset buffer width is compared with the steepness control parameter and used as the input of the soft saturation function to obtain the first transition value. This transition value is then added to a first constant and subjected to subsequent calculations to obtain the first correction value. Similarly, the second difference obtained based on the upper limit value, the preset buffer width, and the joint angle data is processed in a similar manner to obtain the second correction value. Finally, through soft saturation function processing, the correction value changes according to how close the joint angle data is to the boundary of the buffer, achieving smooth adjustment, avoiding abrupt changes in joint angle data, and ensuring the stability and safety of the target robot's motion, thereby improving control accuracy and reliability.

[0066] In practical applications, cross-structure coordination constraints are particularly critical for target robots that include movable structures such as chassis and robotic arm joints. Therefore, this application defines in detail the safety requirements that robotic arm joint angle data must meet when chassis velocity data exceeds a motion stability threshold, to ensure the stability and safety of the target robot. Specifically, the cross-structure coordination constraints include: when chassis velocity data exceeds the motion stability threshold, the joint angle data must fall within a second safe angle range.

[0067] Specifically, it can be expressed as Among them, ||V t ‖ represents the current chassis speed data (the composite modulus of linear velocity and angular velocity), V limit θ represents the preset motion stability threshold. arm Represents joint angle data, [θ fold ,θ safe ] indicates the second safety angle range defined in the structural design.

[0068] By limiting the range of joint angle data when the chassis speed data is too high, it is possible to prevent the robotic arm from making movements beyond the feasible range under the current chassis speed data. This avoids problems such as overall robot imbalance, vibration, or even tipping over due to the lack of coordination between the robotic arm and chassis movements, thereby effectively improving the robot's motion stability and enhancing its adaptability and reliability in practical application scenarios. Figure 1Based on the illustrated embodiment, when the movable structure of the target robot includes a chassis and a robotic arm, this application provides an optional embodiment that uses a soft constraint strategy to specifically adjust the target control commands to achieve coordinated control between chassis speed data and joint angle data. Specifically, this includes: determining the target joint angle data and preset safety angle data of the robotic arm; determining a first weighting function and a second weighting function based on the chassis speed data; determining a first product result of the first weighting function and the target joint angle data, and a second product result of the second weighting function and the safety angle data; and adjusting the joint angle data of the robotic arm based on the sum of the first and second product results.

[0069] The target joint angle data represents the desired angle that the robotic arm joints will achieve. The safe angle data represents the maximum safe angle that the robotic arm joints are allowed to reach under the current chassis speed.

[0070] Optionally, the first weighting function is a velocity-driven smoothing weighting function, denoted as α(v t ), α(v t For values ​​∈ [0,1], a Sigmoid, Tanh, or linear decay function can be used. Correspondingly, the second weighting function can be denoted as 1-α(v t Ultimately, the adjusted joint angle data of the robotic arm can be expressed as α(v). t )×θ goal +(1-α(v t ))×θ safe Where θ goal Represents the target joint angle data, θ safe This represents data from a safety perspective.

[0071] This solution achieves precise adjustment of the robotic arm's joint angles by comprehensively considering the target joint angle data and preset safety angle data, and dynamically determining the weighting function based on the chassis speed data. Specifically, the chassis speed data determines the weighting function, which in turn affects the adjustment range of the joint angle data, enabling close coordination between the chassis and the robotic arm. That is, when the chassis speed data changes, the robotic arm's joint angle data can be adjusted promptly according to the safety angle data, ensuring the stability and safety of the robot's overall movement and avoiding conflicts in the coordinated movement between the robotic arm and the chassis.

[0072] Before performing a safety check on the target control commands of the target robot, this application provides an optional solution to ensure the timeliness and accuracy of the target control commands, specifically including: receiving first control data sent by a virtual reality terminal; determining the delay time of the first control data if the first control data is not outdated; determining second control data of the target robot if the delay time is less than a target time threshold; and obtaining the target control commands of the target robot based on the second control data.

[0073] Optionally, the system first receives the initial control data sent by the virtual reality terminal and checks whether this data is outdated. The initial control data refers to the initial control data sent by the virtual reality terminal, which is the original basis for generating the target control command. Specifically, outdated data indicates that the data receiving order is inconsistent with the sending order; data sent before the currently received data but not received in sequence is outdated data. For example, if the sending order of the initial control data is data A, data B, and data C, while the target robot receives data in the order of data B, data C, and data A, then data A is outdated.

[0074] Then, for the first control data that is not outdated, its delay time is calculated; that is, the time difference between the first control data being sent and received, used to measure whether the data is invalid. If the delay time is less than a target time threshold, it indicates that the network condition is good, and the second control data is then determined. Understandably, if the delay time is greater than the target time threshold, it is considered that there is a potential network anomaly, such as network jitter, bandwidth bottlenecks, or packet retransmission problems, and the first control data is usually discarded. Optionally, the second control data is further determined as valid control data based on the first control data that is not outdated and has a acceptable delay time. Finally, the target control instructions for the target robot are generated based on the second control data.

[0075] For example, in remotely controlled robotic surgery, the initial control data sent from the virtual reality terminal includes the surgeon's instructions, such as joint angle data and movement speed data of the robotic arm. If the initial control data is delayed for too long due to network issues, exceeding a target time threshold (e.g., 0.5 seconds), it is considered no longer accurately reflecting the surgeon's current intentions and is discarded. Only when the delay time of the initial control data is less than the target time threshold will it be further processed to obtain the second control data, ultimately generating the target control instructions to precisely control the target robot, ensuring the accuracy and safety of the surgical operation.

[0076] In this embodiment, the system first checks whether the first control data is outdated to avoid instruction errors caused by disordered data sequence. Then, it determines the delay time and compares it with a target time threshold to identify and eliminate first control data that is unreliable due to severe delays. Finally, the system generates target control commands based on the second control data, effectively improving the control accuracy and response speed of the target robot and enhancing its stability and reliability in complex tasks.

[0077] To accurately calculate the delay time of the first control data, the embodiments of this application further refine the relevant steps, specifically including: determining the original delay time of the first control data based on the reception time of the first control data and the transmission timestamp contained in the first control data; determining the system time offset data between the virtual reality terminal and the target robot; and determining the difference between the original delay time and the system time offset data as the delay time of the first control data.

[0078] The delay time of the first control data is intended to assess whether the communication link between the virtual reality terminal, the network, and the target robot terminal is abnormal, thereby further determining whether the virtual reality terminal is in an acceptable communication state.

[0079] Assume the reception time of the first control data is t. now The first control data includes a transmission timestamp of t. data The system time offset data is t sys Therefore, the initial delay time of the first control data is t. now -t data The system offset time data t sys =min(Δt) sys ,t now -t data That is, the system offset time data t sys By taking the minimum value between the current system time offset data and the historical system time offset data, the delay time is less susceptible to interference from occasional network jitter, effectively avoiding misjudgments. Furthermore, the delay time L of the first control data = (t... now -t data )-t sys The latency here reflects network latency fluctuations, not timestamp errors, and is therefore more meaningful.

[0080] When the delay time L≈0, it indicates that the delay time is stable; L>0 can be used to determine whether to discard the first control data. This strategy is more suitable for decision-making logic in robot control systems.

[0081] In many remote control systems (such as virtual reality and robot execution), the system times of the two devices are often inconsistent, which can lead to direct use of t now -tdata The calculated delay time is inaccurate. This application, however, considers the system offset time, thereby compensating for this fixed time offset and improving the accuracy and robustness of the delay time calculation. Furthermore, compared to synchronization mechanisms such as NTP and ROS Time Synchronizer, this delay time calculation method does not require pre-synchronization or rely on additional synchronization messages. Moreover, it can directly correct delays through the data stream itself, making it ideal for lightweight, low-bandwidth remote control systems.

[0082] After making a preliminary judgment on the timeliness of the first control data, this application further provides an optional embodiment for determining the second control data of the target robot, which specifically includes: determining whether the historical control data sent by the received virtual reality terminal meets the prediction conditions; if the historical control data meets the prediction conditions, predicting the second control data of the target robot based on the historical control data; if the historical control data does not meet the prediction conditions, determining the first control data as the second control data.

[0083] Optionally, the target robot can use a sliding window or a time window to cache historical control data sent from the virtual reality terminal. Assuming the sequence length of the currently cached historical control data is N, the minimum amount of data required to satisfy the prediction conditions is N. min Then, when N≥N min This triggers prediction, that is, predicting the second control data of the target robot based on historical control data. When N <N min At that time, the historical control data accumulated in the instruction manual was insufficient, and even if prediction was made, the accuracy of the predicted control data could not be guaranteed. Therefore, the first control data was directly determined as the second control data.

[0084] In remote control systems (virtual reality terminals), communication latency and data instability can severely impact control accuracy and system responsiveness. To improve the real-time performance and stability of control, this embodiment introduces a state prediction mechanism. However, state prediction typically relies on a certain amount of historical data; therefore, the prediction process needs to be initiated when prediction conditions are met to compensate for data loss or delays, thereby improving the accuracy and continuity of control commands. When historical data does not meet the conditions, the first control data is directly used as the second control data, simplifying the processing flow and ensuring timely response from the control system.

[0085] Below, this application also provides a variety of prediction algorithms to adapt to different types of second control data and prediction requirements, the specific implementation of which is described below.

[0086] In some embodiments, the extended Kalman filter algorithm is used to process historical control data to generate head posture data of the target robot, including pitch angle and yaw angle.

[0087] Specifically, the target robot's attitude angle and angular velocity are used as state variables. Let θ p and θ y Let these represent the pitch angle and yaw angle, respectively, and their state vectors are defined as follows:

[0088]

[0089] in, Indicates the current pitch and yaw angles. This represents the angular velocity corresponding to the current pitch and yaw angles, with the subscript t indicating the current time.

[0090] Assuming that the angular velocity remains constant or changes slowly over a short period of time, the state transition of the head posture can be modeled as follows:

[0091]

[0092] Where Δt is the prediction period, This is process noise.

[0093] Furthermore, the system observations are the attitude angles measured at the current moment:

[0094]

[0095] in To observe noise.

[0096] Furthermore, the extended Kalman filter algorithm mainly includes two stages within each control cycle: state prediction and observation update. This process is used to fuse historical control data with current sensor observation data to obtain the optimal estimate of the target robot's head posture, which can then be used for posture prediction and control compensation in the next time step.

[0097] First, in the state prediction phase, based on the state estimate from the previous moment and combined with the known motion model, the state at the current moment is extrapolated and predicted. Specifically, the process is as follows: Based on the current angular velocity, the changes in pitch and yaw angles within the current cycle are estimated. Assuming the angular velocity remains constant or changes slowly over a short period, the angular velocity from the previous cycle is used as the predicted value for the current cycle. Simultaneously, the increase in uncertainty introduced by the prediction is assessed, and the state covariance matrix is ​​updated to reflect the confidence level of the current prediction. Finally, the predicted state of the current head attitude and the associated covariance matrix are output.

[0098] Secondly, in the observation update phase, the main focus is on correcting prediction errors and improving estimation accuracy. Specifically, this involves comparing the actual head pose observed by the sensors with the predicted head pose, calculating the difference between the two, known as the observation residual. Then, based on the uncertainty of the prediction result and the accuracy of the observation data, the weight ratio of the two in the update is dynamically calculated. Optionally, this process is automatically completed using Kalman gain to ensure the target robot achieves an optimal balance between prediction and observation. Next, based on the residual and weights, the previous prediction state is corrected to obtain a more accurate estimate of the head pose at the current moment. Simultaneously, the covariance estimate of the current head pose is updated to reflect that the uncertainty of the current prediction has been partially corrected by the observations. Finally, the optimal pose estimate (including pitch and yaw angles) and the latest confidence estimate are output.

[0099] This method can effectively predict short-term head posture changes smoothly, and it has good suppression capabilities for noise and jitter. It has low computational cost and is suitable for real-time operation.

[0100] In some embodiments, a long short-term memory neural network model is used to process historical control data to generate joint angle data of the target robot's robotic arm.

[0101] Robotic arms typically have multiple degrees of freedom at their joints, and their angle control exhibits highly nonlinear and temporally correlated characteristics. Furthermore, significant differences exist among users in their control styles and gesture habits, limiting the effectiveness of traditional physics-based prediction methods in practical applications. To address this, this application introduces a sequence modeling method based on long short-term memory neural networks. This method utilizes historical time-series data to learn the user's control intentions, achieving high-precision prediction of the next joint angle.

[0102] Specifically, suppose the robotic arm has n joints, and the joint angle vector observed at each time t is: The objective is to determine the state sequence S based on the past N consecutive observations. t {θ t-N+1 ,……,θ t}, predict the joint angle data for the next moment. Among them, f LSTM This represents the long short-term memory neural network model obtained through training.

[0103] For example, the input dimension of a Long Short-Term Memory (LSTM) neural network model is a sequence tensor with a time window length of N and n-dimensional joint angle data per frame, for a total input of N×n. The output dimension is the joint angle vector at the next time step, with a dimension of n. For example, for a 6-joint robotic arm, if N=20 frames are selected, the input dimension is 20×6, and the output is a 6-dimensional vector.

[0104] For example, the Long Short-Term Memory (LSTM) neural network model employs a standard LSTM encoder structure, paired with a fully connected prediction head. The specific structure includes:

[0105] Input embedding layer: performs a linear mapping on the n-dimensional angle vector of each frame;

[0106] LSTM encoder: processes time series data and learns the time-dependent characteristics of joint control;

[0107] Time feature compression module: retrieves the hidden state h of the last time step. t Represented as a whole;

[0108] Output layer (fully connected): This will connect h... t The mapping is to the predicted joint angle vector.

[0109] For example, the training process of the long short-term memory neural network model adopts a supervised learning approach, uses a large amount of historical manipulation data to fit the model, and uses the mean squared error as the regression objective function.

[0110] In some embodiments, historical control data is processed using a differential-driven dynamic model and a Kalman filter algorithm to obtain the chassis speed data of the target robot.

[0111] In virtual reality operating systems, the chassis's motion response is smoother and its control frequency is more stable compared to that of a robotic arm, typically following a behavior pattern of approximately uniform speed or smooth acceleration. This characteristic gives it good physical modelability and state predictability. Therefore, a prediction strategy based on a differential drive dynamic model and a Kalman filter algorithm is adopted to perform short-time prediction and filtering of the chassis's linear velocity v and angular velocity ω.

[0112] Specifically, for a differential-driven robot, its motion state is mainly described by linear velocity v and angular velocity ω. Therefore, the following simplified first-order motion model is used in the prediction:

[0113] Linear velocity prediction Angular velocity prediction Among them, a t With α t These are the current linear acceleration and angular acceleration, respectively.

[0114] In state modeling using the Kalman filter algorithm, the following KF state vector is constructed:

[0115]

[0116] Among them, v t a represents linear velocity. t ω represents linear acceleration. t α represents angular velocity. tIt represents angular acceleration.

[0117] Next, the state transition model (prediction) is represented as: x t+1 =A×x t +w t Where Δt is the discrete time step, This represents process noise, indicating uncertainty or system jitter during modeling.

[0118] The observation model (where velocity and angular velocity are measurable) can be expressed as: z t =H×x t +v t .

[0119] Observation vector Among them, observation noise This indicates the speed sensor or estimation error.

[0120] As mentioned earlier, the Kalman filter algorithm's execution process consists of two main stages: the prediction stage and the update stage. In the prediction stage, the current state is predicted based on the state estimation results from the previous time step, combined with the state transition relationships of the differential-driven dynamic model. Simultaneously, the uncertainty of the state (i.e., covariance) is forward-propagated. Then, in the update stage, upon receiving new observation data, the Kalman filter first calculates the Kalman gain for the current time step to measure the degree to which the new observation corrects the current prediction. Subsequently, the state prediction result is corrected using the difference between the observed and predicted values, combined with the Kalman gain. Finally, the covariance matrix is ​​updated again to reflect the latest estimate of the state uncertainty.

[0121] This iterative process enables stable, continuous, and robust estimation of chassis speed data even in the presence of noise and latency, providing reliable support for short-term predictions of future moments.

[0122] Optionally, if the system has a delay δ, iterative prediction can be further performed to obtain...

[0123] In some embodiments, historical control data are processed using a sliding window averaging method and / or a uniform acceleration model to obtain waist state data of the target robot, which includes height data and pitch angle.

[0124] The state of the robot's waist includes two continuous variables: the vertical height h. t With pitch angle θ t This part is controlled by the operational task, has a low frequency of change and weak dynamics, and has good short-term predictability. Considering its stationarity and low noise characteristics, a sliding window averaging method and a uniform acceleration model are used to jointly predict the state at the next moment.

[0125] Specifically, the moving average method uses the state values ​​of the previous N frames at the current time to perform an equal-weighted or weighted average, thereby estimating the state at the next time step. Its prediction form is as follows: x t ∈{h t θ t} represents the current height or tilt angle, and N represents the window length. This represents the predicted state value at the next moment.

[0126] In the uniform acceleration model prediction and compensation, the current velocity and acceleration are first estimated, where the current velocity (using finite difference) is: Current acceleration: Based on this, a uniform acceleration model is used to estimate the state at the next time step.

[0127] Specifically, the prediction results of the velocity prediction model (without acceleration) are expressed as follows:

[0128] The prediction results of the acceleration compensation model (with acceleration) are expressed as follows:

[0129] In practical applications, to achieve accurate prediction and fusion of lumbar state data, this application further refines the processing flow based on the uniform acceleration model and the sliding window averaging method. Specifically, the uniform acceleration model is used to process historical control data to obtain the first lumbar state data; the sliding window averaging method is used to process historical control data to obtain the second lumbar state data; if the first lumbar state data meets the prediction accuracy condition, then the first lumbar state data is determined as the lumbar state data; if the first lumbar state data does not meet the prediction accuracy condition, then the first weight corresponding to the first lumbar state data and the second weight corresponding to the second lumbar state data are determined respectively; the sum of the product of the first lumbar state data and the first weight, and the product of the second lumbar state data and the second weight is determined, and the sum is determined as the lumbar state data.

[0130] For example, when the first state data does not meet the prediction accuracy condition, a first weight λ1 and a second weight λ2 are determined. Then, the waist state data can be represented as... Optionally, λ2 = 1 - λ1. Wherein, λ1 can be dynamically adjusted according to the rate of change of velocity or error to achieve a smoother or more agile prediction effect.

[0131] In this embodiment, by combining the uniform acceleration model with the sliding window averaging method, when predicting waist state data, the uniform acceleration model is first used to capture the trend of state change, and then the sliding window averaging method is used to smooth the data and reduce the impact of noise. When the prediction result of the uniform acceleration model meets the accuracy condition, it is directly adopted; otherwise, the results of the two methods are corrected by weighted fusion, thereby improving the prediction accuracy and enhancing the system's adaptability and reliability.

[0132] The data processing method embodiments of this application have been described in detail above. The following will be combined with… Figure 3 This application provides a detailed description of embodiments of the data processing apparatus. It should be understood that the descriptions of the data processing method embodiments correspond to the descriptions of the data processing apparatus embodiments; therefore, any parts not described in detail can be found in the foregoing method embodiments.

[0133] Figure 3 The diagram shown is a structural schematic of a data processing apparatus provided in an embodiment of this application. Figure 3 As shown, the data processing apparatus 30 provided in this embodiment includes:

[0134] The inspection module 310 is used to perform a safety check on the target control instructions of the target robot. The safety check refers to checking whether the target robot is safe if it is controlled based on the target control instructions.

[0135] The determination module 320 is used to determine the movable structure of the target robot involved in the target control command and the cross-structure cooperative constraints corresponding to the movable structure, provided that the target control command passes the safety check.

[0136] The adjustment module 330 is used to adjust the target control commands through a soft constraint strategy when the target control commands do not meet the cross-structure coordination constraints.

[0137] In one embodiment of this application, the target control command includes at least one of joint angle data, joint angular velocity data, and chassis speed data. The checking module 310 is further configured to: check whether the joint angle data falls within the first safe angle range; check whether the difference between joint angular velocity data at adjacent times is less than the target safe angular velocity difference; and check whether the chassis speed data is less than the target safe speed.

[0138] In one embodiment of this application, the inspection module 310 is further configured to process the joint angle data using a soft saturation strategy if the joint angle data exceeds the first safe angle range, so as to control the joint angle data to be smoothly attenuated within the target buffer of the first safe angle range.

[0139] In one embodiment of this application, the inspection module 310 is further configured to: determine a first target buffer located on the lower limit side and a second target buffer located on the upper limit side in the first safe angle interval based on the boundary values ​​on both sides of the first safe angle interval and the preset buffer width; determine the lower limit value of the first target buffer and the upper limit value of the second target buffer; within the first target buffer, adjust the joint angle data to the sum of the lower limit value and a first correction amount, wherein the first correction amount is used to characterize the value of smooth transition adjustment of the joint angle data within the first target buffer; within the second target buffer, adjust the joint angle data to the difference between the upper limit value and the second correction amount, wherein the second correction amount is used to characterize the value of smooth transition adjustment of the joint angle data within the second target buffer.

[0140] In one embodiment of this application, the inspection module 310 is further configured to: obtain a first difference by subtracting a lower limit value and a preset buffer width from the joint angle data; determine the ratio of the first difference to the steepness control parameter as the input value of the soft saturation function to obtain a first transition value; add the first transition value to a first constant and multiply it by the preset buffer width and a first coefficient to obtain a first correction amount; obtain a second difference by subtracting the preset buffer width and the joint angle data from the upper limit value; determine the ratio of the second difference to the steepness control parameter as the input value of the soft saturation function to obtain a second transition value; add the second transition value to a second constant and multiply it by the preset buffer width and a second coefficient to obtain a second correction amount.

[0141] In one embodiment of this application, the inspection module 310 is further configured to: determine acceleration limit data if the difference between joint angular velocity data at adjacent times is greater than or equal to the target safe angular velocity difference; and adjust the joint angular velocity data at adjacent times so that the difference between joint angular velocity data at adjacent times is less than or equal to the product of the acceleration limit data and the time difference between adjacent times.

[0142] In one embodiment of this application, if the movable structure includes a chassis and a robotic arm joint, the cross-structure coordination constraint includes: when the chassis speed data is greater than the motion stability threshold, the joint angle data falls within the second safe angle range.

[0143] In one embodiment of this application, if the movable structure includes a chassis and a robotic arm, the adjustment module 330 is further configured to: determine target joint angle data and preset safety angle data of the robotic arm; determine a first weighting function and a second weighting function based on the chassis speed data of the chassis; determine a first product result of the first weighting function and the target joint angle data, and a second product result of the second weighting function and the safety angle data; and adjust the joint angle data of the robotic arm based on the sum of the first product result and the second product result.

[0144] In one embodiment of this application, the inspection module 310 is further configured to: receive first control data sent by the virtual reality terminal; determine the delay time of the first control data if the first control data is not outdated; determine the second control data of the target robot if the delay time is less than a target time threshold; and obtain the target control command of the target robot based on the second control data.

[0145] In one embodiment of this application, the inspection module 310 is further configured to: determine whether the historical control data sent by the received virtual reality terminal meets the prediction conditions; if the historical control data meets the prediction conditions, predict the second control data of the target robot based on the historical control data; if the historical control data does not meet the prediction conditions, determine the first control data as the second control data.

[0146] In one embodiment of this application, the inspection module 310 is further configured to: process historical control data using an extended Kalman filter algorithm to generate head posture data of the target robot, the head posture data including pitch angle and yaw angle; process historical control data using a long short-term memory neural network model to generate joint angle data of the target robot's robotic arm; process historical control data using a differential drive-based dynamics model and a Kalman filter algorithm to obtain chassis speed data of the target robot; and process historical control data using a sliding window averaging method and / or a uniform acceleration model to obtain waist state data of the target robot, the waist state data including height data and pitch angle.

[0147] In one embodiment of this application, the inspection module 310 is further configured to: process historical control data using a uniform acceleration model to obtain first state data of the waist; process historical control data using a sliding window averaging method to obtain second state data of the waist; if the first state data meets the prediction accuracy condition, then determine the first state data as waist state data; if the first state data does not meet the prediction accuracy condition, then determine the first weight corresponding to the first state data and the second weight corresponding to the second state data respectively; determine the sum of the product of the first state data and the first weight, and the product of the second state data and the second weight, and determine the sum as waist state data.

[0148] In one embodiment of this application, the first control data includes a transmission timestamp. The inspection module 310 is further configured to: determine the original delay time of the first control data based on the reception time and transmission timestamp of the first control data; determine the system time offset data between the virtual reality terminal and the target robot; and determine the difference between the original delay time and the system time offset data as the delay time of the first control data.

[0149] Below, for reference Figure 4 This describes an electronic device according to embodiments of the present application. Figure 4The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application.

[0150] like Figure 4 As shown, the electronic device 40 includes one or more processors 401 and memory 402.

[0151] The processor 401 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 40 to perform desired functions.

[0152] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement the data processing methods of the various embodiments of this application described above and / or other desired functions. The computer-readable storage medium may also store various contents such as target control instructions, cross-structure cooperative constraints, soft constraint strategies, etc.

[0153] In one example, the electronic device 40 may also include an input device 403 and an output device 404, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0154] The input device 403 may include, for example, a keyboard, a mouse, etc.

[0155] The output device 404 can output various information to the outside, including target control commands, cross-structure cooperative constraints, soft constraint strategies, etc. The output device 404 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0156] Of course, for the sake of simplicity, Figure 4 Only some of the components of the electronic device 40 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 40 may include any other suitable components depending on the specific application.

[0157] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of the data processing methods according to various embodiments of this application as described above.

[0158] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0159] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the data processing methods according to various embodiments of this application described above.

[0160] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0161] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0162] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0163] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0164] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0165] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A data processing method, characterized in that, include: Receive the first control data sent by the virtual reality terminal; If the first control data is not outdated, determine the delay time of the first control data; If the delay time is less than the target time threshold, determine whether the historical control data sent by the received virtual reality terminal meets the prediction conditions; If the historical control data meets the prediction conditions, the second control data of the target robot is predicted based on the historical control data; if the historical control data does not meet the prediction conditions, the first control data is determined as the second control data. Based on the second control data, the target control command of the target robot is obtained; A safety check is performed on the target control instructions for the target robot. The safety check refers to checking whether the target robot is safe if it is controlled based on the target control instructions. If the target control command passes the safety check, determine the movable structure of the target robot involved in the target control command, and the cross-structure cooperative constraints corresponding to the movable structure; If the target control command does not meet the cross-structure coordination constraints, the target control command is adjusted using a soft constraint strategy.

2. The data processing method according to claim 1, characterized in that, The target control command includes at least one of joint angle data, joint angular velocity data, and chassis speed data. The safety check of the target control command of the target robot includes at least one of the following: Check whether the joint angle data falls within the first safe angle range; Check whether the difference between the joint angular velocity data at adjacent time points is less than the target safe angular velocity difference; Check whether the chassis speed data is less than the target safe speed.

3. The data processing method according to claim 2, characterized in that, Also includes: If the joint angle data exceeds the first safe angle range, a soft saturation strategy is used to process the joint angle data in order to control the joint angle data to decay smoothly within the target buffer of the first safe angle range.

4. The data processing method according to claim 3, characterized in that, The process of processing the joint angle data using a soft saturation strategy includes: Based on the boundary values ​​on both sides of the first safety angle interval and the preset buffer width, determine the first target buffer located on the lower limit side and the second target buffer located on the upper limit side in the first safety angle interval; Determine the lower limit of the first target buffer and the upper limit of the second target buffer; Within the first target buffer, the joint angle data is adjusted to the sum of the lower limit value and the first correction amount, where the first correction amount is used to characterize the value of smooth transition adjustment of the joint angle data within the first target buffer. Within the second target buffer, the joint angle data is adjusted to the difference between the upper limit value and the second correction amount, whereby the second correction amount characterizes the value used for smooth transition adjustment of the joint angle data within the second target buffer.

5. The data processing method according to claim 4, characterized in that, Also includes: The first difference is obtained by subtracting the lower limit value and the preset buffer width from the joint angle data. The ratio of the first difference to the steepness control parameter is determined as the input value of the soft saturation function to obtain the first transition value. The first transition value is added to the first constant and then multiplied by the preset buffer width and the first coefficient to obtain the first correction amount. The second difference is obtained by subtracting the preset buffer width and the joint angle data from the upper limit value. The ratio of the second difference to the steepness control parameter is determined as the input value of the soft saturation function to obtain the second transition value. The second transition value is added to the second constant and then multiplied by the preset buffer width and the second coefficient to obtain the second correction amount.

6. The data processing method according to claim 2, characterized in that, Also includes: If the difference between the joint angular velocity data at adjacent time points is greater than or equal to the target safe angular velocity difference, then the acceleration limit data is determined. Adjust the joint angular velocity data at adjacent time points so that the difference between the joint angular velocity data at adjacent time points is less than or equal to the product of the acceleration limit data and the time difference between the adjacent time points.

7. The data processing method according to any one of claims 1 to 6, characterized in that, If the movable structure includes a chassis and robotic arm joints, then the cross-structure collaboration constraints include: If the chassis speed data is greater than the motion stability threshold, the joint angle data falls within the second safe angle range.

8. The data processing method according to any one of claims 1 to 6, characterized in that, If the movable structure includes a chassis and a robotic arm, then adjusting the target control command through a soft constraint strategy includes: Determine the target joint angle data and the preset safety angle data of the robotic arm; Based on the chassis speed data, a first weighting function and a second weighting function are determined; Determine the first product result of the first weighting function and the target joint angle data, and the second product result of the second weighting function and the safety angle data; Based on the sum of the first product result and the second product result, the joint angle data of the robotic arm is adjusted.

9. The data processing method according to claim 1, characterized in that, The second control data for the target robot predicted based on the historical control data includes at least one of the following: The historical control data is processed using the extended Kalman filter algorithm to generate the head posture data of the target robot, which includes pitch angle and yaw angle. The historical control data is processed using a long short-term memory neural network model to generate joint angle data of the robotic arm of the target robot; The historical control data is processed using a dynamic model based on differential drive and a Kalman filter algorithm to obtain the chassis speed data of the target robot; The historical control data is processed using a sliding window averaging method and / or a uniform acceleration model to obtain the waist state data of the target robot, which includes height data and pitch angle.

10. The data processing method according to claim 9, characterized in that, The process of processing the historical control data using the sliding window averaging method and / or a uniform acceleration model to obtain the waist state data of the target robot includes: The historical control data is processed using the uniform acceleration model to obtain the first state data of the waist. The historical control data is processed using the sliding window averaging method to obtain the second state data of the waist. If the first state data meets the prediction accuracy condition, then the first state data is determined as the waist state data; If the first state data does not meet the prediction accuracy condition, then the first weight corresponding to the first state data and the second weight corresponding to the second state data are determined respectively. The sum of the product of the first state data and the first weight, and the product of the second state data and the second weight, is determined, and the sum is determined as the waist state data.

11. The data processing method according to claim 1, characterized in that, The first control data includes a transmission timestamp, and determining the delay time of the first control data includes: Based on the reception time and transmission timestamp of the first control data, the original delay time of the first control data is determined; Determine the system time offset data between the virtual reality terminal and the target robot; The difference between the original delay time and the system time offset data is determined as the delay time of the first control data.

12. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the data processing method according to any one of claims 1 to 11.

13. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the data processing method according to any one of claims 1 to 11.

14. A computer program product, characterized in that, The computer program product includes instructions that, when executed on an electronic device, cause the electronic device to perform the data processing method according to any one of claims 1 to 11.

Images