Information processing methods and related devices

Abstract

This application provides an information processing method and related apparatus. The method includes: acquiring joint information of a target object; constructing a virtual object corresponding to the target object in a virtual scene based on the joint information; performing a running simulation of the arm joints of the virtual object based on the running speed of the virtual object during the gait cycle to obtain the angle rotation curve of the arm joints of the virtual object during the gait cycle; and performing motion mapping on the angle rotation curve to obtain the motion sequence of the arm joints of the target object. This application ensures the coordination between arm movement and gait, enabling the reproduction of the arm joint movements of the target object during the gait cycle.

Description

Technical Field

[0001] This application relates to robotics, and more particularly to an information processing method and related apparatus. Background Technology

[0002] With the continuous advancement of robotics technology, bionic robots have been widely used in many fields, especially in service, medical, and rescue scenarios. Bionic robots can mimic human behavior, improving the naturalness of operation and the comfort of interaction. However, when performing humanoid movements, bionic robots simplify arm movements, which reduces the overall stability and coordination of movement. Summary of the Invention

[0003] This application provides an information processing method and related apparatus that can ensure the coordination between arm movements and gait, and realize the reproduction of the arm joint movements of the target object within the gait cycle.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides an information processing method, the method comprising:

[0006] Obtain the joint information of the target object's joints;

[0007] Based on the joint information, a virtual object corresponding to the target object is constructed in the virtual scene;

[0008] Based on the running speed of the virtual object in the gait cycle, the arm joint of the virtual object is simulated to obtain the angle rotation curve of the arm joint of the virtual object in the gait cycle, wherein the angle rotation curve represents the rotation angle of the arm joint at each time point in the gait cycle.

[0009] The angle rotation curve is mapped to obtain the action sequence of the target object's arm joint. The action sequence includes the action of the target object at each time point within the gait cycle, and each action is obtained by the target object's arm joint performing the corresponding rotation angle.

[0010] This application provides an information processing apparatus, including:

[0011] The data acquisition module is used to acquire joint information of the target object's joints;

[0012] An object construction module is used to construct a virtual object corresponding to the target object in a virtual scene based on the joint information;

[0013] The motion simulation module is used to simulate the operation of the arm joint of the virtual object based on the running speed of the virtual object in the gait cycle, and obtain the angle rotation curve of the arm joint of the virtual object in the gait cycle, wherein the angle rotation curve represents the rotation angle of the arm joint at each time point in the gait cycle.

[0014] The motion mapping module is used to perform motion mapping on the angle rotation curve to obtain the motion sequence of the target object's arm joint. The motion sequence includes the action of the target object at each time point within the gait cycle, and each action is obtained by the target object's arm joint performing the corresponding rotation angle.

[0015] This application provides an electronic device, including:

[0016] Memory is used to store executable instructions for a computer;

[0017] The processor, when executing computer-executable instructions stored in the memory, implements the information processing method provided in the embodiments of this application.

[0018] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the information processing method provided in this application.

[0019] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the information processing method provided in this application.

[0020] The embodiments of this application have the following beneficial effects:

[0021] Based on the joint information of the target object, a virtual object corresponding to the target object is constructed in the virtual scene. In this way, by constructing a virtual object corresponding to the target object, the target object can be directly controlled to perform actions according to the joint rotation angle of the virtual object obtained from the simulation, thus achieving consistency between the target object and the virtual object. Based on the running speed of the virtual object in the gait cycle, the arm joints of the virtual object are simulated to obtain the angle rotation curve of the arm joints of the virtual object in the gait cycle. The angle rotation curve is then mapped to obtain the action sequence of the arm joints of the target object. In this way, the coordination between the arm movement and the gait can be ensured, and the action of the arm joints of the target object in the gait cycle can be reproduced. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the architecture of the information processing system 100 provided in an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0024] Figure 3A This is a first flowchart illustrating the information processing method provided in the embodiments of this application;

[0025] Figure 3B This is a second flowchart illustrating the information processing method provided in the embodiments of this application;

[0026] Figure 3C This is a schematic diagram of the third process of the information processing method provided in the embodiments of this application;

[0027] Figure 3D This is a schematic diagram of the fourth process of the information processing method provided in the embodiments of this application;

[0028] Figure 3E This is a schematic diagram of the fifth process of the information processing method provided in the embodiments of this application;

[0029] Figure 4 This is a flowchart illustrating the processing of joint information in a bionic robot provided in an embodiment of this application.

[0030] Figure 5 This is a schematic diagram of the personalized model provided in the embodiments of this application;

[0031] Figure 6 This is a flowchart of the forward prediction simulation provided in the embodiments of this application;

[0032] Figure 7 This is a schematic diagram of the movement trajectory of the shoulder joint in one gait cycle provided in an embodiment of this application;

[0033] Figure 8 This is a schematic diagram of software processing provided in an embodiment of this application;

[0034] Figure 9 This is a schematic diagram of joint reduction provided in an embodiment of this application.

[0035] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0037] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0038] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0039] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0040] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0041] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0042] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0043] 1) The target object is a machine object in a real-world scenario. This application does not limit the machine object; the machine object can be a robot, a robot dog, etc.

[0044] 2) Joint information, used to record relevant information about the joint. This application embodiment does not limit the joint information. The joint information can be at least one of the following: the name of the joint, the purpose of the joint, the location of the joint, and the length between adjacent joints.

[0045] 3) Virtual objects are the images of various people and objects used for gait simulation in a virtual scene. The image of the virtual object is consistent with that of the target object. Virtual objects can be virtual characters, virtual animals, etc., such as the people and animals displayed in the virtual scene. The virtual object can be a virtual image representing the user in the virtual scene. The virtual scene can include multiple virtual objects, each with its own shape and volume, occupying a portion of the space in the virtual scene.

[0046] 4) Virtual Scene: The program presents a virtual scene when running on the terminal. This virtual scene is a simulation of the real world, used to simulate various conditions and situations in the real world, such as real-world physical rules, objects, lighting conditions, terrain, and other environmental factors. Based on the simulated conditions and situations, the program tests, verifies, and trains the performance and behavior of virtual objects in different environments. Users can control the movement of virtual objects within this virtual scene. This virtual scene can be a simulation of the real world, a semi-simulated / semi-fictional virtual environment, or a purely fictional virtual environment. The virtual scene can be any of a two-dimensional, 2.5-dimensional, or three-dimensional virtual scene; this application does not limit the dimension of the virtual scene.

[0047] 5) Gait cycle refers to the complete process a person goes through while walking, from the moment the heel of one leg touches the ground until the heel of the same leg touches the ground again. This process can be subdivided into the following phases: initial contact phase, weight-bearing phase, single-leg support phase, and swing phase. The initial contact phase is the beginning of the gait cycle and represents the first contact of the walker's foot with the ground. The weight-bearing phase represents the period from initial contact to the opposite foot leaving the ground, during which the body's weight is transferred from one foot to the other. The single-leg support phase represents the point where the body is fully supported by one foot after the opposite foot leaves the ground; this phase includes heel lift-off and toe propulsion. The swing phase represents the period from toe lift-off to the foot's re-contact with the ground; this phase includes acceleration, neutralization, and deceleration, during which the walker's leg swings forward in preparation for the next contact with the ground. Gait cycles are used to study movement patterns during human walking to assess and treat gait abnormalities.

[0048] In related technologies, arm movements increase control complexity and may cause stability problems. Although the target object's arm has a high degree of freedom, control strategies are usually simplified. Especially during the target object's walking process, arm movements are often ignored and not integrated into the walking process, reducing the overall stability and coordination of the movement. To address the above problems, this application provides an information processing method, device, electronic device, computer-readable storage medium, and computer program product that can ensure the coordination of arm movements and gait, and realize the reproduction of the target object's arm joint movements within the gait cycle, thereby improving the overall stability and coordination of the movement.

[0049] The information processing method described in the embodiments of this application can be applied to various fields, such as the field of robot motion simulation. That is, the information processing method in the embodiments of this application is not limited to a certain field.

[0050] The following describes exemplary applications of the electronic device provided in the embodiments of this application. The device provided in the embodiments of this application can be implemented as a terminal or as a server. The following will describe exemplary applications when the device is implemented as a server.

[0051] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the information processing system 100 provided in the embodiments of this application. In order to support an information processing application, the terminal (terminal 400 is shown as an example) connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0052] Terminal 400 is used to send the joint information of the target object's joints to server 200 via network 300. Server 200 is used to construct a virtual object corresponding to the target object in a virtual scene based on the joint information. Based on the running speed of the virtual object in the gait cycle, it performs running simulation on the arm joints of the virtual object to obtain the angle rotation curve of the arm joints of the virtual object in the gait cycle. It performs motion mapping on the angle rotation curve to obtain the motion sequence of the target object's arm joints and returns the motion sequence to terminal 400. Terminal 400 controls the arm joints of the target object to execute each action in the motion sequence.

[0053] The following is an example of information processing performed by terminal 400.

[0054] In some embodiments, the terminal 400 can independently complete information processing tasks. For example, the terminal 400 can acquire joint information of the target object's joints, construct a virtual object corresponding to the target object in a virtual scene based on the joint information, simulate the operation of the virtual object's arm joints based on the running speed of the virtual object in the gait cycle, obtain the angle rotation curve of the virtual object's arm joints in the gait cycle, perform motion mapping on the angle rotation curves to obtain the motion sequence of the target object's arm joints, and control the target object's arm joints to execute each action in the motion sequence.

[0055] In one implementation scenario, a server or terminal can process the joint information of a real robot, obtain the joint information of the real robot, construct a simulated robot corresponding to the real robot in the simulation scenario, simulate the operation of the arm joints of the simulated robot based on the running speed of the simulated robot in the gait cycle, obtain the angle rotation curve of the arm joint of the simulated robot in the gait cycle, perform motion mapping on the angle rotation curve to obtain the motion sequence of the arm joint of the real robot, and control the arm joint of the real robot to execute each action in the motion sequence.

[0056] In some embodiments, server 200 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0057] Terminal 400 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, smart voice interaction device, smart home appliance, vehicle terminal, aircraft, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0058] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 2 The electronic device 500 shown can be Figure 1The terminal 400 or server 200, and the electronic device 500 include: at least one processor 510, memory 550, and at least one network interface 520. The various components in server 200 are coupled together via a bus system 540. It is understood that the bus system 540 is used to implement communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 540.

[0059] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0060] User interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls;

[0061] In some embodiments, when the terminal 400 independently completes the information processing task, the server 200 provided in this application embodiment does not include the user interface 530.

[0062] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices physically located away from the processor 510.

[0063] The memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory.

[0064] In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0065] Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0066] The network communication module 552 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0067] Presentation module 553 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 531 (e.g., a display screen, a speaker, etc.) associated with user interface 530;

[0068] In some embodiments, when the information processing task is completed independently by the terminal 400, the server 200 provided in this application embodiment may not include the presentation module 553.

[0069] The input processing module 554 is used to detect and translate one or more user inputs or interactions from one or more input devices 532; in some embodiments, when the terminal 400 independently completes the information processing task, the server 200 provided in this application embodiment may not include the presentation module 553.

[0070] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 An information processing device 555 stored in memory 550 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a data acquisition module 5551, an object construction module 5552, a motion simulation module 5553, and a motion mapping module 5554. These modules are logically connected and can therefore be arbitrarily combined or further separated according to the functions they implement. The functions of each module will be described below.

[0071] It should be noted that, based on the understanding of the following information processing examples, those skilled in the art can apply the information processing methods provided in the embodiments of this application to robot simulation.

[0072] See Figure 3A , Figure 3A This is a first flowchart illustrating the information processing method provided in the embodiments of this application, which will be combined with... Figure 3AThe steps shown will be explained below. The information processing method provided in this application embodiment can be implemented by the server or the terminal alone, or by the server and the terminal working together. The following will be an example of the server and the terminal working together.

[0073] In step 101, the joint information of the target object's joints is obtained.

[0074] Here, the target object is a machine object in a real-world scenario. This application embodiment does not limit the machine object; it can be a robot, a robot dog, etc. The real-world scenario is a scene occurring in real life or a real-world work environment. In a real-world scenario, the machine object can be controlled to perform corresponding actions, such as controlling a robot to perform a dance in a real-world square. The real-world scenario describes the actual situation relative to the virtual scenario. The joints of the target object are important components of the skeletal system, used to drive the bones to perform various movements. This application embodiment does not limit the type of joint; joints can be shoulder joints, elbow joints, and wrist joints, etc. The shoulder joint is used to drive the entire arm to perform a wide range of movements, including at least one of the following: flexion, extension, adduction, abduction, and rotation; the elbow joint is used to drive the flexion and extension of the forearm; the wrist joint is used to drive the movement between the hand and forearm. Joint information is used to record relevant information about the joint. This application embodiment does not limit the joint information; joint information can be at least one of the following: the name of the joint, the purpose of the joint, the location of the joint, and the length between adjacent joints.

[0075] In some embodiments, step 101 can be implemented by: constructing marker points at the joints of the target object, the marker points being used to reflect infrared light; in response to the reflection of preset infrared light, determining the position information of the joint based on the reflected infrared light, and determining the position information of the joint as the joint information.

[0076] It should be noted that the markers are used to reflect the preset infrared light emitted by the motion capture system and reflect it back to the motion capture system. The motion capture system is used to emit and receive infrared light and determine the position of the markers based on the received infrared light.

[0077] Following the above embodiments, the above-mentioned "determining the position information of the joint based on the reflected infrared light" can be achieved in the following way: the infrared camera of the motion capture system emits infrared light of a specific wavelength. After these lights illuminate the marker point, they are reflected back to the infrared camera. After the sensor matrix on the infrared camera receives the reflected infrared light, it obtains the coordinates of the marker point through signal processing.

[0078] In step 102, a virtual object corresponding to the target object is constructed in the virtual scene based on the joint information.

[0079] Here, the program presents a virtual scene when running on the terminal. This virtual scene is a simulation of the real world, used to simulate various conditions and situations in the real world, such as real-world physical rules, objects, lighting conditions, terrain, and other environmental factors. It tests, verifies, and trains the performance and behavior of virtual objects in different environments based on the simulated conditions and situations. Users can control the movement of virtual objects within this virtual scene. This virtual scene can be a simulation of the real world, a semi-simulated / semi-fictional virtual environment, or a purely fictional virtual environment. The virtual scene can be any of a two-dimensional, 2.5-dimensional, or three-dimensional virtual scene; this application embodiment does not limit the dimension of the virtual scene. Virtual objects are the images of various people and objects in the virtual scene that undergo gait simulation. The image of the virtual object is consistent with the image of the target object. Virtual objects can be virtual characters, virtual animals, etc., such as the characters and animals displayed in the virtual scene. A virtual object can be a virtual image representing the user within the virtual scene. The virtual scene can include multiple virtual objects, each with its own shape and volume, occupying a portion of the virtual scene's space.

[0080] In some embodiments, joint information includes the position of the joints of the target object, see [link to relevant documentation]. Figure 3B , Figure 3B This is a second flowchart illustrating the information processing method provided in the embodiments of this application, specifically for... Figure 3A Step 102 shown can be achieved through... Figure 3B Steps 1021 to 1025 are implemented, and will be explained in detail below.

[0081] In step 1021, the initial virtual object in the virtual scene and the virtual parts of the initial virtual object connected to the joint are obtained, and the real scene in which the target object is located is obtained.

[0082] Here, the initial virtual object is a virtual object with a specific size in the virtual scene. This application embodiment does not limit the initial virtual object; it can be a virtual object of a preset size or a virtual object that has already been constructed in the virtual scene. The initial virtual object contains multiple virtual parts, which are connected through joints of the initial virtual object.

[0083] In step 1022, based on the position mapping relationship between the real scene and the virtual scene, the position of the joints of the target object is mapped to obtain the target position of the joints of the initial virtual object.

[0084] Here, the target position of the joint of the initial virtual object is the position of the joint of the virtual object.

[0085] It should be noted that the real-world and virtual scenes have a positional relationship. The coordinate system of the real-world scene is established based on GPS or a local positioning system (such as an indoor positioning system). For example, assuming a Cartesian coordinate system with the ground as the reference, where the origin is located at a fixed position in the scene, the X-axis points north, the Y-axis points east, and the Z-axis points vertically upward. In a 3D virtual environment, the coordinate system of the virtual scene is a coordinate system with a point on the virtual terrain as the origin. This application does not limit the position mapping process; position mapping can be translation, rotation, and scaling, etc. Translation requires determining the distance and direction between the origins of the two coordinate systems; rotation requires determining the angular relationship between the axes of the two coordinate systems; scaling requires determining the scaling factor, which characterizes the scaling ratio between the two coordinate systems.

[0086] In some embodiments, taking the position mapping process of coordinate system translation as an example, step 1022 can be implemented in the following way: determine the distance (e.g., 5) and direction (e.g., east) between the origins of two coordinate systems; move the position of the joint of the target object in that direction by the corresponding distance to obtain the target position of the joint of the initial virtual object.

[0087] In some embodiments, taking the position mapping process of coordinate system rotation as an example, step 1022 can be implemented in the following way: determine the rotation angle between the two coordinate system axes (e.g., 5); rotate the position of the joint of the target object according to the rotation angle to obtain the target position of the joint of the initial virtual object.

[0088] In some embodiments, taking the coordinate system scaling position mapping process as an example, step 1022 can be implemented in the following way: determine the scaling factor (e.g., 0.5); and determine the target position of the joint of the initial virtual object by multiplying the initial position of the joint of the target object by the scaling factor.

[0089] In some embodiments, the following steps may be performed: based on the target position and the initial position of the joints of the initial virtual object, construct a virtual object in the virtual scene corresponding to the target object. The above-mentioned step of "constructing a virtual object in the virtual scene corresponding to the target object based on the target position and the initial position of the joints of the initial virtual object" can be implemented through steps 1023 to 1025.

[0090] In step 1023, the virtual part is aligned based on the target position and the initial position to obtain the aligned virtual part.

[0091] Here, the initial position of the joints of the initial virtual object is in response to the position of the joints of the initial virtual object displayed when the program starts, and the image of the virtual object is consistent with the image of the target object in the real scene.

[0092] In some embodiments, the above-mentioned "aligning the virtual part based on the target position and the initial position to obtain the aligned virtual part" can be achieved in the following way: determining a rotation matrix, which is used to rotate the virtual part from the direction aligned with the initial position to the direction aligned with the target position; applying the rotation matrix to the virtual part, updating the transformation matrix of the virtual part to obtain the updated transformation matrix of the virtual part, and determining the aligned virtual part based on the updated transformation matrix of the virtual part, wherein the transformation matrix of the virtual part is used to describe the position of the virtual part.

[0093] It's important to note that the transformation matrix for virtual parts is used to represent and perform geometric transformations, such as rotation, translation, scaling, and shearing, and can be applied to points, vectors, or the entire object. In 3D graphics, the transformation matrix is ​​typically a 4x4 matrix. The first 3x3 part of the matrix usually describes rotation, consisting of three Euler angles (rotation angles about the X, Y, and Z axes). The last column of the matrix (the first three elements of the fourth column) describes translation, i.e., displacement along the X, Y, and Z axes. The last row of the matrix is ​​used to implement perspective transformations, affecting the viewing distance and size of the object.

[0094] Following the above embodiments, the above-mentioned "applying the rotation matrix to the virtual part, updating the transformation matrix of the virtual part, and obtaining the updated transformation matrix of the virtual part" can be achieved in the following way: the product of the rotation matrix and the transformation matrix of the virtual part is determined as the updated transformation matrix of the virtual part.

[0095] For example, given a rotation matrix (such as [[cos(30), 0, sin(30), 0], [0, 1, 0, 0], [-sin(30), 0, cos(30), 0], [0, 0, 0, 1]]), where sin represents the sine of an angle of 30 degrees and cos represents the cosine of an angle of 30 degrees, this rotation matrix is ​​used to represent a virtual part rotating 30 degrees around the Y-axis. The product of the rotation matrix and the transformation matrix of the virtual part (e.g., [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 1.0]) is determined as the updated transformation matrix of the virtual part (e.g., [[0.86, 0, 0.5, 0], [0, 1, 0, 0], [-0.5, 0, 0.86, 0], [0, 0, 0, 1]]).

[0096] Following the above embodiments, the above "determining the aligned virtual part based on the updated transformation matrix of the virtual part" can be achieved in the following way: the initial coordinates of each point of the virtual part in the virtual scene are determined by multiplying the initial coordinates of the virtual part with the rotation matrix and the transformation matrix of the virtual part as the target coordinates of each point of the virtual part in the virtual scene, and each point of the virtual part in the virtual scene is adjusted from the initial coordinates to the target coordinates to obtain the aligned virtual part.

[0097] In step 1024, the joints of the initial virtual object are adjusted from their initial positions to their target positions to obtain the adjusted joints.

[0098] It should be noted that the embodiments of this application do not limit the adjustment process; the adjustment can be automatic, manual, or other methods.

[0099] In step 1025, the adjusted joints and aligned virtual parts are combined to obtain a virtual object corresponding to the target object.

[0100] Through the embodiments of this application, virtual objects in a virtual scene are constructed based on joint information, which effectively realizes the accurate mapping between the virtual environment and the real world. By obtaining the joint position of the target object and performing position mapping, the object in the real scene is accurately converted into the virtual scene, ensuring the consistency between the virtual object and the target object. This not only optimizes the gait simulation effect of the virtual object, but also enhances the interactivity and immersion of the virtual reality system.

[0101] In step 103, based on the running speed of the virtual object during the gait cycle, the running simulation of the arm joint of the virtual object is performed to obtain the angle rotation curve of the arm joint of the virtual object during the gait cycle.

[0102] The angular rotation curve represents the rotation angle of the arm joint at each time point within the gait cycle.

[0103] Here, the running speed of the virtual object in the gait cycle is a preset running speed, which does not exceed the maximum running speed of the target object in the real scene. The gait cycle refers to the complete process experienced by a person walking, from the moment the heel of one leg touches the ground until the heel of the same leg touches the ground again. This process can be subdivided into the following stages: the initial contact stage, the weight-bearing stage, the single-leg support stage, and the swing stage. The initial contact stage is the beginning of the gait cycle and is used to characterize the first contact of the walker's foot with the ground. The weight-bearing stage is used to characterize the stage from the initial contact to the moment the opposite foot leaves the ground, at which point the body's weight is transferred from one foot to the other. The single-leg support stage is used to characterize the point where the body is completely supported by one foot after the opposite foot leaves the ground. This stage includes the heel leaving the ground and the toes propelling the body forward. The swing stage is used to characterize the stage from the moment the toes leave the ground to the moment the foot touches the ground again. This stage includes three phases: acceleration, neutralization, and deceleration. The walker's leg swings forward, preparing for the next contact with the ground. Gait cycles are used to study movement patterns during human walking to assess and treat gait abnormalities. The duration of a gait cycle is typically expressed as a percentage, with a complete gait cycle being 100%. For example, the initial contact phase accounts for 0% to 2% of the gait cycle, while the swing phase accounts for 60% to 70%. This application does not limit the arm joints of the virtual object. The arm joints include at least one of the following joints: the shoulder joint and the elbow joint. The rotation angles of the arm joints include rotation angles along the X, Y, and Z axes. The arm joints perform actions corresponding to these rotation angles by executing rotation angles in different directions.

[0104] In some embodiments, before the above-described step of "simulating the operation of the arm joint of the virtual object", the running speed of the virtual object in the gait cycle is obtained by the following method: obtaining the running speed of M target objects in the gait cycle, and averaging the M running speeds to obtain the running speed of the virtual object in the gait cycle, where M>1, and M is used to characterize the number of target objects.

[0105] For example, given three target objects (M=3), their running speeds during the gait cycle are 1.2 m / s, 1.5 m / s, and 1.8 m / s, respectively. Weights are assigned to these running speeds (e.g., 0.4, 0.3, and 0.3), and the three running speeds are weighted and averaged to obtain the virtual object's running speed during the gait cycle as 1.47 m / s.

[0106] In some embodiments, see Figure 3C , Figure 3C This is a schematic diagram of the third process of the information processing method provided in the embodiments of this application, targeting... Figure 3A Step 103 shown can be achieved through... Figure 3CSteps 1031 to 1036 are implemented, and will be explained in detail below.

[0107] In step 1031, an objective function for controlling the movement of the virtual object is constructed based on the running speed of the virtual object during the gait cycle.

[0108] Here, the running speed of a virtual object in the gait cycle is the speed at which the virtual object moves as a whole in the virtual scene.

[0109] In some embodiments, step 1031 can be implemented in the following way: for each preset time point in the gait cycle, perform the following processing: based on the running speed of the virtual object in the gait cycle, determine the angular velocity of the joint of the virtual object at the time point; based on the angular velocity of the joint of the virtual object at the time point, determine the motion acceleration of the joint of the virtual object at the time point; and determine the sum of squares of the motion accelerations corresponding to the preset time points as the objective function for controlling the virtual object to move.

[0110] Here, angular velocity describes the angle of rotation of a joint per unit time. The ratio of the rotation angle to time is defined as angular velocity. The angular velocity of a joint is also used to determine the linear velocity of a joint (i.e., the distance the joint moves along its trajectory per unit time). The linear velocity of a joint is the product of the distance (radius) from the joint to the axis of rotation and the angular velocity of the joint. The linear velocity of the joint is used to determine the distance the joint moves along its trajectory. Taking the shoulder and elbow joints of the arm as an example, the elbow joint follows the rotation of the shoulder joint, and its position changes accordingly. Therefore, the distance between the shoulder and elbow joints is the distance between the joint and the axis of rotation.

[0111] It should be noted that the axis of rotation is a reference axis that describes the straight line around which an object rotates during rotational motion. In three-dimensional space, any rotation can be described by an axis of rotation, around which the object rotates.

[0112] In some embodiments, the above-mentioned "determining the angular velocity of the joint of the virtual object at a given time point based on the running speed of the virtual object in the gait cycle" can be implemented in the following way: obtaining the rotation angle of the joint of the virtual object at a given time point, and determining the ratio of the rotation angle to the unit time as the first angular velocity of the joint of the virtual object at a given time point, wherein the unit time is the time between two adjacent time points; decomposing the running speed of the virtual object in the gait cycle to obtain the decomposed linear velocity in the direction perpendicular to the rotation axis of the joint, determining the ratio of the decomposed linear velocity to the length of the rotation axis as the second angular velocity of the joint of the virtual object at a given time point, and fusing the first angular velocity and the second angular velocity of the joint of the virtual object at a given time point to obtain the angular velocity of the joint of the virtual object at a given time point.

[0113] It should be noted that the rotation angle of the virtual object's joint at a given time point is an unknown value to be solved. The angular velocity of the virtual object's joint at a given time point can be determined based on the rotation angle of the virtual object's joint at that time point.

[0114] For example, the running speed of the virtual object in the gait cycle (the running speed in the horizontal direction is 1 m / s) is decomposed to obtain the decomposed linear velocity (e.g., 0.8 m / s) that is consistent with the rotation axis of the joint at the time point. The ratio of the decomposed linear velocity to the length of the rotation axis (e.g., 0.5 m) (e.g., 1.6 m / s) is determined as the second angular velocity of the virtual object's joint at the time point. The first angular velocity (e.g., 0.8 m / s) and the second angular velocity of the virtual object's joint at the time point are fused to obtain the angular velocity of the virtual object's joint at the time point (e.g., 2.4 m / s).

[0115] In some embodiments, the above-mentioned "determining the motion acceleration of the joint of the virtual object at a given time point based on the angular velocity of the joint of the virtual object at a given time point" can be achieved by: approximating the motion acceleration of the joint of the virtual object by calculating the ratio of the change in angular velocity of the joint between adjacent time points to the change in time.

[0116] For example, given two adjacent time points t1 and t2, with corresponding angular velocities ω1 and ω2 respectively, calculate the difference ω3 between ω1 and ω2, calculate the difference t3 between t1 and t2, and determine the ratio between ω3 and t3 as the motion acceleration of the virtual object's joint at time t2.

[0117] Through the embodiments of this application, by simulating the movement of the arm joints of a virtual object, this invention successfully constructs the rotation curve of the arm joint angles within the gait cycle, thereby improving the realism and naturalness of the gait simulation of the virtual object. By constructing the objective function based on the running speed of the virtual object during the gait cycle, the kinematic characteristics of the human body are effectively simulated, making the movement of the virtual object closer to the target object in a real scene.

[0118] In step 1032, the constraints of the joints of the virtual object are obtained.

[0119] Here, the constraints on the joints of the virtual object are used to limit the rotation angle of the joints of the virtual object, for example, the rotation angle of the joints of the virtual object does not exceed 120 degrees; the objective solution is obtained by performing calculations on multiple solutions in the calculation process through the objective function, and the solution corresponding to the minimum value among the multiple values ​​is used to characterize the low work done by the virtual object when it moves within the gait cycle. The work done by the virtual object when it moves within the gait cycle is the product of the force and the distance to which the force acts. The force is proportional to the motion acceleration of the joints of the virtual object, and the distance to which the force acts is proportional to the motion acceleration of the joints of the virtual object. Therefore, the solution obtained by performing calculations on the solution through the objective function is also proportional to the motion acceleration of the joints of the virtual object.

[0120] In some embodiments, the above Figure 3C The step 1032 shown above, which involves obtaining the constraints of the joints of the virtual object, can be achieved in at least one of the following ways: based on the joint information, query a condition table, and determine the candidate conditions corresponding to the joint information as the constraints of the joints of the virtual object. The condition table is used to characterize the correspondence between different candidate joint information and different candidate conditions; or, encode the joint information to obtain encoded features, and decode the encoded features to obtain the constraints of the joints of the virtual object.

[0121] It should be noted that the embodiments of this application do not limit the correspondence between different candidate conditions and different candidate joint information. The correspondence can be multiple candidate conditions corresponding to one candidate joint information, or each candidate condition corresponding to its own candidate joint information. The condition table is a data structure used to store the correspondence between candidate conditions and candidate joint information. The embodiments of this application do not limit the condition table. The condition table can be a data structure implemented as a hash table, a data structure implemented as an array, etc., used to quickly perform lookup, insertion, and deletion operations on the correspondence between candidate conditions and candidate joint information. The hash table uses a hash function to calculate the index value and maps the input joint information to the position in the hash table to access the candidate condition.

[0122] For example, determine the candidate condition A for candidate joint information A, determine the candidate condition B for candidate joint information B, and determine the candidate condition (such as candidate condition A) corresponding to the joint information (such as candidate joint information A) as the constraint condition of the virtual object's joint.

[0123] In some embodiments, the above-mentioned "encoding the joint information to obtain the encoded features corresponding to the joint information" can be achieved by performing convolution processing on the joint information to obtain convolutional features, performing pooling processing on the convolutional features to obtain pooled features, and performing mapping processing on the pooled features to obtain encoded features.

[0124] For example, convolution processing is performed on joint information (such as "[0.8, 1.7, 2.2]", where 0.8 is used to represent the position of the joint in the X-axis direction, 1.7 is used to represent the position of the joint in the Y-axis direction, and 2.2 is used to represent the position of the joint in the Z-axis direction) to obtain convolutional features (such as "[0.14, 0.13, -0.06, 0.15, 0.03, 0.06]). Pooling processing is then performed on the convolutional features to obtain pooled features (such as "[0.14, 0.13, -0.06]"). Mapping processing is then performed on the pooled features to obtain encoded features (such as "[0.52, 0.13, -0.06, 0.15]").

[0125] In some embodiments, the above-mentioned "decoding the encoded features to obtain {decoded data} corresponding to the encoded features" can be implemented in the following way: mapping the encoded features to obtain mapped features, upsampling the mapped features to obtain upsampled features, and deconvolving the upsampled features to obtain the constraints of the joints.

[0126] For example, the encoded features (e.g., [0.52, 0.13, -0.06, 0.15]) are mapped to obtain the mapped features (e.g., [0.14, 0.13, -0.06]). The mapped features are then upsampled to obtain the upsampled features (e.g., [0.14, 0.13, -0.06, 0.15, 0.03, 0.06]). Finally, the upsampled features are deconvolved to obtain the constraints of the joint (e.g., "the rotation angle of the joint does not exceed 120 degrees").

[0127] In some embodiments, the constraints of the joints of a virtual object can also be obtained by: determining the similarity between the joint information and known joint information, wherein the known joint information is used to construct a known virtual object in the virtual scene; when the similarity is greater than a similarity threshold, the constraints of the joints of the known virtual object are determined as the constraints of the joints of the virtual object.

[0128] Following the above embodiments, the above-mentioned "determining known joint information and the similarity between joint information" can also be achieved in the following way: Encoding the known joint information (e.g., "[0.8, 1.7, 2.2]") to obtain a first encoding feature (e.g., "[0.14, 0.13, -0.06]"); Encoding the joint information (e.g., "[0.8, 1.7, 2.3]") to obtain a second encoding feature (e.g., "[0.14, 0.13, -0.06, 0.15, 0.03, 0.06]"); Determining the similarity between the first encoding feature and the second encoding feature (e.g., 0.85) as the similarity between the known joint information and the joint information.

[0129] It should be noted that the embodiments of this application do not limit the similarity. The similarity can be the cosine similarity, Euclidean distance, etc. between the first coding feature and the second coding feature.

[0130] For example, when the similarity (e.g., 0.85) is greater than the similarity threshold (e.g., 0.8), the known constraints of the virtual object's joints (e.g., the joint rotation angle is less than 120 degrees) are determined as the constraints of the virtual object's joints.

[0131] In some embodiments, the following steps are performed: Based on constraints, the objective function is calculated to obtain the objective solution of the objective function that satisfies the constraints. The step of “calculating the objective function based on constraints to obtain the objective solution of the objective function that satisfies the constraints” can be implemented through steps 1033 to 1035.

[0132] In step 1033, based on the penalty factor, the constraint conditions and objective function are constructed to obtain the penalty function.

[0133] Here, the penalty factor is used to incorporate the constraints into the objective function, transforming the constrained optimization problem into an unconstrained optimization problem, which can then be solved using an unconstrained optimization algorithm.

[0134] In some embodiments, step 1033 can be implemented by: performing factor transformation on the constraint conditions to obtain constraint factors; and summing the product of the constraint factors and the penalty factors with the objective function to determine the penalty function.

[0135] Following the above embodiments, the constraint conditions include at least one inequality, which includes a left-side factor and a right-side factor, used to characterize the correspondence between the left-side factor and the right-side factor. The above-mentioned "performing factor transformation on the constraint conditions to obtain constraint factors" can be achieved by determining the difference between the left-side factor and the right-side factor as the constraint factor.

[0136] For example, given a constraint (such as the inequality X>4), where X is the left-hand factor of the inequality and 4 is the right-hand factor of the inequality, the difference between the left-hand factor and the right-hand factor is determined as the constraint factor (such as X-4).

[0137] In step 1034, the initial solution that satisfies the constraints and the initial penalty factor are obtained.

[0138] Here, the initial solution is the rotation angle of the joint at a given time point. This application embodiment does not limit the method of obtaining the initial solution. The initial solution can be the initial value of a randomly generated solution, or the maximum rotation angle of the joint that satisfies the constraints, etc.; the initial penalty factor is a preset penalty factor.

[0139] In some embodiments, obtaining an initial solution that satisfies the constraints can be achieved by: determining the maximum and minimum rotation angles that satisfy the constraints; obtaining a random number between 0 and 1; calculating the product of the difference between the maximum and minimum rotation angles and the random number, and summing the product with the minimum rotation angle to determine the initial solution that satisfies the constraints.

[0140] For example, determine the maximum rotation angle (e.g., 180) and the minimum rotation angle (e.g., -100) that satisfy the constraints; obtain a random number between 0 and 1 (e.g., 0.5); calculate the product of the difference between the maximum and minimum rotation angles (e.g., 280) and the random number (e.g., 140), and determine the sum of the product and the minimum rotation angle (e.g., 40) as the initial solution that satisfies the constraints.

[0141] In step 1035, based on the initial solution and the initial penalty factor, the penalty function is iteratively calculated to obtain the objective solution of the objective function that satisfies the constraints.

[0142] Here, iterative operations are used to determine multiple solutions to the objective function so that the final solution minimizes the objective function that satisfies the constraints.

[0143] In some embodiments, step 1035 can be implemented as follows: performing the following operations for the iteration, substituting the solution of round (i-1) and the penalty factor of round (i-1) into the penalty function to obtain the penalty value of round (i); determining the penalty factor of round (i-1) as the product of the penalty factor and the decreasing factor; determining the solution of round (i-1) as the product of the penalty factor and the solution of round (i-1); when the difference between the penalty value of round (i) and the penalty value of round (i-1) is less than the penalty threshold, determining the solution of round (i) as the objective solution of the objective function that satisfies the constraints, and stopping the iteration.

[0144] Where i>0, i is a positive integer that increases sequentially, the solution of round 0 is the initial solution, and the penalty factor of round 0 is the initial penalty factor.

[0145] It should be noted that the embodiments of this application do not impose restrictions on the reduction factor, which is used to control the frequency of the reduction of the penalty factor.

[0146] In some embodiments, the above-mentioned "substituting the solution of round i-1 and the penalty factor of round i-1 into the penalty function to obtain the penalty value of round i" can be achieved as follows: substituting the solution of round i-1 and the penalty factor of round i-1 as independent variables into the penalty function to obtain the penalty value of round i.

[0147] For example, taking the solution for round i-1 as a rotation angle of 60 degrees and a penalty factor of 0.2 as an example, the ratio of the rotation angle (e.g., 60) to the unit time (e.g., 1) is determined as the first angular velocity (e.g., 60). The running speed of the virtual object in the gait cycle (horizontal running speed is 100) is decomposed to obtain the decomposed linear velocity (e.g., 50) in the direction perpendicular to the rotation axis of the joint at the time point. Here, the decomposed linear velocity is the product of the sine of the angle between the decomposed linear velocity and the horizontal running speed and the movement speed. The angle between the horizontal running speeds is equivalent to the rotation angle; the ratio of the decomposed linear velocity to the length of the rotation axis (e.g., 0.5) (e.g., 100) is determined as the second angular velocity. The first and second angular velocities of the virtual object's joints at a given time point are fused to obtain the angular velocity of the virtual object's joints at that time point (e.g., 200). By calculating the ratio of the change in angular velocity of the joints between adjacent time points (e.g., 20) to the change in time (e.g., 1), the motion acceleration of the virtual object's joints (e.g., 20) is approximately obtained. The sum of the squares of the motion accelerations is determined as the value of the objective function (400).

[0148] Following the example above, the constraint factor is determined based on the rotation angle (e.g., 60°) (e.g., "60-40=20"). The product of the constraint factor and the penalty factor is added to the value of the objective function in the i-th round to determine the penalty value in the i-th round (e.g., 404).

[0149] By introducing joint constraints and constructing a penalty function, this application ensures that the motion simulation of the virtual object's arm joints achieves low work output while meeting actual physiological limitations. By transforming the nonlinear programming problem with joint kinematic constraints into a linear programming problem and using an iterative algorithm to solve for the minimum objective function, the accuracy and efficiency of gait simulation are effectively improved. This ensures that the virtual object's motion within the gait cycle conforms to biomechanical principles and completes the action with minimal energy consumption. Optimizing the coordination between the arm and gait ensures that the arm's trajectory is synchronized with the gait, resulting in smoother arm movement reproduction. This not only enhances the realism and naturalness of the simulation technology but also contributes to human motion analysis and rehabilitation training.

[0150] In step 1036, based on the target solution, the angular rotation curve of the arm joint of the virtual object during the gait cycle is determined.

[0151] In some embodiments, the target solution includes the rotation angle of the virtual object's arm joint at a preset time point, see [link to relevant documentation]. Figure 3D , Figure 3D This is a schematic diagram of the fourth process of the information processing method provided in the embodiments of this application, which is aimed at... Figure 3C Step 1036 shown can be achieved through... Figure 3D Steps 10361 to 10362 are implemented, and will be explained in detail below.

[0152] In step 10361, the rotation angles at other time points are determined based on the rotation angle at a preset time point.

[0153] Other time points are those other than the preset time points within the gait cycle.

[0154] Here, the preset time points are preset time points of number M. Each preset time point has a certain duration, which includes other time points within the gait cycle of a preset number N, excluding the preset time points.

[0155] In some embodiments, the preset time point can be determined by: calculating the ratio of the number of time points within the gait cycle to a preset number, dividing the time points within the gait cycle into multiple time periods based on the ratio, and taking the first time point of each time period as the preset time point.

[0156] For example, the gait cycle has 1000 time points, and a preset time point is determined every 50 time points, resulting in a preset number of 200 preset time points.

[0157] In some embodiments, step 10361 can be implemented by interpolating other time points within the two preset time points based on the rotation angles of two adjacent preset time points to obtain the rotation angles of other time points.

[0158] For example, using linear interpolation, between two preset time points, such as the first time point being 0 degrees and the second time point being 90 degrees, the rotation angle at any intermediate time point changes linearly from 0 degrees to 90 degrees.

[0159] In step 10362, the rotation angle at the preset time point and the rotation angle at other time points are curve-fitted to obtain the angle rotation curve of the arm joint of the virtual object during the gait cycle.

[0160] In some embodiments, step 10362 can be implemented by traversing each time point of the gait cycle, connecting the rotation angle corresponding to the time point with the rotation angle of the next time point using a drawing tool, to obtain the angle rotation curve of the arm joint of the virtual object in the gait cycle, wherein the time points include preset time points and other time points.

[0161] It should be noted that the drawing tool is used for drawing curves. This application embodiment does not limit the drawing tool. The drawing tool can be an online drawing tool (such as Desmos tool) or an offline drawing tool (such as the line chart drawing function in Excel).

[0162] This application's embodiments construct an angular rotation curve of a virtual object's arm joint throughout the entire gait cycle. This curve fitting technique not only makes the virtual object's gait movements smoother and more continuous, but also enhances the realism and naturalness of the virtual reality experience. By performing interpolation calculations between preset time points, it ensures that the joint movements of the virtual object throughout the entire gait cycle can more accurately reflect the continuity and changing patterns of human gait.

[0163] In step 104, motion mapping is performed on the angle rotation curve to obtain the motion sequence of the target object's arm joints.

[0164] The action sequence includes the action of the target object at each time point within the gait cycle, and each action is obtained by the target object's arm joint performing the corresponding rotation angle.

[0165] In some embodiments, see Figure 3E , Figure 3E This is a schematic diagram of the fifth process of the information processing method provided in the embodiments of this application, which is aimed at... Figure 3A Step 104 shown can be achieved through... Figure 3E Steps 1041 to 1043 are implemented, and will be explained in detail below.

[0166] Steps 1041 to 1043 are executed for each time point within the gait cycle.

[0167] In step 1041, the rotation angle corresponding to the time point is obtained by querying the angle rotation curve based on the time point.

[0168] It should be noted that the angle rotation curve represents the rotation angle of the arm joint at each time point within the gait cycle, and each time point corresponds to a unique rotation angle of the arm joint.

[0169] In step 1042, the rotation angle of the target object's arm joint is controlled at the time point to obtain the action corresponding to the time point.

[0170] In some embodiments, step 1042 can be implemented by controlling the target object to change the magnitude and direction of the torque through a driver, controlling the rotating axis to rotate according to the rotation angle corresponding to the time point, and obtaining the action corresponding to the time point.

[0171] For example, at time t = 0 seconds, the action is to adjust the rotation axis angle of the robotic arm to 0 degrees; at time t = 1 second, the action is to adjust the rotation axis angle of the robotic arm to 90 degrees; and at time t = 2 seconds, the action is to adjust the rotation axis angle of the robotic arm to 0 degrees.

[0172] In step 1043, the actions corresponding to each time point are combined to obtain the action sequence of the target object's arm joints.

[0173] For example, given the action corresponding to each time point (such as adjusting the rotation axis angle of the robotic arm to 0 degrees at time t = 0 seconds and adjusting the rotation axis angle of the robotic arm to 90 degrees at time t = 1 second), the action corresponding to each time point is combined to obtain the action sequence of the arm joint of the target object (such as [adjusting the rotation axis angle of the robotic arm to 0 degrees at time t = 0 seconds and adjusting the rotation axis angle of the robotic arm to 90 degrees at time t = 1 second]).

[0174] In some embodiments, after step 104, the arm joints of the target object are controlled to perform each action in the action sequence.

[0175] In some embodiments, the above-mentioned "controlling the arm joint of the target object to perform each action in the action sequence" can be achieved by converting the actions in the action sequence into control signals, and driving the arm joint of the target object to move according to each action in the action sequence based on the control signals.

[0176] It should be noted that the control signal appears in the form of pulses and can be used to control the position or speed of the stepper motor. The frequency of the pulses can represent the speed, and the number of pulses can represent the distance moved. The converted control signal can be a pulse width modulation (PWM) signal of the motor or a control pressure of the hydraulic system. In the target object (robot), the control signal drives the motor, hydraulic or pneumatic actuator, so that the robot's arm joints move according to the action sequence.

[0177] Through the embodiments of this application, by introducing arm swing control planning into the gait cycle of the target object, balance, stability and biomimicry are significantly improved, posture deviations caused by lack of coordinated arm movement are reduced, and the overall walking quality is improved. At the same time, the coordinated movement of the arm and gait enhances the biomimicry effect of the target object, making its movement closer to the natural walking pattern of humans, improving the performance of the target object in human-computer interaction, and making it more suitable for interactive tasks in practical application scenarios.

[0178] The following will describe an exemplary application of the information processing method provided in this application embodiment in a real-world application scenario.

[0179] In related technologies, the movement of the arm increases the complexity of control and may cause stability problems. Although the robot arm has a high degree of freedom, the control strategy is usually simplified. Especially during the robot's walking process, the movement of the arm is often ignored. The movement of the arm is not integrated into the walking process, which reduces the stability and coordination of the overall movement.

[0180] To address the aforementioned issues, this application proposes an information processing method. This method involves collecting information from various segments of a bionic robot's body using a motion capture system, scaling a general model on an open-source platform to obtain a personalized model tailored to the specific bionic robot, and then optimizing the joint states of the personalized model to generate an arm movement trajectory under a stable walking gait. This simulated arm movement trajectory is then mapped onto the bionic robot, ensuring coordination between arm movement and gait, enabling the robot to reproduce arm movements during smooth walking, and improving the balance, stability, and bionic nature of the bionic robot during walking.

[0181] Taking the processing of joint information in a bionic robot as an example, see... Figure 4 , Figure 4 This is a flowchart of the joint information processing of the bionic robot provided in the embodiments of this application. The following is an explanation of the process for processing the joint information of the bionic robot provided in the embodiments of this application.

[0182] In step 401, a real bionic robot is obtained.

[0183] Here, the environment in which the bionic robot is located is a real-world scenario.

[0184] In step 402, a personalized model is constructed.

[0185] Here, a motion capture system (such as Qualisys) is used to record the segmental information of the bionic robot's body. This recorded segmental information includes the length of the bones in the main body parts and the positions of the rotation axes of the arms, hips, and legs during movement. The specific process is as follows: 27 marker points are pasted onto the entire body of the bionic robot (i.e., the target object). Then, the system software records and saves the static position information of the bionic robot. The static position refers to the position information of the marker points pasted on various parts of the body when the robot is standing still with its arms hanging down. Based on the positions of each marker point, the length of the body skeleton in the generic model in open-source software (such as OpenSim) is adjusted to achieve model personalization. The marker points are used to reflect infrared light, and the static position of the bionic robot is determined based on the reflected infrared light. The bionic robot model (i.e., the initial virtual object) is imported into the open-source software. The scaling function in the open-source software is used to scale the generic model to generate a personalized bionic robot model. The generic model is a 170 cm tall adult male. Since everyone's height and bone length are different, the model's size is matched to the robot's size by pasting marker points onto the robot and then scaling the generic model in the open-source software. See [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram of the personalized model provided in the embodiments of this application. Figure 5 It includes the joints 501 of the personalized model, the bones 502 of the personalized model, and the muscles 503 of the personalized model.

[0186] In step 403, a personalized forward prediction simulation is performed.

[0187] Here, a personalized forward prediction simulation platform is built to positively predict the motion process of personalized models (i.e., virtual objects). The forward prediction simulation of bionic robots is described as an optimal control problem, using muscle activation dynamics, musculoskeletal geometry, and multi-rigid-body dynamics to describe the human musculoskeletal structure.

[0188] By setting objective functions, constraints, and optimizing solvers, muscle control and joint states are optimized to generate a stable gait. Since the true objective function of human movement is unknown, factors such as energy consumption, gait stability, and muscle performance are considered. Predicting human walking involves using muscle coordination to maintain body stability and smooth movement while minimizing energy consumption. The objective function is shown in Equation 1.

[0189]

[0190] Among them, w i Here, d represents the weighting coefficient, d is the distance the pelvis advances along the sagittal axis, and t is the weighting coefficient. f The time is half the gait cycle, E is the metabolic energy expenditure rate, a is the muscle activation rate, u is the joint acceleration, and T is the time of the gait cycle. PFor passive joint torque and e arms It is an arm-driven stimulus.

[0191] The forward predictive simulation platform describes the motion control process of the personalized model as an optimal control problem. The optimization method is a method of finding the extreme value, which aims to make the objective function of the system reach the extreme value under constraints. The constraints are shown in Equation 2-4.

[0192]

[0193] u(t+1)=c*x(t+1) (3)

[0194] x min ≤x(t)≤x max

[0195] u min ≤u(t)≤u max

[0196] p min ≤p≤p max (4)

[0197] Where f(.) is the dynamic constraint of the system, used to predict the control quantity at the next time point based on the control quantity (such as the running speed of a joint) and state quantity (such as the running acceleration of a joint) at the previous time point, x is the control quantity, u is the state quantity, c is the decrement factor used to control the decrement rate of the state quantity, and p is the parameter of the objective function. These parameters are usually fixed or constants that vary within a certain range, used to characterize the physical or behavioral characteristics of the objective function.

[0198] In step 404, the motion trajectory is simulated.

[0199] Here, joint space constraints are set according to the actual range of motion of the bionic robot. Specifically, the swing range of each joint of the model is set in the simulation according to the range of motion of each servo motor of the bionic robot, as shown in Table 1.

[0200] Table 1 Simulated Joint Swing Range

[0201]

[0202] Table 1 defines the swing range of each joint in the model. A set walking speed is then input, ranging from [0.5-2.3]. Forward prediction simulation is performed using the direct collocation method and the interior point method. See [link / reference]. Figure 6 , Figure 6 This is a flowchart of the forward prediction simulation provided in the embodiments of this application. The flowchart of the forward prediction simulation provided in the embodiments of this application will be explained below.

[0203] In step 601, the optimal control problem is solved.

[0204] Here, the musculoskeletal system is a rigid system, and even a small change in muscle excitation can significantly impact the predicted motion pattern and the objective function. To avoid such influence, a direct collocation method is introduced. By reducing the time range of the integration, the sensitivity of the objective function to the optimization variables is decreased. This transforms the optimal control problem into a series of nonlinear programming (NLP) problems, which are solved using third-order ordinary differential equations. The transformation process is implemented using an open-source optimization program (such as CasaADi). In the simulation, 50 collocation points are set for each gait cycle.

[0205] In step 602, the nonlinear programming problem is solved.

[0206] Here, the interior point method is used to solve the problem. This process uses a nonlinear programming solver (such as IPOPT) to obtain the solution to the problem, which is the movement trajectory of each joint in the whole body during half a gait cycle.

[0207] A symmetrical strategy is adopted. Based on the motion trajectories of all joints in the entire gait cycle obtained from the solution of half a gait cycle, the motion trajectory of each joint in the entire gait cycle is determined. Specifically, based on the solution of half a gait cycle, the other half of the gait cycle is completed in a completely symmetrical manner. The motion trajectory is a flexion angle curve obtained by connecting the flexion angles corresponding to each time point in the gait cycle. See [link to relevant documentation] Figure 6 , Figure 7 This is a schematic diagram of the movement trajectory of the shoulder joint in a gait cycle provided in this application embodiment. The movement trajectory 701 of the shoulder joint is a flexion angle curve obtained by connecting the flexion angles of the shoulder joint at each time point of the gait cycle. The walking speed is set to 0.8m / s.

[0208] In step 405, the motion trajectory of the bionic robot arm is determined.

[0209] Here, the trajectory output by the arm in the simulation includes the angles of the shoulder joint in the X, Y, and Z directions and the elbow joint in the Y direction at different time points. Since the bionic robot arm has a total of 6 degrees of freedom, the two degrees of freedom of the wrist joint during walking are set to 0 degrees. The joint mapping from the simulated arm to the real robot arm for the remaining 4 degrees of freedom is shown in Table 2.

[0210] Table 2. Joint mapping relationship between simulated arm and real machine arm

[0211] Joints in simulation Joints in the real machine Simulated shoulder joint X-direction degrees of freedom X-axis degrees of freedom of the shoulder joint on a real machine Simulated shoulder joint Y-direction degrees of freedom The Y-axis degree of freedom of the shoulder joint on the actual machine Simulated shoulder joint Z-direction degrees of freedom The Z-axis degree of freedom of the shoulder joint on the actual machine Simulated elbow joint Y-direction degree of freedom The degree of freedom of the elbow joint in the Y direction of the real machine 0 The X-axis degree of freedom of the wrist joint on a real machine 0 The Z-axis degree of freedom of the wrist joint in a real machine

[0212] Among them, the two degrees of freedom of the wrist are the same as those in the real machine, but these two degrees of freedom were not optimized in the simulation. Based on the swing state of the arm when a person walks, these two degrees of freedom were simplified to 0.

[0213] Import the curves output by the simulation platform, and according to the correspondence in Table 1, complete the mapping transformation of the angles of each joint of the arm to obtain the angles of the arm joints of the bionic robot.

[0214] In step 406, the bionic robot performs arm movements.

[0215] Here, based on the set running distance and speed of the bionic robot, the total running time of the bionic robot is obtained, and the motor rotation angle of the arm movement at the corresponding time is output. Specifically, software (such as Maya software) can output the motor rotation angle of each joint at a corresponding time (set frequency of 100Hz). Taking 100Hz as an example, the output shows the motor rotation angle of each joint at 100 time points during the running time. See [link to documentation]. Figure 8 , Figure 8 This is a schematic diagram of software processing provided in an embodiment of this application, such as... Figure 8 As shown, the frame rate is used to edit the motion of each frame of the left forearm node 802 in the personalized model 801, that is, the flexion angle corresponding to each frame on the flexion angle curve 803 corresponding to rotation X. The left forearm node 802 can rotate in three directions, namely rotation X, rotation Y, and rotation Z. After obtaining the flexion angle curve of each joint, the flexion angle curve is smoothed. The smoothed flexion angle curve of each joint can be exported from the system to generate the motion table of each joint. According to the time sequence, the motion table (i.e., the motion sequence) is generated based on the motor rotation angle of each joint of the arm at the corresponding time. The motion table of the joint corresponding to the motor URL number "28:1266" is shown in Table 3.

[0216] Table 3 shows the motion table of the joints corresponding to the motor URL number "28:1266".

[0217] Motor rotation arc time -1.66 0.00 -1.66 0.01 -1.65 0.02 …… …… -1.53 0.39 -1.52 0.4

[0218] Before performing the corresponding arm movements from the action table on the actual bionic robot, reset the rotation angles of each joint on the actual bionic robot. See [link to relevant documentation]. Figure 9 , Figure 9 This is a schematic diagram of joint reset provided in the embodiment of this application, in which the joint 902 of the bionic robot 901 is adjusted to a preset initial position.

[0219] By reading the motion table corresponding to each motor, the bionic robot can move according to the simulation-optimized trajectory and complete the corresponding arm movements (i.e., actions) in the motion table on the actual bionic robot. Each action in the motion table is the rotation angle of the motor of each joint of the arm at the corresponding time. The rotation angle is sent to the corresponding joint of the arm to realize the rotation of the arm.

[0220] In summary, the embodiments of this application introduce control planning for arm swing during the walking process of a bionic robot, and optimize the coordination between the arm and gait through a positive prediction simulation platform, ensuring that the arm movement trajectory is synchronized with the gait, thereby achieving smoother arm movement reproduction. In addition, the bionic robot can significantly improve balance and stability when walking, reduce posture deviations caused by lack of coordinated arm movement, and improve the overall walking quality. At the same time, the coordinated movement of the arm and gait enhances the bionic effect of the bionic robot, making its movement closer to the natural walking pattern of humans, improving the performance of the bionic robot in human-computer interaction, and making it more suitable for interactive tasks in practical application scenarios. Through the method provided by the embodiments of this application, the balance, stability and bionic nature of the bionic robot during walking can be improved.

[0221] The following description continues to illustrate the exemplary structure of the information processing device 555 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the information processing device 555 of the memory 550 may include:

[0222] The data acquisition module 5551 is used to acquire joint information of the target object's joints.

[0223] Object construction module 5552 is used to construct virtual objects corresponding to the target object in a virtual scene based on joint information.

[0224] The motion simulation module 5553 is used to simulate the movement of the arm joints of the virtual object based on the running speed of the virtual object in the gait cycle, and obtain the angle rotation curve of the arm joint of the virtual object in the gait cycle. The angle rotation curve represents the rotation angle of the arm joint at each time point in the gait cycle.

[0225] The motion mapping module 5554 is used to perform motion mapping on the angle rotation curve to obtain the motion sequence of the target object's arm joint. The motion sequence includes the action of the target object at each time point within the gait cycle, and each action is obtained by the target object's arm joint performing the corresponding rotation angle.

[0226] In some embodiments, the object construction module 5552 is further configured to obtain an initial virtual object in a virtual scene and virtual parts connected to joints in the initial virtual object, and obtain the real scene where the target object is located; based on the position mapping relationship between the real scene and the virtual scene, perform position mapping on the joints of the target object to obtain the target position of the joints of the initial virtual object; perform alignment processing on the virtual parts based on the target position and the initial position to obtain the aligned virtual parts; adjust the joints of the initial virtual object from the initial position to the target position to obtain the adjusted joints; combine the adjusted joints and the aligned virtual parts to obtain the virtual object corresponding to the target object.

[0227] In some embodiments, the motion simulation module 5553 is further configured to: construct an objective function for controlling the virtual object's motion based on the virtual object's running speed during the gait cycle; obtain the constraints of the virtual object's joints; construct a penalty function based on the penalty factor for the constraints and the objective function; obtain an initial solution and an initial penalty factor that satisfy the constraints; perform iterative calculations on the penalty function based on the initial solution and the initial penalty factor to obtain the objective solution of the objective function that satisfies the constraints; and determine the angle rotation curve of the arm joints of the virtual object during the gait cycle based on the objective solution.

[0228] In some embodiments, the motion simulation module 5553 is further configured to perform the following processing for each preset time point in the gait cycle: determine the angular velocity of the nodes of the virtual object at the time point based on the running speed of the virtual object in the gait cycle; determine the motion acceleration of the joints of the virtual object at the time point based on the angular velocity of the nodes of the virtual object at the time point; and determine the sum of squares of the motion accelerations corresponding to the preset time points as the objective function for controlling the motion of the virtual object.

[0229] In some embodiments, the motion simulation module 5553 is further configured to determine the constraints of the joints of the virtual object by at least one of the following methods: based on the joint information, querying a condition table, determining the candidate conditions corresponding to the joint information as the constraints of the joints of the virtual object, wherein the condition table is used to characterize the correspondence between different candidate joint information and different candidate conditions; encoding the joint information to obtain encoded features, and decoding the encoded features to obtain the constraints of the joints of the virtual object.

[0230] In some embodiments, the motion simulation module 5553 is further configured to perform the following operations for the iteration: substituting the solution of the (i-1)th round and the penalty factor of the (i-1)th round into the penalty function to obtain the penalty value of the i-th round; determining the penalty factor of the i-th round as the product of the penalty factor of the (i-1)th round and the decreasing factor; determining the solution of the i-th round as the product of the penalty factor of the i-th round and the solution of the (i-1)th round; when the difference between the penalty value of the i-th round and the penalty value of the (i-1)th round is less than the penalty threshold, determining the solution of the i-th round as the objective solution of the objective function that satisfies the constraint conditions, and stopping the iteration, where i is a positive integer that increases sequentially, the solution of the 0th round is the initial solution, and the penalty factor of the 0th round is the initial penalty factor.

[0231] In some embodiments, the motion simulation module 5553 is further configured to determine the rotation angle at other time points based on the rotation angle at a preset time point, wherein the other time points are time points other than the preset time point within the gait cycle, and the target solution includes the rotation angle of the arm joint of the virtual object at the preset time point; and to perform curve fitting on the rotation angle at the preset time point and the rotation angle at other time points to obtain the angle rotation curve of the arm joint of the virtual object within the gait cycle.

[0232] In some embodiments, the motion mapping module 5554 is further configured to perform the following processing for each time point within the gait cycle: query the angle rotation curve based on the time point to obtain the rotation angle corresponding to the time point; control the arm joint of the target object to execute the rotation angle corresponding to the time point to obtain the action corresponding to the time point; combine the actions corresponding to each time point to obtain the action sequence of the arm joint of the target object.

[0233] In some embodiments, the motion mapping module 5554 is also used to control the arm joints of the target object to perform each action in the motion sequence.

[0234] This application provides a computer program product, which includes computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the information processing method described in this application embodiment.

[0235] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the information processing method provided in this application, for example... Figures 3A to 3E The information processing method shown.

[0236] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0237] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0238] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0239] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0240] In summary, based on the joint information of the target object, a virtual object corresponding to the target object is constructed in the virtual scene. In this way, by constructing a virtual object corresponding to the target object, the target object's actions can be directly controlled according to the joint rotation angles of the virtual object obtained from the simulation, achieving consistency between the target object and the virtual object. Based on the running speed of the virtual object in the gait cycle, the arm joints of the virtual object are simulated to obtain the angle rotation curves of the arm joints of the virtual object in the gait cycle. The angle rotation curves are then mapped to obtain the action sequence of the target object's arm joints. In this way, the coordination between the arm movement and the gait can be ensured, and the action of the target object's arm joints in the gait cycle can be reproduced.

[0241] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An information processing method characterized by comprising: The method comprises: obtaining joint information of a joint of a target object; based on the joint information, constructing a virtual object corresponding to the target object in a virtual scene; based on the running speed of the virtual object in a gait cycle, constructing a target function for controlling the movement of the virtual object; obtaining the constraint condition of the joint of the virtual object, and constructing a penalty function based on the constraint condition and the target function by using a penalty factor; obtaining an initial solution and an initial penalty factor that satisfy the constraint condition, and performing iterative operation on the penalty function based on the initial solution and the initial penalty factor to obtain a target solution of the target function that satisfies the constraint condition; based on the target solution, determining an angle rotation curve of the arm joint of the virtual object in the gait cycle, wherein the angle rotation curve represents the rotation angle of the arm joint corresponding to each time point in the gait cycle; mapping the angle rotation curve to obtain a motion sequence of the arm joint of the target object, wherein the motion sequence includes the motion of the target object at each time point in the gait cycle, and each motion is obtained by the target object executing the corresponding rotation angle.

2. The method of claim 1, wherein, The joint information includes the position of the joint of the target object, and the construction of the virtual object corresponding to the target object in the virtual scene based on the joint information comprises: obtaining an initial virtual object in the virtual scene and a virtual part connected to the joint in the initial virtual object, and obtaining a real scene in which the target object is located; based on the position mapping relationship between the real scene and the virtual scene, performing position mapping on the position of the joint of the target object to obtain a target position of the joint of the initial virtual object; based on the target position and the initial position, performing alignment processing on the virtual part to obtain the aligned virtual part; adjusting the joint of the initial virtual object from the initial position to the target position to obtain the adjusted joint; combining the adjusted joint and the aligned virtual part to obtain the virtual object corresponding to the target object.

3. The method of claim 1, wherein, The construction of the target function for controlling the movement of the virtual object based on the running speed of the virtual object in the gait cycle comprises: for each time point in the preset time points of the gait cycle, the following processing is performed: based on the running speed of the virtual object in the gait cycle, determining the angular velocity of the node of the virtual object at the time point; based on the angular velocity of the node of the virtual object at the time point, determining the motion acceleration of the joint of the virtual object at the time point; determining the sum of squares of the motion accelerations corresponding to the preset time points as the target function for controlling the movement of the virtual object.

4. The method of claim 1, wherein, The constraint condition of the joint of the virtual object comprises: the constraint condition of the joint of the virtual object is determined by at least one of the following methods: query a condition table based on the joint information, determine a candidate condition corresponding to the joint information as a constraint condition of a joint of the virtual object, the condition table being used to represent a corresponding relationship between different candidate joint information and different candidate conditions; encode the joint information to obtain an encoded feature, and decode the encoded feature to obtain the constraint condition of the joint of the virtual object.

5. The method of claim 1, wherein, the iterative operation of the penalty function based on the initial solution and an initial penalty factor to obtain a target solution of the target function satisfying the constraint condition, including: perform the following operations for iteration: substitute the solution of the i-1th round and the penalty factor of the i-1th round into the penalty function to obtain a penalty value of the ith round; determine the product of the penalty factor of the i-1th round and a decrement factor as the penalty factor of the ith round; determine the product of the penalty factor of the ith round and the solution of the i-1th round as the solution of the ith round; when the difference between the penalty value of the ith round and the penalty value of the i-1th round is less than a penalty threshold, determine the solution of the ith round as the target solution of the target function satisfying the constraint condition, and stop the iteration, wherein i>0, i is a positive integer that increases sequentially, the solution of the 0th round is the initial solution, and the penalty factor of the 0th round is the initial penalty factor.

6. The method of claim 1, wherein, the target solution includes a rotation angle of an arm joint of the virtual object at a preset time point, and determining an angle rotation curve of the arm joint of the virtual object in the gait cycle based on the target solution includes: determining rotation angles of other time points based on the rotation angle of the preset time point, wherein the other time points are time points in the gait cycle other than the preset time point; performing curve fitting on the rotation angle of the preset time point and the rotation angles of the other time points to obtain the angle rotation curve of the arm joint of the virtual object in the gait cycle.

7. The method of claim 1, wherein, the action mapping of the angle rotation curve to obtain the action sequence of the arm joint of the target object includes: perform the following processing for each time point in the gait cycle: query the angle rotation curve based on the time point to obtain a rotation angle corresponding to the time point; control the arm joint of the target object to perform the rotation angle corresponding to the time point to obtain an action corresponding to the time point; combine the actions corresponding to each time point to obtain the action sequence of the arm joint of the target object; after the action mapping of the angle rotation curve to obtain the action sequence of the arm joint of the target object, the method further includes: control the arm joint of the target object to perform each action in the action sequence.

8. An information processing apparatus, characterized by comprising: the device includes: a data acquisition module configured to acquire joint information of a joint of a target object; an object construction module configured to construct a virtual object corresponding to the target object in a virtual scene based on the joint information; The motion simulation module is configured to construct a target function for controlling the motion of the virtual object based on a running speed of the virtual object in a gait cycle, obtain a constraint condition of a joint of the virtual object, construct a penalty function based on a penalty factor, the constraint condition and the target function, obtain an initial solution satisfying the constraint condition and an initial penalty factor, perform iterative operation on the penalty function based on the initial solution and the initial penalty factor, obtain a target solution of the target function satisfying the constraint condition, and determine an angle rotation curve of an arm joint of the virtual object in the gait cycle based on the target solution, where the angle rotation curve represents a rotation angle of the arm joint corresponding to each time point in the gait cycle. The action mapping module is configured to perform action mapping on the angle rotation curve to obtain an action sequence of the arm joint of the target object, where the action sequence includes an action of the target object at each time point in the gait cycle, and each action is obtained by the arm joint of the target object performing a corresponding rotation angle.

9. An electronic device, comprising: The electronic device includes: a memory configured to store computer executable instructions; a processor configured to execute the computer executable instructions or computer programs stored in the memory to implement the information processing method of any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, The computer executable instructions or computer programs are executed by the processor to implement the information processing method of any one of claims 1 to 7.

11. A computer program product comprising computer-executable instructions or a computer program, characterized in that, The computer executable instructions or computer programs are executed by the processor to implement the information processing method of any one of claims 1 to 7. The computer executable instructions or computer programs are executed by the processor to implement the information processing method of any one of claims 1 to 7.

Images