Information processing method and related device

Abstract

The invention provides an information processing method and a related device. The method comprises the steps of obtaining joint information of joints of a target object; constructing a virtual object corresponding to the target object in the virtual scene based on the joint information; based on the running speed of the virtual object in the gait cycle, performing running simulation on the arm joint of the virtual object to obtain an angle rotation curve of the arm joint of the virtual object in the gait cycle; and performing action mapping on the angle rotation curve to obtain an action sequence of the arm joint of the target object. According to the method and the device, the coordination of the action and the gait of the arm movement can be ensured, and the action reproduction of the arm joint of the target object in the gait period is realized.

Description

Technical Field

[0001] The present application relates to robotics technology, and in particular to an information processing method and related devices. Background Art

[0002] With the continuous advancement of robotics technology, bionic robots have been widely used in many fields, especially in service, medical, rescue and other scenarios. Bionic robots can imitate human behavior, improve the naturalness of operation and the comfort of interaction. When performing anthropomorphic movements, bionic robots simplify arm movements and reduce the stability and coordination of the overall movement. Summary of the invention

[0003] The embodiments of the present application provide an information processing method and related devices, which can ensure the coordination of arm movement and gait, and realize the reproduction of arm joint movements of a target object during the gait cycle.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides an information processing method, the method comprising:

[0006] Get the joint information of the joints of the target object;

[0007] Based on the joint information, construct a virtual object corresponding to the target object in a virtual scene;

[0008] Based on the running speed of the virtual object in the gait cycle, the arm joint of the virtual object is simulated to obtain an angle rotation curve of the arm joint of the virtual object in the gait cycle, wherein the angle rotation curve represents the rotation angle of the arm joint corresponding to each time point in the gait cycle;

[0009] Performing motion mapping on the angle rotation curve, obtaining a motion sequence of the arm joints of the target object, wherein the motion sequence includes the motion of the target object at each time point in the gait cycle, and each motion is obtained by the arm joints of the target object executing the corresponding rotation angle.

[0010] The present application provides an information processing device, including:

[0011] A data acquisition module, used to obtain joint information of the joints of the target object;

[0012] An object construction module, used to construct a virtual object corresponding to the target object in a virtual scene based on the joint information;

[0013] A motion simulation module, used for performing motion simulation on the arm joints of the virtual object based on the running speed of the virtual object in the gait cycle, and obtaining an angle rotation curve of the arm joints of the virtual object in the gait cycle, wherein the angle rotation curve represents the rotation angle of the arm joint corresponding to each time point in the gait cycle;

[0014] An action mapping module is used to perform action mapping on the angle rotation curve to obtain an action sequence of the arm joints of the target object, wherein the action sequence includes the action of the target object at each time point in the gait cycle, and each action is obtained by the arm joints of the target object executing the corresponding rotation angle.

[0015] An embodiment of the present application provides an electronic device, including:

[0016] A memory for storing computer executable instructions;

[0017] The processor is used to implement the information processing method provided in the embodiment of the present application when executing the computer executable instructions stored in the memory.

[0018] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the information processing method provided in the embodiment of the present application when executed by a processor.

[0019] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the information processing method provided by the embodiment of the present application is implemented.

[0020] The embodiments of the present application have the following beneficial effects:

[0021] Based on the joint information of the target object's joints, a virtual object corresponding to the target object in the virtual scene is constructed. In this way, by constructing the virtual object corresponding to the target object, the target object can be directly controlled to perform actions according to the joint rotation angle of the virtual object obtained by running simulation, thereby achieving consistency between the target object and the virtual object. Based on the running speed of the virtual object in the gait cycle, the arm joints of the virtual object are run simulated to obtain the angle rotation curve of the arm joints of the virtual object in the gait cycle, and the angle rotation curve is action mapped to obtain the action sequence of the arm joints of the target object. In this way, the coordination of the arm movement and the gait can be ensured, and the action reproduction of the arm joints of the target object in the gait cycle can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a schematic diagram of the architecture of an information processing system 100 provided in an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0024] Figure 3A It is a first flow chart of the information processing method provided by an embodiment of the present application;

[0025] Figure 3B is a second flow chart of the information processing method provided in an embodiment of the present application;

[0026] Figure 3C is a third flow chart of the information processing method provided in the embodiment of the present application;

[0027] Figure 3D is a fourth flow chart of the information processing method provided in an embodiment of the present application;

[0028] Figure 3E is a fifth flow chart of the information processing method provided in the embodiment of the present application;

[0029] Figure 4 is a flowchart of processing joint information of a bionic robot provided in an embodiment of the present application;

[0030] Figure 5 is a schematic diagram of a personalized model provided in an embodiment of the present application;

[0031] Figure 6 is a forward prediction simulation flow chart provided in an embodiment of the present application;

[0032] Figure 7 is a schematic diagram of the motion trajectory of the shoulder joint in a gait cycle provided by an embodiment of the present application;

[0033] Figure 8 It is a schematic diagram of software processing provided by an embodiment of the present application;

[0034] Fig. 9 It is a schematic diagram of joint reduction provided in an embodiment of the present application.

[0035] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of superiority or inferiority of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0037] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0038] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0039] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0040] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0041] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0042] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0043] 1) The target object is a machine object in a real scene. The embodiment of the present application does not limit the machine object. The machine object can be a robot, a robot dog, etc.

[0044] 2) Joint information, which is used to record relevant information of the joints. The embodiment of the present application does not limit the joint information of the joints. The joint information can be at least one of the following information: the name of the joint, the purpose of the joint, the location of the joint, and the length between adjacent joints.

[0045] 3) Virtual objects are images of various people and objects in the virtual scene for gait simulation. The image of the virtual object is consistent with the image of the target object. The virtual object can be a virtual person, a virtual animal, etc., such as a person or an animal displayed in a virtual scene. The virtual object can be a virtual image in the virtual scene that represents the user. The virtual scene can include multiple virtual objects. Each virtual object has its own shape and volume in the virtual scene and occupies a part of the space of the virtual scene.

[0046] 4) Virtual scene: When the program is running on the terminal, it presents a virtual scene. The virtual scene is a simulation of the real world, used to simulate various conditions and situations in the real world, such as the physical rules, objects, lighting conditions, terrain and other environmental factors of the real world, and to test, verify and train the performance and behavior of virtual objects in different environments according to the simulated conditions and situations. Users can control the movement of virtual objects in the virtual scene. The virtual scene can be a simulation of the real world, a semi-simulated and semi-fictional virtual environment, or a purely fictional virtual environment. The virtual scene can be any one of a two-dimensional virtual scene, a 2.5-dimensional virtual scene or a three-dimensional virtual scene. The embodiment of the present application does not limit the dimension of the virtual scene.

[0047] 5) The gait cycle refers to the complete process that a person goes through when walking, starting from the heel of one leg touching the ground to the heel of the same leg touching the ground again. This process can be subdivided into the following stages: initial contact stage, weight-bearing stage, single-leg support stage and swing stage. The initial contact stage is the beginning stage of the gait cycle, which is used to characterize the first contact of the walker's foot with the ground; the weight-bearing stage is used to characterize the stage from initial contact to the opposite foot leaving the ground, at which time the body weight is transferred from one foot to the other foot; the single-leg support stage is used to characterize the node when the body is fully supported by one foot after the opposite foot leaves the ground. This stage includes the heel leaving the ground and the toes pushing the body forward; the swing stage is used to characterize the stage from the toe leaving the ground to the foot contacting the ground again. This stage includes three stages: acceleration, neutralization and deceleration. The walker's legs swing forward to prepare for the next contact with the ground. The gait cycle is used to study the movement patterns of the human body when walking in order to evaluate and treat gait abnormalities.

[0048] In the related art, since the movement of the arms increases the complexity of control and may cause stability problems, although the arms of the target object have a high degree of freedom, the control strategy is usually simplified. In particular, during the walking process of the target object, the movement of the arms is often ignored, and the moving parts of the arms are not integrated into the walking process, which reduces the stability and coordination of the overall movement. To address the above problems, the embodiments of the present application provide an information processing method, device, electronic device, computer-readable storage medium and computer program product, which can ensure the coordination of arm movement and gait, and achieve the reproduction of the movement of the arm joints of the target object during the gait cycle, so as to improve the stability and coordination of the overall movement.

[0049] The information processing method described in the embodiments of the present application can be applied to various fields, for example, the field of robot motion simulation, that is, the information processing method in the embodiments of the present application is not limited to a certain field.

[0050] The following describes an exemplary application of the electronic device provided in the embodiment of the present application. The device provided in the embodiment of the present application can be implemented as a terminal or a server. The following describes an exemplary application when the device is implemented as a server.

[0051] See also Figure 1 , Figure 1 It is a schematic diagram of the architecture of the information processing system 100 provided in an embodiment of the present application. To support an information processing application, a terminal (terminal 400 is shown as an example) is connected to a server 200 via a network 300. The network 300 may be a wide area network or a local area network, or a combination of the two.

[0052] Terminal 400 is used to send the joint information of the joints of the target object to server 200 through network 300. Server 200 is used to construct a virtual object corresponding to the target object in a virtual scene based on the joint information, and based on the running speed of the virtual object in the gait cycle, perform operation simulation on the arm joints of the virtual object to obtain the angle rotation curve of the arm joints of the virtual object in the gait cycle, perform action mapping on the angle rotation curve to obtain the action sequence of the arm joints of the target object, and return the action sequence to terminal 400. Terminal 400 controls the arm joints of the target object to perform each action in the action sequence.

[0053] Next, an example of information processing performed by terminal 400 will be described.

[0054] In some embodiments, terminal 400 can independently complete information processing tasks. For example, terminal 400 is used to obtain joint information of the joints of the target object, and based on the joint information, construct a virtual object corresponding to the target object in a virtual scene, and based on the running speed of the virtual object in the gait cycle, perform operation simulation on the arm joints of the virtual object to obtain the angle rotation curve of the arm joints of the virtual object in the gait cycle, perform action mapping on the angle rotation curve to obtain the action sequence of the arm joints of the target object, and control the arm joints of the target object to perform each action in the action sequence.

[0055] In one implementation scenario, the server or terminal can process the joint information of the real robot, obtain the joint information of the real robot, and based on the joint information, construct a simulated robot corresponding to the real robot in the simulation scenario, and based on the running speed of the simulated robot in the gait cycle, perform operation simulation on the arm joints of the simulated robot to obtain the angle rotation curve of the arm joints of the simulated robot in the gait cycle, perform action mapping on the angle rotation curve to obtain the action sequence of the arm joints of the real robot, and control the arm joints of the real robot to perform each action in the action sequence.

[0056] In some embodiments, server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0057] The terminal 400 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0058] See also Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 2 The electronic device 500 shown may be Figure 1In the terminal 400 or server 200, the electronic device 500 includes: at least one processor 510, a memory 550, and at least one network interface 520. The various components in the server 200 are coupled together through a bus system 540. It can be understood that the bus system 540 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in Figure 2 Various buses are labeled as bus system 540 .

[0059] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0060] The user interface 530 includes one or more output devices 531 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls;

[0061] In some embodiments, when the terminal 400 independently completes the information processing task, the server 200 provided in the embodiment of the present application does not include the user interface 530 .

[0062] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, a hard disk drive, an optical disk drive, etc. The memory 550 may optionally include one or more storage devices that are physically located away from the processor 510.

[0063] The memory 550 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 550 described in the embodiments of the present application is intended to include any suitable type of memory.

[0064] In some embodiments, the memory 550 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0065] Operating system 551, including system programs for processing various basic system services and performing hardware-related tasks, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0066] A network communication module 552, for reaching other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB);

[0067] a presentation module 553 for enabling presentation of information via one or more output devices 531 (e.g., display screen, speaker, etc.) associated with the user interface 530 (e.g., a user interface for operating peripherals and displaying content and information);

[0068] In some embodiments, when the information processing task is completed independently by the terminal 400, the server 200 provided in the embodiment of the present application may not include the presentation module 553.

[0069] The input processing module 554 is used to detect one or more user inputs or interactions from one of the one or more input devices 532 and translate the detected inputs or interactions; in some embodiments, when the embodiment independently completes the information processing task by the terminal 400, the server 200 provided in the embodiment of the present application may not include the presentation module 553.

[0070] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 The information processing device 555 stored in the memory 550 is shown, which can be software in the form of programs and plug-ins, including the following software modules: data acquisition module 5551, object construction module 5552, motion simulation module 5553 and action mapping module 5554. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0071] It should be noted that in the information processing examples below, those skilled in the art may apply the information processing method provided in the embodiments of the present application to robot simulation based on their understanding of the following.

[0072] See also Figure 3A , Figure 3A is a first flow chart of the information processing method provided in the embodiment of the present application, which will be combined with Figure 3AThe steps shown are used for explanation. The information processing method provided in the embodiments of the present application can be implemented by a server or a terminal alone, or by a server and a terminal in collaboration. The following will take the collaborative implementation by a server and a terminal as an example for explanation.

[0073] In step 101 , joint information of a joint of a target object is obtained.

[0074] Here, the target object is a machine object in a real scene. The embodiment of the present application does not limit the machine object. The machine object can be a robot, a robot dog, etc. The real scene is a scene that occurs in real life or an actual working environment. In the real scene, the machine object can be controlled to perform corresponding actions, such as controlling a robot to perform a dance performance in a square in real life. The real scene is used to describe the actual situation relative to the virtual scene. The joints of the target object are an important part of the skeletal system and are used to drive the skeleton to perform various movements. The embodiment of the present application does not limit the types of joints. The joints can be shoulder joints, elbow joints, and wrist joints. The shoulder joint is used to drive the arm as a whole to perform a wide range of movements, and the movements include at least one of the following: flexion, extension, adduction, abduction and rotation, etc.; the elbow joint is used to drive the flexion and extension of the forearm; the wrist joint is used to drive the movement between the hand and the forearm. The joint information is used to record the relevant information of the joint. The embodiment of the present application does not limit the joint information of the joint. The joint information can be at least one of the following information: the name of the joint, the use of the joint, the position of the joint, and the length between adjacent joints.

[0075] In some embodiments, step 101 can be implemented by: constructing mark points at the joints of the target object, the mark points are used to reflect infrared light; in response to the preset infrared light being reflected, the position information of the joint is determined based on the reflected infrared light, and the position information of the joint is determined as the joint information of the joint.

[0076] It should be noted that when the mark point is used to reflect the preset infrared light emitted by the motion capture system and reflect it back to the motion capture system, the motion capture system is used to emit and receive infrared light and determine the position of the mark point according to the received infrared light.

[0077] Continuing from the above embodiment, the above-mentioned “determining the position information of the joint based on the reflected infrared light” can be achieved in the following way: the infrared camera of the motion capture system emits infrared light of a specific wavelength, which is reflected back to the infrared camera after hitting the mark point, and the sensor matrix on the infrared camera receives the reflected infrared light and obtains the coordinates of the mark point through signal processing.

[0078] In step 102, a virtual object corresponding to the target object in the virtual scene is constructed based on the joint information.

[0079] Here, the program presents a virtual scene when the terminal is running. The virtual scene is a simulation scene of the real world, which is used to simulate various conditions and situations in the real world, such as the physical rules, objects, lighting conditions, terrain and other environmental factors of the real world, and test, verify and train the performance and behavior of virtual objects in different environments according to the simulated conditions and situations. Users can control virtual objects to move in the virtual scene. The virtual scene can be a simulation environment of the real world, a semi-simulated and semi-fictitious virtual environment, or a purely fictitious virtual environment. The virtual scene can be any one of a two-dimensional virtual scene, a 2.5-dimensional virtual scene or a three-dimensional virtual scene. The embodiment of the present application does not limit the dimension of the virtual scene; the virtual object is the image of various people and objects in the virtual scene for gait simulation, and the image of the virtual object is consistent with the image of the target object. The virtual object can be a virtual person, a virtual animal, etc., such as a person, an animal, etc. displayed in the virtual scene. The virtual object can be a virtual image in the virtual scene that represents the user. The virtual scene can include multiple virtual objects, each of which has its own shape and volume in the virtual scene and occupies a part of the space of the virtual scene.

[0080] In some embodiments, the joint information includes the positions of the joints of the target object, see Figure 3B , Figure 3B is a second flow chart of the information processing method provided in the embodiment of the present application, for Figure 3A Step 102 shown can be performed by Figure 3B Steps 1021 to 1025 are implemented as described in detail below.

[0081] In step 1021, an initial virtual object in a virtual scene and a virtual part of the initial virtual object connected to the joint are obtained, and a real scene in which the target object is located is obtained.

[0082] Here, the initial virtual object is a virtual object with a specific size in a virtual scene. The embodiment of the present application does not limit the initial virtual object. The initial virtual object can be a virtual object of a preset size or a virtual object that has been constructed in the virtual scene. The initial virtual object includes multiple virtual parts, and the virtual parts are connected through the joints of the initial virtual object.

[0083] In step 1022, based on the position mapping relationship between the real scene and the virtual scene, the positions of the joints of the target object are mapped to obtain the target positions of the joints of the initial virtual object.

[0084] Here, the target positions of the joints of the initial virtual object are the positions of the joints of the virtual object.

[0085] It should be noted that the real scene and the virtual scene have a position-related relationship, and the coordinate system of the real scene is established based on GPS or a local positioning system (such as an indoor positioning system). For example, assume a Cartesian coordinate system based on the ground, in which the origin is located at a fixed position of the scene, the X-axis points to the north, the Y-axis points to the east, and the Z-axis is vertically upward. In a 3D virtual environment, the coordinate system of the virtual scene is a coordinate system with a point on the virtual terrain as the origin. The embodiment of the present application does not limit the process of position mapping, and the position mapping can be translation, rotation, and scaling, etc., wherein translation requires determination of the distance and direction between the origins of the two coordinate systems; rotation requires determination of the angular relationship between the axes of the two coordinate systems; scaling requires determination of the scaling factor, which is used to characterize the scaling ratio between the two coordinate systems.

[0086] In some embodiments, taking the position mapping process of coordinate system translation as an example, step 1022 can be implemented by: determining the distance (such as 5) and direction (such as east) between the origins of the two coordinate systems; moving the position of the joint of the target object in that direction by a corresponding distance to obtain the target position of the joint of the initial virtual object.

[0087] In some embodiments, taking the position mapping process of coordinate system rotation as an example, step 1022 can be implemented by: determining the rotation angle between the two coordinate system axes (such as 5); rotating the position of the joint of the target object according to the rotation angle to obtain the target position of the joint of the initial virtual object.

[0088] In some embodiments, taking the position mapping process of coordinate system scaling as an example, step 1022 can be implemented by: determining a scaling factor (such as 0.5); multiplying the initial position of the joint of the target object by the scaling factor to determine the target position of the joint of the initial virtual object.

[0089] In some embodiments, the following steps may be performed: based on the target position and the initial positions of the joints of the initial virtual object, a virtual object corresponding to the target object in the virtual scene is constructed. The above-mentioned step of “based on the target position and the initial positions of the joints of the initial virtual object, a virtual object corresponding to the target object in the virtual scene is constructed” may be implemented through steps 1023 to 1025.

[0090] In step 1023, the virtual part is aligned based on the target position and the initial position to obtain an aligned virtual part.

[0091] Here, the initial positions of the joints of the initial virtual object are positions of the joints of the initial virtual object displayed in response to program startup, and the image of the virtual object is consistent with the image of the target object in the real scene.

[0092] In some embodiments, the above-mentioned "aligning the virtual part based on the target position and the initial position to obtain the aligned virtual part" can be achieved in the following ways: determining a rotation matrix, the rotation matrix is ​​used to rotate the virtual part from a direction aligned with the initial position to a direction aligned with the target position; applying the rotation matrix to the virtual part, updating the transformation matrix of the virtual part, obtaining the updated transformation matrix of the virtual part, and determining the aligned virtual part based on the updated transformation matrix of the virtual part, wherein the transformation matrix of the virtual part is used to describe the position of the virtual part.

[0093] It should be noted that the transformation matrix of the virtual part is used to represent and perform geometric transformations, such as rotation, translation, scaling and shearing, etc., and is applied to points, vectors or entire objects. In 3D graphics, the transformation matrix is ​​usually a 4*4 matrix. The first 3*3 parts of the matrix are usually used to describe rotations, consisting of three Euler angles (rotation angles around the X-axis, Y-axis, and Z-axis). The last column of the matrix (the first three elements of the fourth column) is used to describe translation, that is, displacement along the X, Y, and Z axes. The last row of the matrix is ​​used to implement perspective transformation, which affects the viewing distance and size of the object.

[0094] Continuing from the above embodiment, the above “applying the rotation matrix to the virtual part, updating the transformation matrix of the virtual part, and obtaining the updated transformation matrix of the virtual part” can be achieved in the following way: multiplying the rotation matrix and the transformation matrix of the virtual part to determine the updated transformation matrix of the virtual part.

[0095] For example, given a rotation matrix (such as [[cos(30), 0, sin(30), 0], [0, 1, 0, 0], [-sin(30), 0, cos(30), 0], [0, 0, 0, 1]]), where sin is used to represent the sine value of an angle of 30 degrees and cos is used to represent the cosine value of an angle of 30 degrees, the rotation matrix is ​​used to represent the rotation of the virtual part around the Y axis by 30 degrees. The product of the rotation matrix and the transformation matrix of the virtual part (such as [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 1.0]) is determined as the updated transformation matrix of the virtual part (such as [[0.86, 0, 0.5, 0], [0, 1, 0, 0], [-0.5, 0, 0.86, 0], [0, 0, 0, 1]]).

[0096] Continuing from the above embodiment, the above-mentioned "determining the aligned virtual part according to the updated transformation matrix of the virtual part" can be achieved in the following way: the initial coordinates of each point of the virtual part in the virtual scene are determined as the product of the rotation matrix and the transformation matrix of the virtual part as the target coordinates of each point of the virtual part in the virtual scene, and each point of the virtual part in the virtual scene is adjusted from the initial coordinates to the target coordinates to obtain the aligned virtual part.

[0097] In step 1024, the joints of the initial virtual object are adjusted from the initial positions to the target positions to obtain adjusted joints.

[0098] It should be noted that the embodiment of the present application does not limit the adjustment process, and the adjustment can be automatic adjustment, manual adjustment, etc.

[0099] In step 1025, the adjusted joints and the aligned virtual parts are combined to obtain a virtual object corresponding to the target object.

[0100] Through the embodiments of the present application, virtual objects in virtual scenes are constructed based on joint information, which effectively realizes accurate mapping of virtual environments and the real world. By acquiring the joint positions of target objects and performing position mapping, objects in real scenes are accurately converted to virtual scenes, ensuring the consistency of the images of virtual objects and target objects. This not only optimizes the gait simulation effect of virtual objects, but also enhances the interactivity and immersion of virtual reality systems.

[0101] In step 103, based on the running speed of the virtual object in the gait cycle, the arm joints of the virtual object are simulated to obtain the angle rotation curve of the arm joints of the virtual object in the gait cycle.

[0102] The angle rotation curve represents the rotation angle of the arm joint corresponding to each time point in the gait cycle.

[0103] Here, the running speed of the virtual object in the gait cycle is a preset running speed, which does not exceed the maximum running speed of the target object in the real scene. The gait cycle refers to the complete process experienced by a person when walking, starting from the heel of one leg touching the ground to the heel of the same leg touching the ground again. This process can be subdivided into the following stages: initial contact stage, weight-bearing stage, single-leg support stage and swing stage. The initial contact stage is the beginning stage of the gait cycle, which is used to represent the first contact of the walker's foot to the ground; the weight-bearing stage is used to represent the stage from initial contact to the opposite foot leaving the ground, at which time the weight of the body is transferred from one foot to the other foot; the single-leg support stage is used to represent the node when the opposite foot leaves the ground and the body is fully supported by one foot. This stage includes the action of the heel leaving the ground and the toes pushing the body forward; the swing stage is used to represent the stage from the toe leaving the ground to the foot touching the ground again. This stage includes three stages: acceleration, neutralization and deceleration. The walker's legs swing forward and prepare for the next contact with the ground. The gait cycle is used to study the movement patterns of the human body when walking in order to evaluate and treat gait abnormalities. The duration of the gait cycle is usually expressed as a percentage, with a complete gait cycle being 100%. For example, the initial contact phase occupies 0% to 2% of the gait cycle, while the swing phase occupies 60% to 70% of the gait cycle. The embodiment of the present application does not limit the arm joints of the virtual object. The arm joints include at least one of the following joints: a shoulder joint and an elbow joint; the rotation angle of the arm joint includes the rotation angle of the arm joint in the X-axis, Y-axis and Z-axis directions, and the arm joint realizes the action corresponding to the rotation angle by executing the rotation angle in different directions.

[0104] In some embodiments, before the above-mentioned step of "performing running simulation on the arm joints of the virtual object", the running speed of the virtual object in the gait cycle is obtained in the following manner: the running speeds of M target objects in the gait cycle are obtained, and the M running speeds are weighted averaged to obtain the running speed of the virtual object in the gait cycle, where M>1, and M is used to characterize the number of target objects.

[0105] For example, given three target objects (M=3), their running speeds in the gait cycle are 1.2 m / s, 1.5 m / s, and 1.8 m / s, respectively. Weights (such as 0.4, 0.3, and 0.3) are assigned to these running speeds respectively, and the three running speeds are weighted averaged to obtain the running speed of the virtual object in the gait cycle of 1.47 m / s.

[0106] In some embodiments, see Figure 3C , Figure 3C is a third flow chart of the information processing method provided in the embodiment of the present application, for Figure 3A Step 103 shown can be performed by Figure 3CSteps 1031 to 1036 are implemented as described in detail below.

[0107] In step 1031, based on the running speed of the virtual object in the gait cycle, an objective function for controlling the movement of the virtual object is constructed.

[0108] Here, the running speed of the virtual object in the gait cycle is the speed at which the virtual object as a whole walks in the virtual scene.

[0109] In some embodiments, step 1031 can be implemented in the following manner: perform the following processing for each time point in the preset time points of the gait cycle: determine the angular velocity of the joints of the virtual object at the time point based on the running speed of the virtual object in the gait cycle; determine the motion acceleration of the joints of the virtual object at the time point based on the angular velocity of the joints of the virtual object at the time point; and determine the sum of the squares of the motion accelerations corresponding to the preset time points as the objective function for controlling the movement of the virtual object.

[0110] Here, angular velocity describes the angle of rotation of the joint in unit time. The ratio of the rotation angle to time is determined as the angular velocity. The angular velocity of the joint is also used to determine the linear velocity of the joint (that is, the distance the joint moves along the motion trajectory in unit time). The product of the distance from the joint to the rotation axis (radius) and the angular velocity of the joint is determined as the linear velocity of the joint. The linear velocity of the joint is used to determine the distance the joint moves along the motion trajectory. Taking the shoulder joint and elbow joint of the arm as an example, the elbow joint follows the rotation of the shoulder joint, and its position changes accordingly. Therefore, the distance between the shoulder joint and the elbow joint is the distance between the joint and the rotation axis.

[0111] It should be noted that the rotation axis is the reference axis that describes the straight line around which an object rotates in rotational motion. In three-dimensional space, any rotation can be described by a rotation axis around which the object rotates.

[0112] In some embodiments, the above-mentioned "determining the angular velocity of the joints of the virtual object at a point in time based on the running speed of the virtual object in the gait cycle" can be achieved in the following way: obtaining the rotation angle of the joints of the virtual object at the time point, and determining the ratio of the rotation angle to the unit time as the first angular velocity of the joints of the virtual object at the time point, wherein the unit time is the time between two adjacent time points; decomposing the running speed of the virtual object in the gait cycle to obtain a decomposed linear velocity consistent with the vertical direction of the rotation axis of the joint, determining the ratio of the decomposed linear velocity to the length of the rotation axis as the second angular velocity of the joints of the virtual object at the time point, and fusing the first angular velocity and the second angular velocity of the joints of the virtual object at the time point to obtain the angular velocity of the joints of the virtual object at the time point.

[0113] It should be noted that the rotation angle of the joint of the virtual object at a time point is an unknown value to be solved, and the angular velocity of the joint of the virtual object at a time point can be determined according to the rotation angle of the joint of the virtual object at a time point.

[0114] For example, the running speed of the virtual object in the gait cycle (the running speed in the horizontal direction is 1 m / s) is decomposed to obtain a decomposed linear velocity (e.g., 0.8 m / s) that is consistent with the vertical direction of the rotation axis of the joint at the time point, and the ratio of the decomposed linear velocity to the length of the rotation axis (e.g., 0.5 m) (e.g., 1.6 m / s) is determined as the second angular velocity of the joint of the virtual object at the time point, and the first angular velocity (e.g., 0.8 m / s) and the second angular velocity of the joint of the virtual object at the time point are fused to obtain the angular velocity (e.g., 2.4 m / s) of the joint of the virtual object at the time point.

[0115] In some embodiments, the above-mentioned "determining the movement acceleration of the joints of the virtual object at a point in time based on the angular velocity of the joints of the virtual object at that point in time" can be achieved in the following way: by calculating the ratio of the change in the angular velocity of the joints between adjacent time points to the change in time, the movement acceleration of the joints of the virtual object can be approximated.

[0116] For example, given two adjacent time points t1 and t2, their corresponding angular velocities are ω1 and ω2 respectively, the difference ω3 between ω1 and ω2 is calculated, the difference t3 between t1 and t2 is calculated, and the ratio between ω3 and t3 is determined as the motion acceleration of the joint of the virtual object at the time point t2.

[0117] Through the embodiments of the present application, the present invention successfully realizes the accurate construction of the arm joint angle rotation curve in the gait cycle by simulating the operation of the virtual object's arm joint, thereby improving the authenticity and naturalness of the virtual object's gait simulation. By constructing the target function through the running speed of the virtual object in the gait cycle, the kinematic characteristics of the human body are effectively simulated, making the movement of the virtual object closer to the target object in the real scene.

[0118] In step 1032, the constraint conditions of the joints of the virtual object are obtained.

[0119] Here, the constraints of the joints of the virtual object are used to limit the rotation angle of the joints of the virtual object, for example, the rotation angle of the joints of the virtual object does not exceed 120 degrees; the target solution is calculated by the objective function on multiple solutions in the calculation process, and the solution corresponding to the minimum value among the multiple values ​​is used to characterize the low work done by the virtual object when it moves in the gait cycle, wherein the work done by the virtual object when it moves in the gait cycle is the product of the force and the distance over which the force acts, the force is proportional to the motion acceleration of the joints of the virtual object, and the distance over which the force acts is proportional to the motion acceleration of the joints of the virtual object. Therefore, the solution obtained by calculating the solution through the objective function is also proportional to the motion acceleration of the joints of the virtual object.

[0120] In some embodiments, the above Figure 3C Obtaining the constraints of the joints of the virtual object in step 1032 shown can be achieved in at least one of the following ways: based on the joint information, querying the condition table, determining the candidate conditions corresponding to the joint information as the constraints of the joints of the virtual object, and the condition table is used to characterize the correspondence between different candidate joint information and different candidate conditions; or, encoding the joint information to obtain the encoded features, and decoding the encoded features to obtain the constraints of the joints of the virtual object.

[0121] It should be noted that the embodiments of the present application do not impose any restrictions on the correspondence between different candidate conditions and different candidate joint information. The correspondence can be that multiple candidate conditions correspond to one candidate joint information, or that each candidate condition corresponds to its own candidate joint information. The condition table is a data structure used to store the correspondence between candidate conditions and candidate joint information. The embodiments of the present application do not impose any restrictions on the condition table. The condition table can be a data structure implemented as a hash table, a data structure implemented as an array, etc., which is used to quickly perform search, insertion and deletion operations on the correspondence between candidate conditions and candidate joint information, wherein the hash table uses a hash function to calculate an index value, and maps the input joint information to a position in the hash table to access the candidate conditions.

[0122] For example, determine candidate condition A of candidate joint information A, determine candidate condition B of candidate joint information B, and determine the queried candidate condition (such as candidate condition A) corresponding to the joint information (such as candidate joint information A) as the constraint condition of the joint of the virtual object.

[0123] In some embodiments, the above-mentioned "encoding the joint information to obtain the encoding features corresponding to the joint information" can be achieved in the following ways: performing convolution processing on the joint information to obtain convolution features, performing pooling processing on the convolution features to obtain pooling features, and performing mapping processing on the pooling features to obtain encoding features.

[0124] For example, the joint information (such as "[0.8, 1.7, 2.2]", where 0.8 is used to represent the position of the joint in the X-axis direction, 1.7 is used to represent the position of the joint in the Y-axis direction, and 2.2 is used to represent the position of the joint in the Z-axis direction) is convolved to obtain convolution features (such as [0.14, 0.13, -0.06, 0.15, 0.03, 0.06]), the convolution features are pooled to obtain pooled features (such as [0.14, 0.13, -0.06]), and the pooled features are mapped to obtain encoded features (such as [0.52, 0.13, -0.06, 0.15]).

[0125] In some embodiments, the above-mentioned "decoding the encoded features to obtain {decoded data} corresponding to the encoded features" can be achieved in the following ways: mapping the encoded features to obtain mapping features, upsampling the mapping features to obtain upsampled features, and deconvolving the upsampled features to obtain joint constraints.

[0126] For example, the encoded features (such as [0.52, 0.13, -0.06, 0.15]) are mapped to obtain mapped features (such as [0.14, 0.13, -0.06]), the mapped features are upsampled to obtain upsampled features (such as [0.14, 0.13, -0.06, 0.15, 0.03, 0.06]), the upsampled features are deconvolved to obtain joint constraints (such as "the rotation angle of the joint does not exceed 120 degrees").

[0127] In some embodiments, the constraints of the joints of the virtual object can also be obtained in the following manner: determine the similarity between the joint information and the known joint information, wherein the known joint information is used to construct a known virtual object in the virtual scene; when the similarity is greater than a similarity threshold, determine the constraints of the joints of the known virtual object as the constraints of the joints of the virtual object.

[0128] Continuing from the above embodiment, the above-mentioned "determining the known joint information and the similarity between the joint information" can also be achieved in the following way: encode the known joint information (such as "[0.8, 1.7, 2.2]") to obtain a first coding feature (such as [0.14, 0.13, -0.06]); encode the joint information (such as "[0.8, 1.7, 2.3]") to obtain a second coding feature (such as [0.14, 0.13, -0.06, 0.15, 0.03, 0.06]); determine the similarity between the first coding feature and the second coding feature (such as 0.85) as the similarity between the known joint information and the joint information.

[0129] It should be noted that the embodiment of the present application does not limit the similarity, and the similarity can be the cosine similarity, Euclidean distance, etc. between the first coding feature and the second coding feature.

[0130] For example, when the similarity (such as 0.85) is greater than the similarity threshold (such as 0.8), the known joint constraint conditions of the virtual object (such as the rotation angle of the joint is less than 120 degrees) are determined as the joint constraint conditions of the virtual object.

[0131] In some embodiments, the following steps are performed: based on the constraints, the objective function is operated to obtain the objective solution of the objective function that satisfies the constraints. The above steps of "based on the constraints, the objective function is operated to obtain the objective solution of the objective function that satisfies the constraints" can be implemented through steps 1033 to 1035.

[0132] In step 1033, based on the penalty factor, a function is constructed for the constraint condition and the objective function to obtain a penalty function.

[0133] Here, the penalty factor is used to incorporate the constraints into the objective function, so that the constrained optimization problem is transformed into an unconstrained optimization problem, which can be solved using an unconstrained optimization algorithm.

[0134] In some embodiments, step 1033 may be implemented in the following manner: factor transformation is performed on the constraint condition to obtain the constraint factor; and the penalty function is determined as the sum of the product of the constraint factor and the penalty factor and the objective function.

[0135] Continuing from the above embodiment, the constraint condition includes at least one inequality, and the inequality includes a left-side factor formula and a right-side factor formula, which is used to characterize the correspondence between the left-side factor formula and the right-side factor formula. The above "factor transformation of the constraint condition to obtain the constraint factor" can be achieved in the following way: the difference between the left-side factor formula and the right-side factor formula is determined as the constraint factor.

[0136] For example, given a constraint condition (such as an inequality X>4), where X is a left-side factor of the inequality and 4 is a right-side factor of the inequality, the difference between the left-side factor and the right-side factor is determined as the constraint factor (such as X-4).

[0137] In step 1034, an initial solution satisfying the constraint conditions and an initial penalty factor are obtained.

[0138] Here, the initial solution is the rotation angle of the joint at a point in time. The embodiment of the present application does not limit the method of obtaining the initial solution. The initial solution can be the initial value of a randomly generated solution, or the maximum rotation angle of the joint that meets the constraints, etc. The initial penalty factor is a preset penalty factor.

[0139] In some embodiments, obtaining an initial solution that satisfies the constraints can be achieved in the following manner: determining the maximum rotation angle and the minimum rotation angle that satisfy the constraints; obtaining a random number between 0 and 1; calculating the product of the difference between the maximum rotation angle and the minimum rotation angle and the random number, and determining the sum of the product and the minimum rotation angle as the initial solution that satisfies the constraints.

[0140] For example, determine the maximum rotation angle (such as 180) and the minimum rotation angle (such as -100) that meet the constraints; obtain a random number between 0 and 1 (such as 0.5); calculate the product of the difference between the maximum rotation angle and the minimum rotation angle (such as 280) and the random number (such as 140), and determine the sum of the product and the minimum rotation angle (such as 40) as the initial solution that meets the constraints.

[0141] In step 1035, based on the initial solution and the initial penalty factor, the penalty function is iterated to obtain a target solution of the target function that satisfies the constraint conditions.

[0142] Here, an iterative operation is used to determine multiple solutions of the objective function so that the final solution minimizes the objective function that satisfies the constraints.

[0143] In some embodiments, step 1035 can be implemented in the following manner: perform the following operations for the iteration, substitute the solution of the i-1th round and the penalty factor of the i-1th round into the penalty function to obtain the penalty value of the i-th round; determine the product of the penalty factor of the i-1th round and the decrement factor as the penalty factor of the i-th round; determine the product of the penalty factor of the i-th round and the solution of the i-1th round as the solution of the i-th round; when the difference between the penalty value of the i-th round and the penalty value of the i-1th round is less than the penalty threshold, determine the solution of the i-th round as the target solution of the target function that satisfies the constraint conditions, and stop the iteration.

[0144] Among them, i>0, i is a positive integer that increases successively, the solution of the 0th round is the initial solution, and the penalty factor of the 0th round is the initial penalty factor.

[0145] It should be noted that the embodiment of the present application does not limit the decrement factor, and the decrement factor is used to control the frequency of reduction of the penalty factor.

[0146] In some embodiments, the above “substituting the solution of the i-1th round and the penalty factor of the i-1th round into the penalty function to obtain the penalty value of the i-th round” can be implemented in the following way: substituting the solution of the i-1th round and the penalty factor of the i-1th round as independent variables of the penalty function into the penalty function to obtain the penalty value of the i-th round.

[0147] For example, the solution of the i-1th round is that the rotation angle is equal to 60 degrees, and the penalty factor of the i-1th round is equal to 0.2. The ratio of the rotation angle (such as 60) to the unit time (such as 1) is determined as the first angular velocity (such as 60); the running speed of the virtual object in the gait cycle (the running speed in the horizontal direction is 100) is decomposed to obtain a decomposed linear velocity (such as 50) that is consistent with the vertical direction of the rotation axis of the joint at the time point, wherein the decomposed linear velocity is the product of the sine value of the angle between the decomposed linear velocity and the running speed in the horizontal direction and the movement speed, and the decomposed linear velocity is equal to the sine value of the angle between the decomposed linear velocity and the running speed in the horizontal direction. The angle between the running speeds in the horizontal direction is equivalent to the rotation angle; the ratio (such as 100) of the decomposed linear velocity to the length of the rotation axis (such as 0.5) is determined as the second angular velocity, and the first angular velocity and the second angular velocity of the joints of the virtual object at the time point are integrated to obtain the angular velocity of the joints of the virtual object at the time point (such as 200), and the motion acceleration of the joints of the virtual object is approximately obtained by calculating the ratio of the change in the angular velocity of the joints between adjacent time points (such as 20) and the time change (such as 1), and the sum of the squares of the motion acceleration is determined as the value of the objective function (400).

[0148] Continuing with the above example, the constraint factor (such as "60-40=20") is determined according to the rotation angle (such as 60), and the penalty value of the i-th round is determined as the sum of the product of the constraint factor and the penalty factor and the value of the objective function of the i-th round (such as 404).

[0149] Through the embodiments of the present application, by introducing the constraints of the joints and constructing a penalty function, it is ensured that the motion simulation of the arm joints of the virtual object achieves the goal of low work while meeting the actual physiological limitations. By converting the nonlinear programming problem of the joint kinematics limitations into a linear programming problem, and using an iterative algorithm to solve the solution to the minimized objective function, the accuracy and efficiency of the gait simulation are effectively improved, ensuring that the movement of the virtual object during the gait cycle is in line with the laws of biomechanics and can complete the action with minimal energy consumption. Optimizing the coordination relationship between the arm and the gait, ensuring that the arm movement trajectory is synchronized with the gait, and thus achieving a smoother arm movement reproduction, not only helps to improve the realism and naturalness of the simulation technology, but also helps with human motion analysis and rehabilitation training.

[0150] In step 1036 , based on the target solution, the angular rotation curve of the arm joints of the virtual object during the gait cycle is determined.

[0151] In some embodiments, the target solution includes the rotation angle of the arm joint of the virtual object at a preset time point, see Figure 3D , Figure 3D is a fourth flow chart of the information processing method provided in the embodiment of the present application, for Figure 3C Step 1036 shown may be performed by Figure 3D Steps 10361 to 10362 are implemented as described in detail below.

[0152] In step 10361, based on the rotation angle at a preset time point, the rotation angle at other time points is determined.

[0153] Among them, other time points are time points other than the preset time points in the gait cycle.

[0154] Here, the preset time points are time points of which the preset number is M. There is a certain duration between each preset time point, and the duration includes other time points of which the preset number is N except the preset time points in the gait cycle.

[0155] In some embodiments, the preset time point can be determined by calculating the ratio of the number of time points in the gait cycle to the preset number, dividing the time points in the gait cycle into multiple time periods according to the ratio; and taking the first time point of each time period as the preset time point.

[0156] For example, the gait cycle has 1000 time points, and a preset time point is determined every 50 time points, resulting in a preset number of 200 preset time points.

[0157] In some embodiments, step 10361 can be implemented in the following manner: based on the rotation angles of two adjacent preset time points, interpolation calculations are performed on other time points within the two preset time points to obtain the rotation angles of other time points.

[0158] For example, linear interpolation is used as an example to illustrate that between two preset time points, for example, when the first time point is 0 degrees and the second time point is 90 degrees, the rotation angle at any time point in between changes linearly from 0 degrees to 90 degrees.

[0159] In step 10362, curve fitting is performed on the rotation angle at the preset time point and the rotation angle at other time points to obtain the angle rotation curve of the arm joint of the virtual object in the gait cycle.

[0160] In some embodiments, step 10362 can be implemented in the following manner: traverse each time point in the gait cycle, connect the rotation angle corresponding to the time point with the rotation angle of the next time point of the time point through a drawing tool, and obtain the angle rotation curve of the arm joint of the virtual object in the gait cycle, wherein the time point includes preset time points and other time points.

[0161] It should be noted that the drawing tool is used to draw curves. The embodiment of the present application does not limit the drawing tool. The drawing tool can be an online drawing tool (such as the Desmos tool) or an offline drawing tool (such as the line chart drawing function in Excel).

[0162] Through the embodiment of the present application, an angle rotation curve of the virtual object's arm joint in the entire gait cycle is constructed. This curve fitting technology not only makes the gait movement of the virtual object smoother and more continuous, but also enhances the authenticity and naturalness of the virtual reality experience. By performing interpolation calculations between preset time points, it ensures that the joint movement of the virtual object in the entire gait cycle can more accurately reflect the continuity and change rules of human gait.

[0163] In step 104, motion mapping is performed on the angle rotation curve to obtain a motion sequence of the arm joints of the target object.

[0164] The action sequence includes the action of the target object at each time point in the gait cycle, and each action is obtained by the arm joint of the target object performing a corresponding rotation angle.

[0165] In some embodiments, see Figure 3E , Figure 3E is a fifth flow chart of the information processing method provided in the embodiment of the present application, for Figure 3A Step 104 shown can be performed by Figure 3E Steps 1041 to 1043 are implemented as described in detail below.

[0166] Steps 1041 to 1043 are performed for each time point in the gait cycle.

[0167] In step 1041, the angle rotation curve is queried based on the time point to obtain the rotation angle corresponding to the time point.

[0168] It should be noted that the angle rotation curve represents the rotation angle of the arm joint corresponding to each time point in the gait cycle, and each time point corresponds to a unique rotation angle of the arm joint.

[0169] In step 1042, the arm joints of the target object are controlled to rotate at a corresponding angle at the execution time point to obtain the action corresponding to the time point.

[0170] In some embodiments, step 1042 can be implemented by controlling the target object to change the magnitude and direction of the torque through the driver, controlling the rotation axis to rotate according to the rotation angle corresponding to the time point, and obtaining the action corresponding to the time point.

[0171] For example, at time t=0 seconds, the action is to adjust the rotation axis angle of the robot arm to 0 degrees, at time t=1 second, the action is to adjust the rotation axis angle of the robot arm to 90 degrees, and at time t=2 seconds, the action is to adjust the rotation axis angle of the robot arm to 0 degrees.

[0172] In step 1043, the actions corresponding to each time point are combined to obtain the action sequence of the arm joints of the target object.

[0173] For example, given the action corresponding to each time point (such as adjusting the rotation axis angle of the robotic arm to 0 degrees at time t=0 seconds and adjusting the rotation axis angle of the robotic arm to 90 degrees at time t=1 second), the actions corresponding to each time point are combined to obtain the action sequence of the arm joints of the target object (such as [adjusting the rotation axis angle of the robotic arm to 0 degrees at time t=0 seconds and adjusting the rotation axis angle of the robotic arm to 90 degrees at time t=1 second]).

[0174] In some embodiments, after step 104, the arm joints of the target object are controlled to perform each action in the action sequence.

[0175] In some embodiments, the above-mentioned "controlling the arm joints of the target object to perform each action in the action sequence" can be achieved in the following way: converting the actions in the action sequence into control signals, and driving the arm joints of the target object to move according to each action in the action sequence according to the control signals.

[0176] It should be noted that the control signal appears in the form of pulses and can be used to control the position or speed of the stepper motor, where the frequency of the pulse can represent the speed and the number of pulses can represent the distance moved. The converted control signal can be a pulse width modulation (PWM) signal of the motor or a control pressure of the hydraulic system. In the target object (robot), the control signal drives the motor, hydraulic or pneumatic actuator, so that the arm joints of the robot move according to the actions in the action sequence.

[0177] Through the embodiments of the present application, by introducing the control planning of arm swinging in the gait cycle of the target object, the balance, stability and bionics are significantly improved, the posture deviation caused by the lack of coordinated arm movement is reduced, and the overall walking quality is improved. At the same time, the coordinated movement of the arm and gait enhances the bionic effect of the target object, making its movement closer to the natural walking pattern of humans, improving the performance of the target object in human-computer interaction, and making it more suitable for interactive tasks in actual application scenarios.

[0178] The following is an explanation of an exemplary application of the information processing method provided in an embodiment of the present application in a practical application scenario.

[0179] In related technologies, since the movement of the arms increases the complexity of control and may cause stability problems, although the robot arm has a high degree of freedom, the control strategy is usually simplified. Especially during the walking process of the robot, the movement of the arms is often ignored, and the moving part of the arms is not integrated into the walking process, which reduces the stability and coordination of the overall movement.

[0180] In order to solve the above problems, an embodiment of the present application proposes an information processing method, which collects information of each segment of the bionic robot's body through a motion capture system, and scales the general model in an open source platform to obtain a personalized model that conforms to the specific bionic robot. Subsequently, the joint state of the personalized model is optimized to generate the arm motion trajectory under a stable walking gait, and the simulated arm motion trajectory is mapped to the bionic robot to ensure the coordination of arm movement and gait, realize the reproduction of the robot's arm movements in stable walking, and improve the balance, stability and bionics of the bionic robot when walking.

[0181] Take the processing of joint information of a bionic robot as an example, see Figure 4 , Figure 4 This is a flow chart for processing the joint information of the bionic robot provided in an embodiment of the present application. The following explains the process of processing the joint information of the bionic robot provided in an embodiment of the present application.

[0182] In step 401, a real bionic robot is obtained.

[0183] Here, the environment in which the bionic robot is located is a real scene.

[0184] In step 402, a personalized model is constructed.

[0185] Here, the body segment information of the bionic robot is recorded by a motion capture system (such as Qualisys). The recorded body segment information is the length of the bones of the main parts of the body and the position of each rotation axis when the arms, hips, and legs move. The specific process is as follows: 27 markers are pasted on the whole body of the bionic robot (i.e., the target object), and then the system software is used to record and save the static position information of the bionic robot. The static position here refers to the position information of the markers pasted on various parts of the body when the robot hangs down with its hands and stands still. According to the position of each marker, the length of the body bones of the general model in the open source software (such as OpenSim) is adjusted to achieve model personalization. The markers are used to reflect infrared light to determine the static position of the bionic robot based on the reflected infrared light. The bionic robot model (i.e., the initial virtual object) is imported into the open source software, and the scaling function in the open source software is used to scale the general model to generate a personalized model of the bionic robot. The general model is an adult male with a height of 170 cm. Since the height and bone length of each person are different, the size of the model is matched with the size of the robot by pasting markers on the robot and then scaling the general model in the open source software. See Figure 5 , Figure 5 is a schematic diagram of a personalized model provided in an embodiment of the present application, Figure 5 It includes a joint 501 of a personalized model, a skeleton 502 of a personalized model, and a muscle 503 of a personalized model.

[0186] In step 403, a personalized forward prediction simulation is performed.

[0187] Here, a personalized forward prediction simulation platform is built to forward predict the motion process of the personalized model (i.e., virtual object). The forward prediction simulation of the bionic robot is described as an optimal control problem, using muscle activation dynamics, musculoskeletal geometry, and multi-rigid body dynamics to describe the human musculoskeletal structure.

[0188] By setting the objective function, constraints and optimization solver, muscle control and joint state are optimized to generate a stable gait. Since the true objective function of human motion is unknown, energy consumption, gait stability, muscle performance and other aspects are considered. The predicted walking process of a person is to use muscle coordination to maintain body stability and smooth movements, and minimize energy consumption. The objective function is constructed as shown in Formula 1.

[0189]

[0190] Among them, w i is the weight coefficient, d is the distance the pelvis moves in the sagittal axis direction, t f is half the time of the gait cycle, E is the metabolic energy consumption rate, a is the muscle activation amount, u is the joint acceleration, T Pis the passive joint torque and e arms It is the arm drive excitation.

[0191] The forward prediction simulation platform describes the motion control process of the personalized model as an optimal control problem. The optimization method is a method of finding the extreme value, which makes the objective function of the system reach the extreme value under the constraint conditions. The constraint conditions are shown in Formula 2-4.

[0192]

[0193] u(t+1)=c*x(t+1) (3)

[0194] x min ≤x(t)≤x max

[0195] u min ≤u(t)≤u max

[0196] p min ≤p≤p max (4)

[0197] Among them, f(.) is the dynamic constraint of the system, which is used to predict the control quantity at the next time point through the control quantity (such as the running speed of the joint) and the state quantity (such as the running acceleration of the joint) at the previous time point. x is the control quantity, u is the state quantity, c is the decreasing factor, which is used to control the decreasing speed of the state quantity, and p is the parameter of the objective function. These parameters are usually fixed or constants that vary within a certain range and are used to characterize the physical characteristics or behavioral characteristics of the objective function.

[0198] In step 404, a motion trajectory is simulated.

[0199] Here, the joint space limit is set according to the actual motion range of the bionic robot. Specifically, according to the motion range of each servo of the bionic robot, the swing range of each joint of the model is set in the simulation as shown in Table 1.

[0200] Table 1 Simulation joint swing range

[0201]

[0202] Table 1 limits the swing range of each joint of the model, and then inputs the set walking speed, which can range from [0.5-2.3]. The direct point matching method and the interior point method are introduced to perform forward prediction simulation. Figure 6 , Figure 6 This is a forward prediction simulation flow chart provided in an embodiment of the present application. The process of the forward prediction simulation provided in an embodiment of the present application is explained below.

[0203] In step 601, the optimal control problem is solved.

[0204] Here, the musculoskeletal system is a rigid system, and a small change in muscle excitation can have a great impact on the predicted movement pattern and objective function. In order to avoid such an impact, the direct point matching method is introduced. By reducing the time range of integration and reducing the sensitivity of the objective function to the optimization variable, the optimal control problem is transformed into a series of nonlinear programming problems (NLP). The third-order ordinary differential equation solution method is selected, and the conversion process is implemented through an open source optimization program (such as CasADi). In the simulation, 50 configuration points are set for each gait cycle.

[0205] In step 602, a nonlinear programming problem is solved.

[0206] Here, the interior point method is used to solve the problem. This process uses a nonlinear programming solver (such as IPOPT) to obtain the solution to the problem, that is, the motion trajectory of each joint of the whole body in half a gait cycle.

[0207] A bilaterally symmetrical strategy is adopted to determine the motion trajectory of all joints in the entire gait cycle based on the motion trajectory of all joints in the half gait cycle obtained. Specifically, based on the half gait cycle obtained, the other half gait cycle is completed in a completely symmetrical manner. The motion trajectory is the flexion angle curve obtained by connecting the flexion angles corresponding to each time point in the gait cycle. Figure 6 , Figure 7 It is a schematic diagram of the motion trajectory of the shoulder joint in a gait cycle provided by an embodiment of the present application. The motion trajectory 701 of the shoulder joint is a flexion angle curve obtained by connecting the flexion angles of the shoulder joint corresponding to each time point of the gait cycle, and the walking speed is set to 0.8m / s.

[0208] In step 405, the motion trajectory of the bionic robot arm is determined.

[0209] Here, the trajectory output by the arm in the simulation includes the angles of the shoulder joint in the X, Y, and Z directions and the elbow joint in the Y direction at different time points. Since the bionic robot arm has a total of 6 degrees of freedom, the two degrees of freedom of the wrist joint during walking are set to 0 degrees. The joint mapping of the remaining 4 degrees of freedom from the simulated arm to the real arm is shown in Table 2.

[0210] Table 2 Joint mapping relationship table from simulated arm to real machine arm

[0211] Simulation joints Real machine joints Simulate the freedom of the shoulder joint in the X direction Real machine shoulder joint X-direction degrees of freedom Simulate the freedom of the shoulder joint in the Y direction Real machine shoulder joint Y-direction degree of freedom Simulate the freedom of the shoulder joint in the Z direction Real machine shoulder joint Z-direction degree of freedom Simulate the degree of freedom of the elbow joint in the Y direction Real machine elbow joint Y-direction degree of freedom 0 Real machine wrist joint X-direction degrees of freedom 0 Real machine wrist joint Z-direction degree of freedom

[0212] Among them, the two degrees of freedom of the wrist are the degrees of freedom in the real machine, but they are not optimized in the simulation. According to the swinging state of the arm when a person walks, these two degrees of freedom are simplified and set to 0.

[0213] Import the curve output by the simulation platform, and according to the corresponding relationship in Table 1, complete the mapping conversion of the angle of each joint of the arm to obtain the angle of the arm joint of the real bionic robot.

[0214] In step 406 , the bionic robot performs arm movements.

[0215] Here, according to the set running distance and speed of the bionic robot, the total running time of the bionic robot is obtained, and the motor rotation angle of the arm movement at the corresponding time is output. Specifically, the software (such as Maya software) can output the motor rotation angle of each joint at the corresponding time (the set frequency is 100Hz). The frequency is 100Hz, and the motor rotation angle corresponding to 100 time points of each joint in the running time is output. See Figure 8 , Figure 8 is a schematic diagram of software processing provided by an embodiment of the present application, such as Figure 8 As shown, each frame action of the left forearm node 802 of the personalized model 801 is edited according to the frame rate, that is, the flexion angle corresponding to each frame on the edited flexion angle curve 803 corresponding to rotation X, and the left forearm node 802 can rotate in three directions, namely rotation X, rotation Y and rotation Z. After obtaining the flexion angle curve of each joint, the flexion angle curve is smoothed. The flexion angle curve of each joint after smoothing can be derived from the system to generate an action table for each joint. In chronological order, an action table (i.e., an action sequence) is generated according to the motor rotation angle of each joint of the arm at the corresponding time, wherein the action table of the joint corresponding to the motor URL number "28:1266" is shown in Table 3.

[0216] Table 3 Action table of joints corresponding to motor address number "28:1266"

[0217] Motor rotation arc time -1.66 0.00 -1.66 0.01 -1.65 0.02 …… …… -1.53 0.39 -1.52 0.4

[0218] Before completing the corresponding arm movements in the action table on the real bionic robot, reset the rotation angles of each joint on the real bionic robot. Fig. 9 , Fig. 9 It is a schematic diagram of joint resetting provided in an embodiment of the present application, which adjusts the joint 902 of the bionic robot 901 to a preset initial position.

[0219] By reading the action table corresponding to each motor, the bionic robot can move according to the simulation-optimized trajectory and complete the corresponding arm movements (i.e., actions) in the action table on the real bionic robot. Each action in the action table is the motor rotation angle of each joint of the arm at the corresponding time. The rotation angle is sent to the corresponding joint of the arm to realize the rotation of the arm.

[0220] In summary, the embodiments of the present application introduce arm swing control planning during the walking process of the bionic robot, and optimize the coordination relationship between the arm and gait through the forward prediction simulation platform, thereby ensuring that the arm movement trajectory is synchronized with the gait, thereby achieving smoother arm movement reproduction. In addition, the bionic robot can significantly improve its balance and stability when walking, reduce posture deviations caused by lack of coordinated arm movement, and improve the overall walking quality. At the same time, the coordinated movement of the arm and gait enhances the bionic effect of the bionic robot, making its movement closer to the natural walking pattern of humans, and improving the performance of the bionic robot in human-computer interaction, making it more suitable for interactive tasks in actual application scenarios. The method provided in the embodiments of the present application can improve the balance, stability and bionics of the bionic robot when walking.

[0221] The following is a description of an exemplary structure of the information processing device 555 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the information processing device 555 of the memory 550 may include:

[0222] The data acquisition module 5551 is used to obtain the joint information of the joints of the target object.

[0223] The object construction module 5552 is used to construct a virtual object corresponding to the target object in the virtual scene based on the joint information.

[0224] The motion simulation module 5553 is used to perform motion simulation on the arm joints of the virtual object based on the running speed of the virtual object in the gait cycle, and obtain the angle rotation curve of the arm joints of the virtual object in the gait cycle, wherein the angle rotation curve represents the rotation angle of the arm joint corresponding to each time point in the gait cycle.

[0225] The action mapping module 5554 is used to perform action mapping on the angle rotation curve to obtain the action sequence of the arm joints of the target object, wherein the action sequence includes the action of the target object at each time point in the gait cycle, and each action is obtained by the arm joints of the target object executing the corresponding rotation angle.

[0226] In some embodiments, the object construction module 5552 is also used to obtain the initial virtual object in the virtual scene and the virtual parts connected to the joints in the initial virtual object, and obtain the real scene in which the target object is located; based on the position mapping relationship between the real scene and the virtual scene, the positions of the joints of the target object are mapped to obtain the target positions of the joints of the initial virtual object; based on the target position and the initial position, the virtual parts are aligned to obtain the aligned virtual parts; the joints of the initial virtual object are adjusted from the initial positions to the target positions to obtain the adjusted joints; the adjusted joints and the aligned virtual parts are combined to obtain the virtual object corresponding to the target object.

[0227] In some embodiments, the motion simulation module 5553 is also used to construct an objective function for controlling the movement of the virtual object based on the running speed of the virtual object in the gait cycle; obtain the constraints of the joints of the virtual object; based on the penalty factor, construct a function of the constraints and the objective function to obtain the penalty function; obtain an initial solution and an initial penalty factor that satisfy the constraints; based on the initial solution and the initial penalty factor, iterate the penalty function to obtain a target solution of the objective function that satisfies the constraints; based on the target solution, determine the angle rotation curve of the arm joints of the virtual object in the gait cycle.

[0228] In some embodiments, the motion simulation module 5553 is also used to perform the following processing for each time point in the preset time points of the gait cycle: based on the running speed of the virtual object in the gait cycle, determine the angular velocity of the node of the virtual object at the time point; based on the angular velocity of the node of the virtual object at the time point, determine the motion acceleration of the joint of the virtual object at the time point; and determine the sum of the squares of the motion accelerations corresponding to the preset time points as the objective function for controlling the movement of the virtual object.

[0229] In some embodiments, the motion simulation module 5553 is also used to determine the constraints of the joints of the virtual object in at least one of the following ways: based on the joint information, querying the condition table, determining the candidate conditions corresponding to the joint information as the constraints of the joints of the virtual object, the condition table is used to characterize the correspondence between different candidate joint information and different candidate conditions; encoding the joint information to obtain the encoded features, and decoding the encoded features to obtain the constraints of the joints of the virtual object.

[0230] In some embodiments, the motion simulation module 5553 is also used to perform the following operations for iteration: substitute the solution of the i-1th round and the penalty factor of the i-1th round into the penalty function to obtain the penalty value of the i-th round; determine the product of the penalty factor of the i-1th round and the decrement factor as the penalty factor of the i-th round; determine the product of the penalty factor of the i-th round and the solution of the i-1th round as the solution of the i-th round; when the difference between the penalty value of the i-th round and the penalty value of the i-1th round is less than the penalty threshold, determine the solution of the i-th round as the target solution of the target function that satisfies the constraint conditions, and stop the iteration, wherein i is a successively increasing positive integer, the solution of the 0th round is the initial solution, and the penalty factor of the 0th round is the initial penalty factor.

[0231] In some embodiments, the motion simulation module 5553 is also used to determine the rotation angles at other time points based on the rotation angle at the preset time point, wherein the other time points are time points other than the preset time points in the gait cycle, and the target solution includes the rotation angles of the arm joints of the virtual object at the preset time points; curve fitting is performed on the rotation angles at the preset time points and the rotation angles at other time points to obtain the angle rotation curve of the arm joints of the virtual object in the gait cycle.

[0232] In some embodiments, the action mapping module 5554 is also used to perform the following processing for each time point within the gait cycle: query the angle rotation curve based on the time point to obtain the rotation angle corresponding to the time point; control the arm joints of the target object to execute the rotation angle corresponding to the time point to obtain the action corresponding to the time point; combine the actions corresponding to each time point to obtain the action sequence of the arm joints of the target object.

[0233] In some embodiments, the motion mapping module 5554 is also used to control the arm joints of the target object to perform each motion in the motion sequence.

[0234] An embodiment of the present application provides a computer program product, which includes computer executable instructions. The computer executable instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer executable instructions from the computer-readable storage medium, and the processor executes the computer executable instructions, so that the electronic device executes the above-mentioned information processing method of the embodiment of the present application.

[0235] The present application embodiment provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will be caused to execute the information processing method provided by the present application embodiment, for example, FIG. 3A to FIG. 3E The information processing method shown.

[0236] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0237] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0238] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0239] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0240] To summarize, based on the joint information of the target object's joints, a virtual object corresponding to the target object in a virtual scene is constructed. In this way, by constructing the virtual object corresponding to the target object, the target object can be directly controlled to perform actions according to the joint rotation angle of the virtual object obtained by running simulation, thereby achieving consistency between the target object and the virtual object. Based on the running speed of the virtual object in the gait cycle, the arm joints of the virtual object are run simulated to obtain the angular rotation curve of the arm joints of the virtual object in the gait cycle, and the angle rotation curve is action mapped to obtain the action sequence of the arm joints of the target object. In this way, the coordination of the arm movement and the gait can be ensured, and the action reproduction of the arm joints of the target object in the gait cycle can be achieved.

[0241] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An information processing method, characterized in that: The method comprises: Get the joint information of the joints of the target object; Based on the joint information, construct a virtual object corresponding to the target object in a virtual scene; Based on the running speed of the virtual object in the gait cycle, the arm joint of the virtual object is simulated to obtain an angle rotation curve of the arm joint of the virtual object in the gait cycle, wherein the angle rotation curve represents the rotation angle of the arm joint corresponding to each time point in the gait cycle; Performing motion mapping on the angle rotation curve, obtaining a motion sequence of the arm joints of the target object, wherein the motion sequence includes the motion of the target object at each time point in the gait cycle, and each motion is obtained by the arm joints of the target object executing the corresponding rotation angle.

2. The method according to claim 1, characterized in that The joint information includes positions of joints of the target object, and constructing a virtual object corresponding to the target object in a virtual scene based on the joint information includes: Acquire an initial virtual object in the virtual scene and a virtual part of the initial virtual object connected to the joint, and acquire a real scene where the target object is located; Based on the position mapping relationship between the real scene and the virtual scene, position mapping is performed on the positions of the joints of the target object to obtain the target positions of the joints of the initial virtual object; Based on the target position and the initial position, the virtual part is aligned to obtain the aligned virtual part; Adjusting the joint of the initial virtual object from the initial position to the target position to obtain the adjusted joint; The adjusted joints and the aligned virtual parts are combined to obtain a virtual object corresponding to the target object.

3. The method according to claim 1, characterized in that The step of performing operation simulation on the arm joints of the virtual object based on the operation speed of the virtual object in the gait cycle to obtain an angle rotation curve of the arm joints of the virtual object in the gait cycle includes: Based on the running speed of the virtual object in the gait cycle, construct an objective function for controlling the movement of the virtual object; Obtaining constraint conditions of joints of the virtual object; Based on the penalty factor, constructing a function for the constraint condition and the objective function to obtain a penalty function; Obtaining an initial solution and an initial penalty factor that satisfy the constraint conditions; Based on the initial solution and the initial penalty factor, iteratively calculate the penalty function to obtain a target solution of the objective function that satisfies the constraint condition; Based on the target solution, an angular rotation curve of the arm joint of the virtual object during the gait cycle is determined.

4. The method according to claim 3, characterized in that The constructing of an objective function for controlling the movement of the virtual object based on the running speed of the virtual object in the gait cycle includes: The following processing is performed for each of the preset time points in the gait cycle: Determining the angular velocity of the node of the virtual object at the time point based on the running speed of the virtual object in the gait cycle; Determining the motion acceleration of the joint of the virtual object at the time point based on the angular velocity of the node of the virtual object at the time point; The sum of the squares of the motion accelerations corresponding to the preset time points is determined as the objective function for controlling the movement of the virtual object.

5. The method according to claim 3, characterized in that: The obtaining of the constraint conditions of the joints of the virtual object comprises: The constraint conditions of the joints of the virtual object are determined by at least one of the following methods: Based on the joint information, query a condition table, and determine the queried candidate conditions corresponding to the joint information as the constraint conditions of the joints of the virtual object, wherein the condition table is used to characterize the correspondence between different candidate joint information and different candidate conditions; The joint information is encoded to obtain encoded features, and the encoded features are decoded to obtain constraint conditions of the joints of the virtual object.

6. The method according to claim 3, characterized in that The iterative operation of the penalty function based on the initial solution and the initial penalty factor to obtain a target solution of the objective function that satisfies the constraint condition includes: The following operations are performed on the iterations: Substitute the solution of the i-1th round and the penalty factor of the i-1th round into the penalty function to obtain the penalty value of the i-th round; The product of the penalty factor of the i-1th round and the decrement factor is determined as the penalty factor of the i-th round; The solution of round i is determined as the product of the penalty factor of round i and the solution of round i-1; When the difference between the penalty value of the i-th round and the penalty value of the i-1-th round is less than the penalty threshold, the solution of the i-th round is determined as the target solution of the objective function that satisfies the constraint condition, and the iteration is stopped, wherein i>0, i is a successively increasing positive integer, the solution of the 0th round is the initial solution, and the penalty factor of the 0th round is the initial penalty factor.

7. The method according to claim 3, characterized in that The target solution includes the rotation angle of the arm joint of the virtual object at a preset time point, and determining the angle rotation curve of the arm joint of the virtual object in the gait cycle based on the target solution includes: Based on the rotation angle at the preset time point, determining the rotation angle at other time points, wherein the other time points are time points other than the preset time points in the gait cycle; Curve fitting is performed on the rotation angle at the preset time point and the rotation angle at the other time points to obtain an angle rotation curve of the arm joint of the virtual object in the gait cycle.

8. The method according to claim 1, characterized in that The step of performing motion mapping on the angle rotation curve to obtain a motion sequence of the arm joint of the target object includes: The following processing is performed for each time point in the gait cycle: Querying the angle rotation curve based on the time point to obtain the rotation angle corresponding to the time point; Controlling the arm joint of the target object to perform the rotation angle corresponding to the time point to obtain the action corresponding to the time point; Combining the actions corresponding to each time point to obtain the action sequence of the arm joints of the target object; After performing motion mapping on the angle rotation curve to obtain the motion sequence of the arm joint of the target object, the method further includes: The arm joints of the target object are controlled to perform each action in the action sequence.

9. An information processing device, characterized in that: The device comprises: A data acquisition module, used to obtain joint information of the joints of the target object; An object construction module, used to construct a virtual object corresponding to the target object in a virtual scene based on the joint information; A motion simulation module, used for performing motion simulation on the arm joints of the virtual object based on the running speed of the virtual object in the gait cycle, and obtaining an angle rotation curve of the arm joints of the virtual object in the gait cycle, wherein the angle rotation curve represents the rotation angle of the arm joint corresponding to each time point in the gait cycle; An action mapping module is used to perform action mapping on the angle rotation curve to obtain an action sequence of the arm joints of the target object, wherein the action sequence includes the action of the target object at each time point in the gait cycle, and each action is obtained by the arm joints of the target object executing the corresponding rotation angle.

10. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions; A processor, used to implement the information processing method according to any one of claims 1 to 8 when executing the computer executable instructions or computer programs stored in the memory.

11. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the information processing method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising computer executable instructions or a computer program, characterized in that: When the computer executable instructions or computer programs are executed by a processor, the information processing method according to any one of claims 1 to 8 is implemented.

Images