A data processing method and related apparatus

Abstract

This application discloses a data processing method and related apparatus, comprising: acquiring abnormal motion data of a target object in a real scene environment; determining a virtual scene environment corresponding to the real scene environment and an object simulation model corresponding to the target object; driving the object simulation model to perform simulated motion in the virtual scene environment based on the abnormal motion data; and, during the process of driving the object simulation model to perform the simulated motion, correcting the simulated motion data generated by the object simulation model during the simulated motion based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundaries of the virtual scene environment, thereby obtaining corrected motion data corresponding to the abnormal motion data. This application, based on the correspondence between the real scene environment and the virtual scene environment, realizes the physical constraint correction of abnormal motion data, improving the usability of the corrected motion data.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method and related apparatus. Background Technology

[0002] With the rapid development of spatial 3D perception, artificial intelligence, and motion control technologies, motion data of target objects collected by motion capture or visual perception devices can be provided to downstream applications for digital twin display, behavior analysis, or security assessment. However, in practical applications, due to sensor occlusion, environmental noise, and rapid motion switching, the collected motion data may have defects such as jumps, jitter, and self-masking. To address these defects, data repair processing is usually required before delivering the motion data to downstream applications.

[0003] Currently, there are two existing repair schemes: (1) Perform differential operation, extreme value comparison and smooth fitting on the motion data of adjacent frame coordinates to identify abnormal data frames in the target motion data. Then, the abnormal data frames are stripped from the target motion data, and the smoothing algorithm is called to refit the target motion data after the abnormal data frames are stripped to obtain the corrected motion data. (2) Construct a three-dimensional mesh model based on the motion data, and divide the three-dimensional mesh model into multiple body regions according to the topological relationship. Detect whether geometric overlap occurs between multiple body regions to identify abnormal overlapping areas in the motion data where self-penetration occurs. Then, refit the areas in the motion data where geometric overlap occurs to obtain the corrected motion data. However, the above two repair schemes may still result in inconsistencies between the repaired motion data and the real scene environment. For example, the corrected motion data may still have physical distortions such as feet penetrating the floor, target objects passing through walls or scaffolding members, and target objects standing suspended in the air, thereby reducing the usability of the corrected motion data in downstream applications such as digital twin display, behavior analysis or security assessment. Summary of the Invention

[0004] In view of the above problems, this application provides a data processing method and related apparatus to reduce the inconsistency between corrected motion data and the real scene environment, thereby solving the problem of physical distortion easily generated by corrected motion data and improving the usability of corrected motion data in downstream applications such as digital twin display, behavior analysis, or security assessment. The specific solution is as follows:

[0005] The first aspect of this application provides a data processing method, including:

[0006] Acquire abnormal motion data of target objects in a real-world environment;

[0007] Determine the virtual scene environment corresponding to the real scene environment and the object simulation model corresponding to the target object;

[0008] The abnormal motion data is used to drive the object simulation model to perform simulated motion in the virtual scene environment;

[0009] During the simulation motion of the object simulation model, the simulation motion data generated by the object simulation model during the simulation motion is corrected based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundary of the virtual scene environment, so as to obtain the corrected motion data corresponding to the abnormal motion data.

[0010] In one possible implementation, the correction of the simulation motion data generated by the object simulation model during the simulation motion, based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundaries of the virtual scene environment, includes:

[0011] The simulation motion data of the object simulation model in the previous frame is obtained. Based on the simulation motion data of the previous frame and the abnormal motion data in the abnormal motion data corresponding to the current frame, the control torque parameters that drive the object simulation model to move from the previous frame to the current frame are determined.

[0012] Using the control torque parameters and the dynamic simulation parameters associated with the virtual scene environment, the object simulation model is simulated and advanced to obtain the simulation motion data of the current frame;

[0013] Based on the simulation motion data of the current frame and the boundary parameters of the physical boundary, the clipping data of the object simulation model between the current frame and the physical boundary is determined, and the blocking force parameters of the physical boundary on the object simulation model are determined based on the clipping data.

[0014] The simulation motion data of the current frame of the object simulation model is corrected using the blocking force parameters and the dynamic simulation parameters associated with the virtual scene environment.

[0015] In one possible implementation, correcting the simulation motion data of the current frame of the object simulation model using the blocking force parameters and the dynamic simulation parameters associated with the virtual scene environment includes:

[0016] Based on the simulated motion data of the current frame and the abnormal motion data in the abnormal motion data that corresponds to the simulated motion data of the current frame, determine the spatial position deviation between the simulated motion data of the current frame and the corresponding abnormal motion data.

[0017] Based on the simulation motion data of the current frame, determine whether the object simulation model has fallen in the virtual scene environment;

[0018] If the spatial position deviation meets the preset conditions and / or the object simulation model falls in the virtual scene environment, then control the object simulation model to recover to the spatial position corresponding to the corresponding abnormal motion data in the virtual scene environment;

[0019] After the object simulation model is restored to the spatial position corresponding to the time in the abnormal motion data in the virtual scene environment, the simulation motion data of the current frame of the object simulation model is corrected using the blocking force parameter and the dynamic simulation parameter associated with the virtual scene environment.

[0020] In one possible implementation, determining the control torque parameters that drive the object simulation model to move from the previous frame to the current frame based on the simulation motion data of the previous frame and the abnormal motion data corresponding to the current frame in the abnormal motion data includes:

[0021] Determine whether the angular motion features in the abnormal motion data corresponding to the current frame in the abnormal motion data meet the preset reliability conditions;

[0022] Under the condition of satisfying the reliability, the angular motion features and linear motion features in the simulated motion data of the previous frame are determined, and the first difference features are determined relative to the angular motion features and linear motion features in the abnormal motion data corresponding to the current frame in the abnormal motion data, and the first observation parameters are determined based on the first difference features.

[0023] If the reliability condition is not met, determine the linear motion features in the simulated motion data of the previous frame, relative to the second differential features of the linear motion features in the abnormal motion data corresponding to the current frame, and determine the second observation parameters based on the second differential features.

[0024] Based on the first observation parameter or the second observation parameter, determine the control torque parameters that drive the object simulation model to move from the previous frame to the current frame.

[0025] In one possible implementation, the simulation motion data generated by the object simulation model during the simulation motion is corrected based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundaries of the virtual scene environment, to obtain corrected motion data corresponding to the abnormal motion data, including:

[0026] Based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundary of the virtual scene environment, the simulation motion data generated by the object simulation model during the simulation motion is corrected to obtain the initial corrected simulation motion data.

[0027] Based on the angular and linear motion characteristics in the initially corrected simulated motion data and the angular and linear motion characteristics in the abnormal motion data, the angular motion error and linear motion error between the initially corrected simulated motion data and the abnormal motion data are determined; based on preset adjustment parameters and preset weights, the angular motion error and the linear motion error are calculated exponentially to obtain comprehensive evaluation parameters;

[0028] If the comprehensive evaluation parameter is greater than or equal to the preset target threshold, the initially corrected simulation motion data will be used as the corrected motion data corresponding to the abnormal motion data.

[0029] In one possible implementation, the real-world environment includes at least one of a support surface, obstacles, and a risk area; acquiring abnormal motion data of the target object in the real-world environment includes:

[0030] Obtain motion data of the target object in the real-world scene environment;

[0031] Based on the motion data of the target object, determine the spatial relative positional relationship between the target object and the supporting surface, the obstacle, and / or the risk area;

[0032] Based on the spatial relative positional relationship, the abnormal motion data is identified from the motion data of the target object.

[0033] In one possible implementation, abnormal motion data of the target object in a real-world scene are acquired, including:

[0034] Based on the motion data of the target object, determine the motion continuity characteristics of the target object and the self-penetrating characteristics of each joint structure of the target object;

[0035] Based on the motion continuity feature and the self-wearing mold feature, the abnormal motion data is identified from the motion data of the target object.

[0036] A second aspect of this application provides a data processing apparatus, comprising:

[0037] The acquisition module is used to acquire abnormal motion data of target objects in real-world scenarios.

[0038] The determination module is used to determine the virtual scene environment corresponding to the real scene environment and the object simulation model corresponding to the target object;

[0039] The driving module is used to drive the object simulation model to perform simulated motion in the virtual scene environment based on the abnormal motion data;

[0040] The correction module is used to correct the simulation motion data generated by the object simulation model during the simulation motion process, based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundary of the virtual scene environment, to obtain the corrected motion data corresponding to the abnormal motion data.

[0041] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the data processing method described in the first aspect or any implementation thereof.

[0042] A fourth aspect of this application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0043] The memory is used to store computer programs;

[0044] The processor is used to execute the computer program so that the electronic device can implement the data processing method of the first aspect or any implementation thereof.

[0045] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to perform the data processing method described in the first aspect or any implementation thereof.

[0046] By employing the above technical solution, the data processing method provided in this application, after acquiring abnormal motion data of a target object in a real-world environment, first determines the virtual scene environment corresponding to the real-world environment and then determines the object simulation model corresponding to the target object. Subsequently, the abnormal motion data is used to drive the object simulation model to perform simulated motion in the virtual scene environment. During the simulation motion, the simulated motion data generated by the object simulation model is corrected by combining the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundaries, thus obtaining corrected motion data corresponding to the abnormal motion data. In this way, the corrected motion data is constrained by the dynamic simulation parameters and physical boundaries in the virtual scene environment, reducing inconsistencies between the corrected motion data and the real-world environment. This solves the problem of physical distortion easily generated by corrected motion data and improves the usability of corrected motion data in downstream applications such as digital twin display, behavior analysis, or security assessment. Attached Figure Description

[0047] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0048] Figure 1 A schematic diagram of the architecture of a data processing system provided in this application;

[0049] Figure 2 A flowchart illustrating a data processing method provided in this application;

[0050] Figure 3 A schematic diagram of a model structure for acquiring a real-world scene environment, provided in this application;

[0051] Figure 4 A schematic diagram of the structure of a virtual scene environment provided in this application;

[0052] Figure 5 A schematic diagram of the structure of the object simulation model provided in this application;

[0053] Figure 6 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0054] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0055] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0056] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0057] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0058] See Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include a terminal 100 and a server 200. The server 200 can provide the methods provided in the embodiments of this application to one or more terminals.

[0059] The terminal 100 may have a data processing application installed. The application and the webpage can provide a data processing application interface. The terminal 100 can receive relevant parameters input by the user on the data processing application interface and send the parameters to the server 200. The server 200 can obtain the processing result based on the received parameters and return the processing result to the terminal 100.

[0060] It should be understood that in some optional implementations, the terminal 100 can also complete the action of obtaining the processing result based on the received parameters on its own, without the need for the server to cooperate. This application embodiment is not limited to this.

[0061] The following description Figure 1 The product form of the mid-terminal 100;

[0062] The terminal 100 in this application embodiment can be a mobile phone, tablet computer, wearable device, vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0063] Terminal 100 may include a radio frequency unit, memory, input unit, display unit, camera (optional), audio circuitry (optional), speaker (optional), microphone (optional), headphone jack (optional), processor, external interface, power supply, and other components. Those skilled in the art will understand that the above-mentioned components are merely examples and do not constitute a limitation on the terminal or multifunctional device; it may include more or fewer components, or a combination of certain components, or different components.

[0064] The input unit can be used to receive input numeric or character information, and to generate key signal inputs related to user settings and function control of the portable multi-functional device. Specifically, the input unit may include a touchscreen (optional) and / or other input devices. Other input devices may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, a joystick, etc.

[0065] The input device can receive motion data of the target object in the real scene environment, configuration parameters of the virtual scene environment (such as dynamic simulation parameters and boundary parameters of physical boundaries), model parameters of the object simulation model, etc.

[0066] The display unit can be used to display information input by the user or information provided to the user, various menus of the terminal, interactive interfaces, file display, and / or playback of any multimedia file. In the embodiments of this application, the display unit can be used to display the interface for data processing, processing results, etc.

[0067] The memory can be used to store software code related to the data processing method, the processor can execute the steps of the data processing method, and can also schedule other units (such as the input unit and display unit mentioned above) to achieve the corresponding functions.

[0068] This radio frequency unit (optional) can be used to receive and send signals during information transmission or calls.

[0069] In this embodiment of the application, the radio frequency unit can send data to the server 200 and receive the processing results sent by the server 200.

[0070] It should be understood that this radio frequency unit is optional and can be replaced with other communication interfaces, such as a network port.

[0071] Terminal 100 also includes a power source (such as a battery) for supplying power to the various components.

[0072] Terminal 100 also includes an external interface, which can be a standard Micro USB interface or a multi-pin connector, which can be used to connect terminal 100 to other devices for communication or to connect a charger to charge terminal 100.

[0073] Server 200 includes a bus, a processor, a communication interface, and memory. The processor, memory, and communication interface communicate with each other via the bus.

[0074] The memory can be used to store software code related to data processing methods, the processor can execute the steps of the chip's data processing methods, and it can also schedule other units to achieve corresponding functions.

[0075] To facilitate understanding of the technical solutions provided in the embodiments of this application by those skilled in the art, the relevant technologies are described below: The inventors have discovered that motion data of a target object can be obtained through methods such as 3D reconstruction, motion capture, key point estimation, or visual perception. In real-world environments, due to occlusion, rapid movement, recognition errors, or unstable data acquisition, the motion data of the target object may exhibit problems such as inter-frame jumps, sudden changes in local pose, the target object floating in mid-air, the target object penetrating a support surface, and abnormal contact states. Existing motion restoration methods typically focus on restoring the motion continuity of the target object itself, or on general geometric clipping restoration, which may result in inconsistencies between the restored motion data and the real-world environment. For example, corrected motion data may still exhibit physical distortions such as feet penetrating floors, the target object passing through walls or scaffolding members, or the target object standing suspended in mid-air, thereby reducing the usability of the corrected motion data in downstream applications such as digital twin display, behavior analysis, or security assessment.

[0076] To address the aforementioned problems, embodiments of this application provide a data processing method. The data processing method of this application embodiment will be described in detail below with reference to the accompanying drawings.

[0077] Reference Figure 2 , Figure 2 This is a flowchart illustrating a data processing method provided in an embodiment of this application, such as... Figure 2 As shown, the data processing method provided in this application embodiment may include steps 201 to 204, which are described in detail below.

[0078] 201. Obtain abnormal motion data of target objects in real-world scenarios.

[0079] Specifically, the real-world scene environment can be the physical environment in which the target object actually moves. The real-world scene environment can include indoor or outdoor scenes (e.g., a construction environment). The real-world scene environment is used to define the source of abnormal motion data, ensuring that subsequently obtained corrected motion data corresponds to the actual motion process.

[0080] The target object can be any object that requires motion data processing. Target objects can include humans, animals, and robots, among others. Abnormal motion data can be motion data generated by the target object in a real-world environment that requires correction. Abnormal motion data can include a single abnormal data frame or a segment of multiple consecutive abnormal data frames. Abnormal motion data can include position data, joint data, velocity data, timestamps, and frame numbers. Abnormal motion data does not necessarily mean that all data is erroneous. Abnormal motion data can preserve the basic motion trend of the target object, thus serving as the data basis for subsequent motion simulation of the driving object simulation model.

[0081] Abnormal motion data can be extracted based on abnormal marker information or read from stored abnormal motion data. Furthermore, there are various other methods for obtaining abnormal motion data. The following provides an example of such a method.

[0082] In one possible implementation, the real-world environment includes at least one of a support surface, an obstacle, and a risk area. Step 201 may specifically include: acquiring motion data of a target object in the real-world environment; determining the spatial relative positional relationship between the target object and the support surface, the obstacle, and / or the risk area based on the motion data of the target object; and identifying abnormal motion data from the motion data of the target object based on the spatial relative positional relationship.

[0083] Specifically, a support surface can be used to represent an environmental surface capable of supporting a target object. Support surfaces can include the ground, steps, or equipment bearing surfaces. Obstacles can be used to represent physical structures that restrict the movement space of the target object. Obstacles can include walls, railings, fixed equipment, or stacked objects, etc. Risk areas can be used to represent spatial ranges that restrict the entry or stay of the target object. Risk areas can include prohibited areas, fall-risk areas, equipment operating areas, etc. Support surfaces, obstacles, and risk areas can be determined based on scene models of real-world environments, manually configured data, or environmental perception data.

[0084] The motion data of a target object can include root node pose, keypoint positions, mesh vertex positions, motion trajectory, and time information. Root node pose represents the overall position and orientation of the target object. Keypoint positions represent the positions of local parts of the target object. Mesh vertex positions represent the positions of the surface structures of the target object. The motion trajectory represents the path of position change of the target object in consecutive data frames. Time information represents the sequential order of the various data frames in the target object's motion data. After acquiring the motion data of the target object, it can be transformed into the scene coordinate system corresponding to the real-world environment. The scene coordinate system unifies the spatial positions of the target object, supporting surfaces, obstacles, and risk areas. Through coordinate transformation, the motion data of the target object can be placed under the same spatial reference as the supporting surfaces, obstacles, and risk areas, thus facilitating the calculation of relative spatial positions.

[0085] In one implementation, the spatial relative positional relationship between the target object and the supporting surface can be determined based on the distance between them. This spatial relative positional relationship can include height, distance, and contact relationships. To quantify the spatial relative positional relationship between the target object and the supporting surface, the minimum distance of the target object relative to the supporting surface in frame t can be calculated based on the keypoint positions or mesh vertex positions of the target object. The minimum distance of the target object relative to the supporting surface in frame t can be expressed as:

[0086]

[0087] in, Indicates the first The minimum distance between the target object in the frame and the supporting surface. Indicates the first The first frame target object Key point locations or mesh vertex locations Indicates the supporting surface. This indicates distance calculation.

[0088] like If the value exceeds the preset floating threshold, it can be determined that the target object has a floating anomaly relative to the supporting surface. If the penetration value is less than the preset penetration threshold, it can be determined that there is a penetration anomaly between the target object and the support surface. If the target object has a floating anomaly or a penetration anomaly relative to the support surface, it indicates that the spatial relative positional relationship between the target object and the support surface is mismatched.

[0089] In continuous intervals Furthermore, contact stability can be determined based on changes in the height of the lowest point. The contact stability constraint can be expressed as:

[0090]

[0091] in, Indicates the first The minimum distance between the target object in the frame and the supporting surface. This indicates a highly stable threshold.

[0092] If the change in the lowest point height within a continuous interval I does not satisfy the contact stability constraint, then a contact instability anomaly exists between the target object and the supporting surface. If a contact instability anomaly exists between the target object and the supporting surface, then a mismatch in the spatial relative positional relationship between the target object and the supporting surface can be determined.

[0093] When the target object is a human figure, the relationship between the human figure's center of gravity projection and the foot support area can be used to determine whether the ground contact support between the human figure and the support surface is reasonable. The set of ground contact vertices can be represented as:

[0094]

[0095] in, Let represent the set of ground-touching vertices in frame t. Show the first The first frame of the human body object Each grid vertex position Indicates the supporting surface. Indicates distance calculation, This indicates the height tolerance threshold.

[0096] If the first of the human body objects The distance from each grid vertex to the supporting surface is less than or equal to This set of grid vertices can be identified as ground-contact vertices. After determining the set of ground-contact vertices, the support area can be formed based on the projections of these vertices onto the support surface. And determine whether the center of gravity projection of the human object lies within the support area. The center of gravity support constraint can be expressed as:

[0097]

[0098] in, This represents the centroid of the human object in frame t. This represents the projection of the center of gravity of a human figure onto the supporting surface. This indicates the support area after tolerance expansion.

[0099] If the center of gravity projection of the human body object is not located at Within this framework, an anomaly (unreasonable) in ground contact support between the human object and the supporting surface can be identified. In another implementation, the spatial relative positional relationship between the target object and the obstacle can be determined based on the distance between them. This spatial relative positional relationship can include distance, proximity, and overlap. To quantify the spatial relative positional relationship between the target object and the obstacle, the minimum distance between them can be calculated based on the target object's keypoint positions, mesh vertex positions, or occupied space. The minimum distance between the target object and the obstacle can be expressed as:

[0100]

[0101] in, This represents the minimum distance between the t-th target object and the obstacle. This represents the position of the i-th keypoint or mesh vertex of the target object in frame t. This represents the boundary of the j-th obstacle. ( ) indicates distance calculation.

[0102] like If the distance is less than the preset safety distance, or if the space occupied by the target object overlaps with the space occupied by the obstacle, then a spatial conflict exists between the target object and the obstacle. In cases of spatial conflict, it can be determined that the relative spatial positions of the target object and the obstacle are mismatched.

[0103] In another implementation, the spatial relative positional relationship between the target object and the risk area can be determined based on their positional relationship. This spatial relative positional relationship can include being located within the area, outside the area, near the area boundary, or crossing the area boundary. To quantify the spatial relative positional relationship between the target object and the risk area, it can be determined whether the target object has entered the risk area based on its root node position, centroid projection, or trajectory. The risk area determination result can be expressed as:

[0104]

[0105] in, This represents the risk area assessment result for frame t. R represents the reference position of the target object in frame t, and R represents the risk area. () represents a judgment function that takes the value 1 when the condition is true.

[0106] like =1, and the constraint rule corresponding to the risk area indicates that the target object must not enter the risk area, which confirms that there is a regional constraint conflict between the target object and the risk area. In the case of a regional constraint conflict between the target object and the risk area, it can be determined that the spatial relative positional relationship between the target object and the risk area is mismatched.

[0107] After determining the spatial relative positions of the target object with its supporting surface, obstacles, and risk areas, anomaly detection results can be obtained based on these spatial relative positions (e.g., a mismatch in the spatial relative positions of the target object and the risk area). If the motion data of the target object corresponding to the same data frame satisfies at least one of the anomaly detection conditions: floating anomaly, penetration anomaly, spatial conflict, ground-contact support anomaly, and area constraint conflict, abnormal motion data can be identified from the target object's motion data. If the motion data of the target object corresponding to multiple consecutive data frames all satisfy the anomaly detection conditions, the motion data of the target object corresponding to multiple consecutive data frames can be considered as abnormal motion data.

[0108] In this embodiment, spatial relative position analysis can be performed on the motion data of the target object based on the support surface, obstacles, and risk areas, and abnormal motion data can be identified from the motion data of the target object. This allows the acquired abnormal motion data to not only reflect the motion state of the target object itself, but also reflect the spatial constraint relationship between the target object and the real scene environment, thereby improving the accuracy of abnormal motion data identification and providing a more reliable data foundation for subsequent simulation motion based on the simulation model of the object driven by abnormal motion data.

[0109] In one possible implementation, step 201 may include: determining the motion continuity characteristics of the target object and the self-piercing characteristics of each joint structure of the target object based on the motion data of the target object; and identifying the abnormal motion data from the motion data of the target object based on the motion continuity characteristics and the self-piercing characteristics.

[0110] Specifically, the motion data of the target object can include center position, joint rotation, mesh vertex positions, and time information. Center position represents the overall position of the target object in the real-world scene. Joint rotation represents the pose changes of individual joint structures of the target object. Mesh vertex positions represent the positions of the target object's surface structures in the real-world scene. Time information represents the sequential order of data frames in the target object's motion data.

[0111] Motion continuity features can be used to indicate whether the motion changes of a target object are smooth in consecutive data frames. Motion continuity features can be determined through motion acceleration and joint rotation differences, among others. Motion acceleration represents the degree of change in the overall position of the target object in consecutive data frames. Joint rotation differences represent the degree of rotational change of the same joint structure between adjacent data frames.

[0112] In one implementation, the motion acceleration can be calculated based on the center position of the target object, and the motion acceleration can reflect the continuity of motion. The motion acceleration of the target object in frame t can be expressed as:

[0113]

[0114] in, Represents the motion acceleration in frame t. This indicates the center position of the target object in frame t. This indicates the center position of the target object in frame t+1. Δt represents the center position of the target object in the (t-1)th frame, and Δt represents the time interval between adjacent data frames.

[0115] like If the acceleration exceeds a preset threshold, it can be determined that the motion data of the target object in frame t exhibits an anomaly in motion continuity. This method can identify motion continuity anomalies caused by jumps in the overall position of the target object.

[0116] In another implementation, joint rotation differences can be calculated based on the joint rotation of the target object. The joint rotation difference can be the amount of rotational change of the same joint structure between adjacent data frames. If the joint rotation difference is greater than a preset rotation difference threshold, it can be determined that the motion data of the target object in the corresponding data frame has an abnormal motion continuity. In this way, motion continuity anomalies caused by local joint jitter or sudden posture changes in the target object can be identified.

[0117] Self-penetrating features can be used to indicate whether there is unreasonable spatial penetration between corresponding parts of various joint structures of a target object. Self-penetrating features are determined based on the mesh vertex positions. Self-penetrating features can include the proportion of penetration points, average penetration depth, and maximum penetration depth. The proportion of penetration points represents the percentage of points where spatial penetration occurs among the detected points. The average penetration depth represents the overall penetration degree of the spatially penetrated area. The maximum penetration depth represents the penetration degree at the location of the most severe spatial penetration.

[0118] In one implementation, the severity of self-penetration can be determined based on the proportion of penetration points, the average penetration depth, and the maximum penetration depth. The severity of self-penetration can be expressed as:

[0119]

[0120] in, This indicates the severity of the self-penetrating pattern in frame t. This represents the proportion of clipping points in frame t. This represents the average penetration depth at time t. This represents the maximum clipping depth in frame t. , and This indicates the preset weight parameters.

[0121] like If the motion data of the target object in frame t exceeds a preset clipping threshold, then it can be determined that there is a clipping anomaly. In this way, the clipping anomaly can be judged by combining the spatial penetration range and spatial penetration depth, reducing misjudgments caused by judging based on spatial penetration at only a single detection point.

[0122] If the motion data of the target object exhibits abnormal motion continuity or self-piercing anomaly, the motion data of the target object in the corresponding data frame can be identified as abnormal motion data. If the motion data of the target object in multiple consecutive data frames are all identified as abnormal motion data, the motion data of the target object in multiple consecutive data frames can be treated as abnormal data segments. Then, these abnormal data segments can be merged to obtain the abnormal motion data.

[0123] In this embodiment, abnormal motion data can be identified from the motion data of the target object based on the motion continuity characteristics and self-penetrating characteristics of the target object. This allows for simultaneous consideration of the smoothness of the target object's motion in continuous data frames, as well as the spatial penetration between different parts of the target object, thereby improving the accuracy of abnormal motion data identification.

[0124] 202. Determine the virtual scene environment corresponding to the real scene environment and the object simulation model corresponding to the target object.

[0125] Specifically, such as Figure 4 As shown, a virtual scene environment can be a simulation environment obtained by digitally representing a real scene environment. A virtual scene environment can include a scene coordinate system, scene geometric data, scene scale information, and scene constraint data. The scene coordinate system represents the spatial reference in the virtual scene environment. Scene geometric data represents the spatial structure in the real scene environment. Scene scale information represents the dimensional correspondence between the virtual and real scene environments. Scene constraint data represents the motion restrictions that target objects need to meet in the virtual scene environment.

[0126] The object simulation model can be a driveable model corresponding to the target object. The object simulation model can include skeletal structures, joint structures, mesh structures, and collision structures. The skeletal structure represents the connection relationships between parts of the target object. The joint structure represents the degrees of freedom of movement for different parts of the target object. The mesh structure represents the external surface of the target object. The collision structure detects the contact or overlap between the object simulation model and the virtual scene environment. In one implementation, the virtual scene environment can be determined based on scene data from the real scene environment. For example... Figure 3 As shown, scene data of a real-world environment can be obtained through a pre-set model. Scene data can include 3D model data, point cloud data, mesh data, or building information model data. Scene coordinate systems, scene geometry data, and scene scale information can be extracted from the scene data, and then a virtual scene environment corresponding to the real-world environment can be constructed based on these elements. This approach improves the spatial consistency between the virtual and real-world scene environments. In another implementation, the virtual scene environment can be determined based on pre-set scene data. Pre-set scene data can include pre-configured 3D scene files, scene templates, or scene parameter tables. A corresponding virtual scene environment can be matched from the pre-set scene data based on the scene identifier or size of the real-world scene environment. This method reduces the amount of data processing required for real-time modeling and improves the efficiency of determining the virtual scene environment.

[0127] In one implementation, the object simulation model can be determined based on the object type of the target object. For example... Figure 5 As shown, if the target object is a human body, a simulation model with a human skeletal structure, joint structure, and mesh structure can be determined. If the target object is a device object, the simulation model can be determined based on the device object's structural connections and degrees of freedom. This approach allows the simulation model to match the structural features of the target object. In another implementation, the initial simulation model can be adapted based on the target object's motion data to obtain the corresponding simulation model. Model adaptation can include adjusting bone length, joint positions, mesh size, and collision structure size. This method reduces the dimensional differences between the target object and the simulation model, improving the matching degree when abnormal motion data drives the simulation model.

[0128] By determining the virtual scene environment corresponding to the real scene environment and the object simulation model corresponding to the target object, we can provide a scene foundation and model foundation for subsequent simulation motion based on abnormal motion data-driven object simulation models, thereby improving the consistency between simulated motion and real motion processes.

[0129] 203. Based on the abnormal motion data, drive the object simulation model to perform simulated motion in the virtual scene environment.

[0130] Specifically, abnormal motion data can serve as reference motion input for the object simulation model. The reference motion input indicates the position, posture, or joint state of the object simulation model in different data frames. Driving the object simulation model to simulate motion in a virtual scene environment based on abnormal motion data can be understood as reproducing the motion process of the target object in the virtual scene environment according to the temporal order of the abnormal motion data. Abnormal motion data can include center position, joint rotation, keypoint positions, mesh vertex positions, and time information. The object simulation model can include model root nodes, model joint structures, model mesh structures, and model collision structures. Based on the correspondence between abnormal motion data and the object simulation model, abnormal motion data can be mapped to the object simulation model, allowing the object simulation model to obtain the model state corresponding to the abnormal motion data. In one implementation, based on the correspondence between abnormal motion data and the object simulation model, the center position in the abnormal motion data can be mapped to the model root node of the object simulation model, the joint rotation in the abnormal motion data can be mapped to the model joint structure of the object simulation model, and the keypoint positions or mesh vertex positions in the abnormal motion data can be mapped to the corresponding parts of the object simulation model. In this way, the overall position and local pose of the object simulation model can maintain a correspondence with the abnormal motion data. In another implementation, the object simulation model can be driven frame-by-frame according to the time information in the abnormal motion data. In yet another implementation, the object simulation model can be driven by a controller to approximate the reference motion state corresponding to the abnormal motion data. The controller can determine the target position, target pose, or target joint state of the object simulation model in the current data frame based on the abnormal motion data, and control the object simulation model to perform simulated motion in the virtual scene environment. This approach improves the stability of the object simulation model during simulated motion. During the simulated motion of the object simulation model, the simulated motion data generated by the object simulation model can be recorded. The simulated motion data can include the model root node position, model joint rotation, model keypoint positions, model mesh vertex positions, model velocity, and time information, etc.

[0131] 204. During the simulation motion of the object simulation model, based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundary of the virtual scene environment, the simulation motion data generated by the object simulation model during the simulation motion is corrected to obtain the corrected motion data corresponding to the abnormal motion data.

[0132] Specifically, dynamic simulation parameters can be used to represent the physical constraints that an object simulation model needs to satisfy when moving in a virtual scene environment. Dynamic simulation parameters can include gravity parameters, friction parameters, damping parameters, contact parameters, and rebound parameters. Gravity parameters can be used to constrain the falling trend of the object simulation model. Friction parameters can be used to constrain the sliding trend of the object simulation model when in contact with the environment. Damping parameters can be used to reduce abnormal jitter of the object simulation model during simulated motion.

[0133] Physical boundaries are used to represent the boundaries that restrict the movement range of simulation models of objects in a virtual scene environment. Physical boundaries can include load-bearing boundaries, blocking boundaries, and region boundaries. Boundary parameters can include boundary position, boundary direction, boundary range, boundary normal, and boundary thickness. Boundary parameters are used to determine whether the simulation model of an object crosses the physical boundary or whether it unreasonably overlaps with the physical boundary.

[0134] Simulated motion data can be the motion data generated by the simulated object model during the simulated motion process. Simulated motion data can include the position, attitude, joint states, velocity, angular velocity, and surface position of the simulated object model. Corrected motion data can be the motion data obtained after applying physical constraints to the simulated motion data. Corrected motion data corresponds to abnormal motion data and is used to represent the result of correcting abnormal motion data through simulation.

[0135] In one implementation, the simulated motion data can be corrected based on dynamic simulation parameters. If the simulated object model experiences sudden velocity changes, attitude jitter, or unstable contact states during simulated motion, the model's position, attitude, or velocity can be adjusted based on gravity, friction, and damping parameters. This method makes the simulated motion data more consistent with the laws of physical motion. In another implementation, the simulated motion data can be corrected based on boundary parameters of the physical boundaries. The position of the simulated object model's surface or the location of the collision structure can be used to determine whether the simulated object model has crossed the physical boundary. If the simulated object model has crossed the physical boundary, its position or attitude can be adjusted based on the boundary position and boundary normal. This method reduces clipping issues between the simulated object model and the virtual scene environment. In yet another implementation, the simulated motion data can be corrected simultaneously based on both dynamic simulation parameters and boundary parameters. The simulated object model can be simulated and advanced based on the dynamic simulation parameters first, and then the boundary parameters can be used to determine whether the simulated motion data after the advancement meets the physical boundary constraints. If the simulated motion data does not meet the physical boundary constraints, position, attitude, or velocity corrections can be applied to the simulated motion data. The following provides an example of a scheme for correcting the simulation motion data of an object simulation model.

[0136] In one possible implementation, step 204 may include: acquiring the simulation motion data of the object simulation model in the previous frame; determining, based on the simulation motion data of the previous frame and the abnormal motion data corresponding to the current frame in the abnormal motion data, the control torque parameters driving the object simulation model to move from the previous frame to the current frame; using the control torque parameters and the dynamic simulation parameters associated with the virtual scene environment, simulating the object simulation model to obtain the simulation motion data of the current frame; determining the clipping data of the object simulation model between the current frame and the physical boundary based on the simulation motion data of the current frame and the boundary parameters of the physical boundary, and determining the blocking force parameters of the physical boundary on the object simulation model based on the clipping data; and correcting the simulation motion data of the object simulation model in the current frame using the blocking force parameters and the dynamic simulation parameters associated with the virtual scene environment.

[0137] Specifically, the simulation motion data of the previous frame can represent the motion state of the object simulation model at the end of the previous frame. The abnormal motion data corresponding to the current frame can be used as the reference motion input for the current frame. The control torque parameter can be determined based on the motion difference between the simulation motion data of the previous frame and the abnormal motion data corresponding to the current frame. The control torque parameter is used to drive the object simulation model to move from the motion state of the previous frame to the reference motion state corresponding to the current frame. The specific method for determining the control torque parameter can be further explained in subsequent embodiments in conjunction with angular motion characteristics, linear motion characteristics, and observation parameters.

[0138] After determining the control torque parameters, the simulation model of the object can be advanced using these parameters and dynamic simulation parameters. This simulation advancement allows the object simulation model to approach the abnormal motion data corresponding to the current frame, while also being subject to constraints such as gravity, friction, damping, contact, and rebound within the virtual scene environment. This allows the simulation model to obtain its motion data for the current frame.

[0139] After obtaining the simulation motion data for the current frame, it can be determined whether the object simulation model has crossed the physical boundary based on the simulation motion data and the boundary parameters of the physical boundary. If the model surface position or collision structure position of the object simulation model enters the forbidden space defined by the physical boundary, it can be determined that there is clipping data between the object simulation model and the physical boundary in the current frame. Clipping data can include clipping position, clipping depth, clipping direction, and clipping location, etc.

[0140] After determining the penetration data, the blocking force parameters can be determined based on the penetration depth and boundary normal. These blocking force parameters are used to cause the simulated object model to move in a direction away from the physical boundary. Using the blocking force parameters and dynamic simulation parameters, position, attitude, velocity, or angular velocity corrections can be applied to the simulated motion data of the current frame, resulting in the corrected motion data for that frame.

[0141] In this embodiment, dynamic constraints and physical boundary constraints can be corrected on the simulated motion data during the simulation process where the object simulation model follows abnormal motion data. This reduces unreasonable situations such as the object simulation model penetrating physical boundaries, passing through obstacles, or entering risky areas, thereby obtaining corrected motion data that better matches the virtual scene environment.

[0142] In one possible implementation, correcting the simulation motion data of the current frame of the object simulation model using the blocking force parameter and the dynamic simulation parameters associated with the virtual scene environment includes: determining the spatial position deviation between the simulation motion data of the current frame and the corresponding abnormal motion data in the abnormal motion data; determining whether the object simulation model has fallen in the virtual scene environment based on the simulation motion data of the current frame; if the spatial position deviation meets a preset condition and / or the object simulation model has fallen in the virtual scene environment, then controlling the object simulation model to recover to the spatial position corresponding to the corresponding abnormal motion data in the virtual scene environment; after controlling the object simulation model to recover to the spatial position corresponding to the corresponding moment in the abnormal motion data in the virtual scene environment, correcting the simulation motion data of the current frame of the object simulation model using the blocking force parameter and the dynamic simulation parameters associated with the virtual scene environment.

[0143] Specifically, the simulation motion data of the current frame may include the root node pose of the object simulation model in frame t. The abnormal motion data corresponding to the current frame may include the reference root node pose (the root node pose of the target object) in frame t. Spatial position deviation can be represented by the root node offset distance (the deviation distance between the root node pose of the object simulation model in frame t and the abnormal motion data corresponding to the current frame, which may include the reference root node pose in frame t). The root node offset distance can be expressed as:

[0144]

[0145] in, This represents the offset distance of the root node in frame t. This represents the pose of the root node of the simulation model of the object in frame t. This represents the pose of the reference root node in frame t.

[0146] In one implementation, the distance from the root node can be used as a reference. And the height of the human body in the t-th frame corresponding to the abnormal motion data of the current frame. To determine whether the simulation model of the object has fallen or significantly deviated from the corresponding abnormal motion data, the determination can be expressed as:

[0147]

[0148] in, This indicates the failure status flag for frame t. This indicates that the preset root node deviates from the threshold. This indicates the preset standing height threshold.

[0149] like Greater than This confirms that the spatial position deviation meets the preset conditions. If Less than This can determine whether the object simulation model is unstable in the virtual scene environment, such as falling or standing.

[0150] exist When =1, the object simulation model can be controlled to enter the recovery phase. During the recovery phase, the object simulation model can first be controlled to stand upright and approach the pose of the reference root node corresponding to the abnormal motion data, then switch back to reference motion tracking. After the object simulation model recovers to the spatial position corresponding to the time in the abnormal motion data, the simulation motion data of the current frame of the object simulation model can be further corrected using the blocking force parameters and dynamic simulation parameters.

[0151] In this embodiment, when the object simulation model deviates significantly from the reference position corresponding to the abnormal motion data, or when the object simulation model falls, the spatial position and standing state can be restored first, and then the blocking force and dynamic constraint correction can be continued. This reduces the situation where the simulation motion data continues to deviate from the reference motion and improves the stability of the corrected motion data.

[0152] In one possible implementation, determining the control torque parameters for driving the object simulation model from the previous frame to the current frame based on the simulated motion data of the previous frame and the abnormal motion data corresponding to the current frame in the abnormal motion data includes: determining whether the angular motion features in the abnormal motion data corresponding to the current frame in the abnormal motion data satisfy a preset reliability condition. If the reliability condition is satisfied, determining a first difference feature between the angular motion features and linear motion features in the simulated motion data of the previous frame and the angular motion features and linear motion features in the abnormal motion data corresponding to the current frame in the abnormal motion data, and determining a first observation parameter based on the first difference feature. If the reliability condition is not satisfied, determining a second difference feature between the linear motion features in the simulated motion data of the previous frame and the linear motion features in the abnormal motion data corresponding to the current frame in the abnormal motion data, and determining a second observation parameter based on the second difference feature. The control torque parameters for driving the object simulation model from the previous frame to the current frame are then determined based on either the first observation parameter or the second observation parameter.

[0153] Specifically, angular motion features can be used to represent the rotational motion state of a target object or object simulation model. Angular motion features can include joint rotation and angular velocity, etc. Linear motion features can be used to represent the translational motion state of a target object or object simulation model. Linear motion features can include keypoint positions and linear velocities, etc.

[0154] In one implementation, it can be first determined whether the angular motion features in the abnormal motion data corresponding to the current frame meet preset reliability conditions. Reliability conditions may include that the angular motion features are not missing, that the angular motion features do not exhibit significant jumps, and that the confidence level corresponding to the angular motion features is greater than a preset confidence threshold. By determining reliability, unreliable joint rotations or angular velocities can be avoided from directly influencing the determination of control torque parameters.

[0155] Under the condition of satisfying reliability requirements, the first difference feature can be determined simultaneously based on angular motion characteristics and linear motion characteristics, and the first observation parameter can be determined based on the first difference feature. The first observation parameter can be expressed as:

[0156]

[0157] in, This represents the first observation parameter currently being observed. This indicates the joint rotations in the abnormal motion data corresponding to the current frame. This represents the joint rotation in the simulated motion data of the previous frame. This indicates the location of key points in the abnormal motion data corresponding to the current frame. This indicates the location of key points in the simulated motion data of the previous frame. This represents the linear velocity in the abnormal motion data corresponding to the current frame. This represents the linear velocity in the simulated motion data of the previous frame. This represents the angular velocity in the abnormal motion data corresponding to the current frame. This represents the angular velocity in the simulated motion data of the previous frame.

[0158] By using the first observation parameter, the control torque parameter can simultaneously reflect the degree of deviation of the object simulation model in angular motion and linear motion.

[0159] If the reliability condition is not met, the influence of angular motion characteristics on the control process can be reduced. A second difference characteristic is determined based on the linear motion characteristics, and then a second observation parameter is determined based on the second difference characteristic. The second observation parameter can be expressed as:

[0160]

[0161] in, This represents the second observation parameter corresponding to the current frame. This indicates the location of key points in the abnormal motion data corresponding to the current frame. This indicates the location of key points in the simulated motion data of the previous frame. This represents the linear velocity in the abnormal motion data corresponding to the current frame. This represents the linear velocity in the simulated motion data of the previous frame.

[0162] By using the second observation parameter, when angular motion features are unreliable, the simulation model of the object can be driven to approach the motion state corresponding to the current frame by using the key point position and linear velocity, thereby reducing the impact of abnormal angular motion features on the simulation progress.

[0163] After obtaining the first or second observation parameter, the control torque parameter can be determined based on the first or second observation parameter. The control torque parameter can be applied to the joint structure of the object simulation model, causing the object simulation model to move from the simulated motion state of the previous frame to the reference motion state corresponding to the current frame.

[0164] In this embodiment, different observation parameter determination methods can be selected based on whether the angular motion features in the abnormal motion data corresponding to the current frame are reliable. This allows for the use of angular motion features and linear motion features to improve driving accuracy when the angular motion features are reliable, and also reduces the impact of abnormal angular motion features on control torque parameters when the angular motion features are unreliable, thereby improving the stability of the simulated propulsion process.

[0165] In one possible implementation, step 204 may include: correcting the simulation motion data generated by the object simulation model during the simulation motion based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundary of the virtual scene environment, to obtain initially corrected simulation motion data. Based on the angular and linear motion characteristics in the initially corrected simulation motion data and the angular and linear motion characteristics in the abnormal motion data, determining the angular motion error and linear motion error between the initially corrected simulation motion data and the abnormal motion data. Based on preset adjustment parameters and preset weights, performing an exponential weighted calculation on the angular motion error and the linear motion error to obtain a comprehensive evaluation parameter. If the comprehensive evaluation parameter is greater than or equal to a preset target threshold, using the initially corrected simulation motion data as the corrected motion data corresponding to the abnormal motion data.

[0166] Specifically, based on the angular motion features (joint rotation and angular velocity) and linear motion features (keypoint position and linear velocity) in the initially corrected simulation motion data, and the angular motion features (joint rotation and angular velocity) and linear motion features (keypoint position and linear velocity) in the abnormal motion data, the angular motion error and linear motion error between the initially corrected simulation motion data and the abnormal motion data can be determined. Then, based on preset adjustment parameters and preset weights, the angular motion error and the linear motion error are calculated exponentially to obtain comprehensive evaluation parameters. The preset adjustment parameters may include preset position error scale coefficients, preset attitude error scale coefficients, preset linear velocity error scale coefficients, and preset angular velocity error scale coefficients. The preset weights may include preset position tracking weights, preset linear velocity tracking weights, preset attitude or joint rotation tracking weights, and preset angular velocity tracking weights. The comprehensive evaluation parameters can be expressed as:

[0167]

[0168] in, This represents the comprehensive evaluation parameters for frame t. Indicates the preset position tracking weight. This indicates the preset posture or joint rotation tracking weights. This indicates the preset linear velocity tracking weight. This indicates the preset angular velocity tracking weight. This represents the preset position error scale coefficient. This represents the preset attitude error scale coefficient. This represents the preset linear velocity error scale coefficient. This represents the preset angular velocity error scale coefficient. This indicates the joint rotations in the abnormal motion data corresponding to the current frame. This represents the joint rotations in the simulated motion data of the current frame. This indicates the location of key points in the abnormal motion data corresponding to the current frame. This indicates the location of key points in the simulated motion data of the current frame. This represents the linear velocity in the abnormal motion data corresponding to the current frame. This represents the linear velocity in the simulated motion data of the current frame. This represents the angular velocity in the abnormal motion data corresponding to the current frame. This represents the angular velocity in the simulated motion data of the current frame. This represents the base of the natural exponential function.

[0169] like If the value is greater than or equal to a preset target threshold, it can be determined that the initial corrected simulation motion data meets the requirements for tracking abnormal motion data, and the initial corrected simulation motion data is used as the corrected motion data corresponding to the abnormal motion data. If the value is less than the preset target threshold, the control torque parameter, resistance force parameter, or dynamic simulation parameter can be adjusted, and the simulation motion data can be corrected again.

[0170] In this embodiment, after obtaining the initially corrected simulation motion data, the initial correction results can be further evaluated based on the errors in position, joint rotation, linear velocity, and angular velocity. This ensures that the final output corrected motion data satisfies both the dynamic constraints and physical boundary constraints of the virtual scene environment, while also preserving as much of the original motion trend as possible from the abnormal motion data.

[0171] The above describes a data processing method provided by an embodiment of this application. The following describes an apparatus for performing the above data processing method.

[0172] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Figure 7 As shown, the data processing device 600 includes:

[0173] The acquisition module 601 is used to acquire abnormal motion data of target objects in a real scene environment;

[0174] The determining module 602 is used to determine the virtual scene environment corresponding to the real scene environment and the object simulation model corresponding to the target object;

[0175] The driving module 603 is used to drive the object simulation model to perform simulated motion in the virtual scene environment based on the abnormal motion data;

[0176] The correction module 604 is used to correct the simulation motion data generated by the object simulation model during the simulation motion process based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundary of the virtual scene environment, so as to obtain the corrected motion data corresponding to the abnormal motion data.

[0177] In one possible implementation, the correction module 604 is specifically used for:

[0178] The simulation motion data of the object simulation model in the previous frame is obtained. Based on the simulation motion data of the previous frame and the abnormal motion data in the abnormal motion data corresponding to the current frame, the control torque parameters that drive the object simulation model to move from the previous frame to the current frame are determined.

[0179] Using the control torque parameters and the dynamic simulation parameters associated with the virtual scene environment, the object simulation model is simulated and advanced to obtain the simulation motion data of the current frame;

[0180] Based on the simulation motion data of the current frame and the boundary parameters of the physical boundary, the clipping data of the object simulation model between the current frame and the physical boundary is determined, and the blocking force parameters of the physical boundary on the object simulation model are determined based on the clipping data.

[0181] The simulation motion data of the current frame of the object simulation model is corrected using the blocking force parameters and the dynamic simulation parameters associated with the virtual scene environment.

[0182] In one possible implementation, correcting the simulation motion data of the current frame of the object simulation model using the blocking force parameters and the dynamic simulation parameters associated with the virtual scene environment includes:

[0183] Based on the simulated motion data of the current frame and the abnormal motion data in the abnormal motion data that corresponds to the simulated motion data of the current frame, determine the spatial position deviation between the simulated motion data of the current frame and the corresponding abnormal motion data.

[0184] Based on the simulation motion data of the current frame, determine whether the object simulation model has fallen in the virtual scene environment;

[0185] If the spatial position deviation meets the preset conditions and / or the object simulation model falls in the virtual scene environment, then control the object simulation model to recover to the spatial position corresponding to the corresponding abnormal motion data in the virtual scene environment;

[0186] After the object simulation model is restored to the spatial position corresponding to the time in the abnormal motion data in the virtual scene environment, the simulation motion data of the current frame of the object simulation model is corrected using the blocking force parameter and the dynamic simulation parameter associated with the virtual scene environment.

[0187] In one possible implementation, determining the control torque parameters that drive the object simulation model to move from the previous frame to the current frame based on the simulation motion data of the previous frame and the abnormal motion data corresponding to the current frame in the abnormal motion data includes:

[0188] Determine whether the angular motion features in the abnormal motion data corresponding to the current frame in the abnormal motion data meet the preset reliability conditions;

[0189] Under the condition of satisfying the reliability, the angular motion features and linear motion features in the simulated motion data of the previous frame are determined, and the first difference features are determined relative to the angular motion features and linear motion features in the abnormal motion data corresponding to the current frame in the abnormal motion data, and the first observation parameters are determined based on the first difference features.

[0190] If the reliability condition is not met, determine the linear motion features in the simulated motion data of the previous frame, relative to the second differential features of the linear motion features in the abnormal motion data corresponding to the current frame, and determine the second observation parameters based on the second differential features.

[0191] Based on the first observation parameter or the second observation parameter, determine the control torque parameters that drive the object simulation model to move from the previous frame to the current frame.

[0192] In one possible implementation, the correction module 604 is specifically used for:

[0193] Based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundary of the virtual scene environment, the simulation motion data generated by the object simulation model during the simulation motion is corrected to obtain the initial corrected simulation motion data.

[0194] Based on the angular and linear motion characteristics in the initially corrected simulated motion data and the angular and linear motion characteristics in the abnormal motion data, the angular motion error and linear motion error between the initially corrected simulated motion data and the abnormal motion data are determined; based on preset adjustment parameters and preset weights, the angular motion error and the linear motion error are calculated exponentially to obtain comprehensive evaluation parameters;

[0195] If the comprehensive evaluation parameter is greater than or equal to the preset target threshold, the initially corrected simulation motion data will be used as the corrected motion data corresponding to the abnormal motion data.

[0196] In one possible implementation, the real-world environment includes at least one of a support surface, obstacles, and a risk area; the acquisition module 601 is specifically used for:

[0197] Obtain motion data of the target object in the real-world scene environment;

[0198] Based on the motion data of the target object, determine the spatial relative positional relationship between the target object and the supporting surface, the obstacle, and / or the risk area;

[0199] Based on the spatial relative positional relationship, the abnormal motion data is identified from the motion data of the target object.

[0200] In one possible implementation, module 601 is specifically used for:

[0201] Based on the motion data of the target object, determine the motion continuity characteristics of the target object and the self-penetrating characteristics of each joint structure of the target object;

[0202] Based on the motion continuity feature and the self-wearing mold feature, the abnormal motion data is identified from the motion data of the target object.

[0203] This application also provides an electronic device in its embodiments. (See reference...) Figure 7 The diagram illustrates a structural schematic of an electronic device suitable for implementing the data processing method in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0204] like Figure 7 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. When the electronic device is powered on, the RAM 703 also stores various programs and data required for the operation of the electronic device. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0205] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, memory cards, hard drives, etc.; and communication devices 709. Communication device 709 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0206] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the data processing methods provided in this application.

[0207] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the data processing methods provided in this application.

[0208] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0209] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0210] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0211] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A data processing method, characterized in that, include: Acquire abnormal motion data of target objects in a real-world environment; Determine the virtual scene environment corresponding to the real scene environment and the object simulation model corresponding to the target object; The abnormal motion data is used to drive the object simulation model to perform simulated motion in the virtual scene environment; During the simulation motion of the object simulation model, the simulation motion data generated by the object simulation model during the simulation motion is corrected based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundary of the virtual scene environment, so as to obtain the corrected motion data corresponding to the abnormal motion data. The step of correcting the simulation motion data generated by the object simulation model during the simulation motion, based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundary of the virtual scene environment, includes: The simulation motion data of the object simulation model in the previous frame is obtained. Based on the simulation motion data of the previous frame and the abnormal motion data in the abnormal motion data corresponding to the current frame, the control torque parameters that drive the object simulation model to move from the previous frame to the current frame are determined. Using the control torque parameters and the dynamic simulation parameters associated with the virtual scene environment, the object simulation model is simulated and advanced to obtain the simulation motion data of the current frame; Based on the simulation motion data of the current frame and the boundary parameters of the physical boundary, the clipping data of the object simulation model between the current frame and the physical boundary is determined, and the blocking force parameters of the physical boundary on the object simulation model are determined based on the clipping data. The simulation motion data of the current frame of the object simulation model is corrected using the blocking force parameters and the dynamic simulation parameters associated with the virtual scene environment.

2. The method according to claim 1, characterized in that, The step of correcting the simulation motion data of the current frame of the object simulation model using the blocking force parameters and the dynamic simulation parameters associated with the virtual scene environment includes: Based on the simulated motion data of the current frame and the abnormal motion data in the abnormal motion data that corresponds to the simulated motion data of the current frame, determine the spatial position deviation between the simulated motion data of the current frame and the corresponding abnormal motion data. Based on the simulation motion data of the current frame, determine whether the object simulation model has fallen in the virtual scene environment; If the spatial position deviation meets the preset conditions and / or the object simulation model falls in the virtual scene environment, then control the object simulation model to recover to the spatial position corresponding to the corresponding abnormal motion data in the virtual scene environment; After the object simulation model is restored to the spatial position corresponding to the time in the abnormal motion data in the virtual scene environment, the simulation motion data of the current frame of the object simulation model is corrected using the blocking force parameter and the dynamic simulation parameter associated with the virtual scene environment.

3. The method according to claim 1, characterized in that, The determination of the control torque parameters that drive the object simulation model to move from the previous frame to the current frame, based on the simulation motion data of the previous frame and the abnormal motion data corresponding to the current frame in the abnormal motion data, includes: Determine whether the angular motion features in the abnormal motion data corresponding to the current frame in the abnormal motion data meet the preset reliability conditions; Under the condition of satisfying the reliability, the angular motion features and linear motion features in the simulated motion data of the previous frame are determined, and the first difference features are determined relative to the angular motion features and linear motion features in the abnormal motion data corresponding to the current frame in the abnormal motion data, and the first observation parameters are determined based on the first difference features. If the reliability condition is not met, determine the linear motion features in the simulated motion data of the previous frame, relative to the second differential features of the linear motion features in the abnormal motion data corresponding to the current frame, and determine the second observation parameters based on the second differential features. Based on the first observation parameter or the second observation parameter, determine the control torque parameters that drive the object simulation model to move from the previous frame to the current frame.

4. The method according to claim 1, characterized in that, The dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundaries of the virtual scene environment are used to correct the simulation motion data generated by the object simulation model during the simulation motion, resulting in corrected motion data corresponding to the abnormal motion data, including: Based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundary of the virtual scene environment, the simulation motion data generated by the object simulation model during the simulation motion is corrected to obtain the initial corrected simulation motion data. Based on the angular and linear motion characteristics in the initially corrected simulated motion data and the angular and linear motion characteristics in the abnormal motion data, the angular motion error and linear motion error between the initially corrected simulated motion data and the abnormal motion data are determined; based on preset adjustment parameters and preset weights, the angular motion error and the linear motion error are calculated exponentially to obtain comprehensive evaluation parameters; If the comprehensive evaluation parameter is greater than or equal to the preset target threshold, the initially corrected simulation motion data will be used as the corrected motion data corresponding to the abnormal motion data.

5. The method according to claim 1, characterized in that, The real-world environment includes at least one of a support surface, obstacles, and a risk area; acquiring abnormal motion data of the target object in the real-world environment includes: Obtain motion data of the target object in the real-world scene environment; Based on the motion data of the target object, determine the spatial relative positional relationship between the target object and the supporting surface, the obstacle, and / or the risk area; Based on the spatial relative positional relationship, the abnormal motion data is identified from the motion data of the target object.

6. The method according to claim 1, characterized in that, The acquisition of abnormal motion data of target objects in a real-world environment includes: Based on the motion data of the target object, determine the motion continuity characteristics of the target object and the self-penetrating characteristics of each joint structure of the target object; Based on the motion continuity feature and the self-wearing mold feature, the abnormal motion data is identified from the motion data of the target object.

7. A data processing apparatus, characterized in that, include: The acquisition module is used to acquire abnormal motion data of target objects in real-world scenarios. The determination module is used to determine the virtual scene environment corresponding to the real scene environment and the object simulation model corresponding to the target object; The driving module is used to drive the object simulation model to perform simulated motion in the virtual scene environment based on the abnormal motion data; The correction module is used to correct the simulation motion data generated by the object simulation model during the simulation motion process, based on the dynamic simulation parameters associated with the virtual scene environment and the boundary parameters of the physical boundary of the virtual scene environment, to obtain the corrected motion data corresponding to the abnormal motion data. The correction module is specifically used to acquire the simulation motion data of the object simulation model in the previous frame; based on the simulation motion data of the previous frame and the abnormal motion data corresponding to the current frame in the abnormal motion data, determine the control torque parameters that drive the object simulation model to move from the previous frame to the current frame; use the control torque parameters and the dynamic simulation parameters associated with the virtual scene environment to simulate and propel the object simulation model to obtain the simulation motion data of the current frame; based on the simulation motion data of the current frame and the boundary parameters of the physical boundary, determine the clipping data of the object simulation model between the current frame and the physical boundary, and determine the blocking force parameters of the physical boundary on the object simulation model based on the clipping data; The simulation motion data of the current frame of the object simulation model is corrected using the blocking force parameters and the dynamic simulation parameters associated with the virtual scene environment.

8. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to perform the data processing method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the data processing method as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the data processing method as described in any one of claims 1 to 6.

Images