Real-time rendering method and device

Abstract

The invention discloses a real-time rendering method and device. The method comprises the following steps: carrying out multi-dimensional filtering processing on preprocessed data by utilizing a Kalman filtering method based on multi-level and multi-scale analysis to obtain filtered multi-dimensional data; analyzing the filtered multi-dimensional data to obtain target object information and corresponding event information in the filtered multi-dimensional data; classifying the target object information, matching a classification result with a corresponding model in the virtual scene, and dynamically adjusting the posture, action and position of a virtual object according to a matching result; and based on the event information, fusing the dynamically adjusted virtual object with a preset interaction event in the virtual scene to generate optimized rendering input data, and based on the optimized rendering input data, rendering the virtual object and the virtual scene. The technical problem that in the prior art, the rendering speed is low is solved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a real-time rendering method and device. Background Art

[0002] With the development of technologies such as virtual reality and augmented reality, real-time rendering of virtual objects and virtual scenes has become a key technology for related applications. However, in the existing technology, the real-time rendering of virtual objects and scenes faces the problem of slow speed, especially when it involves complex actions, posture changes, and a large number of interactive events. Therefore, there is an urgent need for a real-time rendering method that can improve rendering speed and rendering quality to solve the relevant problems in the existing technology, optimize the interactive performance of virtual objects and scenes, and achieve a smoother real-time rendering effect.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present invention provide a real-time rendering method and device to at least solve the technical problem of slow rendering speed in the prior art.

[0005] According to one aspect of an embodiment of the present invention, a real-time rendering method is provided, comprising: acquiring raw motion capture data from a motion capture device in real time, and preprocessing the raw motion capture data to obtain preprocessed data, wherein the preprocessing includes data cleaning, outlier detection and denoising; performing multidimensional filtering on the preprocessed data using a Kalman filtering method based on multi-level and multi-scale analysis to obtain filtered multidimensional data; parsing the filtered multidimensional data to obtain target object information and corresponding event information in the filtered multidimensional data; classifying the target object information, matching the classification results with the corresponding models in a virtual scene, and dynamically adjusting the posture, action and position of the virtual object according to the matching results; based on the event information, fusing the dynamically adjusted virtual object with a preset interactive event in the virtual scene to generate optimized rendering input data, and rendering the virtual object and the virtual scene based on the optimized rendering input data.

[0006] According to another aspect of an embodiment of the present invention, a real-time rendering device is provided, comprising: a preprocessing module configured to acquire raw motion capture data from a motion capture device in real time, and preprocess the raw motion capture data to obtain preprocessed data, wherein the preprocessing includes data cleaning, outlier detection and denoising; a filtering module configured to perform multidimensional filtering on the preprocessed data using a Kalman filtering method based on multi-level and multi-scale analysis to obtain filtered multidimensional data; a parsing module configured to parse the filtered multidimensional data to obtain target object information and corresponding event information in the filtered multidimensional data; a matching module configured to classify the target object information, match the classification result with the corresponding model in the virtual scene, and dynamically adjust the posture, action and position of the virtual object according to the matching result; a rendering module configured to fuse the dynamically adjusted virtual object with a preset interactive event in the virtual scene based on the event information to generate optimized rendering input data, and render the virtual object and the virtual scene based on the optimized rendering input data.

[0007] In an embodiment of the present invention, raw motion capture data is acquired from a motion capture device in real time, and the raw motion capture data is preprocessed to obtain preprocessed data, wherein the preprocessing includes data cleaning, outlier detection and denoising; the preprocessed data is multidimensionally filtered using a Kalman filtering method based on multi-level and multi-scale analysis to obtain filtered multidimensional data; the filtered multidimensional data is parsed to obtain target object information and corresponding event information in the filtered multidimensional data; the target object information is classified, the classification result is matched with the corresponding model in the virtual scene, and the posture, action and position of the virtual object are dynamically adjusted according to the matching result; based on the event information, the dynamically adjusted virtual object is fused with the preset interactive event in the virtual scene to generate optimized rendering input data, and the virtual object and the virtual scene are rendered based on the optimized rendering input data. The above scheme solves the technical problem of slow rendering speed in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0009] Figure 1 is a flow chart of a real-time rendering method according to an embodiment of the present application;

[0010] Figure 2 is a flowchart of another real-time rendering method according to an embodiment of the present application;

[0011] Figure 3 is a flowchart of a multidimensional Kalman filter process according to an embodiment of the present application;

[0012] Figure 4 is a flow chart of a target object information classification and model matching method according to an embodiment of the present application;

[0013] Figure 5 is a flowchart of another real-time rendering method according to an embodiment of the present application;

[0014] Figure 6 is a flow chart of a method for performing Kalman filtering processing according to an embodiment of the present application;

[0015] Figure 7 is a flow chart of a method for parsing filtered multidimensional data according to an embodiment of the present application;

[0016] Figure 8 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0019] According to an embodiment of the present invention, a method embodiment of a real-time rendering method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0020] Figure 1 is a real-time rendering method according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0021] Step S102, acquiring raw motion capture data from the motion capture device in real time, and preprocessing the raw motion capture data to obtain preprocessed data, wherein the preprocessing includes data cleaning, outlier detection and denoising.

[0022] Step S104, using a Kalman filtering method based on multi-level and multi-scale analysis, multi-dimensional filtering is performed on the pre-processed data to obtain filtered multi-dimensional data.

[0023] For example, based on multi-level analysis, the time, space and frequency features in the preprocessed data are extracted as multi-scale features; based on the extracted multi-scale features, Kalman filtering is applied at different scales to predict and update the data at each scale to obtain filtering results at different scales; the filtering results at different scales are fused to generate the filtered multidimensional data.

[0024] Step S106, parsing the filtered multi-dimensional data to obtain target object information and corresponding event information in the filtered multi-dimensional data.

[0025] From the filtered multidimensional data, the target object corresponding to the preset object in the virtual scene is identified and parsed to obtain the target object information; through the event detection algorithm, the action and interaction information related to the target object is analyzed to extract and generate the event information.

[0026] Step S108, classifying the target object information, matching the classification result with the corresponding model in the virtual scene, and dynamically adjusting the posture, action and position of the virtual object according to the matching result.

[0027] Based on the physical features, action features and interaction information in the target object information, the target object is classified using a classification algorithm; according to the classification result, a preset object model library in the virtual scene is queried to determine a virtual model that matches the target object classification as the corresponding model. Then, the posture, action and position of the virtual object are dynamically adjusted according to the matching result.

[0028] Step S110, based on the event information, the dynamically adjusted virtual object is integrated with the preset interactive event in the virtual scene to generate optimized rendering input data, and based on the optimized rendering input data, the virtual object and the virtual scene are rendered.

[0029] According to the interaction type detected in the event information, the preset interaction event in the virtual scene is selected; the action, posture and position of the dynamically adjusted virtual object and the preset interaction event are merged. Based on the action, posture and position of the virtual object, the real-time animation of the virtual object is rendered; based on the preset interaction event, a pre-made animation sequence matching the event is called; the pre-made animation sequence is merged with the real-time animation of the virtual object to generate a virtual scene containing the virtual object and the corresponding interaction event; the virtual scene is rendered as a whole so that the dynamic effects of the virtual object and the interaction event in the virtual scene are consistent with the environment.

[0030] The present application embodiment provides another real-time rendering method, such as Figure 2 As shown, the method comprises the following steps:

[0031] Step S202, obtaining original motion capture data and performing preprocessing.

[0032] The raw motion capture data is obtained through a real-time capture device. The raw motion capture data contains detailed information such as the position, posture, and movement of the target object. Since the raw motion capture data may be affected by noise, errors, or abnormal points during the acquisition process, the data must be preprocessed to ensure the accuracy of subsequent processing.

[0033] First, the raw data is preliminarily checked to remove duplicate, missing or invalid data during the collection process. The purpose of data cleaning is to ensure the integrity and consistency of the input data. After that, anomaly detection algorithms are used to identify outliers in the data, which are deviations caused by equipment failures or changes in the collection environment. The detected anomalies are marked and excluded from subsequent processing. Finally, filtering techniques are used to denoise the data, such as using median filtering or Gaussian filtering to eliminate data fluctuations caused by equipment noise or external environmental interference, thereby obtaining smoother and more accurate preprocessed data.

[0034] Step S204: multi-dimensional Kalman filtering processing.

[0035] In order to improve the accuracy and stability of the data, the present invention uses a Kalman filtering method based on multi-level and multi-scale analysis to process multi-dimensional motion capture data. Figure 3 As shown, the method comprises the following steps:

[0036] Step S2042, multi-level analysis.

[0037] The preprocessed data is decomposed into time, space and frequency features, where time, space and frequency features represent the motion changes of the target object in different dimensions. The time feature represents the time series changes of the target object's action, the space feature represents the spatial position and posture changes of the target object in the virtual scene, and the frequency feature is used to identify the periodicity of the target object's motion.

[0038] Step S2044, extracting multi-scale features.

[0039] The extracted features are analyzed at different scales (such as time scale and spatial scale) and multi-scale features are generated. By observing the data at different scales, the complex motion changes of the target object can be captured more carefully.

[0040] Step S2046, Kalman filter processing.

[0041] Based on the Kalman filter algorithm, the feature data at each scale is predicted and updated. Kalman filtering combines historical data and current data for optimal prediction through recursive estimation. The filtering process not only considers the current state of the data, but also refers to past movement trends, thereby obtaining a smoother and more accurate filtering result.

[0042] Step S2048, multi-scale fusion.

[0043] The filtering results generated at different scales are fused and processed to combine the advantages of multi-scale features to generate the final filtered multi-dimensional data. The filtered multi-dimensional data contains comprehensive information in different dimensions such as time, space, and frequency, which can provide a basis for subsequent target object analysis and event analysis.

[0044] Step S206, analyzing the filtered multi-dimensional data.

[0045] The filtered multidimensional data is parsed to extract the target object information and event information. Specifically, the multidimensional data is parsed to identify the target object corresponding to the preset object in the virtual scene. The target object information includes the physical characteristics, action characteristics, and posture changes of the object. Through the event detection algorithm, the action and interaction behavior of the target object are analyzed to extract the relevant event information. The event information describes the dynamic interaction of the target object in the virtual scene, such as the object's action intention, the interaction behavior with other objects in the scene, etc.

[0046] Step S208: target object information classification and model matching.

[0047] like Figure 4As shown, the method for target object information classification and model matching includes the following steps:

[0048] Step S2082, target object classification.

[0049] Based on the physical features, motion features and interaction information of the target object, a classification algorithm (such as support vector machine, decision tree or neural network, etc.) is used to classify the target object. For example, the target object is classified into different types (such as virtual human, animal or mechanical device) according to its skeleton structure, motion mode, physical parameters, etc.

[0050] Step S2084, model matching.

[0051] According to the classification results, the corresponding virtual model is queried from the object model library in the virtual scene. Each virtual model is pre-designed to adapt to the dynamic behavior and appearance characteristics of different types of target objects. By matching the classification results with the virtual models in the model library, the virtual object model that best matches the current state of the target object can be found.

[0052] Step S2086, dynamic adjustment.

[0053] After finding a matching virtual model, the system can dynamically adjust the virtual object’s posture, movement, and position based on the classification results and motion characteristics of the target object. The dynamic adjustment is based on the real-time motion data of the target object to ensure that the virtual object’s posture is synchronized with the target object’s actual movement.

[0054] Step S210: rendering virtual objects and virtual scenes.

[0055] Render real-time animation of virtual objects based on optimized rendering input data. Specifically, the action, posture, and position data of virtual objects are calculated and rendered in real time to ensure that the animation of virtual objects is synchronized with the occurrence time of interactive events and dynamic changes in the scene. For example, if the user triggers a button, the hand movement of the virtual object will immediately present the animation effect of pressing the button.

[0056] While rendering the real-time animation of the virtual object, according to the preset interactive event, the pre-made animation sequence that matches the event is called. For example, when a collision event is detected, a pre-made explosion effect or scene change animation can be called. These pre-made animation sequences are usually designed and produced in advance to enhance the interactive effect. They ensure the consistency and efficiency of the performance of complex events and improve the rendering speed.

[0057] The real-time animation of virtual objects is integrated with pre-made animation sequences. For example, when a virtual object collides with an object in the scene, the dynamic movement and posture of the virtual object need to be displayed synchronously with the preset explosion animation or object decomposition animation. Through this integration, a complete virtual scene containing virtual objects and corresponding interactive events is finally generated, achieving deep interaction and visual optimization between virtual objects and virtual scenes.

[0058] Through the above steps, the real-time rendering method of the present invention not only improves the interactive performance between virtual objects and scenes, but also significantly optimizes the rendering speed and quality. This method overcomes the rendering delay and processing bottleneck in the prior art through Kalman filtering processing of multi-dimensional data, fusion analysis of multi-scale features, and dynamic adjustment and event fusion rendering strategies, and can achieve efficient and realistic virtual object and scene rendering, which is widely applicable to virtual reality, augmented reality and other fields.

[0059] The present application also provides another real-time rendering method, which significantly improves the dynamic rendering effect of objects in the virtual scene through the steps of real-time acquisition of motion capture data, multi-dimensional preprocessing of data, complex Kalman filter processing, data analysis and classification, dynamic object matching and rendering fusion processing, etc., and is particularly suitable for virtual reality scenes that require efficient and accurate rendering. Figure 5 As shown, the method comprises the following steps:

[0060] Step S502: collecting original motion capture data.

[0061] Use real-time motion capture equipment to capture the motion data of the target object. For example, a camera system based on optical tracking, an inertial sensor, or other equipment that can track the target object's movements, positions, and postures in real time can be used for motion capture. Motion capture equipment can capture the target object's complex motion data, including its position in three-dimensional space, posture information represented by quaternions or Euler angles, and dynamic characteristics such as the object's motion speed and acceleration.

[0062] Step S504, perform preprocessing.

[0063] Motion capture equipment may collect inaccurate data due to external interference or insufficient sensitivity. In addition, factors such as light and object occlusion in the environment may also affect the accuracy of the data. Therefore, it is necessary to pre-process the raw motion capture data to ensure data quality and improve the processing efficiency of subsequent steps.

[0064] First, data cleaning is performed. Data cleaning is used to remove redundant or invalid parts in the data to ensure the integrity of the input data. During the data cleaning process, the original motion capture data needs to be initially checked to identify missing data, duplicate data points, or invalid acquisition data. For example, during multi-frame acquisition, if the capture information of a frame is missing or the data is incomplete, the frame will be marked and discarded. By cleaning these redundant and invalid data, the complexity of subsequent processing can be reduced.

[0065] Next, outlier detection is performed. Outlier detection is used to identify outliers in the data. Anomaly detection algorithms, such as statistical analysis, cluster analysis, or machine learning-based anomaly detection models, can identify outliers that do not conform to the overall trend of the data. Typically, outliers show abrupt changes compared to normal data, such as extremely high or extremely low values. The system automatically marks these outliers and excludes them in subsequent processing.

[0066] Then, denoising is performed. The original motion capture data is usually accompanied by noise. In order to ensure the smoothness and accuracy of the data, a median filtering denoising algorithm can be used for denoising. Median filtering can effectively remove noise while maintaining edge features. For burst noise in motion capture data (such as erroneous data caused by sensor jitter), median filtering can smooth the data well. In some other embodiments, Gaussian filtering can also be used for denoising. Gaussian filtering eliminates high-frequency noise by smoothing the signal, making the data more continuous. By applying a Gaussian kernel function, the intensity of the filter can be adjusted to strike a balance between retaining data details and eliminating noise.

[0067] Step S506: perform Kalman filtering processing.

[0068] In the embodiment of the present invention, a Kalman filtering method based on multi-level and multi-scale analysis is applied to perform filtering processing. Figure 6 As shown, the following steps are included:

[0069] Step S5062, multi-level analysis.

[0070] Multi-level analysis is to decompose the motion capture data and extract motion features in different dimensions such as time, space and frequency. Through this analysis, the changing trends of the target object in different dimensions can be captured respectively. The time feature represents the change of the motion trajectory of the target object over time. By analyzing the time series data, the action pattern of the target object can be captured, such as the speed of the steps, the frequency of the action, etc. The spatial feature describes the position change of the target object in three-dimensional space, including its X, Y, Z coordinates, direction angle, etc. By analyzing the spatial features, the position and direction of the target object in the virtual scene can be accurately located. The frequency feature is used to describe the periodic changes of the target object's action, such as periodically repeated action patterns (such as walking, running, etc.). Through frequency domain analysis, the movement rhythm of the target object can be captured, so as to more accurately predict its next action.

[0071] Step S5064, extracting multi-scale features.

[0072] The temporal, spatial, and frequency characteristics of the motion capture data are processed in layers. The data in each dimension will be decomposed into multiple different scales to capture the details and global characteristics of the motion. For example, in the temporal dimension, the data in the short time window and the long time window can be analyzed separately to capture both short-term motion changes and long-term motion trends; in the spatial dimension, the local position information and the overall spatial changes can be analyzed simultaneously to ensure the full capture of the target object.

[0073] Step S5066, perform Kalman filtering processing.

[0074] After extracting multi-scale features, the Kalman filter algorithm is applied to the data at each scale. Kalman filtering recursively updates the state of the target object at each moment using the current observation value and combines the historical state to make the best estimate. For example, based on the motion model of the target object, its future state is predicted. The prediction step generates an estimate of the state of the target object at the next moment based on previous data and model assumptions. Once new motion capture data is obtained, the current observation data is compared with the prediction result, and the prediction result is corrected by the Kalman gain to generate an updated state estimate.

[0075] After Kalman filtering is performed at different scales, the filtering results at each scale are fused to generate filtered multidimensional data. The purpose of multiscale fusion is to combine the advantages of data at different scales to generate comprehensive data containing multidimensional information such as time, space, and frequency. These data can not only reflect the global motion trend of the target object, but also capture its subtle changes in movement. In this way, the complex movements of the target object can be modeled and tracked more accurately.

[0076] Step S508, analyzing the filtered multi-dimensional data.

[0077] like Figure 7 As shown, parsing the filtered multidimensional data includes the following steps:

[0078] Step S5082, extract target object information.

[0079] By analyzing the filtered multidimensional data, the target object corresponding to the preset object in the virtual scene is identified. By analyzing the position, action, and posture information in the three-dimensional space, the specific position and direction of the target object in the virtual scene are determined. Among them, the target object information includes skeletal motion information, and motion speed and acceleration information. For the characters in the virtual scene, skeletal motion information is the core action feature. By analyzing the filtered data, the corresponding parameters such as the skeletal position and joint rotation angle are generated, and then the action of the virtual character is driven. For example, when the target object performs actions such as walking and running, its detailed actions such as footsteps, knee bending, and arm swinging can be accurately captured. The motion speed and acceleration information are used to indicate the speed and change speed of the action. By analyzing the speed change, a motion trajectory that conforms to reality can be generated. In order to simulate realistic physical effects, the center of gravity and balance state of the target object also need to be analyzed. For example, when the target object suddenly changes the direction of movement, the center of gravity and action posture of its body will be adjusted accordingly. Through the calculation of dynamic balance, it is ensured that the generated virtual action conforms to the laws of physics.

[0080] Step S5084, generate event information.

[0081] While parsing the action features of the target object, relevant event information can also be generated. Event information is usually used to trigger interaction or behavior logic in the virtual scene, mainly including action events and state events. When the target object performs a specific action, a corresponding action event is generated. For example, when the character completes a jump, attack, or interaction, the system marks these actions as specific events and triggers subsequent scene interactions. These action events can be used for interactive plot design in virtual reality and augmented reality to enhance the immersiveness of the user experience. In addition to action events, state events can also be generated based on the motion state of the target object. For example, the system can detect that the character enters a running state, a fatigue state, or a falling state, and generate corresponding state events. These events can be used to adjust the feedback mechanism in the virtual scene, such as changing the character's motion performance or interaction with the environment.

[0082] By analyzing the filtered multi-dimensional data, not only can the motion information of the target object be generated, but also events related to scene interaction can be automatically generated, thus providing rich dynamic input for real-time rendering in virtual scenes.

[0083] Step S510: Dynamic object matching based on multi-dimensional data.

[0084] First, predefine a virtual object template. A virtual object template is a three-dimensional model created in advance in the system, including characters, objects or other dynamic objects in a virtual scene. Each virtual object template contains geometric data and material information, and needs to be able to match the motion characteristics of the target object. In an embodiment of the present invention, the virtual object template usually contains a preset skeletal structure, including information such as joints and connecting rods. These skeletal structures correspond to the parsed motion capture data, thereby achieving accurate motion matching. In addition to the skeletal structure, the virtual object template may also include a predefined action animation library. Each action clip in the animation library is associated with a specific motion mode, such as walking, running, jumping, etc. These animation clips are carefully designed to ensure the natural and smooth movement.

[0085] Next, dynamic matching is performed. In the process of matching the motion capture data with the virtual object template, the dynamic matching mechanism is used to ensure that the motion data of the target object can be accurately mapped to the virtual object to achieve natural motion performance. Specifically, the parsed motion capture data is first converted into motion parameters that drive the virtual object. Motion parameters usually include joint rotation angles, bone positions, speeds, etc. These driving parameters are mapped to the skeletal structure of the virtual object in a one-to-one correspondence. For example, the arm raising action in the motion capture data will be mapped to the arm bones of the virtual character to drive the virtual character to make corresponding actions. In order to ensure the continuity and smoothness of the action, motion interpolation calculation can also be performed. The purpose of interpolation calculation is to generate transition frames between two different action states to ensure that there is no abrupt feeling when the action switches. Interpolation calculation is usually based on interpolation algorithms, such as linear interpolation, curve interpolation, etc., to generate natural motion performance through smooth transition. Since there may be certain errors between the motion characteristics of the target object and the virtual object template (for example, the motion amplitude and rhythm are not completely consistent), the error correction algorithm can be used to adjust the motion driving parameters in real time. For example, the motion amplitude can be automatically adjusted according to the body proportion of the virtual character to ensure that the matching result meets the visual expectations. The error correction process is dynamic and will be continuously adjusted according to the real-time motion changes of the target object.

[0086] Step S512: determine a preset interaction event.

[0087] Select preset interaction events based on event information. According to the parsed event information, detect the interaction type, such as click, collision, drag and other specific interaction methods or special effect scenes. According to the detected interaction type, select the preset interaction event corresponding to the interaction type from the virtual scene. Preset interaction events may include collisions between virtual objects and other objects in the scene, changes in object states (such as changes in color or shape), or specific animation effects triggered by users.

[0088] After determining the interactive event, the virtual object is dynamically adjusted according to its current state (including posture, action, and position) to ensure that it can be effectively combined with the interactive event. For example, the posture of the virtual object can be modified according to the interactive event to meet the actual needs of the scene. Then, the dynamically adjusted virtual object is merged with the preset interactive event to obtain optimized rendering input data.

[0089] Step S514, rendering.

[0090] First, multiple virtual object instances are generated through instantiation operations based on the optimized rendering input data. Each instance is assigned independent spatial positioning information and cropping ID, which enables each virtual object to be independently arranged and managed according to different spatial information. Based on the physical position and display parameters of the virtual object, a rendering ID is generated, and the cropping operation is performed by matching the rendering ID with the cropping ID. Subsequently, an orthogonal camera is constructed to set the rendering scene and configure the cropping ratio parameters for the camera. These parameters include the associated settings of the camera's aspect ratio and screen resolution to ensure dynamic adjustment of the viewing angle, thereby ensuring real-time rendering of virtual objects on the screen, and real-time updates based on the cropped rendering data to ensure that the rendering results are highly consistent with the display parameters of the virtual objects.

[0091] After the action, posture, and position data of the virtual object are calculated in real time, the animation of the rendered virtual object is synchronized with the occurrence time of the interactive event. For example, when the user triggers a button, the hand movement of the virtual object will present the animation effect of pressing the button in real time. In addition, the corresponding pre-made animation sequence can be called according to the preset interactive event. For example, when a collision event is detected, the preset explosion effect or scene change animation will be triggered. By calling these pre-made animations, the system can achieve efficient and coherent performance effects in complex interactive events, thereby improving rendering speed and user experience.

[0092] To enhance the flexibility of the system, the rendering process also incorporates the optimization of the vertex offset matrix. When constructing the vertex offset matrix, the vertex coordinates of each set of rendering data are dynamically adjusted with reference to the preset near clipping plane and the offset of the model. In this way, the rendering data can maintain accuracy and continuity during the display process. In addition, the real-time animation of virtual objects is fused with pre-made animation sequences. When a virtual object collides with an object in the scene, its dynamic movements and postures are displayed synchronously with the preset explosion animation or object decomposition animation. Through this fusion method, a complete virtual scene containing virtual objects and corresponding interactive events can be generated, thereby achieving deep interaction and visual optimization between virtual objects and virtual scenes.

[0093] The present application also provides a real-time rendering device, including: a preprocessing module, configured to obtain raw motion capture data from a motion capture device in real time, and preprocess the raw motion capture data to obtain preprocessed data, wherein the preprocessing includes data cleaning, outlier detection and denoising; a filtering module, configured to perform multidimensional filtering on the preprocessed data using a Kalman filtering method based on multi-level and multi-scale analysis to obtain filtered multidimensional data; a parsing module, configured to parse the filtered multidimensional data to obtain target object information and corresponding event information in the filtered multidimensional data; a matching module, configured to classify the target object information, match the classification result with the corresponding model in the virtual scene, and dynamically adjust the posture, action and position of the virtual object according to the matching result; a rendering module, configured to fuse the dynamically adjusted virtual object with a preset interactive event in the virtual scene based on the event information, generate optimized rendering input data, and render the virtual object and the virtual scene based on the optimized rendering input data.

[0094] It should be noted that the real-time rendering device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the real-time rendering device provided in the above embodiment and the real-time rendering method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0095] Figure 8 FIG. 1 shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure. It should be noted that: Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0096] like Figure 8As shown, the electronic device includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0097] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0098] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A real-time rendering method, characterized in that: include: Acquire raw motion capture data from a motion capture device in real time, and preprocess the raw motion capture data to obtain preprocessed data, wherein the preprocessing includes data cleaning, outlier detection and denoising; Using a Kalman filtering method based on multi-level and multi-scale analysis, the pre-processed data is subjected to multi-dimensional filtering to obtain filtered multi-dimensional data; parsing the filtered multidimensional data to obtain target object information and corresponding event information in the filtered multidimensional data; Classifying the target object information, matching the classification results with the corresponding models in the virtual scene, and dynamically adjusting the posture, action and position of the virtual object according to the matching results; Based on the event information, the dynamically adjusted virtual object is merged with a preset interactive event in the virtual scene to generate optimized rendering input data, and based on the optimized rendering input data, the virtual object and the virtual scene are rendered.

2. The method according to claim 1, characterized in that The pre-processed data is subjected to multi-dimensional filtering by using a Kalman filtering method based on multi-level and multi-scale analysis to obtain filtered multi-dimensional data, including: Based on multi-level analysis, extracting time, space and frequency features from the preprocessed data as multi-scale features; Based on the extracted multi-scale features, the Kalman filter method is applied at different scales to predict and update the data at each scale to obtain filtering results at different scales. The filtering results at different scales are fused to generate the filtered multi-dimensional data.

3. The method according to claim 1, characterized in that Parsing the filtered multidimensional data to obtain target object information and corresponding event information in the filtered multidimensional data includes: Identifying and parsing a target object corresponding to a preset object in the virtual scene from the filtered multidimensional data to obtain the target object information; The event detection algorithm is used to analyze the actions and interaction information related to the target object, and the event information is extracted and generated.

4. The method according to claim 1, characterized in that: Classifying the target object information and matching the classification result with the corresponding model in the virtual scene, including: Based on the physical features, action features and interaction information in the target object information, using a classification algorithm to classify the target object; According to the classification result, a preset object model library in the virtual scene is queried to determine a virtual model that matches the classification of the target object as the corresponding model.

5. The method according to claim 1, characterized in that Based on the event information, the dynamically adjusted virtual object is integrated with a preset interactive event in the virtual scene, including: Selecting the preset interaction event in the virtual scene according to the interaction type detected in the event information; The dynamically adjusted posture, action and position of the virtual object and the preset interaction event are integrated.

6. The method according to claim 5, characterized in that Rendering the virtual object and the virtual scene based on the optimized rendering input data includes: Rendering a real-time animation of the virtual object based on the action, posture and position of the virtual object in the optimized rendering input data; Based on the preset interactive event, calling a pre-made animation sequence matching the event information; The pre-made animation sequence is merged with the real-time animation of the virtual object to generate the virtual scene including the virtual object and the corresponding preset interaction event.

7. A real-time rendering device, characterized in that: include: A preprocessing module is configured to obtain raw motion capture data from the motion capture device in real time, and preprocess the raw motion capture data to obtain preprocessed data, wherein the preprocessing includes data cleaning, outlier detection and denoising; A filtering module is configured to perform multi-dimensional filtering on the pre-processed data using a Kalman filtering method based on multi-level and multi-scale analysis to obtain filtered multi-dimensional data; A parsing module is configured to parse the filtered multidimensional data to obtain target object information and corresponding event information in the filtered multidimensional data; A matching module is configured to classify the target object information, match the classification result with the corresponding model in the virtual scene, and dynamically adjust the posture, action and position of the virtual object according to the matching result; The rendering module is configured to fuse the dynamically adjusted virtual object with a preset interactive event in the virtual scene based on the event information, generate optimized rendering input data, and render the virtual object and the virtual scene based on the optimized rendering input data.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

9. A computer device, characterized in that: include: Memory and processor, The memory stores a computer program; The processor is used to execute the computer program stored in the memory, and when the computer program is run, the processor is enabled to execute the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images