Information processing method and device, equipment and storage medium

Abstract

Embodiments of the present disclosure disclose an information processing method, device and equipment, and a storage medium, wherein the method comprises: obtaining acquisition information of an object to be rendered; processing the acquisition information through a first application end to obtain initial texture information of the object to be rendered; wherein the initial texture information carries a texture identifier; processing the initial texture information through a second application end based on the texture identifier to obtain a rendering result of the object to be rendered. Embodiments of the present disclosure can improve rendering efficiency and information processing security, etc.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of computer technology, and in particular to an information processing method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of mobile internet and the increasing demand for Augmented Reality (AR) and Virtual Reality (VR) projects, more and more manufacturers are focusing on the research and development of Software Development Kits (SDKs) to support AR / VR projects. Since the Unity game engine offers relatively comprehensive support for 3D modeling, rendering, and skeletal animation, manufacturers tend to migrate their SDKs to the Unity platform. Therefore, secondary development of the Unity version of the SDK and the reintegration of algorithms into Unity have become major development goals for many manufacturers. However, while third-party vendors or suppliers provide self-developed SDKs (including camera-related interfaces) to facilitate secondary development for customers, directly performing algorithm processing within Unity presents challenges such as long event processing times and lower security. Summary of the Invention

[0003] In view of the above, the present disclosure provides at least one information processing method, apparatus, device, and storage medium.

[0004] The technical solution of this disclosure embodiment is implemented as follows: On one hand, this disclosure provides an information processing method, including: acquiring collection information of an object to be rendered; processing the collection information through a first application terminal to obtain initial texture information of the object to be rendered; wherein the initial texture information carries a texture identifier; and processing the initial texture information through a second application terminal based on the texture identifier to obtain a rendering result of the object to be rendered.

[0005] On the other hand, embodiments of this disclosure provide an information processing apparatus, including: a first acquisition module, configured to acquire collection information of an object to be rendered; a first processing module, configured to process the collection information through a first application terminal to obtain initial texture information of the object to be rendered; wherein the initial texture information carries a texture identifier; and a second processing module, configured to process the initial texture information through a second application terminal based on the texture identifier to obtain a rendering result of the object to be rendered.

[0006] In another aspect, embodiments of this disclosure provide a computer device including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above-described method.

[0007] In another aspect, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above-described method.

[0008] In another aspect, embodiments of this disclosure provide a computer program including computer-readable code, which, when executed in a computer device, causes a processor in the computer device to perform some or all of the steps in the above-described method.

[0009] In another aspect, embodiments of this disclosure provide a computer program product, the computer program product including a non-transitory computer-readable storage medium storing a computer program, wherein when the computer program is read and executed by a computer, it implements some or all of the steps in the above method.

[0010] In this embodiment, the initial texture information of the object to be rendered is obtained by acquiring the collection information of the object to be rendered through a first application terminal; wherein the initial texture information carries a texture identifier; based on the texture identifier, the initial texture information is processed through a second application terminal to quickly and accurately obtain the rendering result of the object to be rendered; thus, by implementing offline drawing on the first application terminal, the initial texture information can be obtained quickly and accurately, so that the second application terminal can simplify the rendering process by sharing the initial texture information generated by the first application terminal, realize the rapid rendering of the object to be rendered, and help improve rendering efficiency and information processing security, etc.

[0011] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0013] Figure 1 A schematic diagram illustrating the implementation flow of the first information processing method provided in this embodiment of the disclosure; Figure 2 A schematic diagram illustrating the implementation flow of the second information processing method provided in this embodiment of the disclosure; Figure 3 A schematic diagram illustrating the implementation flow of the third information processing method provided in this embodiment of the disclosure; Figure 4 A schematic diagram illustrating the implementation flow of the fourth information processing method provided in this embodiment of the disclosure; Figure 5 This is a schematic diagram of the composition structure of an information processing device provided in an embodiment of the present disclosure; Figure 6 This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0015] In the following description, references to "some embodiments" describe a subset of all possible embodiments; however, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict. The terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein.

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this disclosure.

[0017] This disclosure provides an information processing method that can be executed by a processor of a computer device. The computer device refers to any device with information processing capabilities, such as a server, laptop computer, tablet computer, desktop computer, smart TV, set-top box, or mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device). Figure 1 This is a schematic diagram illustrating the implementation flow of an information processing method provided in an embodiment of this disclosure, such as... Figure 1 As shown, the method includes the following steps S101 to S103: Step S101: Obtain the collection information of the object to be rendered.

[0018] Here, the object to be rendered can refer to any object capable of being rendered, such as virtual objects or physical objects, and can include vehicles, animals, and people. Acquired information can refer to information obtained by using acquisition devices such as cameras and LiDAR to capture images of the object to be rendered, perform detection, and other processing, such as image information, point cloud information, and motion information. For example, using a pre-set camera or other image acquisition device to capture images of the object to be rendered yields the acquired information of the object; alternatively, it can receive a video stream uploaded by a user that contains vehicle objects, identify the vehicle object as the object to be rendered, and use the video stream containing the vehicle object as the acquired information of the object to be rendered.

[0019] Step S102: The collected information is processed by the first application terminal to obtain the initial texture information of the object to be rendered.

[0020] Here, the first application terminal can refer to the application terminal that processes the acquired information. The application terminal can be a runtime environment capable of performing detection, rendering, and other processing; for example, the first application terminal could be a programming language (Java) application terminal. The initial texture information can be texture information obtained in the first application terminal. Texture information can refer to information used to represent the surface structure organization and arrangement attributes of an object's surface that have slowly changing or periodically changing characteristics, such as statistical texture features or signal processing texture features. The initial texture information carries a texture identifier, which can be an identifier used to characterize the initial texture information. Accessing this texture identifier allows for the reading and / or processing of the initial texture information.

[0021] For example: the object to be rendered is a human figure, and the collected information is a human figure image containing the human figure. Based on the human figure image, the key points of the human figure are identified by the key point recognition algorithm in the first application terminal to obtain the position coordinates of the key points of the human figure in the human figure image. Based on the position coordinates of the key points, the human figure is rendered by the rendering algorithm in the first application terminal to obtain initial texture information, and a texture identifier corresponding to the initial texture information is generated based on the processing space corresponding to the initial texture information in the first application terminal. The processing space corresponding to the first application terminal can be a central processing unit (CPU), etc.

[0022] In some embodiments, the first application can process the collected information using offline rendering to obtain the initial texture information of the object to be rendered, and generate a texture identifier corresponding to the initial texture information, so that the second application can share the initial texture information based on the texture identifier.

[0023] Step S103: Based on the texture identifier, the initial texture information is processed by the second application terminal to obtain the rendering result of the object to be rendered.

[0024] Here, the second application can refer to the application that processes the initial texture information. The programming language corresponding to the first application is different from that of the second application. For example, the second application might be a Unity application, which corresponds to the C# programming language, or the first application might be a Java application, which corresponds to the Java programming language. Rendering can refer to adjusting the lighting, shadows, roughness, and surface texture of the object to be rendered. Different rendering methods can be used to render the object, and different rendering methods can refer to using different rendering parameters to obtain different rendering effects. The rendering result of the object to be rendered can refer to the rendered object with special effects, such as a vehicle object with fire effects, smoke effects, or lightning effects. For example, the second application obtains the texture identifier from the first application so that it can directly obtain the initial texture information from the processing space corresponding to the first application (i.e., texture sharing). The initial texture information is then used as the rendering source or external texture of the object to be rendered, and the rendering algorithm in the second application is used to quickly and easily render the object to be rendered based on the initial texture information.

[0025] In this embodiment, the initial texture information of the object to be rendered is obtained by acquiring the collection information of the object to be rendered through a first application terminal; wherein the initial texture information carries a texture identifier; based on the texture identifier, the initial texture information is processed through a second application terminal to quickly and accurately obtain the rendering result of the object to be rendered; thus, by implementing offline drawing on the first application terminal, the initial texture information can be obtained quickly and accurately, so that the second application terminal can simplify the rendering process by sharing the initial texture information generated by the first application terminal, realize the rapid rendering of the object to be rendered, and help improve rendering efficiency and information processing security, etc.

[0026] This disclosure provides an information processing method, wherein the collected information includes: inertial measurement information and image information; such as Figure 2 As shown, the method includes the following steps S201 to S206: Step S201 corresponds to the aforementioned step S101, and can be implemented with reference to the specific implementation of the aforementioned step S101; Step S206 corresponds to the aforementioned step S103, and can be implemented with reference to the specific implementation of the aforementioned step S103.

[0027] Step S202: The acquired information is split to obtain the inertial measurement information and the image information.

[0028] Here, we will use the example of capturing information by a camera on the object to be rendered. This captured information can include inertial measurement information and image information. Inertial measurement information refers to the three-axis attitude angles and acceleration information of the camera or the object to be rendered, obtained using an Inertial Measurement Unit (IMU) (i.e., a velocity sensor). Image information refers to the images captured by the camera, such as color images or grayscale matrices. Since the captured information obtained by the first application is a whole, different parts of the captured information can have different functions. The captured information can be broken down into different parts. For example, the first part of the captured information can be used to determine the orientation of the object to be rendered, while the second part can be used to determine the external attributes of the object to be rendered.

[0029] In some embodiments, when the acquired information is a string, a first string at a first preset position can be read from the acquired information, and the first string can be identified as inertial measurement information; a second string at a second preset position can be read from the acquired information, and the second string can be identified as image information, etc. When the acquired information consists of multiple data packets, the header of each data packet is read; based on the information identifier carried in each header, the part to which each data packet belongs is determined; wherein, the information identifier is used to characterize the part to which the data packet belongs; for example, if the information identifier of the first data packet is determined to be a first identifier, then the first data packet is determined to be inertial measurement information; if the information identifier of the second data packet is determined to be a second identifier, then the second data packet is determined to be image information, etc.

[0030] Inertial measurement information can carry a first timestamp, and image information can carry a second timestamp. For example, inertial measurement information includes multiple segments, with the first timestamp corresponding to the first segment of inertial measurement information being 1 second, the first timestamp corresponding to the second segment of inertial measurement information being 2 seconds, and so on. Image information includes multiple images, with the second timestamp corresponding to the first image being 1 second, the second image being 1.5 seconds, the third image being 2 seconds, and so on.

[0031] Step S203: Synchronize the inertial measurement information and the image information based on the first timestamp and the second timestamp to obtain the synchronization result between the inertial measurement information and the image information.

[0032] Here, since both inertial measurement information and image information are continuous in time, accurate synchronization between them is necessary to accurately render the object being rendered. The synchronization result characterizes the temporal correspondence between inertial measurement information and image information. For example, if the acquired inertial measurement information is a first value, the corresponding image information is the first image. For instance, if the inertial measurement information consists of multiple segments, with the first segment corresponding to a first timestamp of 1 second, the second segment to a first timestamp of 2 seconds, and so on; and the image information consists of multiple images, with the first image corresponding to a second timestamp of 1 second, the second image to a timestamp of 1.5 seconds, and the third image to a timestamp of 2 seconds, then the synchronization result can be characterized as the first segment of inertial measurement information corresponding to the first image, and the second segment corresponding to the third image.

[0033] Step S204: Based on the synchronization result, process the inertial measurement information and the image information respectively to obtain the evaluation result of the object to be rendered.

[0034] Here, the evaluation result can refer to the information used to render the object to be rendered. Taking a virtual character performing a dance as an example, the evaluation result could be the type of the virtual character's dance movement. For example, using a multi-threaded approach, the virtual character's orientation information is determined based on the relational measurement information at the first moment, and the key point information of the virtual character is determined based on the image information at the first moment. Then, using a trained dance recognition model, the type of the virtual character's dance movement is determined based on the orientation information and key point information. The dance recognition model can be a pre-set machine learning model, such as a neural network model used to perform movement type recognition.

[0035] Step S205: Draw the evaluation results to obtain the initial texture information of the object to be rendered.

[0036] Here, drawing can be understood as the process of obtaining a visual image based on abstract information. Drawing can include drawing layer attributes such as color, shadow, mask, and rounded corners. For example, using the drawing algorithm preset in the first application, the object to be rendered is drawn based on information such as abstract evaluation results to obtain the initial texture information of the visualized object to be rendered.

[0037] In this embodiment of the disclosure, different types of inertial measurement information and image information can be obtained by splitting the acquired information. Then, the synchronously processed inertial measurement information and image information can be processed separately in a multi-threaded manner to quickly and accurately obtain evaluation information, so that the first application can obtain initial texture information based on the evaluation information.

[0038] In some embodiments, the collected information is located at the first application layer; step S202 above may include the following steps S2021 to S2022: Step S2021: Based on a preset mapping relationship, the collected information in the first application layer is transmitted to the second application layer.

[0039] Here, the programming languages ​​used to develop the first application layer and the second application layer are different. For example, the first application is the Java side in Android, the first application layer is the Java application layer, and the second application layer is the C & C++ layer. The interaction between the first and second application layers requires predefined relationships, such as interfaces (Java Native Interface, JNI). The JNI layer is a function mapping layer used to represent the predefined relationships between the Java application layer and the C & C++ layer.

[0040] In step S2022, in the second application layer, the acquired information is split and processed using a preset multi-threaded method to obtain the inertial measurement information and the image information.

[0041] For example, the collected information is transmitted from the Java application layer of the first application to the C application layer. Since the splitting algorithm can be quickly and easily set in the C application layer, the splitting algorithm preset by the C programming language is used in the C application layer to perform multi-threaded splitting processing on the collected information to obtain inertial measurement information and image information.

[0042] In this embodiment of the disclosure, by using a preset multi-threading method in the second application layer to split and process the collected information, inertial measurement information and image information can be obtained simply and quickly.

[0043] In some embodiments, the object to be rendered is a human body object, and the evaluation result includes the three-dimensional point coordinates and position information of the skeleton corresponding to the human body object; the above step S204 may include the following step S2041: Step S2041: Based on the synchronization result, key point recognition processing is performed on the inertial measurement information and the image information respectively to obtain the three-dimensional point coordinates and bone node position information of the skeleton corresponding to the human object.

[0044] Here, inertial measurement information and image information at the same moment can be input into the trained key point recognition model to obtain the three-dimensional point coordinates and bone node position information of the skeleton corresponding to the human object. For example, the three-dimensional point coordinates of the skeleton are composed of the coordinates of one or more intermediate points of the bones, and the position information of the bone nodes includes the coordinates of the joints between two bones.

[0045] In this embodiment of the disclosure, by performing key point recognition processing on inertial measurement information and image information, the three-dimensional point coordinates and skeletal node position information of the skeleton corresponding to the human object can be obtained quickly and accurately.

[0046] In some embodiments, step S205 may include the following steps S2051 to S2054: Step S2051: Obtain the current context of the preset rendering thread.

[0047] Here, in the rendering software (OpenGL) of the first application, a preset rendering thread can be used to process the evaluation results, such as drawing, to obtain the initial texture information of the object to be rendered. The context of the rendering thread can refer to a data structure used to store information such as rendering parameters. OpenGL can save various state information of the rendering thread in the context, including vertex information, texture information, compiled shader information, etc. Subsequent drawing work is based on the information in the rendering thread's context. When a context is destroyed, the corresponding OpenGL rendering work will also end. All threads can be traversed, and by reading the identifier of each thread, the thread type matching the identifier can be determined, thereby identifying the rendering thread. Then, the current context corresponding to that rendering thread can be determined from the preset storage space.

[0048] Step S2052: Obtain the memory object determined based on the current context of the rendering thread.

[0049] Here, a memory object can refer to the storage space where the first application performs off-screen drawing or off-screen rendering. For example, when creating a rendering thread, a storage space that satisfies rendering parameters and other rendering attributes is preset, and this storage space is determined as the memory object determined by the current context of the rendering thread.

[0050] Step S2053: Bind the memory object and the rendering thread to obtain the bound rendering thread.

[0051] Here, the memory object determined by the current context of the rendering thread can be bound to the rendering thread, so that the rendering thread can perform rendering or drawing operations in the memory object, thus making the rendering or drawing operations effective.

[0052] Step S2054: Use the bound rendering thread to draw the evaluation result to obtain the initial texture information of the object to be rendered.

[0053] Here, rendering includes at least one of the following: point rendering, line logic rendering, and model transformation logic rendering. Point rendering can refer to the rendering of pixels on the surface of the object to be rendered; line logic rendering can be understood as fitting the pixels on the surface of the object to be rendered to obtain the lines on the surface of the object to be rendered; model transformation logic rendering can be understood as drawing the three-dimensional structure of the object to be rendered based on the lines on the surface of the object to be rendered. Model transformation can include linear transformations and nonlinear transformations. For example, in the first application, using the bound rendering thread, the object to be rendered is first rendered by point rendering, then by line logic rendering, and finally by model transformation logic rendering to obtain the initial texture information of the object to be rendered.

[0054] In this embodiment of the disclosure, the memory object is determined by determining the current context of the rendering thread, and then the memory object is bound to the rendering thread so that rendering or drawing processes can take effect.

[0055] In some embodiments, the following steps S211 to S212 may be included after step S2054: Step S211: Store the initial texture information of the object to be rendered into the frame buffer corresponding to the memory object.

[0056] Here, a framebuffer can refer to storage space used to store initial texture information. For example, after drawing and / or rendering an object to be rendered in a memory object, the initial texture information is obtained and stored in a preset framebuffer. A pre-defined mapping between the rendering thread identifier and the memory object and framebuffer can be established. Given a defined rendering thread identifier, the memory object and framebuffer matching the rendering thread identifier are determined based on this mapping.

[0057] Step S212: Determine the texture identifier that matches the frame buffer.

[0058] Here, the texture identifier can refer to the identifier corresponding to the initial texture information. Each application can read and call the corresponding initial texture information based on this texture identifier. The texture identifier corresponding to the initial texture information can also be generated based on the position information of the framebuffer.

[0059] In this embodiment of the disclosure, by determining the frame buffer corresponding to the memory object, the texture identifier matching the frame buffer can be determined quickly and accurately.

[0060] This disclosure provides an information processing method, such as... Figure 3 As shown, the method includes the following steps S301 to S307: Steps S301 to S302 correspond to the aforementioned steps S101 to S102, and can be implemented with reference to the specific implementation of the aforementioned steps S101 to S102.

[0061] Step S303: Obtain the target startup class and the corresponding runtime resources from the second application.

[0062] Here, the target startup class can refer to the class file used to launch the second application, and runtime resources can refer to resource files such as function libraries involved in the target startup class. Taking the first application as a Java application and the second application as a Unity application for illustrative purposes, the target startup class can be the base class startup class file (UnityPlayerActivity.java), and runtime resources can be resources such as the Java Archive File (Jar) (i.e., the Res package) and resource files (Res) that UnityPlayerActivity.java depends on.

[0063] Step S304: Based on the target startup class and the runtime resources corresponding to the target startup class, determine the first module of the first application.

[0064] Here, the first module can refer to the basic model in the first application that is used to process the collected information, such as compiling the target startup class and the runtime resources corresponding to the target startup class to obtain the first model.

[0065] Step S305: Obtain the target interface and target library file from the first application.

[0066] Here, the target interface can refer to the interface through which the first application interacts with the second application, and the target library file can refer to the library file in the first application that processes the collected information. The target interface and target library file can be determined based on pre-defined requirements for processing the collected information.

[0067] Step S306: Based on the target interface and the target library file, determine the second module of the first application.

[0068] Here, the second module can refer to the architectural functional module that processes the acquired information. The second module can include multiple units, each of which performs partial processing of the acquired information. For example, a camera unit is used to acquire the acquired information of the object to be rendered; a data processing unit is used to perform processing such as splitting the acquired information; an algorithm processing unit is used to process inertial measurement information and image information to obtain the evaluation result of the object to be rendered; and a rendering unit is used to draw the evaluation result to obtain the initial texture information of the object to be rendered.

[0069] Step S307: When the first module is started, the second module processes the collected information to obtain the initial texture information of the object to be rendered.

[0070] Here, the first module can be started first. If the first module is started successfully, the camera unit, data processing unit, algorithm processing unit, and rendering unit of the second module can be used to process the acquired information in sequence to obtain the initial texture information of the object to be rendered.

[0071] In this embodiment of the disclosure, by presetting a first module and a second module, and then using the first module and the second module to process the collected information, the initial texture information of the object to be rendered is obtained. This helps to achieve modularity, reduce the coupling between operations, and simplify the complexity of cross-platform development.

[0072] The following describes the application of the information processing method provided in this disclosure in a real-world scenario, using a scenario where the first application is a Java application and the second application is a Unity application as an example. Addressing the issue in related technologies where Unity development avoids existing camera interfaces, resulting in an inability to meet the needs of various camera types, this disclosure addresses this problem. In this embodiment, a Unity plugin is constructed. A first module is generated based on the target startup class and its corresponding runtime resources in the Unity application. Rendering units, data processing units, etc., are generated based on the target interfaces and target library files in the first application. The rendering units, data processing units, camera units, and algorithm processing units are then packaged together to obtain a second module with functional architecture. For the Unity application, the plugin package is exposed to the Unity application via a Java interface, facilitating the rendering of evaluation results, texture sharing, and data transmission on the Java application. This contributes to modular, decoupled, and cross-platform development.

[0073] This disclosure provides an information processing method, such as... Figure 4 As shown, the method may include the following steps S401 to S415: Step S401: Start the Unity application.

[0074] Here, you can launch Unity applications on the Android system, such as Unity game applications.

[0075] Step S402: Obtain the binary archive file of the plugin.

[0076] Here, the binary archive (Android Archive, AAR) can refer to a file in an Android library project. For example, the first and second modules can be packaged to obtain a binary archive, and a Java application interface can be exposed. The Unity application can use this Java application interface to call the binary archive and perform other processing, and put the binary archive as a plugin into the project created in the Unity application.

[0077] Step S403: Obtain the camera unit from the binary archive file.

[0078] Here, the binary archive file includes a first module and a second module. The second module includes a camera unit, a data processing unit, etc. Based on a preset unit or module identifier, the function library and other files corresponding to the camera unit can be obtained from the binary archive file.

[0079] Step S404: Initialize the camera unit.

[0080] Here, the camera unit can replace the native Unity camera in the Android system, merging third-party cameras and system cameras together, and uniformly exposing them to Java applications through external interfaces. The camera unit can perform processing such as camera initialization, camera list, and camera preview data stream configuration. For example, the camera unit can be initialized and resource configuration can be completed using the preset initialization function in the camera unit.

[0081] Step S405: Acquire information using the camera unit.

[0082] Here, the camera unit can be used to control the camera to capture images of the object to be rendered, thereby obtaining the collected information.

[0083] Step S406: The collected information is split using the data processing unit to obtain inertial measurement information and image information.

[0084] Here, the data processing unit can include a Java application layer, a JNI application layer, a C&C++ application layer, and encapsulation files for the splitting algorithm. The acquired information, such as image information and inertial measurement information, required by the splitting algorithm is transmitted from the Java application layer to the JNI application layer for mapping processing, and then to the C&C++ application layer. A multi-threaded processing approach can be used, where the acquired information is split in the C application layer to obtain image information and inertial measurement information (e.g., YUV format image information). Furthermore, a timestamp synchronization mechanism is added during the splitting process to determine the synchronization result of the image information and inertial measurement information, which helps improve data processing efficiency. For example, the acquired information is exported from the camera unit so that the data processing unit obtains a data buffer in native data format (NV21); this data buffer represents the acquired information; the data buffer is then passed to the C application layer through the JNI application layer to complete the data splitting and realize the mapping from the Java application layer to the C application layer.

[0085] Step S407: The algorithm processing unit processes the relationship measurement information and image information to obtain the evaluation result.

[0086] Here, the algorithm processing unit can include encapsulating the algorithm library, redefining the interface, and re-exporting new interfaces, which helps improve the security of the algorithm library and reduce the impact of decompilation. The collected information is split and transmitted to the algorithm processing unit. After processing, the algorithm processing unit outputs the evaluation results corresponding to this frame of image. For example, if the object to be rendered is a human figure, the evaluation information includes the 3D point coordinates of the human skeleton and the skeletal node information of the model, estimated through a keypoint recognition algorithm.

[0087] Step S408: Generate a data packet carrying the evaluation results.

[0088] Here, the evaluation results can be packaged to generate a data package, and the data package can be transmitted to the rendering unit of the Java application layer for drawing, rendering and other processing.

[0089] Step S409: Perform a rendering self-loop process based on the data packet to obtain the initial texture information.

[0090] Here, the rendering unit in the second module can be used to perform a rendering self-loop process to obtain the initial texture information.

[0091] Step S410: Initialize the preset rendering function.

[0092] Here, before step S409, the rendering unit can be initialized, for example, by using a rendering function (Java Render) to obtain resources for compilation.

[0093] Step S411: Initialize the preset window function.

[0094] Here, before step S409, initialization can also be performed through window functions (Java EGL) to provide a rendering environment for the library file (OpenGL ES) to be called.

[0095] Step S412: Create a memory object, store the initial texture information in the frame buffer managed by the memory object, and generate a texture identifier.

[0096] Here, after the rendering unit is initialized, it can be called via Unity scripts. For example: obtain the current context information (GLContext) of the rendering thread (OpenGL), and create a shared context (SharedContext) using the obtained context; then create a hardware device-related display object (Display) (i.e., a corresponding hardware resource object) using the shared context; then create an off-screen rendering memory object (EglCreatePbufferSurface) using the new display object, and then bind the memory object to the current rendering thread to enable drawing; the rendering unit draws the evaluation information, such as drawing point, line logic, model transformation logic, etc., to obtain the initial texture information; finally, the initial texture information is drawn into the framebuffer managed by the off-screen memory object, and a texture identifier corresponding to the framebuffer is generated. This texture identifier is used to share with the Unity application as a rendering source.

[0097] Step S413: Obtain the texture identifier.

[0098] Here, the Java application can send the texture identifier to the Unity application so that the Unity application can share the initial texture information.

[0099] Step S414: Perform self-initialization on the Unity application.

[0100] Here, you can use the preset initialization function in the Unity application to perform initialization.

[0101] Step S415: Based on the texture identifier, obtain the initial texture information, and perform rendering processing based on the initial texture information to obtain the rendering result.

[0102] Here, after obtaining the shared initial texture information through the AAR package, the Unity application can directly create an external texture. For example, the initial texture information can be determined as the external texture of the object to be rendered. Based on the external texture, the object to be rendered can be rendered directly, which helps to reduce the heavy rendering logic in the Unity application.

[0103] Compared to related technologies that directly perform algorithm processing within the Unity application using traditional development methods, such as developing for single-channel color (Red, Green, Blue, RGB) images or dual-channel infrared (IR) images, several problems arise. For example, regarding security, it's impossible to encrypt the algorithm library and encapsulated information, increasing the risk of data leakage. Secondly, in Unity application development, all script execution occurs in the rendering thread, but some time-consuming, multi-frame algorithms are not suitable for direct rendering, and this impacts frame rate when using multi-threaded rendering in Unity applications. Furthermore, regarding camera handling, the Unity application's camera module (Web Camera) cannot meet the needs of various camera types, such as USB Camera architecture (Universal Serial Bus, USB) and Camera2 architecture cameras; and the solutions for adjusting camera parameters, setting preview data types (such as NV21, YV12, RGB565, etc.), and camera lifecycle management are incomplete. Finally, regarding interface operations, the Unity application takes over all Android touch messages, causing external controls (Views) added on the Android system to be unable to receive data.

[0104] The proposed data integration, offline rendering, and shared texture solutions in this disclosure can help improve program running efficiency and data security. For example, in AR / VR projects, the information processing methods in this disclosure can be used to render the object to be rendered. At the same time, by setting the camera unit in the second module, the camera can be controlled more conveniently and accurately, thereby improving development efficiency.

[0105] Based on the foregoing embodiments, this disclosure provides an information processing device, which includes various units and modules included in each unit, and can be implemented by a processor in a computer device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0106] Figure 5This is a schematic diagram of the composition structure of an information processing device provided in an embodiment of the present disclosure, such as... Figure 5 As shown, the information processing device 500 includes: a first acquisition module 510, a first processing module 520, and a second processing module 530, wherein: The first acquisition module 510 is used to acquire the collection information of the object to be rendered; The first processing module 520 is used to process the collected information through the first application terminal to obtain the initial texture information of the object to be rendered; wherein the initial texture information carries a texture identifier; The second processing module 530 is used to process the initial texture information through a second application terminal based on the texture identifier to obtain the rendering result of the object to be rendered.

[0107] In some embodiments, the acquired information includes: inertial measurement information and image information; the first processing module is further configured to: split the acquired information to obtain the inertial measurement information and the image information; wherein the inertial measurement information carries a first timestamp and the image information carries a second timestamp; perform synchronization processing on the inertial measurement information and the image information based on the first timestamp and the second timestamp to obtain a synchronization result between the inertial measurement information and the image information; process the inertial measurement information and the image information respectively based on the synchronization result to obtain an evaluation result of the object to be rendered; and draw the evaluation result to obtain the initial texture information of the object to be rendered.

[0108] In some embodiments, the acquired information is located in a first application layer; the first processing module is further configured to: transmit the acquired information in the first application layer to a second application layer based on a preset mapping relationship; wherein the programming language used to develop the first application layer is different from the programming language used to develop the second application layer; in the second application layer, the acquired information is split and processed using a preset multi-threading method to obtain the inertial measurement information and the image information.

[0109] In some embodiments, the object to be rendered is a human body object, and the evaluation result includes the three-dimensional point coordinates and position information of the skeleton corresponding to the human body object; the first processing module is further configured to: perform key point recognition processing on the inertial measurement information and the image information respectively based on the synchronization result to obtain the three-dimensional point coordinates and position information of the skeleton corresponding to the human body object.

[0110] In some embodiments, the first processing module is further configured to: obtain the current context of a preset rendering thread; obtain a memory object determined based on the current context of the rendering thread; bind the memory object and the rendering thread to obtain a bound rendering thread; and use the bound rendering thread to draw the evaluation result to obtain the initial texture information of the object to be rendered; wherein the drawing includes at least one of the following: drawing points, drawing line logic, and drawing model transformation logic.

[0111] In some embodiments, the apparatus further includes: a storage module for storing initial texture information of the object to be rendered into a frame buffer corresponding to the memory object; and a first determination module for determining a texture identifier that matches the frame buffer.

[0112] In some embodiments, the apparatus further includes: a second acquisition module, configured to acquire a target startup class and runtime resources corresponding to the target startup class from the second application terminal; a second determination module, configured to determine a first module of the first application terminal based on the target startup class and runtime resources corresponding to the target startup class; a third acquisition module, configured to acquire a target interface and a target library file from the first application terminal; a third determination module, configured to determine a second module of the first application terminal based on the target interface and the target library file; and the second processing module, further configured to: when the first module is started, process the acquired information through the second module to obtain the initial texture information of the object to be rendered.

[0113] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this disclosure can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0114] It should be noted that, in the embodiments of this disclosure, if the above-described information processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this disclosure are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0115] This disclosure provides a computer device including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0116] This disclosure provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium may be transient or non-transient.

[0117] This disclosure provides a computer program including computer-readable code, wherein when the computer-readable code is executed in a computer device, a processor in the computer device performs some or all of the steps in the above-described method.

[0118] This disclosure provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0119] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referenced interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0120] It should be noted that, Figure 6 This is a schematic diagram of a hardware entity of a computer device in an embodiment of this disclosure, such as... Figure 6 As shown, the hardware entity of the computer device 600 includes: a processor 601, a communication interface 602, and a memory 603, wherein: Processor 601 typically controls the overall operation of computer device 600.

[0121] Communication interface 602 enables computer devices to communicate with other terminals or servers via a network.

[0122] The memory 603 is configured to store instructions and applications executable by the processor 601, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 601 and various modules in the computer device 600. It can be implemented using flash memory or random access memory (RAM). Data transfer between the processor 601, the communication interface 602, and the memory 603 can be performed via bus 604.

[0123] It should be understood that the phrase "an embodiment" or "one embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this disclosure. Therefore, "in one embodiment" or "one embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above embodiments of this disclosure are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0124] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0125] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0126] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0127] In addition, each functional unit in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0128] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0129] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0130] The methods disclosed in the several method embodiments provided in this disclosure can be arbitrarily combined without conflict to obtain new method embodiments.

[0131] If the embodiments of this disclosure involve personal information, the products using these embodiments have clearly informed the users of the personal information processing rules and obtained their voluntary consent before processing the personal information. If the embodiments of this disclosure involve sensitive personal information, the products using these embodiments have obtained the individual's separate consent before processing the sensitive personal information, and the requirement of "express consent" is also met.

[0132] The above description is merely an embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An information processing method, characterized in that, include: Acquire the acquisition information of the object to be rendered; wherein, the acquisition information includes: inertial measurement information and image information; The acquired information is split to obtain the inertial measurement information and the image information; wherein the inertial measurement information carries a first timestamp and the image information carries a second timestamp. Based on the first timestamp and the second timestamp, the inertial measurement information and the image information are synchronized to obtain the synchronization result between the inertial measurement information and the image information; Based on the synchronization results, the inertial measurement information and the image information are processed respectively to obtain the evaluation results of the object to be rendered; The evaluation results are plotted to obtain the initial texture information of the object to be rendered; wherein, the initial texture information carries a texture identifier; Based on the texture identifier, the initial texture information is processed by the second application terminal to obtain the rendering result of the object to be rendered.

2. The method according to claim 1, characterized in that, The acquired information is located in the first application layer; the process of splitting the acquired information to obtain the inertial measurement information and the image information includes: Based on a preset mapping relationship, the collected information in the first application layer is transmitted to the second application layer; wherein the programming language used to develop the first application layer is different from the programming language used to develop the second application layer. In the second application layer, a preset multi-threaded method is used to split and process the acquired information to obtain the inertial measurement information and the image information.

3. The method according to claim 1, characterized in that, The object to be rendered is a human body object, and the evaluation result includes the three-dimensional point coordinates and the position information of the bone nodes of the skeleton corresponding to the human body object. Based on the synchronization result, the inertial measurement information and the image information are processed respectively to obtain the evaluation result of the object to be rendered, including: Based on the synchronization results, key point recognition processing is performed on the inertial measurement information and the image information respectively to obtain the three-dimensional point coordinates and skeletal node position information of the skeleton corresponding to the human object.

4. The method according to claim 1, characterized in that, The step of rendering the evaluation results to obtain the initial texture information of the object to be rendered includes: Get the current context of the preset rendering thread; Obtain the memory object determined based on the current context of the rendering thread; The memory object and the rendering thread are bound together to obtain the bound rendering thread; The evaluation result is drawn using the bound rendering thread to obtain the initial texture information of the object to be rendered; wherein the drawing includes at least one of the following: drawing points, drawing line logic, and drawing model transformation logic.

5. The method according to claim 4, characterized in that, The method further includes: The initial texture information of the object to be rendered is stored in the frame buffer corresponding to the memory object; Determine the texture identifier that matches the framebuffer.

6. The method according to claim 1, characterized in that, The method further includes: Obtain the target startup class and the corresponding runtime resources from the second application. Based on the target startup class and the runtime resources corresponding to the target startup class, the first module of the first application is determined; Obtain the target interface and target library file from the first application. Based on the target interface and the target library file, the second module of the first application is determined; When the first module is activated, the second module processes the collected information to obtain the initial texture information of the object to be rendered.

7. An information processing device, characterized in that, include: The first acquisition module is used to acquire the collection information of the object to be rendered; wherein, the collection information includes: inertial measurement information and image information; A first processing module is used to split the acquired information to obtain inertial measurement information and image information; wherein the inertial measurement information carries a first timestamp and the image information carries a second timestamp; to perform synchronization processing on the inertial measurement information and the image information based on the first timestamp and the second timestamp to obtain a synchronization result between the inertial measurement information and the image information; to process the inertial measurement information and the image information respectively based on the synchronization result to obtain an evaluation result of the object to be rendered; and to draw the evaluation result to obtain the initial texture information of the object to be rendered; wherein the initial texture information carries a texture identifier. The second processing module is used to process the initial texture information through a second application terminal based on the texture identifier to obtain the rendering result of the object to be rendered.

8. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, the computer program product comprising a non-transitory computer-readable storage medium storing a computer program, characterized in that, When the computer program is read and executed by a computer, it implements the steps of the method according to any one of claims 1 to 6.

Images