Data rendering method and related products

Abstract

The application discloses a data rendering method and related products. The method comprises the following steps: rendering 3DGS data by using a rendering pipeline corresponding to the operating system type of a first device to obtain a space photo, wherein the 3DGS data is generated based on a two-dimensional picture, the first device supports calling a 3DGS rendering library, the 3DGS rendering library comprises a plurality of rendering pipelines corresponding to a plurality of operating system types respectively, and a plurality of rendering view components corresponding to the plurality of operating system types respectively, and the operating system type of the first device is one of the plurality of operating system types; and displaying the space photo by using the rendering view component corresponding to the operating system type of the first device. The application is used for improving the rendering efficiency of the space photo and expanding the application scene of the space photo.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data rendering method and related products. Background Technology

[0002] Spatial photography refers to the three-dimensional (3D) display of two-dimensional (2D) images. It breaks through the planar limitations of traditional images, recording detailed depth information and presenting dynamic parallax effects, achieving a fundamental leap from "planar viewing" to "spatial perception," bringing users a more immersive and interactive content experience. However, currently, the rendering and display of spatial photos is limited to certain proprietary devices. For example, it may be rendered and displayed in specific applications (such as photo albums) on terminal devices that support specific operating system types, and is not applicable to terminal devices that support other operating system types.

[0003] Furthermore, if a user wants to display a 2D image in other applications (such as social media) on a proprietary device in 3D, they can first save the image from the social media to a specific application and then perform localized 3D rendering in that application. This process is cumbersome and reduces the rendering efficiency of spatial photos. Summary of the Invention

[0004] This application provides a data rendering method and related products, wherein the related products include a data rendering device, an electronic device, and a computer-readable storage medium, which can improve the rendering efficiency of spatial photographs and expand the application scenarios of spatial photographs.

[0005] Firstly, a data rendering method is provided, the method comprising: The rendering pipeline corresponding to the operating system type of the first device is used to render three-dimensional Gaussian splatting (3DGS) data to obtain a spatial photograph. The 3DGS data is generated based on two-dimensional images. The first device supports calling the 3DGS rendering library, which includes rendering pipelines corresponding to multiple operating system types and rendering view components corresponding to the multiple operating system types. The operating system type of the first device is one of the multiple operating system types. The spatial photo is displayed using the rendering view component corresponding to the operating system type of the first device.

[0006] In any embodiment of this application, the rendering pipeline corresponding to the operating system type of the first device is obtained after initialization based on the target view parameters, wherein the target view parameters are the view parameters of the rendering view component corresponding to the operating system type of the first device.

[0007] In any embodiment of this application, the 3DGS rendering library further includes a pose acquisition component, which is used to acquire the device pose of the first device. The device pose is used to control camera viewpoint parameters, and the camera viewpoint parameters and the 3DGS data are used to obtain the spatial photograph.

[0008] In any embodiment of this application, the 3DGS rendering library includes a 3DGS data processing component, which is used to determine auxiliary rendering data corresponding to the 3DGS data. The auxiliary rendering data and the 3DGS data are used to obtain the spatial photograph.

[0009] In any embodiment of this application, the auxiliary rendering data includes at least one of the following: shader calculation results, rasterization processing results, physical data mixing results, or camera viewpoint parameters.

[0010] In conjunction with any embodiment of this application, the 3DGS data processing component is further configured to acquire the 3DGS data.

[0011] In conjunction with any embodiment of this application, the method further includes: The device receives shared data from a second device, the shared data including the 3DGS data, the second device supports calling the 3DGS rendering library, and the operating system type of the second device is one of the plurality of operating system types.

[0012] In any embodiment of this application, the shared data further includes auxiliary rendering data corresponding to the 3DGS data, and the auxiliary rendering data and the 3DGS data are used to obtain the spatial photograph.

[0013] In any embodiment of this application, the operating system type of the second device is different from that of the first device.

[0014] Secondly, a data rendering apparatus is provided, the data rendering apparatus comprising: The rendering unit is used to render 3D Gaussian splash 3DGS data using the rendering pipeline corresponding to the operating system type of the first device to obtain a spatial photograph. The first device supports calling the 3DGS rendering library, which includes rendering pipelines corresponding to multiple operating system types and rendering view components corresponding to the multiple operating system types. The operating system type of the first device is one of the multiple operating system types.

[0015] The display unit is used to display the spatial photo using a rendering view component corresponding to the operating system type of the first device.

[0016] In any embodiment of this application, the rendering pipeline corresponding to the operating system type of the first device is obtained after initialization based on the target view parameters, wherein the target view parameters are the view parameters of the rendering view component corresponding to the operating system type of the first device.

[0017] In any embodiment of this application, the 3DGS rendering library further includes a pose acquisition component, which is used to acquire the device pose of the first device. The device pose is used to control camera viewpoint parameters, and the camera viewpoint parameters and the 3DGS data are used to obtain the spatial photograph.

[0018] In any embodiment of this application, the 3DGS rendering library includes a 3DGS data processing component, which is used to determine auxiliary rendering data corresponding to the 3DGS data. The auxiliary rendering data and the 3DGS data are used to obtain the spatial photograph.

[0019] In any embodiment of this application, the auxiliary rendering data includes at least one of the following: shader calculation results, rasterization processing results, physical data mixing results, or camera viewpoint parameters.

[0020] In conjunction with any embodiment of this application, the 3DGS data processing component is further configured to acquire the 3DGS data.

[0021] In conjunction with any embodiment of this application, the apparatus further includes: A receiving unit is configured to receive shared data from a second device, the shared data including the 3DGS data, the second device supporting the use of the 3DGS rendering library, and the operating system type of the second device being one of the plurality of operating system types.

[0022] In any embodiment of this application, the shared data further includes auxiliary rendering data corresponding to the 3DGS data, and the auxiliary rendering data and the 3DGS data are used to obtain the spatial photograph.

[0023] In any embodiment of this application, the operating system type of the second device is different from that of the first device.

[0024] Thirdly, an electronic device is provided, characterized in that it includes: a processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions; When the processor executes the computer instructions, the electronic device performs the first aspect and any of its embodiments as described above; when the processor executes the computer instructions, the electronic device also performs the second aspect and any of its embodiments as described above.

[0025] Fourthly, another electronic device is provided, comprising: a processor, a transmitting device, an input device, an output device, and a memory, the memory being used to store computer program code, the computer program code including computer instructions; When the processor executes the computer instructions, the electronic device performs the first aspect and any of its embodiments as described above; when the processor executes the computer instructions, the electronic device also performs the second aspect and any of its embodiments as described above.

[0026] Fifthly, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, the computer program including program instructions; When the program instructions are executed by the processor, the processor is caused to perform the first aspect and any embodiment thereof as described above; or when the program instructions are executed by the processor, the processor is caused to perform the second aspect and any embodiment thereof as described above.

[0027] In a sixth aspect, a computer program product is provided, the computer program product comprising a computer program or instructions; when the computer program or instructions are executed on a computer, causing the computer to perform the first aspect and any embodiment thereof described above; when the program instructions are executed by a processor, causing the processor to perform the second aspect and any embodiment thereof described above.

[0028] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application.

[0029] In this application, the first device supports calling the 3DGS rendering library. The 3DGS rendering library includes rendering pipelines corresponding to multiple operating system types, as well as rendering view components corresponding to multiple operating system types. By using the rendering pipeline in the 3DGS rendering library corresponding to the operating system type of the first device, 3DGS data can be rendered to obtain a spatial photograph. The spatial photograph can then be displayed using the rendering view components in the 3DGS rendering library corresponding to the operating system type of the first device. In other words, the display of the spatial photograph in this embodiment does not depend on a proprietary device. Instead, by supporting the calling of the 3DGS rendering library, it achieves adaptive rendering and efficient display of 3DGS data on devices corresponding to different operating system types, improving the compatibility, rendering efficiency, and display effect of the spatial photograph in multi-device environments, thereby expanding the application scenarios of the spatial photograph. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0032] Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application; Figure 2 A flowchart illustrating a data rendering method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a 3DGS rendering library provided in an embodiment of this application; Figure 4 This is a schematic diagram of a spatial photograph display scene provided in an embodiment of this application. Figure 1 ; Figure 5 This is a schematic diagram of a spatial photograph display scene provided in an embodiment of this application. Figure 2 ; Figure 6 This is a schematic diagram of the structure of a data rendering device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0034] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0035] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0036] It should be understood that this application provides a rendering method for 3DGS data, applicable to multimedia transmission production and consumption scenarios, such as customized advertising embedding, multi-view live streaming, immersive social interaction, and the 3D display of 2D images in social media. In this application, the rendering method can be applied to the field of artificial intelligence (AI). Artificial intelligence refers to the theories, methods, technologies, and application systems that use digital computers or computer-controlled computing to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. Artificial intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making functions.

[0037] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0038] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing, tracking, and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional (3D) object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0039] Before proceeding with the following explanation, let's define the technical terms that will appear in the text.

[0040] 1. 3DGS data 3DGS data can be understood as higher-order point cloud data. It's a technique that uses Gaussian distributions to model and render 3D scenes. The core idea is to represent the scene as a set of three-dimensional Gaussian distributions, which effectively approximate the scene's geometry and appearance. Each point in 3DGS data (called a Gaussian point) can contain parameters such as position (mean), covariance matrix (shape), transparency, and spherical harmonic coefficients (color). The spherical harmonic coefficients represent the color distribution of the Gaussian point under different viewpoints, capturing complex lighting and texture information. Compared to points in point cloud data, it has more dimensions.

[0041] Point cloud data is a collection of data consisting of a large number of discrete points in three-dimensional space. Each point can contain attribute information such as three-dimensional position, color, or reflection intensity. Point cloud data has high precision, high resolution, and high-dimensional geometric information, and can intuitively represent the shape, surface, and texture of digital objects in space.

[0042] 2. Rendering library A rendering library is a software development library that encapsulates the core logic of graphics rendering, the implementation of the rendering pipeline, and the calls to the underlying graphics application programming interface (API). Its core purpose is to highly abstract and encapsulate the complex low-level work of graphics rendering (such as GPU calls, rendering pipeline configuration, and shader management), and provide a set of simple, easy-to-use, and standardized high-level APIs. This allows developers to quickly implement high-performance graphics rendering and image drawing functions without needing to deeply understand the underlying graphics engine and hardware details.

[0043] 3. Rendering Pipeline The rendering pipeline, also known as the graphics rendering pipeline or graphics pipeline, is the core processing flow of a graphics rendering engine. It transforms abstract three-dimensional (3D) models, textures, lighting, and other data into a two-dimensional pixel image that can be displayed on the screen through a series of structured, pipelined, and fixed steps. A graphics rendering engine refers to the software framework or core component used to control and execute the rendering process.

[0044] 4. Rendering view component The rendering view component is the container and presentation layer in the rendering process. It is responsible for providing the graphics context environment required for graphics drawing and acts as the graphics rendering window to present the rendering results to the user. The graphics rendering window refers to the rectangular area provided by the operating system for displaying images.

[0045] The following are explanations of some key terms used in this application: "In response to" indicates the state in which a corresponding event occurs or a condition is met. The timing of subsequent actions performed in response to this event or condition is not necessarily strongly correlated with the time when the event occurs or the condition is met. For example, in some cases, subsequent actions may be performed immediately when the event occurs or the condition is met; while in other cases, subsequent actions may be performed some time after the event occurs or the condition is met.

[0046] Triggered operation: This refers to an operation performed on information (such as controls) provided by the terminal device. This operation sends a corresponding instruction to the terminal device, triggering it to execute the next task. The next task triggered by different triggered operations for different information can all be pre-set in the program. This triggered operation can be a manual, contact-based operation performed by the user, such as clicking, double-tapping, long-pressing, or swiping on content displayed on the terminal device screen. It can also be a non-contact operation such as voice or gestures; these will not be limited here. In some cases, the triggered operation can also be executed by the terminal device based on a pre-defined program.

[0047] Please see Figure 1 , Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application; the network architecture may include a server 10d and a terminal cluster, and the terminal cluster may include one or more terminal devices, without limiting the number of terminal devices included in the terminal cluster. Figure 1 As shown, the terminal cluster may specifically include terminal device 10a, terminal device 10b, and terminal device 10c, etc.; all terminal devices in the terminal cluster (e.g., may include terminal device 10a, terminal device 10b, and terminal device 10c, etc.) can connect to server 10d via a network, so that each terminal device can interact with server 10d through the network connection. The network connection method is not limited here; it can be a direct or indirect connection via wired communication, a direct or indirect connection via wireless communication, or other methods, which are not limited herein.

[0048] in, Figure 1 The terminal devices in the terminal cluster shown may include, but are not limited to: smartphones, tablets, laptops, PDAs, desktop computers, wearable devices (such as smartwatches, smart bracelets, etc.), smart voice interaction devices, smart home appliances (such as smart TVs, etc.), in-vehicle devices, aircraft, and other electronic devices. This application does not limit the type of terminal device. It should be understood that... Figure 1 Each terminal device in the terminal cluster shown can install an application client. When the application client runs on each terminal device, it can interact with... Figure 1 Data interaction occurs between the servers 10d shown. The application client running on each terminal device can be a standalone client or an embedded subroutine integrated into a standalone client (e.g., a game client); this application does not impose any limitations on this. It is understood that... Figure 1The server 10d shown can be the backend server corresponding to the application client installed on various terminal devices. Here, the application client can be a social media platform, and the corresponding business data platform for social media refers to a content production and exchange platform based on user relationships on the Internet. For example, this social media platform can be used to display two-dimensional images.

[0049] Figure 1 The server 10d shown can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. This application does not limit the type of server.

[0050] For ease of understanding, the embodiments of this application may be described in detail below. Figure 1 One of the multiple terminal devices shown is selected as the first device. For example, in the embodiments of this application, a terminal device can be selected as the first device. Figure 1 The terminal device 10a shown serves as a first device, which may integrate social media. This first device can interact with the server 10d through the business data platform corresponding to the social media platform.

[0051] It is understood that the first device supports calling the 3DGS rendering library, which includes rendering pipelines corresponding to multiple operating system types, as well as rendering view components corresponding to multiple operating system types. In one possible implementation, if the first device's operating system type is one of the multiple operating system types, then when the first device obtains 3DGS data (generated based on the aforementioned 2D image), it can use the rendering pipeline corresponding to the first device's operating system type to render the 3DGS data, obtaining a spatial photograph; and then use the rendering view component corresponding to the first device's operating system type to display the spatial photograph. This rendering method can improve the rendering efficiency of spatial photographs and expand their application scenarios.

[0052] It should be understood that the executing entity of the embodiments of this application is the first device, which can be any electronic device capable of executing the technical solutions disclosed in the embodiments of the method of this application. It should also be understood that the embodiments of the method of this application can also be implemented by a processor executing computer program code. The embodiments of this application will now be described in conjunction with the accompanying drawings.

[0053] Please see Figure 2 , Figure 2This is a flowchart illustrating a data rendering method provided in an embodiment of this application. The method mainly describes how a first device efficiently renders a 2D image into 3D by calling components (including rendering pipeline and rendering view components) from the 3DGS rendering library that match its operating system. This method can be executed by the first device, which can be one of the aforementioned... Figure 1 Any terminal device in the terminal cluster shown.

[0054] like Figure 2 As shown, this data rendering method includes, but is not limited to, the following steps: Step S101: The 3DGS data is rendered using the rendering pipeline corresponding to the operating system type of the first device to obtain a spatial photograph. The 3DGS data is generated based on a two-dimensional image. The first device supports calling the 3DGS rendering library. The 3DGS rendering library includes rendering pipelines corresponding to multiple operating system types and rendering view components corresponding to multiple operating system types. The operating system type of the first device is one of the multiple operating system types.

[0055] It is understandable that the above operating system types refer to categories based on the kernel derivation relationship of the operating system. An operating system (OS) is a built-in software program used to cooperate with various computer hardware components to enable interaction with the user.

[0056] In different scenarios, the operating system may be replaced by other names (such as platform), and there is no limitation here. For example, the operating system is the core foundational software of the platform, and the platform is the complete execution environment in which the operating system resides. Based on this, "operating system" in the embodiments of this application can also be replaced by "platform". "Cross-operating system" can also be understood as cross-platform.

[0057] Step S102: Use the rendering view component corresponding to the operating system type of the first device to display the spatial photo.

[0058] To facilitate understanding of steps S101 and S102 above, the 3DGS rendering library will be introduced below: The rendering pipeline in the 3DGS rendering library is mainly responsible for resource allocation, rendering scheduling, and unified encapsulation of cross-system (or cross-platform) rendering interfaces, which can achieve consistency in rendering spatial photos across systems.

[0059] For example, the rendering pipeline in the 3DGS rendering library may include rendering pipeline A, rendering pipeline B, and rendering pipeline C; wherein, rendering pipeline A is the rendering pipeline corresponding to operating system type A, rendering pipeline B is the rendering pipeline corresponding to operating system type B, and rendering pipeline C is the rendering pipeline corresponding to operating system type C.

[0060] The rendering view components in the 3DGS rendering library can include rendering view component A, rendering view component B, and rendering view component C. Rendering view component A corresponds to operating system type A, rendering view component B corresponds to operating system type B, and rendering view component C corresponds to operating system type C.

[0061] In one feasible implementation, the 3DGS rendering library can also include a 3DGS data processing component, which is specifically responsible for parsing, transforming, organizing, and efficiently inputting and outputting 3DGS point cloud data. This 3DGS data processing component supports physical data blending, shader calculations, rasterization processing, and camera viewpoint parameter control, providing an optimized data source for the backend rendering pipeline to achieve the generation and real-time rendering of high-quality spatial images.

[0062] In another feasible implementation, the 3DGS rendering library can also include an attitude acquisition component responsible for acquiring the device's (e.g., the first device) attitude in three-dimensional space, enabling interactive browsing and immersive experiences of spatial photographs (e.g., immersive operations such as movement and rotation). Device attitude refers to the orientation and rotation of the device in three-dimensional space relative to a reference coordinate system (usually the geographic coordinate system or the direction of gravity). The attitude acquisition component can include, but is not limited to, an inertial measurement unit (IMU), an attitude and heading reference system (AHRS), and a gyroscope. For example, the IMU includes a 3-axis gyroscope and a 3-axis accelerometer. The AHRS includes a 3-axis gyroscope, a 3-axis accelerometer, and a 3-axis magnetometer.

[0063] For example, please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a 3DGS rendering library provided in an embodiment of this application. For example... Figure 3 As shown, the 3DGS rendering library can include a pose acquisition component, a 3DGS core rendering module, and a rendering view module.

[0064] The 3DGS Core Rendering Module is primarily responsible for the main rendering process of spatial photographs, including data decoding, rendering scheduling, and view synchronization. It is the core of the 3DGS rendering library. For example, the 3DGS Core Rendering Module may include the 3DGS data processing component and rendering pipelines corresponding to three operating system types (such as rendering pipeline A, rendering pipeline B, and rendering pipeline C).

[0065] The rendering view module includes rendering view components corresponding to multiple operating system types, supporting the visualization needs of different operating system types to achieve a unified 3D display. For example, the rendering view module includes rendering view components corresponding to three operating system types (such as rendering view component A, rendering view component B, and rendering view component C).

[0066] For example, the 3DGS rendering library here can be built on the Vulkan rendering framework. Vulkan is a high-performance, low-overhead, cross-platform graphics API. Compared with traditional open graphics libraries (OpenGL) and direct extensions (DirectX), Vulkan has more efficient resource management, finer-grained hardware access capabilities, and better multi-threading utilization. It is widely used in 3D rendering and image processing scenarios on mobile, desktop, and embedded devices. The Vulkan rendering framework can support complex 3D scenes, spatial photographs, real-time high-quality rendering, multi-terminal compatibility, and a more optimized interactive experience. At the same time, the Vulkan rendering framework provides basic capabilities for 3D content display and interaction in social media through low-level interfaces, offering advantages such as strong versatility and ease of expansion.

[0067] In one possible implementation, the 3DGS rendering library can also integrate a conversion library (such as the moltenvk conversion library). When the rendering pipeline B (such as the metal rendering pipeline) corresponding to the operating system type of a subsequent terminal device (e.g., operating system type B) cannot be called, the conversion library can be called to achieve a transparent conversion from the Vulkan API to the metal underlying layer, enabling the Vulkan rendering framework to adapt to the aforementioned terminal devices, thereby achieving consistency in the cross-platform display of spatial photos.

[0068] In one possible implementation, before executing step S101, the first device can acquire first data, input the first data into the rendering pipeline (such as rendering pipeline A) corresponding to the operating system type of the first device, and then render the first data through rendering pipeline A to obtain a spatial photo.

[0069] The first data may include 3DGS data. The following provides examples illustrating several possible methods for obtaining 3DGS data.

[0070] Implementation method a: 3DGS data can be generated by the first device based on two-dimensional images.

[0071] For example, the first device can reconstruct a 3D scene from a 2D image using geometric algorithms, generating 3DGS data. Alternatively, the first device can also generate 3DGS data using a trained artificial intelligence model (e.g., a feedforward neural network or a diffusion model) and a 2D image.

[0072] Implementation method b: 3DGS data can also be obtained by the first device based on the 3DGS data processing component.

[0073] For example, the first device can read the data file corresponding to the 2D image based on the 3DGS data processing component and parse the 3DGS data in the data file. The file format of the data file may include, but is not limited to: polygon file format (PLY), SPLAT, and DRC format.

[0074] The PLY file format is used to store image objects described as collections of polygons. It's a common point cloud storage format that supports text or binary storage. A PLY file includes a header and a data area. The header records the encoding method, encoding version, element categories (such as vertices, faces, edges, etc., and their numbers), and attribute information (including the storage type and attribute name of attribute fields). The data area stores 3D data, for example, storing each point line by line. While the PLY file format supports custom fields and types, making it flexible, it is also highly redundant, resulting in large file sizes and slow loading speeds.

[0075] The SPLAT file format is a binary file format with a compact structure. The diffraction of each splat point (i.e., the point to be rendered) is stored in a fixed order, such as position: 3×float32; size: 3×float32; color: 4×uint8; rotation: 4×uint8. The file size is relatively small, but it is not easily expanded.

[0076] DRC format is a compressed 3D file format that is compatible with PLY and SPLAT file formats, but does not support the insertion of custom fields.

[0077] Implementation method c: 3DGS data can also be received by the first device from other terminal devices (such as the second device).

[0078] For example, when social media is running on a second device, the user of the second device (also known as the object of operation) can perform a 3D display operation on a 2D image in the social media, thereby displaying a spatial photo corresponding to the 2D image on the second device. In response to a sharing operation for the spatial photo, the second device can send the sharing data associated with the spatial photo to the first device, the sharing data including 3DGS data.

[0079] In one possible implementation, the first data may also include auxiliary rendering data corresponding to the 3DGS data, so as to facilitate the generation and real-time rendering of high-quality spatial photos based on rendering pipeline A.

[0080] The auxiliary rendering data here is determined by the first device through the 3DGS data processing component. The auxiliary rendering data here may include at least one of the following: shader calculation results, rasterization processing results, physical data mixing results, or camera view parameters.

[0081] The physical data mixing result refers to the result obtained by performing data format conversion, parameter normalization, and numerical activation on 3DGS data using 3DGS data processing components. Data format conversion refers to parsing and preprocessing the file format of the 3DGS data; parameter normalization refers to adopting a unified data storage structure to achieve efficient storage and access of step size parameters; numerical activation refers to using the Sigmoid function for range constraints, quaternion normalization to achieve rotational unity, and exponential scaling to restore the original scale, thus completing the mapping from 3DGS data to the effective parameter space.

[0082] Shader computation results refer to the results obtained by performing depth calculations and sorting on the physical data mixture results through the 3DGS data processing component. Shaders include, but are not limited to, computational shaders and graphics shaders. Computational shaders are used for spatial depth calculations and sorting, while graphics shaders are used for vertex or fragment shading. For example, shader computation results may include a normalized Gaussian parameter array and a depth sorting index.

[0083] The rasterization process refers to mapping the shader calculation results to the screen space through projection transformation using the 3DGS data processing component, generating a corresponding two-dimensional elliptical representation, and completing the rasterization through ellipse traversal and fragment shading.

[0084] Camera viewpoint parameters refer to the viewpoint used to control the virtual camera in 3D space. These parameters can be used to precisely adjust the display area and viewpoint of 3D content, allowing users to browse spatial photos by rotating the primary device, thus achieving interactive browsing and an immersive experience. For example, the camera viewpoint parameters here can be determined based on the device posture acquired by the posture acquisition component, or they can be system default settings; this application does not limit this.

[0085] In one possible implementation, before executing step S101, in order to achieve dynamic adaptation between the spatial photo and the device screen and improve the rendering effect of the spatial photo, the rendering pipeline corresponding to the operating system type of the first device is obtained after initialization based on the target view parameters. For ease of explanation, in this embodiment, the rendering pipeline corresponding to the operating system type of the first device before initialization can be referred to as the first rendering pipeline, and the rendering pipeline corresponding to the operating system type of the first device after initialization can be referred to as the second rendering pipeline.

[0086] For example, the first device can obtain target view parameters, which are view parameters of the rendering view component (such as rendering view component A) corresponding to the operating system type of the first device. These view parameters can be understood as window parameters, which may include, but are not limited to, window position, window size (e.g., window width and height). Based on these target view parameters, the first rendering pipeline can be initialized to obtain the second rendering pipeline (such as rendering pipeline A).

[0087] Furthermore, after the first device obtains the spatial photograph (generated by rendering through the second rendering pipeline (such as rendering pipeline A), it can use a rendering view component (such as rendering view component A) that corresponds to the operating system type of the first device to display the spatial photograph.

[0088] For ease of understanding, please refer to the example provided. Figure 4 , Figure 4 This is a schematic diagram of a spatial photograph display scene provided in an embodiment of this application. Figure 1 .like Figure 4 As shown, interface 20a is a content overview page for social media on the first device, wherein interface 20a includes at least one piece of published content. For example... Figure 4 As shown, at least one published content may include published content 1, published content 2, published content 3, and published content 4.

[0089] Published content refers to information posted by users on social media, either in advance or in real-time. This content can include text, images, audio, and video, and can be displayed in formats such as notes, articles, video files, and video streams. The specific content information included in published content can be adjusted for different scenarios, and the content information in the published content can be determined by the publisher. Published content is stored on the server corresponding to the first device (as described above). Figure 1 The server shown (10d) distributes content to users browsing social media through a content distribution algorithm deployed on the server.

[0090] like Figure 4 As shown, when a user performs a trigger operation on published content 2, the first device responds to the trigger operation on published content 2 and can display the content details information of published content 2 on the content details page 20b, for example, it can display a two-dimensional image 20c.

[0091] In one feasible implementation, the content details page 20b may also display a business control 20e (such as a "3D display" control). When a user performs a trigger operation on the business control 20e, the first device, in response to the trigger operation on the business control 20e, may display a spatial photo 20f corresponding to the two-dimensional image 20c. The spatial photo 20f may be displayed on the content details page 20b in the form of a floating window, a mask, or a semi-transparent layer. Alternatively, it may be displayed on a page whose size can be changed by dragging and is collapsible, with the size of the page being smaller than that of the content details page 20b. This application does not limit the specific display method of the spatial photo 20f.

[0092] It is understandable that when the first device responds to a trigger operation on the business control 20e, the first device can obtain the 3DGS data corresponding to the 2D image 20c. For example, the first device can call a 3DGS rendering library and, through its 3DGS data processing component, obtain the 3DGS data corresponding to the 2D image 20c and the corresponding auxiliary rendering data. Further, using the rendering pipeline (such as rendering pipeline A) corresponding to the first device's operating system type, the 3DGS data is rendered based on the auxiliary rendering data to obtain a spatial photograph. The spatial photograph 20f corresponding to the 2D image 20c is displayed using a rendering view component (such as rendering view component A) corresponding to the first device's operating system type.

[0093] It should be noted that, Figure 4 The pages and controls shown are merely some reference formats. In actual business scenarios, developers can make relevant designs according to product requirements. This application embodiment does not limit the specific forms of the interfaces and controls involved.

[0094] In this embodiment, the display of spatial photos on social media does not rely on proprietary devices. Instead, by supporting the call of the 3DGS rendering library, spatial photos on social media can be rendered and displayed on first devices corresponding to different operating system types. In other words, spatial photos can be adapted and rendered efficiently on devices corresponding to different operating system types, improving the compatibility, rendering efficiency and display effect of spatial photos in multi-device environments, thereby expanding the application scenarios of spatial photos.

[0095] For ease of understanding, please refer to the example provided. Figure 5 , Figure 5 This is a schematic diagram of a spatial photograph display scene provided in an embodiment of this application. Figure 2 .like Figure 5 As shown, the second device is the terminal device corresponding to operation object 1, and the first device is the terminal device corresponding to operation object 2. Both the first and second devices can install social media apps, and both support calling the 3DGS rendering library.

[0096] The second device responds to a 3D display operation for the object being operated on (e.g., Figure 4 The trigger operation shown for business control 20e can display a spatial photograph (such as spatial photograph 1) corresponding to the 2D image, based on the operating system type of the second device. Spatial photograph 1 can be generated based on 3DGS data and auxiliary rendering data 1. The auxiliary rendering data is obtained by the second device calling the 3DGS rendering library and using its 3DGS data processing component.

[0097] like Figure 5 As shown, the operation object 1 performs a sharing operation on the spatial photo 1 (to trigger the sharing of the spatial photo 1). In response to the sharing operation, the second device can send the sharing data corresponding to the spatial photo 1 to the first device. The first device can call the 3DGS rendering library to display the spatial photo corresponding to the above two-dimensional image based on the first device's operating system type and the sharing data.

[0098] The shared data may include 3DGS data, or 3DGS data and auxiliary rendering data 1, or spatial photograph 1.

[0099] The following will describe the different scenarios: (1) Shared data includes 3DGS data For example, after receiving 3DGS data, the first device can call the 3DGS rendering library, use the rendering pipeline (such as rendering pipeline A) in the 3DGS rendering library that corresponds to the operating system type of the first device to render the 3DGS data, obtain spatial photo 1, and use the rendering view component (such as rendering view component A) in the 3DGS rendering library that corresponds to the operating system type of the first device to display spatial photo 1.

[0100] Optionally, after receiving the 3DGS data, the first device can call the 3DGS data processing component in the 3DGS rendering library to generate auxiliary rendering data 2 corresponding to the 3DGS data. Then, the 3DGS data and auxiliary rendering data 2 can be transmitted to the rendering pipeline (e.g., rendering pipeline A) corresponding to the operating system type of the first device. Rendering pipeline A is used to render the 3DGS data based on the auxiliary rendering data 2 to obtain spatial photograph 2. The spatial photograph 2 is then displayed using the rendering view component (e.g., rendering view component A) corresponding to the operating system type of the first device.

[0101] (2) Shared data includes 3DGS data and auxiliary rendering data 1 For example, after receiving 3DGS data and auxiliary rendering data 1, the first device can call the 3DGS rendering library and use the rendering pipeline (such as rendering pipeline A) in the 3DGS rendering library that corresponds to the operating system type of the first device to render the 3DGS data based on the auxiliary rendering data 1, thereby obtaining spatial photograph 1. Using the rendering view component (such as rendering view component A) corresponding to the operating system type of the first device, spatial photograph 1 can be displayed efficiently.

[0102] Optionally, after receiving the 3DGS data and auxiliary rendering data 1, the first device can call the 3DGS data processing component in the 3DGS rendering library to regenerate the auxiliary rendering data 2 corresponding to the 3DGS data. Then, using the rendering pipeline (such as rendering pipeline A) in the 3DGS rendering library that corresponds to the operating system type of the first device, the 3DGS data is rendered based on the auxiliary rendering data 2 to obtain the spatial photograph 2. The spatial photograph 2, which is more adapted to the device screen, is then displayed using a rendering view component (such as rendering view component A) in the 3DGS rendering library that corresponds to the operating system type of the first device.

[0103] (3) Shared data includes spatial photos 1 After receiving spatial photo 1, if the operating system type of the first device is the same as that of the second device (e.g., both are operating system type A), the first device can directly call the rendering view component (e.g., rendering view component A) in the 3DGS rendering library that corresponds to operating system type A to display spatial photo 1.

[0104] In this embodiment, the first device receives shared data sent by another device and calls the 3GDS rendering library to render and display the shared data. This enables the rendering and display of spatial photos corresponding to the same two-dimensional image on multiple devices (e.g., the first device and the second device), achieving cross-terminal display of spatial photos. Furthermore, sharing spatial photos from social media across multiple terminal devices improves the efficiency of content dissemination on social media, enhances user experience, and strengthens user engagement.

[0105] In one possible implementation, before sending shared data to the first device, the second device can obtain the operating system type of the first device to determine the type of shared data to send. For example, if the operating system type of the first device is the same as that of the second device, spatial photo 1 can be sent directly, improving the propagation efficiency of the spatial photo. If the operating system types of the first device and the second device are different, 3DGS data, or 3DGS data and auxiliary rendering data 1, can be sent to the first device.

[0106] It is understood that in the specific implementation of this application, data such as the operating system type of the terminal device of the target may be involved. When the embodiments of this application are applied to specific products or technologies, permission or consent from relevant institutions or departments, or the user himself or himself, is required. The collection, use and processing of relevant data must comply with the relevant laws and standards of the relevant regions.

[0107] For example, before the second device sends shared data to the first device, the second device sends an authorization request to the first device. This authorization request is used to request the operating system type of the first device. When the first device receives the authorization request, it can display a prompt message corresponding to the authorization request. This prompt message can be displayed on the first device in the form of a floating window, a mask, or a semi-transparent layer, or it can be displayed on the first device as a page; this application does not limit this. The prompt message can be used to indicate to the second device that it is requesting the operating system type of the second device. For example, the prompt message could be "The other party is requesting the operating system type of your terminal device. Do you agree?" If the second device confirms the authorization, it can obtain the operating system type of the first device.

[0108] In this embodiment, spatial photos from social media can be shared and displayed between terminal devices with different operating system types. In other words, spatial photos can be rendered and displayed on multiple terminal devices with different operating system types, realizing cross-system rendering and display of spatial photos, which can improve the adaptability of spatial photos across systems.

[0109] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0110] The methods of the embodiments of this application have been described in detail above, and the apparatus of the embodiments of this application is provided below.

[0111] Please see Figure 6 , Figure 6 This is a schematic diagram of a data rendering apparatus provided in an embodiment of this application. The data rendering apparatus includes: a rendering unit 10 and a display unit 20. The rendering unit 10 is used to render the 3D Gaussian splash 3DGS data using the rendering pipeline corresponding to the operating system type of the first device to obtain a spatial photograph. The first device supports calling the 3DGS rendering library. The 3DGS rendering library includes multiple rendering pipelines corresponding to multiple operating system types and rendering view components corresponding to the multiple operating system types. The operating system type of the first device is one of the multiple operating system types. Display unit 20 is used to display the spatial photo using a rendering view component corresponding to the operating system type of the first device.

[0112] In any embodiment of this application, the rendering pipeline corresponding to the operating system type of the first device is obtained after initialization based on the target view parameters, wherein the target view parameters are the view parameters of the rendering view component corresponding to the operating system type of the first device.

[0113] In any embodiment of this application, the 3DGS rendering library further includes a pose acquisition component, which is used to acquire the device pose of the first device. The device pose is used to control camera viewpoint parameters, and the camera viewpoint parameters and the 3DGS data are used to obtain the spatial photograph.

[0114] In any embodiment of this application, the 3DGS rendering library includes a 3DGS data processing component, which is used to determine auxiliary rendering data corresponding to the 3DGS data. The auxiliary rendering data and the 3DGS data are used to obtain the spatial photograph.

[0115] In any embodiment of this application, the auxiliary rendering data includes at least one of the following: shader calculation results, rasterization processing results, physical data mixing results, or camera viewpoint parameters.

[0116] In conjunction with any embodiment of this application, the 3DGS data processing component is further configured to acquire the 3DGS data.

[0117] In conjunction with any embodiment of this application, the apparatus further includes: The receiving unit 30 is used to receive shared data from the second device, the shared data including the 3DGS data, the second device supporting the use of the 3DGS rendering library, and the operating system type of the second device being one of the plurality of operating system types.

[0118] In any embodiment of this application, the shared data further includes auxiliary rendering data corresponding to the 3DGS data, and the auxiliary rendering data and the 3DGS data are used to obtain the spatial photograph.

[0119] In any embodiment of this application, the operating system type of the second device is different from that of the first device.

[0120] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0121] Please see Figure 7 , Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device 6 includes a processor 61 and a memory 62. Optionally, the electronic device 6 also includes an input device 63 and an output device 64. The processor 61, memory 62, input device 63, and output device 64 are coupled together via connectors, which include various interfaces, transmission lines, or buses, etc., and are not limited in this embodiment of the application. It should be understood that in the various embodiments of this application, coupling refers to mutual connection in a specific way, including direct connection or indirect connection through other devices, such as through various interfaces, transmission lines, buses, etc.

[0122] Processor 61 may include one or more processors, such as one or more central processing units (CPUs). If the processor is a CPU, it may be a single-core CPU or a multi-core CPU. Optionally, processor 61 may be a processor group consisting of multiple CPUs, with the multiple processors coupled to each other via one or more buses. Optionally, the processor may also be other types of processors, etc., which are not limited in this embodiment.

[0123] The memory 62 can be used to store computer program instructions, as well as various types of computer program code, including program code for executing the present application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), which is used for related instructions and data.

[0124] Input device 63 is used to input data and / or signals, and output device 64 is used to output data and / or signals. Input device 63 and output device 64 can be independent devices or an integrated device.

[0125] It is understood that in this embodiment of the application, the memory 62 can be used not only to store related instructions, but also to store related data. This embodiment of the application does not limit the specific data stored in the memory.

[0126] Understandable Figure 7 This is merely a simplified design of an electronic device. In practical applications, the electronic device may also include other necessary components, including, but not limited to, any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of this application are within the protection scope of this application.

[0127] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0128] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will also readily understand that the various embodiments of this application have different focuses, and for the sake of convenience and brevity, the same or similar parts may not be repeated in different embodiments. Therefore, parts not described or not described in detail in one embodiment can be referred to the descriptions in other embodiments.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0131] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0132] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0133] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A data rendering method, characterized in that, The method includes: The rendering pipeline corresponding to the operating system type of the first device is used to render the 3D Gaussian splash 3DGS data to obtain a spatial photograph. The 3DGS data is generated based on a 2D image. The first device supports calling the 3DGS rendering library. The 3DGS rendering library includes rendering pipelines corresponding to multiple operating system types and rendering view components corresponding to the multiple operating system types. The operating system type of the first device is one of the multiple operating system types. The spatial photo is displayed using the rendering view component corresponding to the operating system type of the first device.

2. The method according to claim 1, characterized in that, The rendering pipeline corresponding to the operating system type of the first device is obtained after initialization based on the target view parameters, which are the view parameters of the rendering view component corresponding to the operating system type of the first device.

3. The method according to claim 1 or 2, characterized in that, The 3DGS rendering library also includes a pose acquisition component, which is used to acquire the device pose of the first device. The device pose is used to control the camera viewpoint parameters, and the camera viewpoint parameters and the 3DGS data are used to obtain the spatial photograph.

4. The method according to any one of claims 1-3, characterized in that, The 3DGS rendering library includes a 3DGS data processing component, which is used to determine the auxiliary rendering data corresponding to the 3DGS data. The auxiliary rendering data and the 3DGS data are used to obtain the spatial photograph.

5. The method according to claim 4, characterized in that, The auxiliary rendering data includes at least one of the following: shader calculation results, rasterization processing results, physical data mixing results, or camera viewpoint parameters.

6. The method according to claim 4 or 5, characterized in that, The 3DGS data processing component is also used to acquire the 3DGS data.

7. The method according to any one of claims 1-5, characterized in that, The method further includes: The device receives shared data from a second device, the shared data including the 3DGS data, the second device supports calling the 3DGS rendering library, and the operating system type of the second device is one of the plurality of operating system types.

8. The method according to claim 7, characterized in that, The shared data also includes auxiliary rendering data corresponding to the 3DGS data, and the auxiliary rendering data and the 3DGS data are used to obtain the spatial photograph.

9. The method according to claim 8, characterized in that, The second device has a different operating system type than the first device.

10. A data rendering apparatus, characterized in that, The device includes: The rendering unit is used to render the 3D Gaussian splash 3DGS data using the rendering pipeline corresponding to the operating system type of the first device to obtain a spatial photograph. The first device supports calling the 3DGS rendering library, which includes rendering pipelines corresponding to multiple operating system types and rendering view components corresponding to the multiple operating system types. The operating system type of the first device is one of the multiple operating system types. The display unit is used to display the spatial photo using a rendering view component corresponding to the operating system type of the first device.

11. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions; When the processor executes the computer instructions, the electronic device performs the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions; When the program instructions are executed by the processor, the processor is caused to perform the method described in any one of claims 1 to 9.

Images