Graphics processing device, system and method for graphics processing

Abstract

Disclosed are a graphics processing device, a system and a method for graphics processing. A graphics processing device includes a computing resource; and a neural rendering pipeline that invokes the computing resource to perform the following steps: obtaining a plurality of view parameters, wherein the plurality of view parameters are respectively associated with a plurality of view cameras, the plurality of view cameras are configured to take a same scene at different view angles; based on the plurality of view parameters, calculating an equivalent monocular view parameter associated with an equivalent monocular camera; filtering out a set of neural features within a view cone space associated with the equivalent monocular view parameter; decoding the set of neural features into a set of neural primitives; and rendering the set of neural primitives into a plurality of images based on the plurality of view parameters, respectively.

Description

Technical Field

[0001] This invention relates generally to the field of graphics processing, and more specifically to multi-view stereo rendering. Background Technology

[0002] In recent years, Virtual Reality (VR) and Augmented Reality (AR) technologies have developed rapidly and are widely used in various fields, such as education, film and television, games, and even industrial design and production. High-resolution, high-speed, and continuous rendering are essential requirements for VR / AR technologies and are also important drivers for their further development.

[0003] Traditional stereoscopic rendering typically employs an alternating rendering method, rendering the left-eye view first, followed by the right-eye view. While simple to implement, this method results in a theoretical 50% performance drop in frame rate compared to monocular rendering, and may cause image flickering issues due to asynchronous binocular display. For traditional 3D rendering technologies, this performance degradation is usually still sufficient to meet the smooth experience requirement of 60-72 frames per second (FPS).

[0004] Existing research has proposed several solutions to improve the speed and resource utilization efficiency of stereoscopic rendering in VR / AR applications, achieving higher rendering frame rates and a better user experience.

[0005] Some studies have proposed a single-pass stereo approach, which accelerates the rendering process of traditional 3D rendering methods in a stereoscopic context by building a dedicated rendering pipeline at the GPU level through hardware optimization. This method aims to optimize the rasterization process based on meshes and reduce the time spent on off-GPU stages by merging or reducing texture computation. However, it is not applicable to neural rendering processes that do not include texture computation steps and whose main computational stages are located on the GPU.

[0006] Some studies have proposed foveated rendering methods, which identify high-perception regions and improve rendering detail by tracking the binocular gaze points of the human eye, while reducing rendering detail outside these regions, thus achieving efficient utilization of rendering resources. While this method improves the utilization of rendering resources, it is still not applicable to neural rendering workflows that do not include texture calculation steps and whose main computational phase is performed on the GPU.

[0007] Cutting-edge neural rendering models (such as Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS)) can build highly realistic and detailed virtual scenes for real-time interactive rendering applications.

[0008] The rendering workflow of neural rendering technology differs significantly from that of traditional 3D rendering. The neural rendering workflow includes key stages such as frustum filtering, neural feature decoding, and neural primitive rendering. The neural feature decoding stage is unique to neural rendering, consuming substantial computational resources and taking significantly longer. Furthermore, this stage cannot be rendered using the hardware rendering pipeline built for traditional 3D rendering on the GPU, thus requiring specific software and hardware support. Neural feature decoding is the primary computational bottleneck in the neural rendering workflow, limiting the practical application of neural rendering technology. Compared to traditional 3D rendering, neural rendering technology has significant shortcomings in computational efficiency and resource utilization. This performance bottleneck is particularly pronounced in VR / AR applications requiring stereoscopic rendering. For neural rendering, with its higher computational costs, the performance loss caused by separate rendering for each eye is unacceptable.

[0009] Chinese patent application CN 117094877 A discloses a VR multi-GPU rendering method and system. This method improves the stereoscopic rendering speed of the system by dividing and distributing the single-frame image rendering task to multiple GPUs and utilizing parallel computing across devices. While this method may be applicable to accelerating neural rendering, it still suffers from computational redundancy in stereoscopic rendering and introduces inter-device synchronization overhead.

[0010] Chinese patent application CN 116883631 A discloses a dual-GPU collaborative rendering method and device based on the UE engine DX12 MultiAdapter. The method uses two GPUs to render in parallel, rendering images from the left and right eyes of the binoculars respectively. However, there is still computational redundancy in binocular rendering and the problem of inter-device synchronization overhead still exists.

[0011] Chinese patent application CN 113209604 A discloses a method and system for large-view VR rendering. This method uses preset screen frames and frame generation technology to reduce the frequency of performing complete rendering, thereby improving the speed of binocular stereo rendering. However, the problem of computational redundancy in binocular rendering still exists.

[0012] Given that the existing technologies described above have the following drawbacks in real-time rendering: 1) low rendering efficiency; 2) redundant computation; and 3) unsuitability for neural rendering, there is a need in the art for VR / AR technology solutions that improve at least one aspect of rendering efficiency, eliminate computational redundancy, and are suitable for neural rendering. Summary of the Invention

[0013] This invention is provided to offer an improved multi-view stereo rendering solution.

[0014] One aspect of the present invention provides a graphics processing apparatus comprising: a computing resource; and a neural rendering pipeline, the neural rendering pipeline invoking the computing resource to perform the following steps: acquiring a plurality of viewpoint parameters, wherein the plurality of viewpoint parameters are respectively associated with a plurality of viewpoint cameras configured to capture the same scene from different viewpoints; calculating equivalent monocular viewpoint parameters associated with an equivalent monocular camera based on the plurality of viewpoint parameters; filtering out a set of neural features in a frustum space associated with the equivalent monocular viewpoint parameters; decoding the set of neural features into a set of neural primitives; and rendering the set of neural primitives into a plurality of images based on the plurality of viewpoint parameters.

[0015] In the graphics processing apparatus described above, each of the plurality of viewpoint parameters includes at least the pose matrix and projection matrix of the corresponding viewpoint camera among the plurality of viewpoint cameras; the equivalent monocular viewpoint parameter includes at least the pose matrix and projection matrix of the equivalent monocular camera.

[0016] The graphics processing apparatus as described in any of the preceding claims, wherein the neural rendering pipeline invokes the computing resources to perform the following steps: using a minimum fit method to compute the equivalent monocular viewpoint parameters associated with the equivalent monocular camera.

[0017] The graphics processing apparatus as described in any of the preceding claims, wherein the neural rendering pipeline invokes the computing resources to perform the following steps: determining the frustum space associated with the equivalent monocular viewpoint parameters by making the boundaries of the frustums of the equivalent monocular camera parallel to the boundaries of the frustums of the plurality of viewpoint cameras, and making the observation range of the equivalent monocular camera no greater than the sum of the observation ranges of the plurality of viewpoint cameras.

[0018] In a graphics processing apparatus as described in any of the preceding claims, the neural rendering pipeline invokes the computing resources to perform the following steps: rendering the set of neural primitives into multiple images using sequential computation or fusion computation.

[0019] A graphics processing apparatus as described in any of the preceding claims, wherein the neural rendering pipeline invokes the computing resources to perform the following steps: enabling the graphics processing apparatus to compute in parallel with at least one other graphics processing apparatus to render the set of neural primitives into multiple images.

[0020] The graphics processing apparatus as described in any of the preceding claims, wherein the plurality of view parameters include a first view parameter and a second view parameter, the first view parameter being associated with a left-eye view camera and the second view parameter being associated with a right-eye view camera.

[0021] The graphics processing apparatus as described in any of the preceding claims, wherein the neural rendering pipeline invokes the computing resources to perform the following steps: calculating equivalent monocular view parameters associated with an equivalent monocular camera based on the first view parameter and the second view parameter; rendering the set of neural primitives as a left-eye image based on the first view parameter, and rendering the set of neural primitives as a right-eye image based on the second view parameter.

[0022] The graphics processing apparatus as described in any of the preceding claims, wherein the plurality of viewpoint parameters includes at least three viewpoint parameters, the at least three viewpoint parameters being associated with at least three viewpoint cameras respectively; the neural rendering pipeline invokes the computing resources to perform the following steps: calculating, in an iterative manner, equivalent monocular viewpoint parameters associated with an equivalent monocular camera based on the at least three viewpoint parameters.

[0023] The graphics processing apparatus as described in any of the preceding claims, wherein the at least three viewpoint parameters include a first viewpoint parameter, a second viewpoint parameter, and a third viewpoint parameter, the first viewpoint parameter, the second viewpoint parameter, and the third viewpoint parameter being associated with three viewpoint cameras respectively; the neural rendering pipeline invokes the computing resources to perform the following steps: determining an intermediate viewpoint parameter based on the first viewpoint parameter and the second viewpoint parameter; and determining the equivalent monocular viewpoint parameter based on the intermediate viewpoint parameter and the third viewpoint parameter.

[0024] The graphics processing apparatus as described in any of the preceding claims, wherein the at least three viewpoint parameters include a first viewpoint parameter, a second viewpoint parameter, a third viewpoint parameter, and a fourth viewpoint parameter, the first viewpoint parameter, the second viewpoint parameter, the third viewpoint parameter, and the fourth viewpoint parameter being associated with four viewpoint cameras respectively; the neural rendering pipeline invokes the computing resources to perform the following steps: determining a first intermediate viewpoint parameter based on the first viewpoint parameter and the second viewpoint parameter; determining a second intermediate viewpoint parameter based on the third viewpoint parameter and the fourth viewpoint parameter; and determining the equivalent monocular viewpoint parameter based on the first intermediate viewpoint parameter and the second intermediate viewpoint parameter.

[0025] Another aspect of the present invention provides a system for graphics processing, comprising: a graphics processing device as described in any of the preceding embodiments; and a plurality of viewpoint cameras coupled to the graphics processing device.

[0026] Another aspect of the present invention provides a method for graphics processing, comprising: acquiring a plurality of viewpoint parameters, wherein the plurality of viewpoint parameters are respectively associated with a plurality of viewpoint cameras, the plurality of viewpoint cameras being configured to capture the same scene from different viewpoints; calculating equivalent monocular viewpoint parameters associated with an equivalent monocular camera based on the plurality of viewpoint parameters; filtering out a set of neural features in a frustum space associated with the equivalent monocular viewpoint parameters; decoding the set of neural features into a set of neural primitives; and rendering the set of neural primitives into a plurality of images based on the plurality of viewpoint parameters respectively.

[0027] As described above, each of the plurality of viewpoint parameters includes at least the pose matrix and projection matrix of the corresponding viewpoint camera among the plurality of viewpoint cameras; the equivalent monocular viewpoint parameter includes at least the pose matrix and projection matrix of the equivalent monocular camera.

[0028] The method described in any of the above embodiments, wherein calculating the equivalent monocular viewpoint parameters associated with the equivalent monocular camera based on the plurality of viewpoint parameters includes: using a minimum fit method to calculate the equivalent monocular viewpoint parameters associated with the equivalent monocular camera.

[0029] The method described in any of the above further includes: determining the frustum space associated with the equivalent monocular viewpoint parameters by making each boundary of the frustum of the equivalent monocular camera parallel to each boundary of the frustum of the plurality of viewpoint cameras, and making the observation range of the equivalent monocular camera no greater than the sum of the observation ranges of the plurality of viewpoint cameras.

[0030] The method described in any of the above embodiments, wherein rendering the set of neural primitives into multiple images based on the plurality of viewpoint parameters includes: rendering the set of neural primitives into multiple images using sequential computation or fusion computation.

[0031] The method described in any of the above embodiments, wherein the plurality of viewpoint parameters includes a first viewpoint parameter and a second viewpoint parameter, the first viewpoint parameter being associated with a left-eye viewpoint camera and the second viewpoint parameter being associated with a right-eye viewpoint camera, wherein calculating the equivalent monocular viewpoint parameter associated with the equivalent monocular camera includes: calculating the equivalent monocular viewpoint parameter associated with the equivalent monocular camera based on the first viewpoint parameter and the second viewpoint parameter; and rendering the set of neural primitives into multiple images based on the plurality of viewpoint parameters includes: rendering the set of neural primitives into a left-eye image based on the first viewpoint parameter; and rendering the set of neural primitives into a right-eye image based on the second viewpoint parameter.

[0032] The method described in any of the above embodiments, wherein the plurality of view parameters includes at least three view parameters, the at least three view parameters being associated with at least three view cameras respectively, wherein calculating the equivalent monocular view parameter associated with the equivalent monocular camera includes: calculating the equivalent monocular view parameter associated with the equivalent monocular camera in an iterative manner based on the at least three view parameters.

[0033] The method described in any of the above embodiments, wherein the at least three viewpoint parameters include a first viewpoint parameter, a second viewpoint parameter, and a third viewpoint parameter, wherein the first viewpoint parameter, the second viewpoint parameter, and the third viewpoint parameter are respectively associated with three viewpoint cameras, wherein calculating the equivalent monocular viewpoint parameter associated with the equivalent monocular camera includes: determining an intermediate viewpoint parameter based on the first viewpoint parameter and the second viewpoint parameter; and determining the equivalent monocular viewpoint parameter based on the intermediate viewpoint parameter and the third viewpoint parameter.

[0034] As described in any of the above methods, wherein the at least three viewpoint parameters include a first viewpoint parameter, a second viewpoint parameter, a third viewpoint parameter, and a fourth viewpoint parameter, wherein the first viewpoint parameter, the second viewpoint parameter, the third viewpoint parameter, and the fourth viewpoint parameter are respectively associated with four viewpoint cameras, wherein calculating the equivalent monocular viewpoint parameter associated with the equivalent monocular camera includes: determining a first intermediate viewpoint parameter based on the first viewpoint parameter and the second viewpoint parameter; determining a second intermediate viewpoint parameter based on the third viewpoint parameter and the fourth viewpoint parameter; and determining the equivalent monocular viewpoint parameter based on the first intermediate viewpoint parameter and the second intermediate viewpoint parameter.

[0035] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0036] Another aspect of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0037] The apparatus, system, and method according to embodiments of the present invention can eliminate redundancy in rendering computation during neural multi-view stereo rendering on general-purpose computing devices, thereby saving rendering overhead and improving rendering efficiency. Attached Figure Description

[0038] Various embodiments of the present invention are described in conjunction with the accompanying drawings.

[0039] Figure 1 This is a schematic diagram showing the observation range of the left-eye and right-eye cameras in a binocular stereo camera.

[0040] Figure 2 This illustrates the processing flow of the neural rendering pipeline.

[0041] Figure 3 This is a schematic diagram illustrating the alternating execution of neural rendering processes by the left-eye and right-eye cameras in existing binocular stereo rendering technology.

[0042] Figure 4 A block diagram of a graphics processing apparatus according to some embodiments of the present invention is shown.

[0043] Figure 5 A schematic diagram illustrating the relationship between the observation range of a binocular camera and an equivalent monocular camera according to some embodiments of the present invention.

[0044] Figure 6 The process of performing neural rendering for the viewpoint of an equivalent monocular camera is illustrated according to some embodiments of the present invention.

[0045] Figure 7 A schematic diagram illustrating the relationship between the observation range of a trinocular camera and an equivalent monocular camera according to some embodiments of the present invention.

[0046] Figure 8 A schematic diagram illustrating the relationship between the observation range of a quad-camera and an equivalent monocular camera according to some embodiments of the present invention.

[0047] Figure 9 A block diagram of a system for graphics processing according to some embodiments of the present invention is shown.

[0048] Figure 10 A flowchart illustrating a method for graphics processing according to some embodiments of the present invention is shown.

[0049] Figure 11 A block diagram illustrating the process of calculating equivalent monocular view parameters for multiple view cameras according to some embodiments of the present invention.

[0050] Figure 12 A flowchart illustrating the process of rendering a set of neural primitives into left and right eye images during neural rendering for the perspective of a binocular stereo camera according to some embodiments of the present invention is shown.

[0051] Figure 13 A flowchart illustrating the process of calculating equivalent monocular view parameters in an iterative manner based on three view parameters according to some embodiments of the present invention is shown.

[0052] Figure 14 A flowchart illustrating the process of iteratively calculating equivalent monocular view parameters based on four view parameters according to some embodiments of the present invention is shown.

[0053] Figure 15 A flowchart illustrating the process of continuously performing neural rendering calculations on multiple scenes for multiple viewpoint cameras according to some embodiments of the present invention is shown.

[0054] Figure 16 A block diagram of a computer-readable storage medium according to some embodiments of the present invention is shown.

[0055] Figure 17 A block diagram of a computer program product according to some embodiments of the present invention is shown. Detailed Implementation

[0056] In multi-view stereo rendering, such as the binocular stereo rendering required for VR / AR applications, it is usually necessary to have both a left-eye camera and a right-eye camera capture the same scene.

[0057] Figure 1 This is a schematic diagram showing the observation range of the left-eye view camera 110 and the right-eye view camera 120 in a binocular stereo camera.

[0058] like Figure 1 As shown, the observation range of the left-eye camera 110 is defined by lines 112 and 114, while the observation range of the right-eye camera 120 is defined by lines 122 and 124. Typically, the relative position parameters between the left and right eye cameras remain constant. For example, the range is usually between 58 and 70 millimeters, and the rotation parameters remain consistent.

[0059] like Figure 1 As shown, the observation range of the left-eye camera 110 may include scene region 130 and scene region 140, and the observation range of the right-eye camera 120 may include scene region 130 and scene region 150. Among them, the observation range of the left-eye camera 110 and the observation range of the right-eye camera 120 have a redundant scene region 130 (i.e., scene regions that overlap with each other).

[0060] Figure 2 The processing procedure 200 of the neural rendering pipeline is shown. The processing procedure 200 can be implemented by the neural rendering pipeline of the graphics processing device calling computing resources.

[0061] At position 210, input the camera parameters in the view frustum space associated with the current viewpoint. As an example, camera parameters may include the pose matrix, projection matrix, resolution, etc., associated with the camera.

[0062] At position 220, frustum filtering is performed to filter out the neural features to be used in subsequent rendering calculations, and a neural feature mask is calculated to index the location of the neural features. For example... Figure 2As shown, the view frustum may include neural features 221, 222, 223, 224, 225, and 226. Neural features 221, 222, and 223 are filtered out to participate in subsequent rendering calculations. The view frustum filtering stage in neural rendering primarily aims to filter out the neural features most relevant to the currently captured scene, avoiding the need for subsequent decoding and rendering of all neural features, thereby improving rendering efficiency and saving computational resources.

[0063] At position 230, neural feature decoding is performed to decode the filtered neural features into corresponding neural primitives. For example... Figure 2 As shown, neural feature 221 is decoded into neural primitives 231, 232, and 233; neural feature 222 is decoded into neural primitives 234, 235, and 236; and neural feature 223 is decoded into neural primitives 237, 238, and 239. The neural feature decoding stage is unique to neural rendering compared to traditional 3D rendering. This stage consumes significant computational resources and takes longer, making it a major computational bottleneck in neural rendering.

[0064] At position 240, neural primitives 231, 232, 233, 234, 235, 236, 237, 238, and 239 are output to the rendering buffer.

[0065] At position 250, neural primitives in the render cache are rendered to generate render image 260.

[0066] Figure 3 This is a schematic diagram illustrating the alternating execution of neural rendering processing 300 by the left-eye camera 110 and the right-eye camera 120 in existing binocular stereo rendering technology. The neural rendering processing 300 can be... Figure 2 The diagram illustrates the processing flow 200 of the neural rendering pipeline. The neural rendering process 300 can be implemented by the neural rendering pipeline of the graphics processing unit calling upon computational resources. Figure 3 The same legend is used to represent the correspondence between neural features and neural primitives.

[0067] like Figure 3 As shown, in existing binocular stereo rendering technology, the viewpoints of the left-eye camera 110 and the right-eye camera 120 require separate neural rendering processes. In other words, the neural rendering process needs to be executed twice. (See reference...) Figure 2 The processing steps of the neural rendering pipeline are described in detail, and will not be repeated here.

[0068] It is worth noting that at 320a or 320b, frustum filtering is performed to filter out the neural features to be involved in subsequent rendering calculations, and a neural feature mask is calculated to index the location of the neural features. For example... Figure 3 As shown, at 320a, neural features 321a, 322a, and 323a are filtered out. At 320b, neural features 321b, 322b, and 323b are filtered out. Notably, neural features 321a and 321b are duplicated neural features. In other words, in existing binocular stereo rendering technology, redundancy occurs in the cone-space filtering stages at 320a and 320b for neural features 321a associated with the left-eye camera 110 and 322b associated with the right-eye camera 120.

[0069] Additionally, it is worth noting that in the subsequent neural decoding stage at 330a or 330b, the filtered neural features will be decoded into corresponding neural primitives. For example... Figure 3 As shown, neural feature 321a is decoded into neural primitives 331a, 332a, and 333a; neural feature 321b is decoded into neural primitives 334b, 335b, and 336a. Notably, neural primitives 331a, 332a, and 333a decoded from neural feature 321a are duplicates of neural primitives 334b, 335b, and 336b decoded from neural feature 322b. In other words, in existing binocular stereo rendering technology, redundancy occurs in the neural feature decoding stages at points 330a and 330b for neural feature 321a associated with the left-eye camera 110 and neural feature 322b associated with the right-eye camera 120.

[0070] Figure 4 A block diagram of a graphics processing apparatus 400 according to some embodiments of the present invention is shown.

[0071] The graphics processing device 400 may be a GPU or other graphics processing device. The graphics processing device 400 may include computing resources 410 and a neural rendering pipeline 420. The computing resources 410 may include various computing units (e.g., integer computing units, floating-point computing units, etc.), computing cores (e.g., ray tracing cores, tensor cores, etc.), etc.

[0072] The neural rendering pipeline 420 can be implemented via software programming to invoke computing resources 410 to perform the neural rendering process (e.g., in conjunction with the above). Figure 2 and Figure 3 (Description of the neural rendering process).

[0073] The neural rendering pipeline 420 can call upon computing resources 410 to obtain multiple viewpoint parameters, which are associated with multiple viewpoint cameras. These multiple viewpoint cameras can be configured to capture the same scene from different perspectives. As an example, the multiple viewpoint cameras can be binocular stereo cameras, trinocular cameras, or tetranocular cameras, and the camera parameters can include pose matrices, projection matrices, resolutions, etc., associated with the cameras.

[0074] The neural rendering pipeline 420 can call upon computing resources 410 to calculate equivalent monocular view parameters associated with an equivalent monocular camera, based on multiple view parameters. As an example, the equivalent monocular view parameters associated with an equivalent monocular camera can be calculated based on the left and right eye view parameters of a binocular stereo camera, or based on multiple view parameters from multiple binocular cameras, as will be discussed below. Figures 5-8 Detailed description.

[0075] The neural rendering pipeline 420 can call upon computing resources 410 to filter out a set of neural features within the frustum space associated with the equivalent monocular viewpoint parameter.

[0076] The neural rendering pipeline 420 can call upon computing resources 410 to decode the set of neural features into a set of neural primitives.

[0077] The neural rendering pipeline 420 can call upon computing resources 410 to render the collection of neural primitives into multiple images based on the multiple viewpoint parameters.

[0078] Some implementations fuse multiple independent neural rendering processes from multi-view cameras in each image frame into a single complete rendering flow, reducing rendering time and improving rendering efficiency, while eliminating computational redundancy introduced by overlapping viewing fields (including references). Figure 3 The redundancy in the discussed frustum space filtering stage and neural feature decoding stage is reduced, and rendering overhead is lowered (i.e., only monocular rendering overhead is required). Furthermore, unlike existing improvements to traditional 3D rendering methods that are not applicable to neural rendering, the embodiments of this disclosure are suitable for neural rendering and effectively improve the rendering efficiency of the neural rendering pipeline.

[0079] In some embodiments, each of the plurality of viewpoint parameters may include at least the pose matrix and projection matrix of the corresponding viewpoint camera among the plurality of viewpoint cameras. In some embodiments, the equivalent monocular viewpoint parameter may include at least the pose matrix and projection matrix of the equivalent monocular camera.

[0080] In some embodiments, a minimum fitting method can be used to calculate the equivalent monocular viewpoint parameters associated with the equivalent monocular camera. In other embodiments, a non-minimum fitting method can also be used to calculate the equivalent monocular viewpoint parameters associated with the equivalent monocular camera.

[0081] In some embodiments, the neural rendering pipeline may invoke computational resources to determine the frustum space associated with the equivalent monocular viewpoint parameters by ensuring that the boundaries of the frustums of the equivalent monocular camera are parallel to the boundaries of the frustums of the multiple viewpoint cameras, and that the observation range of the equivalent monocular camera is no greater than the sum of the observation ranges of the multiple viewpoint cameras.

[0082] The following will combine Figures 5-7 Describes the method for determining the observation range of an equivalent monocular camera in the case of stereo observation with multi-view cameras.

[0083] Figure 5 A schematic diagram illustrating the relationship between the observation range of a binocular camera and an equivalent monocular camera 560 according to some embodiments of the present invention is shown. The binocular camera may include a left-eye view camera 510 and a right-eye view camera 520, wherein the left-eye view camera 510 may be a reference... Figure 1 and Figure 2 The description of the left-eye view camera 110 and the right-eye view camera 520 can be used as a reference. Figure 1 and Figure 2 The camera described is a 120-degree right-eye view camera.

[0084] like Figure 5 As shown, the observation range of the left-eye camera 510 can be defined by lines 512 and 514, and the observation range of the right-eye camera 520 can be defined by lines 122 and 124. Typically, the relative position parameters between the left and right eye cameras remain constant. For example, the range is usually between 58 and 70 mm, and the rotation parameters remain consistent. The observation range of the left-eye camera 510 may include scene area 530 and scene region 540, and the observation range of the right-eye camera 520 may include scene region 530 and scene region 550. Notably, the observation ranges of the left-eye camera 510 and right-eye camera 520 share a redundant scene region 530.

[0085] In some embodiments, the multiple view parameters may include a first view parameter and a second view parameter, the first view parameter being associated with a left-eye view camera 510 and the second view parameter being associated with a right-eye view camera 520.

[0086] In some embodiments, the neural rendering pipeline may invoke computational resources to compute equivalent monocular view parameters associated with the equivalent monocular camera 560 based on first and second view parameters.

[0087] In some embodiments, the neural rendering pipeline may invoke computational resources to determine the frustum space associated with the equivalent monocular viewpoint parameters by ensuring that the boundaries of the frustums of the equivalent monocular camera 560 are parallel to the boundaries of the frustums of the left and right eye viewpoint cameras, and that the observation range of the equivalent monocular camera 560 is no greater than the sum of the observation ranges of the multiple viewpoint cameras. For example... Figure 5 As shown, the view frustum of the equivalent monocular camera 560 can be defined by line 512 of the left-eye view camera and line 524 of the right-eye view camera 520. Compared to a binocular stereo camera, the view frustum of the equivalent monocular camera 560 does not have a large degree of overlap, thus avoiding the introduction of a large amount of unnecessary redundancy in the rendering calculation process.

[0088] In some embodiments, the plurality of view parameters may further include at least three view parameters, which are respectively associated with at least three view cameras.

[0089] In some embodiments, for cases with at least three viewpoint parameters, equivalent monocular viewpoint parameters can be calculated using an iterative method. The following will refer to... Figures 7-8 The paper further describes the calculation of equivalent monocular viewpoint parameters based on trinocular and tetranocular cameras using an iterative method.

[0090] Figure 6 The present invention illustrates a process 600 for performing neural rendering for the viewpoint of an equivalent monocular camera according to some embodiments thereof. Process 600 can be derived from the foregoing. Figure 4 The graphics processing device 400 described is executed, but the scope of the invention is not limited thereto. As an example, process 600 may be performed by... Figure 4 The neural rendering pipeline 420 of the graphics processing unit 400 calls upon computing resources 410 to execute. Figure 6 The same legend is used to represent the correspondence between neural features and neural primitives.

[0091] First, the equivalent monocular viewpoint parameter 620 is determined. The equivalent monocular viewpoint parameter 620 can be calculated based on viewpoint parameter 612 associated with one viewpoint camera and viewpoint parameter 614 associated with another viewpoint camera. These two viewpoint cameras can be configured to capture the same scene from different viewpoints. As an example, viewpoint parameter 612 may include pose matrix, projection matrix, resolution, etc., associated with one viewpoint camera, and viewpoint parameter 614 may include pose matrix, projection matrix, resolution, etc., associated with the other viewpoint camera. In some embodiments, the two viewpoint cameras can be a left-eye viewpoint camera and a right-eye viewpoint camera, respectively. In some embodiments, the equivalent monocular viewpoint parameter 620 can also be calculated based on three or more viewpoint parameters associated with three or more viewpoint cameras, respectively. The equivalent monocular viewpoint parameter 620 can be based on the above combined with... Figure 5 Calculate in a descriptive way

[0092] In some embodiments, the viewing angle parameter 612 can be an intermediate viewing angle parameter, which can be calculated from the viewing angle parameters associated with two or more viewing angle cameras. The viewing angle parameter 614 can be a viewing angle parameter associated with a monocular viewing angle camera. This can correspond to the case of a trinocular camera, which will be discussed below. Figure 7 The process of calculating the equivalent monocular viewpoint parameter in this case is further described. In some embodiments, viewpoint parameter 612 can be an intermediate viewpoint parameter, which can be calculated from viewpoint parameters associated with two or more viewpoint cameras. Viewpoint parameter 614 can also be an intermediate viewpoint parameter, which can be calculated from viewpoint parameters associated with two or more viewpoint cameras. This can correspond, for example, to the case of a quad-camera setup, as will be discussed below. Figure 8 The process of calculating the equivalent monocular viewpoint parameters in this case is further described.

[0093] In some embodiments, a minimum fitting method may be used to calculate the equivalent monocular viewing angle parameter 620. In other embodiments, a non-minimum fitting method may be used to calculate the equivalent monocular viewing angle parameter 620.

[0094] At position 630, input the equivalent monocular viewpoint parameter 620. As an example, the equivalent monocular viewpoint parameter 620 may include the pose matrix, projection matrix, resolution, etc., associated with the corresponding equivalent monocular camera.

[0095] Subsequently, at position 640, a set 647 of neural features can be filtered out within the cone space associated with the equivalent monocular visual parameter 620. For example... Figure 6 As shown, the view cone space may include neural features 641, 642, 643, 644, 645, and 646. Neural features 641, 642, and 643 are filtered out and formed as a set of neural features 647 to participate in subsequent rendering calculations. The positions of neural features 641, 642, and 643 can be indexed using corresponding neural feature masks.

[0096] Subsequently, at position 650, the set of neural features 647 can be decoded into a set of neural primitives. The filtered neural features 641, 642, and 643 are then decoded at position 650 into their corresponding neural primitives. For example... Figure 6 As shown, neural feature 641 is decoded into neural primitives 651, 652, and 653; neural feature 642 is decoded into neural primitives 654, 655, and 656; and neural feature 643 is decoded into neural primitives 657, 658, and 659.

[0097] At position 660, neural primitives 651, 652, 653, 654, 655, 656, 657, 658, and 659 are output to the rendering buffer.

[0098] Subsequently, at 670, the neural primitives in the rendering cache can be rendered as a rendered image 682 based on the viewpoint parameter 612, and the neural primitives in the rendering cache can be rendered as a rendered image 684 based on the viewpoint parameter 614. The rendered image 682 can be associated with a viewpoint camera (e.g., a left-eye viewpoint camera) associated with the viewpoint parameter 612, and the rendered image 684 can be associated with a viewpoint camera (e.g., a right-eye viewpoint camera) associated with the viewpoint parameter 614.

[0099] In an embodiment where equivalent monocular view parameters are calculated based on three or more view parameters, the set of neural primitives in the rendering cache can be rendered into three or more corresponding rendered images based on the three or more view parameters respectively.

[0100] In some embodiments, sequential computation or fusion computation can be used to render neural primitives in the rendering cache into rendering images 682 and 684.

[0101] In some embodiments, the rendering stage at 670 may be performed by a single graphics processing device (e.g., Figure 4 The rendering at 670 may be performed by a graphics processing device 400. In some embodiments, the rendering at 670 may be performed by a plurality of graphics processing devices (e.g., graphics processing device 400). Figure 4 The graphics processing device 400 and another graphics processing device, or in combination below Figure 9 The graphics processing unit 910 and graphics processing unit 950 in the system 900 described are executed.

[0102] from Figure 6 As can be seen, in the cone-space filtering stage at 640, the filtered neural features 641, 642, and 643 do not contain duplicate neural features. Subsequently, in the neural feature decoding stage at 650, the decoded neural primitives 651, 652, 653, 654, 655, 656, 657, 658, and 659 also do not contain duplicate neural primitives. Compared to the reference... Figure 3In existing binocular stereo rendering techniques, the neural rendering process is performed alternately by the left-eye and right-eye view cameras. In the embodiments of this invention, the computational redundancy caused by independent rendering from multiple viewpoints is eliminated in both the frustum filtering stage at 640 and the subsequent neural feature decoding stage at 650, thus reducing rendering overhead. Furthermore, the independent rendering of multiple viewpoints in each image frame is merged into a single complete rendering process, shortening the rendering time and improving rendering efficiency.

[0103] Figure 7 A schematic diagram illustrating the relationship between the observation range of a trinocular camera and an equivalent monocular camera according to some embodiments of the present invention.

[0104] like Figure 7 As shown, the trinocular camera may include a field-of-view camera 710, a field-of-view camera 720, and a field-of-view camera 730. The observation range of the field-of-view camera 710 can be defined by lines 712 and 714, the observation range of the field-of-view camera 720 can be defined by lines 722 and 724, and the observation range of the field-of-view camera 730 can be defined by lines 732 and 734. Figure 7 As can be seen, the observation range of the viewpoint camera 710 can include scene area 760 and scene area 770, and the observation range of the viewpoint camera 730 can include scene area 780 and scene area 790. The observation range of the viewpoint camera 710 overlaps with the observation range of the viewpoint camera 720 in scene area 770, and the observation range of the viewpoint camera 720 overlaps with the observation range of the viewpoint camera 730 in scene area 780.

[0105] In some embodiments, intermediate view parameters associated with intermediate view cameras 740 can be determined based on view parameters associated with view cameras 710 and 720. Equivalent monocular view parameters associated with equivalent monocular cameras 750 can also be determined based on intermediate view parameters associated with intermediate view cameras 740 and 730. As an example, the equivalent monocular camera parameters can still be performed as described in the reference... Figure 2 The neural rendering process described in 200, such as Figure 6 The neural rendering process described in 600.

[0106] In some embodiments, the frustum space associated with the equivalent monocular viewpoint parameters can be determined by making each boundary of the frustum of the equivalent monocular camera 750 parallel to each boundary of the frustum of the plurality of viewpoint cameras, and making the observation range of the equivalent monocular camera 750 no greater than the sum of the observation ranges of the plurality of viewpoint cameras.

[0107] Figure 8A schematic diagram illustrating the relationship between the observation range of a quad-camera and an equivalent monocular camera according to some embodiments of the present invention.

[0108] like Figure 8 As shown, the four-lens camera system may include a field-view camera 810, a field-view camera 820, a field-view camera 830, and a field-view camera 840. The observation range of the field-view camera 810 can be defined by lines 812 and 814, the observation range of the field-view camera 820 can be defined by lines 822 and 824, the observation range of the field-view camera 830 can be defined by lines 832 and 834, and the observation range of the field-view camera 840 can be defined by lines 842 and 844. Figure 8 As can be seen, the observation range of the perspective camera 810 can include scene regions 880-1 and 890-1, the observation range of the perspective camera 820 can include scene regions 890-1 and 890-2, the observation range of the perspective camera 830 can include scene regions 890-2 and 890-3, and the observation range of the perspective camera 840 can include scene regions 890-3 and 880-2. Among these, the observation ranges of perspective cameras 810 and 820 overlap in scene region 890-1, 820 and 830 overlap in scene region 890-2, and 830 and 840 overlap in scene region 890-3.

[0109] In some embodiments, the intermediate viewing angle parameters associated with the intermediate viewing angle camera 850 may be determined based on the viewing angle parameters associated with the viewing angle camera 810 and the viewing angle parameters associated with the viewing angle camera 820.

[0110] In some embodiments, the intermediate viewing angle parameters associated with the intermediate viewing angle camera 860 may be determined based on the viewing angle parameters associated with the viewing angle camera 830 and the viewing angle parameters associated with the viewing angle camera 840.

[0111] In some embodiments, the equivalent monocular view parameters associated with the equivalent monocular camera 870 can be determined based on the intermediate view parameters associated with the intermediate view camera 850 and the intermediate view parameters associated with the intermediate view camera 860. As an example, the equivalent monocular camera parameters can still be used to perform operations as described in the reference... Figure 2 The neural rendering process described in 200, such as Figure 6 The neural rendering process described in 600.

[0112] In some embodiments, the frustum space associated with the equivalent monocular viewpoint parameters can be determined by making each boundary of the frustum of the equivalent monocular camera 870 parallel to each boundary of the frustum of the plurality of viewpoint cameras, and by making the observation range of the equivalent monocular camera 870 no greater than the sum of the observation ranges of the plurality of viewpoint cameras.

[0113] According to another aspect of the present invention, a system for graphics processing is provided.

[0114] Figure 9 A block diagram of a system 900 for graphics processing according to some embodiments of the present invention is shown.

[0115] System 900 may include a graphics processing unit 910, a camera 920, and a camera 930. The graphics processing unit 910 may be used in conjunction with the above. Figure 4 The graphics processing device 400 described is similar. Cameras 920 and 930 may be coupled to the graphics processing device 910. Cameras 920 and 930 may be configured as stereo cameras and configured to continuously capture scenes. In some embodiments, the graphics processing device 910 may be configured to calculate equivalent monocular camera parameters based on the viewpoint parameters of cameras 920 and 930, and subsequently perform a neural rendering process.

[0116] In some embodiments, system 900 may further include additional cameras, such as camera 940, etc. Camera 940, etc., may be coupled to graphics processing device 910. Cameras 920, 930, ..., 940, etc., may be configured as multi-view stereo cameras and configured to continuously capture the scene from different viewpoints. In some embodiments, graphics processing device 910 may be configured to calculate equivalent monocular camera parameters based on viewpoint parameters associated with camera 920, camera 930, ..., camera 940, respectively, and subsequently perform a neural rendering process.

[0117] In some embodiments, system 900 may further include an additional graphics processing device 950. The graphics processing device 950 may be coupled to the graphics processing device 910. The graphics processing device 950 may be combined with the above. Figure 4 The graphics processing device 400 described is similar. In some embodiments, graphics processing devices 910 and 950 may be configured to compute in parallel to obtain multiple rendered images associated with multiple viewpoint cameras.

[0118] According to another aspect of the present invention, a method for graphics processing is provided.

[0119] Figure 10 A flowchart illustrating a method 1000 for graphics processing according to some embodiments of the present invention is shown. As an example, method 1000 can be performed via... Figure 4The neural rendering pipeline 420 of the device 400 calls its computing resources 410 to perform, but the scope of the invention is not limited thereto.

[0120] Method 1000 may include: at block 1010, acquiring multiple viewpoint parameters, wherein the multiple viewpoint parameters are associated with multiple viewpoint cameras, and the multiple viewpoint cameras are configured to capture the same scene from different viewpoints. As an example, the multiple viewpoint cameras may be a stereo camera, a trinocular camera, or a quad-camera, etc. As an example, the viewpoint parameters may include a pose matrix, projection matrix, resolution, etc., associated with the camera. In some embodiments, each of the multiple viewpoint parameters includes at least the pose matrix and projection matrix of the corresponding viewpoint camera among the multiple viewpoint cameras. In some embodiments, the equivalent monocular viewpoint parameter includes at least the pose matrix and projection matrix of the equivalent monocular camera.

[0121] Method 1000 may include: at block 1020, calculating equivalent monocular view parameters associated with an equivalent monocular camera based on a plurality of view parameters. In some embodiments, calculating equivalent monocular view parameters associated with an equivalent monocular camera based on a plurality of view parameters may include: at block 1021, using a minimum fitting method to calculate equivalent monocular view parameters associated with an equivalent monocular camera. In other embodiments, calculating equivalent monocular view parameters associated with an equivalent monocular camera based on a plurality of view parameters may further include using a non-minimum fitting method to calculate equivalent monocular view parameters associated with an equivalent monocular camera.

[0122] Method 1000 may include: at box 1030, filtering out a set of neural features in a cone space associated with equivalent monocular viewpoint parameters.

[0123] Method 1000 may include: at box 1040, decoding the set of neural features into a set of neural primitives.

[0124] Method 1000 may include: at block 1050, rendering the set of neural primitives into multiple images based on multiple viewpoint parameters, respectively. In some embodiments, rendering the set of neural primitives into multiple images based on multiple viewpoint parameters may include: at block 1051, rendering the set of neural primitives into multiple images using sequential computation or fusion computation.

[0125] In some embodiments, method 1000 may further include: at block 1060, determining the frustum space associated with the equivalent monocular viewpoint parameters by making the boundaries of the frustums of the equivalent monocular camera parallel to the boundaries of the frustums of the plurality of viewpoint cameras, and making the observation range of the equivalent monocular camera no greater than the sum of the observation ranges of the plurality of viewpoint cameras.

[0126] Some implementations fuse multiple independent neural rendering processes from multi-view cameras in each image frame into a single complete rendering flow, reducing rendering time and improving rendering efficiency, while eliminating computational redundancy introduced by overlapping viewing fields (including references). Figure 3 The redundancy in the discussed frustum space filtering stage and neural feature decoding stage is reduced, and rendering overhead is lowered (i.e., only monocular rendering overhead is required). Furthermore, unlike existing improvements to traditional 3D rendering methods that are not applicable to neural rendering, the embodiments of this disclosure are suitable for neural rendering and effectively improve the rendering efficiency of the neural rendering pipeline.

[0127] Figure 11 A block diagram illustrating a process 1100 for calculating equivalent monocular view parameters for multiple view cameras according to some embodiments of the present invention. Process 1100 can be combined with the above description. Figure 10 Step 1020 in the described method 1000.

[0128] In some embodiments, process 1100 may include: at block 1110, calculating equivalent monocular view parameters associated with an equivalent monocular camera based on first view parameters and second view parameters. In some embodiments, the first view parameters are associated with a left-eye view camera, and the second view parameters are associated with a right-eye view camera. Process 1100 may be combined with the above. Figure 5 The description of the observation range based on a binocular camera is related to the observation range of an equivalent monocular camera.

[0129] In some embodiments, process 1100 may include, at block 1120, iteratively calculating the equivalent monocular view parameters associated with the equivalent monocular camera based on at least three view parameters. Process 1200 may be combined with the above. Figure 7 The description of determining the observation range of an equivalent monocular camera based on a trinocular camera, and combining... Figure 8 The description of the observation range based on a quad-camera is related to the observation range of an equivalent monocular camera. The following will refer to... Figure 13-14 The specific description details the process of iteratively calculating the equivalent monocular viewpoint parameters associated with the equivalent monocular camera.

[0130] Figure 12 A flowchart illustrating process 1200, based on some embodiments of the present invention, for rendering a set of neural primitives into left and right eye images during neural rendering for the perspective of a binocular stereo camera, is shown. Process 1200 can be a combination of the above. Figure 10 Step 1050 in method 1000 is described. Process 1200 can be combined with the above. Figure 6The description states that at 670, a set of neural primitives can be rendered into multiple images 680-1 to 680-n based on multiple viewpoint parameters 610-1 to 610-n, respectively.

[0131] Process 1200 may include: at box 1210, rendering a set of neural primitives as a left-eye image based on first-viewpoint parameters. In some embodiments, the first-viewpoint parameters may be associated with a left-eye view camera.

[0132] Process 1200 may include: at box 1220, rendering a set of neural primitives into an eye image based on second-viewpoint parameters. In some embodiments, the second-viewpoint parameters may be associated with a right-eye view camera. In this way, parallax characteristics in binocular stereo are maintained.

[0133] Figure 13 A flowchart illustrating a process 1300 based on three viewpoint parameters, according to some embodiments of the present invention, iteratively calculates equivalent monocular viewpoint parameters. Process 1300 can be a combination of the above. Figure 10 Step 1020 in method 1000 is described. Process 1300 can be combined with the above. Figure 7 The described observation range based on a trinocular camera is related to the observation range of an equivalent monocular camera.

[0134] Process 1300 may include: at box 1310, determining intermediate view parameters based on first view parameters and second view parameters.

[0135] Process 1300 may include: at box 1320, determining equivalent monocular view parameters based on intermediate view parameters and third view parameters.

[0136] In some embodiments, the first viewpoint parameter, the second viewpoint parameter, and the third viewpoint parameter may be associated with three viewpoint cameras, respectively.

[0137] Figure 14 A flowchart illustrating a process 1400 for iteratively calculating equivalent monocular view parameters based on four view parameters according to some embodiments of the present invention is shown. Process 1400 can be combined with the above. Figure 10 Step 1020 in method 1000 is described. Process 1400 can be combined with the above. Figure 8 The description of the observation range based on a quad-camera is related to the observation range of an equivalent monocular camera.

[0138] Process 1400 may include: at box 1410, determining a first intermediate view parameter based on the first view parameter and the second view parameter.

[0139] Process 1400 may include: at box 1420, determining second intermediate view parameters based on third view parameters and fourth view parameters.

[0140] Process 1400 may include: at block 1430, determining equivalent monocular view parameters based on first intermediate view parameters and second intermediate view parameters.

[0141] In some embodiments, the first viewpoint parameter, the second viewpoint parameter, the third viewpoint parameter, and the fourth viewpoint parameter may be associated with four viewpoint cameras, respectively.

[0142] Figure 15 A flowchart illustrating a process 1500 of performing neural rendering calculations on multiple scenes for multiple viewpoint cameras according to some embodiments of the present invention is shown.

[0143] After starting, process 1500 can proceed to: at box 1510, select the viewpoint of the scene to be rendered.

[0144] Subsequently, process 1500 can proceed to: at box 1520, performing neural rendering associated with the equivalent monocular viewpoint of the scene captured by the camera. The process at box 1520 may include the above-described combination of... Figure 4 The described process of the neural rendering pipeline 420 calling computing resources 410 for execution, combined with Figure 10 The described method 1000 includes each step and combination. Figures 11-14 The methods and processes described, including each step.

[0145] Subsequently, process 1500 can proceed to: at decision box 1530, determine whether to stop rendering.

[0146] If so, process 1500 can end.

[0147] Otherwise, process 1500 can proceed to: at box 1540, proceed to the viewpoint of the next scene.

[0148] After this, process 1500 can continue execution from box 1510.

[0149] Process 1500 enables multiple cameras to continuously perform neural rendering on multiple scenes.

[0150] According to another aspect of the present invention, a computer-readable storage medium is provided.

[0151] Figure 16 A block diagram of a computer-readable storage medium 1600 according to some embodiments of the present invention is shown.

[0152] A computer-readable storage medium 1600 may store a computer program 1650. When executed by a processor, the computer program 1650 implements the above-mentioned combination. Figure 10 Method 1000 described, combined Figure 11 The described process 1100, combined Figure 12 The process described is 1200, combined with Figure 13 The process described is 1300, combined with Figure 14 The process described is 1400, combined with Figure 15 The process described is 1500.

[0153] According to another aspect of the present invention, a computer program product is provided.

[0154] Figure 17 A block diagram of a computer program product 1700 according to some embodiments of the present invention is shown.

[0155] Computer program product 1700 may include computer program 1750. Computer program 1750, when executed by a processor, implements the above-mentioned... Figure 10 Method 1000 described, combined Figure 11 The described process 1100, combined Figure 12 The process described is 1200, combined with Figure 13 The process described is 1300, combined with Figure 14 The process described is 1400, combined with Figure 15 The process described is 1500.

[0156] Various embodiments of the invention have been described with reference to the accompanying drawings. These embodiments are illustrative and not restrictive.

Claims

1. A graphics processing apparatus, comprising: Computing resources; as well as A neural rendering pipeline that invokes the computing resources to perform the following steps: Acquire multiple viewpoint parameters, wherein the multiple viewpoint parameters are respectively associated with multiple viewpoint cameras, and the multiple viewpoint cameras are configured to capture the same scene from different viewpoints; Based on the aforementioned multiple viewpoint parameters, calculate the equivalent monocular viewpoint parameters associated with the equivalent monocular camera; A set of neural features is filtered out within the cone space associated with the equivalent monocular viewpoint parameters; Decode the set of neural features into a set of neural primitives; and The set of neural primitives is rendered into multiple images based on the multiple viewpoint parameters.

2. The graphics processing apparatus as claimed in claim 1, characterized in that, Each of the plurality of viewpoint parameters includes at least the pose matrix and projection matrix of the corresponding viewpoint camera among the plurality of viewpoint cameras; The equivalent monocular viewpoint parameters include at least the pose matrix and projection matrix of the equivalent monocular camera.

3. The graphics processing apparatus as claimed in claim 1, characterized in that, The neural rendering pipeline invokes the computing resources to perform the following steps: The equivalent monocular viewpoint parameters associated with the equivalent monocular camera are calculated using a minimum fit method.

4. The graphics processing apparatus as claimed in claim 1, characterized in that, The neural rendering pipeline invokes the computing resources to perform the following steps: The frustum space associated with the equivalent monocular viewpoint parameters is determined by ensuring that the boundaries of the frustums of the equivalent monocular camera are parallel to the boundaries of the frustums of the plurality of viewpoint cameras, and that the observation range of the equivalent monocular camera is no greater than the sum of the observation ranges of the plurality of viewpoint cameras.

5. The graphics processing apparatus as claimed in claim 1, characterized in that, The neural rendering pipeline invokes the computing resources to perform the following steps: The collection of neural primitives is rendered into multiple images using sequential computation or fusion computation.

6. The graphics processing apparatus as claimed in claim 1, characterized in that, The neural rendering pipeline invokes the computing resources to perform the following steps: The graphics processing device is configured to compute in parallel with at least one other graphics processing device to render the set of neural primitives into multiple images.

7. The graphics processing apparatus as claimed in claim 1, characterized in that, The plurality of view parameters include a first view parameter and a second view parameter, wherein the first view parameter is associated with the left-eye view camera and the second view parameter is associated with the right-eye view camera.

8. The graphics processing apparatus as claimed in claim 7, characterized in that, The neural rendering pipeline invokes the computing resources to perform the following steps: Based on the first viewpoint parameter and the second viewpoint parameter, calculate the equivalent monocular viewpoint parameter associated with the equivalent monocular camera; The set of neural primitives is rendered as a left-eye image based on the first viewpoint parameters, and the set of neural primitives is rendered as a right-eye image based on the second viewpoint parameters.

9. The graphics processing apparatus as claimed in claim 1, characterized in that, The plurality of viewpoint parameters includes at least three viewpoint parameters, each of which is associated with at least three viewpoint cameras; The neural rendering pipeline invokes the computing resources to perform the following steps: Based on the at least three viewpoint parameters, the equivalent monocular viewpoint parameters associated with the equivalent monocular camera are calculated iteratively.

10. The graphics processing apparatus as claimed in claim 9, characterized in that, The at least three viewpoint parameters include a first viewpoint parameter, a second viewpoint parameter, and a third viewpoint parameter, wherein the first viewpoint parameter, the second viewpoint parameter, and the third viewpoint parameter are respectively associated with three viewpoint cameras; The neural rendering pipeline invokes the computing resources to perform the following steps: Based on the first viewpoint parameters and the second viewpoint parameters, determine the intermediate viewpoint parameters; and Based on the intermediate viewpoint parameters and the third viewpoint parameters, the equivalent monocular viewpoint parameters are determined.

11. The graphics processing apparatus as claimed in claim 9, characterized in that, The at least three viewpoint parameters include a first viewpoint parameter, a second viewpoint parameter, a third viewpoint parameter, and a fourth viewpoint parameter, wherein the first viewpoint parameter, the second viewpoint parameter, the third viewpoint parameter, and the fourth viewpoint parameter are respectively associated with four viewpoint cameras; The neural rendering pipeline invokes the computing resources to perform the following steps: Based on the first viewpoint parameters and the second viewpoint parameters, determine the first intermediate viewpoint parameters; Based on the third-view parameters and the fourth-view parameters, the second intermediate-view parameters are determined; as well as The equivalent monocular view parameters are determined based on the first intermediate view parameters and the second intermediate view parameters.

12. A system for graphics processing, comprising: The graphics processing apparatus as described in any one of claims 1-11; as well as Multiple viewpoint cameras, wherein the multiple viewpoint cameras are coupled to the graphics processing device.

13. A method for graphics processing, comprising: Acquire multiple viewpoint parameters, wherein the multiple viewpoint parameters are respectively associated with multiple viewpoint cameras, and the multiple viewpoint cameras are configured to capture the same scene from different viewpoints; Based on the aforementioned multiple viewpoint parameters, calculate the equivalent monocular viewpoint parameters associated with the equivalent monocular camera; A set of neural features is filtered out within the cone space associated with the equivalent monocular viewpoint parameters; Decode the set of neural features into a set of neural primitives; and The set of neural primitives is rendered into multiple images based on the multiple viewpoint parameters.

14. The method as described in claim 13, characterized in that, Each of the plurality of viewpoint parameters includes at least the pose matrix and projection matrix of the corresponding viewpoint camera among the plurality of viewpoint cameras; The equivalent monocular viewpoint parameters include at least the pose matrix and projection matrix of the equivalent monocular camera.

15. The method as described in claim 13, characterized in that, Calculating the equivalent monocular viewpoint parameters associated with the equivalent monocular camera based on the multiple viewpoint parameters includes: using a minimum fit method to calculate the equivalent monocular viewpoint parameters associated with the equivalent monocular camera.

16. The method as described in claim 13, characterized in that, The method further includes: The frustum space associated with the equivalent monocular viewpoint parameters is determined by ensuring that the boundaries of the frustums of the equivalent monocular camera are parallel to the boundaries of the frustums of the plurality of viewpoint cameras, and that the observation range of the equivalent monocular camera is no greater than the sum of the observation ranges of the plurality of viewpoint cameras.

17. The method as described in claim 13, characterized in that, Rendering the set of neural primitives into multiple images based on the multiple viewpoint parameters includes: rendering the set of neural primitives into multiple images using sequential computation or fusion computation.

18. The method as described in claim 13, characterized in that, The plurality of viewpoint parameters include a first viewpoint parameter and a second viewpoint parameter. The first viewpoint parameter is associated with the left-eye view camera, and the second viewpoint parameter is associated with the right-eye view camera. The calculation of equivalent monocular field of view parameters associated with an equivalent monocular camera includes: Based on the first viewpoint parameters and the second viewpoint parameters, calculate the equivalent monocular viewpoint parameters associated with the equivalent monocular camera; and Rendering the set of neural primitives into multiple images based on the multiple viewpoint parameters includes: The set of neural primitives is rendered as a left-eye image based on the first viewpoint parameters; and The set of neural primitives is rendered as a right-eye image based on the second viewpoint parameters.

19. The method as described in claim 13, characterized in that, The plurality of viewpoint parameters includes at least three viewpoint parameters, each of which is associated with at least three viewpoint cameras, wherein... The calculation of equivalent monocular field of view parameters associated with an equivalent monocular camera includes: Based on the at least three viewpoint parameters, the equivalent monocular viewpoint parameters associated with the equivalent monocular camera are calculated iteratively.

20. The method as described in claim 19, characterized in that, The at least three viewpoint parameters include a first viewpoint parameter, a second viewpoint parameter, and a third viewpoint parameter, wherein the first viewpoint parameter, the second viewpoint parameter, and the third viewpoint parameter are respectively associated with three viewpoint cameras. The calculation of equivalent monocular field of view parameters associated with an equivalent monocular camera includes: Based on the first viewpoint parameters and the second viewpoint parameters, determine the intermediate viewpoint parameters; and Based on the intermediate viewpoint parameters and the third viewpoint parameters, the equivalent monocular viewpoint parameters are determined.

21. The method as described in claim 19, characterized in that, The at least three viewpoint parameters include a first viewpoint parameter, a second viewpoint parameter, a third viewpoint parameter, and a fourth viewpoint parameter. The first viewpoint parameter, the second viewpoint parameter, the third viewpoint parameter, and the fourth viewpoint parameter are each associated with four viewpoint cameras. The calculation of equivalent monocular field of view parameters associated with an equivalent monocular camera includes: Based on the first viewpoint parameters and the second viewpoint parameters, determine the first intermediate viewpoint parameters; Based on the third-view parameters and the fourth-view parameters, determine the second intermediate-view parameters; and The equivalent monocular view parameters are determined based on the first intermediate view parameters and the second intermediate view parameters.

22. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method as described in any one of claims 13-21.

23. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 13-21.

Images