Graphics processing device, and system and method for graphics processing

Abstract

The invention discloses a graphics processing device, a system and a method for graphics processing. A graphics processing apparatus includes: a computing resource; and a neural rendering pipeline which calls the computing resources to execute the following steps: obtaining a plurality of view angle parameters, the plurality of view angle parameters being respectively associated with a plurality of view angle cameras, the plurality of view angle cameras being configured to shoot the same scene at different view angles; calculating an equivalent monocular viewing angle parameter associated with the equivalent monocular camera based on the plurality of viewing angle parameters; filtering out a set of neural features within a cone space associated with the equivalent monocular view angle parameter; decoding the set of neural features into a set of neural primitives; and rendering the set of neural primitives into a plurality of images based on the plurality of viewing angle parameters, respectively.

Description

Technical Field

[0001] The present invention relates generally to the field of graphics processing, and more particularly to multi-view stereo rendering. Background Art

[0002] In recent years, Virtual Reality (VR) and Augmented Reality (AR) technologies have developed rapidly and are widely used in various application fields, such as education, film and television, games, and even industrial design and industrial production. High-resolution, high-speed and continuous rendering is the essential requirement of VR / AR technology and an important driver for its further development in the future.

[0003] Traditional binocular stereo rendering usually uses an alternating rendering method, that is, rendering the left eye view first, then the right eye view. Although this method is simple to implement, it will cause the rendering frame rate to drop by 50% of the theoretical performance compared to the monocular rendering method, and may cause image flickering problems when binocular asynchronous display is displayed. For traditional 3D rendering technology, this performance drop can usually still meet the smooth experience requirements of 60-72 frames per second (FPS).

[0004] Existing research work has proposed some solutions to improve the speed and resource utilization efficiency of binocular stereo rendering in VR / AR applications, achieving higher rendering frame rates and better user experience.

[0005] Some studies have proposed a single-pass stereo method (see https: / / developer.nvidia.com / vrworks / graphics / singlepassstereo), which accelerates the rendering process of traditional 3D rendering methods under the premise of binocular stereo by hardware optimization by building a dedicated rendering pipeline at the GPU level. This method aims to optimize the grid-based rasterization process and reduce the time consumption of the GPU-external stage by merging or reducing texture calculations. It is not applicable to the computational process of neural rendering that does not contain any texture calculation steps and the main calculation stage is located in the GPU.

[0006] Some studies have proposed a foveated rendering method (see https: / / dl.acm.org / doi / 10.1145 / 2929464.2929472), which tracks the binocular gaze points of the human eye, identifies high-perception areas and improves rendering precision, while reducing rendering precision outside the area to achieve effective utilization of rendering resources. Although this method improves the utilization of rendering resources, it is still not applicable to the computational process of neural rendering that does not contain any texture calculation steps and the main calculation stage is located in the GPU.

[0007] Cutting-edge neural rendering models (e.g., Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS)) enable building highly realistic and detailed virtual scenes for real-time interactive rendering applications.

[0008] There is a big difference between the rendering process of neural rendering technology and the rendering process of traditional 3D rendering technology. The rendering process of neural rendering technology may include main stages such as cone space filtering, neural feature decoding, and neural primitive rendering. Among them, the neural feature decoding stage is a unique rendering stage in the neural rendering process, which consumes a lot of computing resources and takes a longer computing time. And this stage cannot be rendered through the hardware rendering pipeline built on the GPU for traditional 3D rendering, so it needs to rely on specific software and hardware support. Neural feature decoding is the main computing bottleneck in the neural rendering process, which limits the development of practical applications of neural rendering technology. Compared with traditional 3D rendering technology, neural rendering technology has significant deficiencies in computing efficiency and resource utilization. Especially in the scenario where binocular stereo rendering is required in VR / AR applications, the performance bottleneck of neural rendering is more prominent. For neural rendering technology with higher computing overhead, the performance loss caused by binocular rendering separately will be unacceptable.

[0009] Chinese patent application publication CN 117094877 A discloses a VR multi-graphics card rendering method and system, which divides and distributes the single-frame image rendering task to multiple GPUs, and uses device parallel computing to improve the binocular stereo rendering speed of the system. This method may be suitable for accelerating neural rendering, but there is still computational redundancy in binocular rendering, and it introduces synchronization overhead between devices.

[0010] Chinese patent application publication CN 116883631 A discloses a dual-GPU collaborative rendering method and device based on the UE engine DX12MutiAdapter. The method uses two GPUs for parallel rendering to render binocular left and right eye perspective images respectively, but there is still computational redundancy in binocular rendering and the problem of synchronization overhead between devices is introduced.

[0011] Chinese patent application publication CN 113209604 A discloses a method and system for large-viewing angle VR rendering. The method uses preset picture frames and frame generation technology to reduce the frequency of performing complete rendering, thereby improving the speed of binocular stereo rendering. However, there is still a problem of computational redundancy in binocular rendering.

[0012] In view of the above-mentioned existing technologies in the process of real-time rendering: 1) low rendering efficiency; 2) calculation redundancy; 3) not applicable to neural rendering, there is a need in the art for a VR / AR technology solution that has improvements in at least one of improving rendering efficiency, removing calculation redundancy, and applying neural rendering. Summary of the invention

[0013] In order to provide an improved multi-view stereo rendering solution, the present invention is provided.

[0014] One aspect of the present invention provides a graphics processing device, comprising: computing resources; and a neural rendering pipeline, wherein the neural rendering pipeline calls the computing resources to perform the following steps: obtaining multiple viewing parameters, wherein the multiple viewing parameters are respectively associated with multiple viewing cameras, and the multiple viewing cameras are configured to shoot the same scene from different viewing angles; based on the multiple viewing parameters, calculating equivalent monocular viewing parameters associated with an equivalent monocular camera; filtering out a set of neural features in a viewing cone space associated with the equivalent monocular viewing parameters; decoding the set of neural features into a set of neural primitives; and rendering the set of neural primitives into multiple images based on the multiple viewing parameters, respectively.

[0015] The graphics processing device as described above, wherein each of the multiple viewing angle parameters includes at least the pose matrix and projection matrix of the corresponding viewing angle camera among the multiple viewing angle cameras; and the equivalent monocular viewing angle parameter includes at least the pose matrix and projection matrix of the equivalent monocular camera.

[0016] A graphics processing device as described in any of the above items, wherein the neural rendering pipeline calls the computing resource to perform the following steps: using a minimum fitting method to calculate the equivalent monocular viewing angle parameters associated with the equivalent monocular camera.

[0017] A graphics processing device as described in any of the above items, wherein the neural rendering pipeline calls the computing resource to perform the following steps: determining the viewing cone space associated with the equivalent monocular viewing angle parameter by making each boundary of the viewing cone of the equivalent monocular camera parallel to each boundary of the viewing cone of the multiple viewing angle cameras, and making the observation range of the equivalent monocular camera not greater than the sum of the observation ranges of the multiple viewing angle cameras.

[0018] A graphics processing device as described in any one of the above items, wherein the neural rendering pipeline calls the computing resource to perform the following steps: rendering the set of neural primitives into multiple images using sequential calculations or fused calculations.

[0019] A graphics processing device as described in any of the above items, wherein the neural rendering pipeline calls the computing resource to perform the following steps: causing the graphics processing device to calculate in parallel with at least one other device for graphics processing to render the set of neural primitives into multiple images.

[0020] A graphics processing device as described in any of the above items, wherein the multiple viewing angle parameters include a first viewing angle parameter and a second viewing angle parameter, the first viewing angle parameter is associated with a left-eye viewing angle camera, and the second viewing angle parameter is associated with a right-eye viewing angle camera.

[0021] A graphics processing device as described in any of the above items, wherein the neural rendering pipeline calls the computing resources to perform the following steps: calculating an equivalent monocular perspective parameter associated with an equivalent monocular camera based on the first perspective parameter and the second perspective parameter; rendering the set of neural primitives as a left-eye image based on the first perspective parameter, and rendering the set of neural primitives as a right-eye image based on the second perspective parameter.

[0022] A graphics processing device as described in any of the above items, wherein the multiple viewing angle parameters include at least three viewing angle parameters, and the at least three viewing angle parameters are respectively associated with at least three viewing angle cameras; the neural rendering pipeline calls the computing resource to perform the following steps: based on the at least three viewing angle parameters, iteratively calculate an equivalent monocular viewing angle parameter associated with an equivalent monocular camera.

[0023] A graphics processing device as described in any of the above items, wherein the at least three viewing angle parameters include a first viewing angle parameter, a second viewing angle parameter and a third viewing angle parameter, and the first viewing angle parameter, the second viewing angle parameter and the third viewing angle parameter are respectively associated with three viewing angle cameras; the neural rendering pipeline calls the computing resource to perform the following steps: determining an intermediate viewing angle parameter based on the first viewing angle parameter and the second viewing angle parameter; and determining the equivalent monocular viewing angle parameter based on the intermediate viewing angle parameter and the third viewing angle parameter.

[0024] A graphics processing device as described in any of the above items, wherein the at least three viewing angle parameters include a first viewing angle parameter, a second viewing angle parameter, a third viewing angle parameter and a fourth viewing angle parameter, and the first viewing angle parameter, the second viewing angle parameter, the third viewing angle parameter and the fourth viewing angle parameter are respectively associated with four viewing angle cameras; the neural rendering pipeline calls the computing resource to perform the following steps: determine a first intermediate viewing angle parameter based on the first viewing angle parameter and the second viewing angle parameter; determine a second intermediate viewing angle parameter based on the third viewing angle parameter and the fourth viewing angle parameter; and determine the equivalent monocular viewing angle parameter based on the first intermediate viewing angle parameter and the second intermediate viewing angle parameter.

[0025] Another aspect of the present invention provides a system for graphics processing, comprising: a graphics processing device as described in any one of the above items; and a plurality of view cameras, wherein the plurality of view cameras are coupled to the graphics processing device.

[0026] Another aspect of the present invention provides a method for graphics processing, comprising: obtaining multiple viewing angle parameters, wherein the multiple viewing angle parameters are respectively associated with multiple viewing angle cameras, and the multiple viewing angle cameras are configured to shoot the same scene from different viewing angles; based on the multiple viewing angle parameters, calculating equivalent monocular viewing angle parameters associated with an equivalent monocular camera; filtering out a set of neural features in a visual cone space associated with the equivalent monocular viewing angle parameter; decoding the set of neural features into a set of neural primitives; and rendering the set of neural primitives into multiple images based on the multiple viewing angle parameters, respectively.

[0027] As described above, each of the multiple viewing angle parameters includes at least the pose matrix and projection matrix of the corresponding viewing angle camera among the multiple viewing angle cameras; the equivalent monocular viewing angle parameter includes at least the pose matrix and projection matrix of the equivalent monocular camera.

[0028] A method as described in any of the above items, wherein, based on the multiple viewing angle parameters, calculating the equivalent monocular viewing angle parameters associated with the equivalent monocular camera comprises: using a minimum fitting method to calculate the equivalent monocular viewing angle parameters associated with the equivalent monocular camera.

[0029] The method as described in any of the above items further includes: determining the viewing cone space associated with the equivalent monocular viewing angle parameter by making each boundary of the viewing cone of the equivalent monocular camera parallel to each boundary of the viewing cone of the multiple viewing angle cameras, and making the observation range of the equivalent monocular camera not greater than the sum of the observation ranges of the multiple viewing angle cameras.

[0030] A method as described in any of the above items, wherein rendering the set of neural primitives into multiple images based on the multiple viewing parameters respectively includes: rendering the set of neural primitives into multiple images using sequential calculation or fusion calculation.

[0031] A method as described in any of the above items, wherein the multiple viewing parameters include a first viewing parameter and a second viewing parameter, the first viewing parameter is associated with a left-eye viewing camera, and the second viewing parameter is associated with a right-eye viewing camera, wherein calculating the equivalent monocular viewing parameter associated with the equivalent monocular camera includes: calculating the equivalent monocular viewing parameter associated with the equivalent monocular camera based on the first viewing parameter and the second viewing parameter; and rendering the set of neural graph elements into multiple images based on the multiple viewing parameters respectively includes: rendering the set of neural graph elements into a left-eye image based on the first viewing parameter; and rendering the set of neural graph elements into a right-eye image based on the second viewing parameter.

[0032] A method as described in any of the above items, wherein the multiple viewing angle parameters include at least three viewing angle parameters, and the at least three viewing angle parameters are respectively associated with at least three viewing angle cameras, wherein calculating the equivalent monocular viewing angle parameter associated with the equivalent monocular camera includes: based on the at least three viewing angle parameters, iteratively calculating the equivalent monocular viewing angle parameter associated with the equivalent monocular camera.

[0033] A method as described in any of the above items, wherein the at least three viewing angle parameters include a first viewing angle parameter, a second viewing angle parameter and a third viewing angle parameter, and the first viewing angle parameter, the second viewing angle parameter and the third viewing angle parameter are respectively associated with three viewing angle cameras, wherein calculating the equivalent monocular viewing angle parameter associated with the equivalent monocular camera includes: determining an intermediate viewing angle parameter based on the first viewing angle parameter and the second viewing angle parameter; and determining the equivalent monocular viewing angle parameter based on the intermediate viewing angle parameter and the third viewing angle parameter.

[0034] A method as described in any of the above items, wherein the at least three viewing angle parameters include a first viewing angle parameter, a second viewing angle parameter, a third viewing angle parameter and a fourth viewing angle parameter, and the first viewing angle parameter, the second viewing angle parameter, the third viewing angle parameter and the fourth viewing angle parameter are respectively associated with four viewing angle cameras, wherein calculating the equivalent monocular viewing angle parameter associated with the equivalent monocular camera includes: determining a first intermediate viewing angle parameter based on the first viewing angle parameter and the second viewing angle parameter; determining a second intermediate viewing angle parameter based on the third viewing angle parameter and the fourth viewing angle parameter; and determining the equivalent monocular viewing angle parameter based on the first intermediate viewing angle parameter and the second intermediate viewing angle parameter.

[0035] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0036] Another aspect of the present invention provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any one of the above methods are implemented.

[0037] The apparatus, system and method according to the embodiments of the present invention can realize the redundancy removal of rendering calculations in the process of neural multi-view stereo rendering on a general computing device, thereby saving rendering overhead and improving rendering efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Embodiments of the present invention are described with reference to the accompanying drawings.

[0039] Figure 1 Schematic diagram showing the observation ranges of a left-eye perspective camera and a right-eye perspective camera in a binocular stereo camera.

[0040] Figure 2 Figure 1 shows the processing of the neural rendering pipeline.

[0041] Figure 3 It is a schematic diagram showing a neural rendering process in which the perspectives of a left-eye perspective camera and a right-eye perspective camera are alternately executed in an existing binocular stereo rendering technology.

[0042] Figure 4 A block diagram of a graphics processing device according to some embodiments of the present invention is shown.

[0043] Figure 5 A schematic diagram showing the relationship between the observation range of a binocular camera and an equivalent monocular camera according to some embodiments of the present invention.

[0044] Figure 6 A process of performing neural rendering for an equivalent monocular camera's view according to some embodiments of the present invention is shown.

[0045] Figure 7 A schematic diagram illustrating the relationship between the observation range of a trinocular camera and an equivalent monocular camera according to some embodiments of the present invention.

[0046] Figure 8 A schematic diagram showing the relationship between the observation range of a quadrilateral camera and an equivalent monocular camera according to some embodiments of the present invention.

[0047] Fig. 9 A block diagram of a system for graphics processing according to some embodiments of the present invention is shown.

[0048] Fig.10 A flow chart showing a method for graphics processing according to some embodiments of the present invention.

[0049] Fig.11A block diagram illustrating a process of calculating equivalent monocular view parameters for the views of multiple view cameras according to some embodiments of the present invention.

[0050] Fig.12 A flowchart of a process of rendering a set of neural primitives into left and right eye images in a neural rendering process for the perspective of a binocular stereo camera according to some embodiments of the present invention is shown.

[0051] Fig.13 A flowchart illustrating a process of iteratively calculating an equivalent monocular viewing angle parameter based on three viewing angle parameters according to some embodiments of the present invention.

[0052] Fig.14 A flowchart illustrating a process of iteratively calculating an equivalent monocular viewing angle parameter based on four viewing angle parameters according to some embodiments of the present invention.

[0053] Fig.15 A flowchart illustrating a process of continuously performing neural rendering calculations on multiple scenes for multiple view cameras according to some embodiments of the present invention.

[0054] Fig.16 A block diagram of a computer-readable storage medium according to some embodiments of the invention is shown.

[0055] Fig.17 A block diagram illustrating a computer program product according to some embodiments of the invention. DETAILED DESCRIPTION

[0056] In a multi-eye stereo rendering process, such as a binocular stereo rendering process required by VR / AR applications, a left-eye perspective camera and a right-eye perspective camera are usually involved in shooting the same scene.

[0057] Figure 1 1 is a schematic diagram showing the observation ranges of the left-eye view camera 110 and the right-eye view camera 120 in a binocular stereo camera.

[0058] like Figure 1 As shown, the observation range of the left-eye perspective camera 110 is defined by lines 112 and 114, and the observation range of the right-eye perspective camera 120 can be defined by lines 122 and 124. Generally, the relative position parameters between the left and right eye observation cameras are kept constant. For example, the range is usually between 58 and 70 mm, and the rotation parameters are kept consistent.

[0059] like Figure 1As shown, the observation range of the left-eye perspective camera 110 may include the scene area 130 and the scene area 140, and the observation range of the right-eye perspective camera 120 may include the scene area 130 and the scene area 150. There are redundant scene areas 130 (i.e., scene areas that overlap with each other) in the observation range of the left-eye perspective camera 110 and the observation range of the right-eye perspective camera 120.

[0060] Figure 2 The processing process 200 of the neural rendering pipeline is shown. The processing process 200 can be implemented by calling computing resources of the neural rendering pipeline of the graphics processing device.

[0061] At 210, camera parameters of a viewing cone space associated with a current viewing angle are input. As an example, the camera parameters may include a pose matrix, a projection matrix, a resolution, etc. associated with the camera.

[0062] At 220, a cone space filtering is performed to filter out the neural features to be involved in the subsequent rendering calculation, and a neural feature mask is calculated to index the position of the neural feature. Figure 2 As shown in , the cone space may include neural features 221, neural features 222, neural features 223, neural features 224, neural features 225, and neural features 226, wherein neural features 221, neural features 222, and neural features 223 are filtered out to participate in subsequent rendering calculations. The cone space filtering stage is performed in the neural rendering process mainly to filter out the neural features most relevant to the currently captured scene, avoiding subsequent decoding and rendering of all neural features, so as to improve rendering efficiency and save computing resources.

[0063] At 230, neural feature decoding is performed to decode the filtered neural features into corresponding neural primitives. Figure 2 As shown in , neural feature 221 is decoded into neural grapheme 231, neural grapheme 232, and neural grapheme 233; neural feature 222 is decoded into neural grapheme 234, neural grapheme 235, and neural grapheme 236; neural feature 223 is decoded into neural grapheme 237, neural grapheme 238, and neural grapheme 239. The neural feature decoding stage is a stage unique to the neural rendering process compared to the traditional 3D rendering process. This stage consumes a lot of computing resources and takes a longer computing time. It is the main computing bottleneck in the neural rendering process.

[0064] At 240 , the neural primitives 231 , 232 , 233 , 234 , 235 , 236 , 237 , 238 , and 239 are output to a rendering buffer.

[0065] At 250 , the neural primitives in the render buffer are rendered to generate a rendered image 260 .

[0066] Figure 3 3 is a schematic diagram showing a neural rendering process 300 in which the perspectives of the left-eye perspective camera 110 and the right-eye perspective camera 120 are alternately executed in the existing binocular stereo rendering technology. The neural rendering process 300 may be Figure 2 The processing process 200 of the neural rendering pipeline is shown, and the neural rendering processing process 300 can be implemented by calling computing resources by the neural rendering pipeline of the graphics processing device. Figure 3 The same legend is used to represent the correspondence between neural features and neural graph primitives.

[0067] like Figure 3 As shown, in the existing binocular stereo rendering technology, the perspectives of the left eye perspective camera 110 and the right eye perspective camera 120 need to be subjected to the neural rendering process respectively. In other words, the neural rendering process needs to be performed twice. Figure 2 The processing of the neural rendering pipeline is described in detail and will not be repeated here.

[0068] It is worth noting that at 320a or 320b, a cone space filtering is performed to filter out the neural features to be involved in the subsequent rendering calculation, and a neural feature mask is calculated to index the position of the neural feature. Figure 3 As shown in , at 320a, neural features 321a, 322a, and 323a are filtered out. At 320b, neural features 321b, 322b, and 323b are filtered out. Among them, neural features 321a and 321b are repeated neural features. That is to say, in the existing binocular stereo rendering technology, the neural features 321a associated with the left-eye perspective camera 110 and the neural features 322b associated with the right-eye perspective camera 120 are redundant in the cone space filtering stage at 320a and 320b.

[0069] In addition, it is worth noting that in the subsequent neural decoding stage at 330a or 330b, the filtered neural features will be decoded into corresponding neural primitives. Figure 3As shown in , neural feature 321a is decoded into neural cell 331a, neural cell 332a, and neural cell 333a; neural feature 321b is decoded into neural cell 334b, neural cell 335b, and neural cell 336a. Among them, neural cell 331a, neural cell 332a, and neural cell 333a decoded from neural feature 321a are duplicate neural cells with neural cell 334b, neural cell 335b, and neural cell 336b decoded from neural feature 322b, respectively. That is to say, in the existing binocular stereo rendering technology, the neural feature 321a associated with the left-eye perspective camera 110 and the neural feature 322b associated with the right-eye perspective camera 120 are redundant in the neural feature decoding stage at 330a and 330b.

[0070] Figure 4 A block diagram of a graphics processing device 400 according to some embodiments of the present invention is shown.

[0071] The graphics processing device 400 may be a GPU or other graphics processing device. The graphics processing device 400 may include computing resources 410 and a neural rendering pipeline 420. The computing resources 410 may include various computing units (e.g., integer computing units, floating point computing units, etc.), computing cores (e.g., ray tracing cores, tensor cores, etc.), etc.

[0072] The neural rendering pipeline 420 may be implemented by software programming to call the computing resources 410 to perform the neural rendering process (e.g., Figure 2 and Figure 3 The neural rendering process described in [4].

[0073] The neural rendering pipeline 420 may call the computing resource 410 to obtain a plurality of perspective parameters, wherein the plurality of perspective parameters are respectively associated with a plurality of perspective cameras, and the plurality of perspective cameras may be configured to shoot the same scene from different perspectives. As an example, the plurality of perspective cameras may be a binocular stereo camera, a trinocular camera, or a quadruple camera, and the camera parameters may include a pose matrix, a projection matrix, a resolution, etc. associated with the camera.

[0074] The neural rendering pipeline 420 may call the computing resource 410 to calculate an equivalent monocular viewing angle parameter associated with the equivalent monocular camera based on multiple viewing angle parameters. As an example, the equivalent monocular viewing angle parameter associated with the equivalent monocular camera may be calculated based on the left and right eye viewing angle camera viewing angle parameters of a binocular stereo camera, or may be calculated based on multiple viewing angle parameters of cameras more than binocular, which will be referred to below. Figure 5-Figure 8 Specific description.

[0075] The neural rendering pipeline 420 may call the computing resources 410 to filter out a set of neural features within a viewing cone space associated with the equivalent monocular view parameter.

[0076] The neural rendering pipeline 420 may call the computing resources 410 to decode the set of neural features into a set of neural primitives.

[0077] The neural rendering pipeline 420 may call the computing resource 410 to render the set of neural primitives into a plurality of images based on the plurality of viewing parameters, respectively.

[0078] Some embodiments combine multiple independent neural rendering processes of multi-view cameras in each image frame into a complete rendering process, shortening the rendering time and improving rendering efficiency, while removing the computational redundancy introduced by the overlap of the observation fields (including reference Figure 3 In addition, compared with the improved methods for traditional 3D rendering in the prior art that are not applicable to neural rendering, the embodiments of the present disclosure are applicable to neural rendering and effectively improve the rendering efficiency of the neural rendering process.

[0079] In some embodiments, each of the plurality of viewing angle parameters may include at least a pose matrix and a projection matrix of a corresponding viewing angle camera in the plurality of viewing angle cameras. In some embodiments, the equivalent monocular viewing angle parameter may include at least a pose matrix and a projection matrix of an equivalent monocular camera.

[0080] In some embodiments, a minimum fitting method may be used to calculate the equivalent monocular viewing angle parameters associated with the equivalent monocular camera. In other embodiments, a non-minimum fitting method may also be used to calculate the equivalent monocular viewing angle parameters associated with the equivalent monocular camera.

[0081] In some embodiments, the neural rendering pipeline may call computing resources to determine a viewing cone space associated with an equivalent monocular perspective parameter by making each boundary of the viewing cone of the equivalent monocular camera parallel to each boundary of the viewing cone of multiple view cameras and making the observation range of the equivalent monocular camera no greater than the sum of the observation ranges of the multiple view cameras.

[0082] The following will combine Figure 5-Figure 7 This paper describes the method of determining the observation range of an equivalent monocular camera in the case of multi-camera stereo observation.

[0083] Figure 5 A schematic diagram showing the relationship between the observation range of a binocular camera and an equivalent monocular camera 560 according to some embodiments of the present invention. The binocular camera may include a left-eye perspective camera 510 and a right-eye perspective camera 520, wherein the left-eye perspective camera 510 may be a reference Figure 1 and Figure 2 The left-eye perspective camera 110 and the right-eye perspective camera 520 described above may be referenced. Figure 1and Figure 2 A right eye perspective camera 120 is depicted.

[0084] like Figure 5 As shown, the observation range of the left-eye perspective camera 510 can be limited by lines 512 and 514, and the observation range of the right-eye perspective camera 520 can be limited by lines 122 and 124. Generally, the relative position parameters between the left and right eye observation cameras are kept constant. For example, the range is usually between 58 and 70 mm, and the rotation parameters remain consistent. The observation range of the left-eye perspective camera 510 may include a scene area 530 and a scene area 540, and the observation range of the right-eye perspective camera 520 may include a scene area 530 and a scene area 550. Among them, there is a redundant scene area 530 between the observation range of the left-eye perspective camera 510 and the observation range of the right-eye perspective camera 520.

[0085] In some embodiments, the plurality of perspective parameters may include a first perspective parameter and a second perspective parameter, wherein the first perspective parameter may be associated with the left-eye perspective camera 510 , and the second perspective parameter may be associated with the right-eye perspective camera 520 .

[0086] In some embodiments, the neural rendering pipeline may call computing resources to calculate an equivalent monocular viewing angle parameter associated with the equivalent monocular camera 560 based on the first viewing angle parameter and the second viewing angle parameter.

[0087] In some embodiments, the neural rendering pipeline may call computing resources to determine the viewing cone space associated with the equivalent monocular viewing parameter by making each boundary of the viewing cone of the equivalent monocular camera 560 parallel to each boundary of the viewing cone of the left and right eye viewing cameras, and making the observation range of the equivalent monocular camera 560 no greater than the sum of the observation ranges of multiple viewing cameras. Figure 5 As shown, the viewing cone space of the equivalent monocular camera 560 can be defined by the line 512 of the left eye perspective camera and the line 524 of the right eye perspective camera 520. Compared with the binocular stereo camera, the viewing cone space of the equivalent monocular camera 560 does not overlap in a large range, thereby avoiding the introduction of a large amount of unnecessary redundancy in the rendering calculation process.

[0088] In some embodiments, the plurality of viewing angle parameters may further include at least three viewing angle parameters, where the at least three viewing angle parameters are respectively associated with at least three viewing angle cameras.

[0089] In some embodiments, for the case of at least three viewing angle parameters, the equivalent monocular viewing angle parameter can be calculated by an iterative method. Figure 7-Figure 8 The case of calculating equivalent monocular viewing angle parameters by an iterative method based on a trinocular camera and a quadrocular camera is further described.

[0090] Figure 6FIG. 6 is a process 600 for performing neural rendering for an equivalent monocular camera view according to some embodiments of the present invention. Figure 4 The process 600 may be performed by the graphics processing device 400 described above, but the scope of the present invention is not limited thereto. Figure 4 The neural rendering pipeline 420 of the graphics processing device 400 in the FIG4 calls the computing resource 410 to execute. Figure 6 The same legend is used to represent the correspondence between neural features and neural graph primitives.

[0091] First, determine the equivalent monocular perspective parameter 620. The equivalent monocular perspective parameter 620 can be calculated based on the perspective parameter 612 associated with one perspective camera and the perspective parameter 614 associated with another perspective camera. The two perspective cameras can be configured to shoot the same scene from different perspectives. As an example, the perspective parameter 612 may include a pose matrix, a projection matrix, a resolution, etc. associated with one perspective camera, and the perspective parameter 614 may include a pose matrix, a projection matrix, a resolution, etc. associated with another perspective camera. In some embodiments, the two perspective cameras may be a left-eye perspective camera and a right-eye perspective camera, respectively. In some embodiments, the equivalent monocular perspective parameter 620 can also be calculated based on three or more perspective parameters respectively associated with three or more perspective cameras. The equivalent monocular perspective parameter 620 can be calculated based on the above in combination with the above. Figure 5 Describe the way to calculate

[0092] In some embodiments, the viewing angle parameter 612 may be an intermediate viewing angle parameter, which may be calculated from the viewing angle parameters associated with two or more viewing angle cameras. The viewing angle parameter 614 may be a viewing angle parameter associated with a monocular viewing angle camera. This may correspond to the case of a three-eye camera, which will be described below in conjunction with Figure 7 The process of calculating the equivalent monocular view parameters in this case is further described. In some embodiments, the view parameter 612 may be an intermediate view parameter, which may be calculated by the view parameters associated with two or more view cameras. The view parameter 614 may also be an intermediate view parameter, which may be calculated by the view parameters associated with two or more view cameras. This may correspond to the case of a quad camera, which will be described below in conjunction with Figure 8 The process of calculating the equivalent monocular viewing angle parameters in this case is further described.

[0093] In some embodiments, a minimum fitting method may be used to calculate the equivalent monocular viewing angle parameter 620. In other embodiments, a non-minimum fitting method may be used to calculate the equivalent monocular viewing angle parameter 620.

[0094] At 630, the equivalent monocular view parameters 620 are input. As an example, the equivalent monocular view parameters 620 may include a pose matrix, a projection matrix, a resolution, etc. associated with a corresponding equivalent monocular camera.

[0095] Subsequently, at 640, a set 647 of neural features may be filtered within the viewing cone space associated with the equivalent monocular view parameter 620. Figure 6 As shown in , the visual cone space may include neural features 641, neural features 642, neural features 643, neural features 644, neural features 645, and neural features 646, wherein neural features 641, neural features 642, and neural features 643 are filtered out as a set of neural features 647 to participate in subsequent rendering calculations. Neural features 641, neural features 642, and neural features 643 can index their positions through corresponding neural feature masks.

[0096] Subsequently, at 650, the set of neural features 647 may be decoded into a set of neural graph elements. The filtered neural features 641, 642, and 643 are subjected to neural feature decoding at 650 to be decoded into corresponding neural graph elements. Figure 6 As shown in , neural feature 641 is decoded into neural grapheme 651, neural grapheme 652, and neural grapheme 653; neural feature 642 is decoded into neural grapheme 654, neural grapheme 655, and neural grapheme 656; neural feature 643 is decoded into neural grapheme 657, neural grapheme 658, and neural grapheme 659.

[0097] At 660 , neural primitives 651 , 652 , 653 , 654 , 655 , 656 , 657 , 658 , and 659 are output to a rendering buffer.

[0098] Subsequently, at 670, the neural primitives in the render buffer may be rendered as a rendered image 682 based on the view parameter 612, and the neural primitives in the render buffer may be rendered as a rendered image 684 based on the view parameter 614. The rendered image 682 may be associated with a view camera associated with the view parameter 612 (e.g., a left-eye view camera), and the rendered image 684 may be associated with a view camera associated with the view parameter 614 (e.g., a right-eye view camera).

[0099] In an embodiment where an equivalent monocular viewing angle parameter is calculated based on three or more viewing angle parameters, the set of neural primitives in the rendering buffer may be rendered into corresponding three or more rendered images based on the three or more viewing angle parameters, respectively.

[0100] In some embodiments, the neural primitives in the rendering buffer may be rendered into the rendering image 682 and the rendering image 684 using sequential calculation or fusion calculation.

[0101] In some embodiments, the rendering stage at 670 may be performed by a single graphics processing device (e.g., Figure 4 In some embodiments, the rendering at 670 may be performed by multiple graphics processing devices (e.g., Figure 4 The graphics processing device 400 and another graphics processing device, or the combination of the following Fig. 9 The graphics processing device 910, the graphics processing device 950) in the described system 900 is executed.

[0102] from Figure 6 It can be seen that in the cone space filtering stage at 640, the filtered neural features 641, neural features 642, and neural features 643 do not have repeated neural features. Subsequently, in the neural feature decoding stage at 650, the decoded neural graph 651, neural graph 652, neural graph 653, neural graph 654, neural graph 655, neural graph 656, neural graph 657, neural graph 658, and neural graph 659 also do not have repeated neural graphs. Compared with the reference Figure 3 In the existing binocular stereo rendering technology described, the perspectives of the left-eye view camera and the right-eye view camera alternately perform the neural rendering process. In the embodiment of the present invention, the cone space filtering stage at 640 and the subsequent neural feature decoding stage at 650 remove the computational redundancy generated by the independent rendering of multiple perspectives, thereby reducing the rendering overhead. In addition, the independent rendering of multiple perspectives in each image frame is merged into a complete rendering process, which shortens the time consumption and improves the rendering efficiency.

[0103] Figure 7 A schematic diagram illustrating the relationship between the observation range of a trinocular camera and an equivalent monocular camera according to some embodiments of the present invention.

[0104] like Figure 7 As shown, the three-eye camera may include a viewing angle camera 710, a viewing angle camera 720, and a viewing angle camera 730. The observation range of the viewing angle camera 710 may be defined by lines 712 and 714, the observation range of the viewing angle camera 720 may be defined by lines 722 and 724, and the observation range of the viewing angle camera 730 may be defined by lines 732 and 734. Figure 7As can be seen in FIG, the observation range of the view camera 710 may include the scene area 760 and the scene area 770, and the observation range of the view camera 730 may include the scene area 780 and the scene area 790. The observation range of the view camera 710 overlaps with the observation range of the view camera 720 in the scene area 770, and the observation range of the view camera 720 overlaps with the observation range of the view camera 730 in the scene area 780.

[0105] In some embodiments, the intermediate viewing angle parameters associated with the intermediate viewing angle camera 740 may be determined based on the viewing angle parameters associated with the viewing angle camera 710 and the viewing angle parameters associated with the viewing angle camera 720. The equivalent monocular viewing angle parameters associated with the equivalent monocular camera 750 may also be determined based on the intermediate viewing angle parameters associated with the intermediate viewing angle camera 740 and the viewing angle parameters associated with the viewing angle camera 730. As an example, the method described in reference to the determined equivalent monocular camera parameters may still be performed. Figure 2 The neural rendering process described in 200, such as Figure 6 The neural rendering process 600 is described.

[0106] In some embodiments, the viewing cone space associated with the equivalent monocular viewing angle parameter can be determined by making each boundary of the viewing cone of the equivalent monocular camera 750 parallel to each boundary of the viewing cone of multiple viewing angle cameras, and making the observation range of the equivalent monocular camera 750 no greater than the sum of the observation ranges of the multiple viewing angle cameras.

[0107] Figure 8 A schematic diagram showing the relationship between the observation range of a quadrilateral camera and an equivalent monocular camera according to some embodiments of the present invention.

[0108] like Figure 8 As shown, the four-eye camera may include a viewing angle camera 810, a viewing angle camera 820, a viewing angle camera 830, and a viewing angle camera 840. The observation range of the viewing angle camera 810 may be defined by lines 812 and 814, the observation range of the viewing angle camera 820 may be defined by lines 822 and 824, the observation range of the viewing angle camera 830 may be defined by lines 832 and 834, and the observation range of the viewing angle camera 840 may be defined by lines 842 and 844. Figure 8It can be seen that the observation range of the perspective camera 810 may include the scene area 880-1 and the scene area 890-1, the observation range of the perspective camera 820 may include the scene area 890-1 and the scene area 890-2, the observation range of the perspective camera 830 may include the scene area 890-2 and the scene area 890-3, and the observation range of the perspective camera 840 may include the scene area 890-3 and the scene area 880-2. Among them, the observation range of the perspective camera 810 and the observation range of the perspective camera 820 have an overlapping scene area 890-1, the observation range of the perspective camera 820 and the observation range of the perspective camera 830 have an overlapping scene area 890-2, and the observation range of the perspective camera 830 and the observation range of the perspective camera 840 have an overlapping scene area 890-3.

[0109] In some embodiments, intermediate perspective parameters associated with intermediate perspective camera 850 may be determined based on perspective parameters associated with perspective camera 810 and perspective parameters associated with perspective camera 820 .

[0110] In some embodiments, intermediate perspective parameters associated with intermediate perspective camera 860 may be determined based on perspective parameters associated with perspective camera 830 and perspective parameters associated with perspective camera 840 .

[0111] In some embodiments, the equivalent monocular perspective parameters associated with the equivalent monocular camera 870 may be determined based on the intermediate perspective parameters associated with the intermediate perspective camera 850 and the intermediate perspective parameters associated with the intermediate perspective camera 860. As an example, the equivalent monocular perspective parameters may still be performed as described in reference to the determined equivalent monocular camera parameters. Figure 2 The neural rendering process described in 200, such as Figure 6 The neural rendering process 600 is described.

[0112] In some embodiments, the viewing cone space associated with the equivalent monocular viewing angle parameter can be determined by making each boundary of the viewing cone of the equivalent monocular camera 870 parallel to each boundary of the viewing cone of multiple viewing angle cameras, and making the observation range of the equivalent monocular camera 870 no greater than the sum of the observation ranges of the multiple viewing angle cameras.

[0113] According to another aspect of the present invention, a system for graphics processing is provided.

[0114] Fig. 9 A block diagram of a system 900 for graphics processing according to some embodiments of the present invention is shown.

[0115] System 900 may include a graphics processing device 910, a camera 920, and a camera 930. Graphics processing 910 may be combined with the above Figure 4The graphics processing device 910 is similar to the graphics processing device 400 described above. Cameras 920 and 930 may be coupled to the graphics processing device 910. Cameras 920 and 930 may be configured as binocular stereo cameras and configured to continuously capture scenes. In some embodiments, the graphics processing device 910 may be configured to calculate equivalent monocular camera parameters based on the viewing angle parameters of cameras 920 and 930, and then perform a neural rendering process.

[0116] In some embodiments, the system 900 may further include more cameras, for example, camera 940, etc. Camera 940, etc. may be coupled to the graphics processing device 910. Camera 920, camera 930, ..., camera 940, etc. may be configured as a multi-eye stereo camera and configured to continuously capture scenes from different perspectives. In some embodiments, the graphics processing device 910 may be configured to calculate equivalent monocular camera parameters based on the perspective parameters associated with camera 920, camera 930, ..., camera 940, respectively, and then perform a neural rendering process.

[0117] In some embodiments, the system 900 may further include an additional graphics processing device 950. The graphics processing device 950 may be coupled to the graphics processing device 910. The graphics processing device 950 may be combined with the above Figure 4 The graphics processing device 400 described is similar. In some embodiments, the graphics processing device 910 and the graphics processing device 950 may be configured to compute in parallel to obtain multiple rendered images associated with multiple view cameras.

[0118] According to another aspect of the present invention, a method for graphics processing is provided.

[0119] Fig.10 A flowchart of a method 1000 for image processing according to some embodiments of the present invention is shown. As an example, the method 1000 may be Figure 4 The neural rendering pipeline 420 of the device 400 calls its computing resources 410 to execute, but the scope of the present invention is not limited to this.

[0120] Method 1000 may include: at box 1010, obtaining multiple viewing angle parameters, wherein the multiple viewing angle parameters are respectively associated with multiple viewing angle cameras, and the multiple viewing angle cameras are configured to shoot the same scene from different viewing angles. As an example, the multiple viewing angle cameras may be binocular stereo cameras, three-eye cameras, and four-eye cameras, etc. As an example, the viewing angle parameters may include a pose matrix, a projection matrix, a resolution, etc. associated with the camera. In some embodiments, each of the multiple viewing angle parameters includes at least a pose matrix and a projection matrix of a corresponding viewing angle camera in the multiple viewing angle cameras. In some embodiments, the equivalent monocular viewing angle parameter includes at least a pose matrix and a projection matrix of an equivalent monocular camera.

[0121] Method 1000 may include: at block 1020, calculating an equivalent monocular viewing angle parameter associated with the equivalent monocular camera based on the multiple viewing angle parameters. In some embodiments, calculating the equivalent monocular viewing angle parameter associated with the equivalent monocular camera based on the multiple viewing angle parameters may include: at block 1021, calculating the equivalent monocular viewing angle parameter associated with the equivalent monocular camera using a minimum fitting method. In other embodiments, calculating the equivalent monocular viewing angle parameter associated with the equivalent monocular camera based on the multiple viewing angle parameters may also include calculating the equivalent monocular viewing angle parameter associated with the equivalent monocular camera using a non-minimum fitting method.

[0122] The method 1000 may include, at block 1030 , filtering a set of neural features within a viewing cone space associated with an equivalent monocular view angle parameter.

[0123] The method 1000 may include, at block 1040 , decoding the set of neural features into a set of neural primitives.

[0124] Method 1000 may include, at block 1050, rendering the set of neural primitives into multiple images based on multiple viewing parameters, respectively. In some embodiments, rendering the set of neural primitives into multiple images based on multiple viewing parameters, respectively, may include, at block 1051, rendering the set of neural primitives into multiple images using sequential computation or fusion computation.

[0125] In some embodiments, method 1000 may further include: at box 1060, determining the viewing cone space associated with the equivalent monocular viewing parameter by making each boundary of the viewing cone of the equivalent monocular camera parallel to each boundary of the viewing cone of the multiple viewing cameras, and making the observation range of the equivalent monocular camera no greater than the sum of the observation ranges of the multiple viewing cameras.

[0126] Some embodiments combine multiple independent neural rendering processes of multi-view cameras in each image frame into a complete rendering process, shortening the rendering time and improving rendering efficiency, while removing the computational redundancy introduced by the overlap of the observation fields (including reference Figure 3 In addition, compared with the improved methods for traditional 3D rendering in the prior art that are not applicable to neural rendering, the embodiments of the present disclosure are applicable to neural rendering and effectively improve the rendering efficiency of the neural rendering process.

[0127] Fig.11 A block diagram of a process 1100 for calculating equivalent monocular view parameters for the view angles of multiple view cameras according to some embodiments of the present invention is shown. Process 1100 may be combined with the above Fig.10 Step 1020 in method 1000 is described.

[0128] In some embodiments, process 1100 may include: at block 1110, calculating an equivalent monocular perspective parameter associated with an equivalent monocular camera based on the first perspective parameter and the second perspective parameter. In some embodiments, the first perspective parameter is associated with a left-eye perspective camera, and the second perspective parameter is associated with a right-eye perspective camera. Process 1100 may be combined with the above Figure 5 The description is related to determining the observation range of an equivalent monocular camera based on the observation range of a binocular camera.

[0129] In some embodiments, process 1100 may include, at block 1120, iteratively calculating an equivalent monocular view parameter associated with the equivalent monocular camera based on at least three view parameters. Process 1200 may be combined with the above Figure 7 Determine the observation range of the equivalent monocular camera based on the observation range of the trinocular camera and combine Figure 8 The description of determining the observation range of an equivalent monocular camera based on the observation range of a quadruple camera is related. Figure 13-14 The present invention specifically describes a process of iteratively calculating the equivalent monocular viewing angle parameters associated with the equivalent monocular camera.

[0130] Fig.12 FIG. 1 is a flowchart of a process 1200 for rendering a set of neural primitives into left and right eye images in a neural rendering process according to some embodiments of the present invention for a binocular stereo camera perspective. The process 1200 may be a process of combining the above Fig.10 Step 1050 of method 1000 described above. Process 1200 can be combined with the above Figure 6 At 670 , the depicted set of neural primitives may be rendered as a plurality of images 680 - 1 to 680 - n based on a plurality of view parameters 610 - 1 to 610 - n , respectively.

[0131] Process 1200 may include rendering the set of neural primitives as a left-eye image based on a first view parameter at block 1210. In some embodiments, the first view parameter may be associated with a left-eye view camera.

[0132] Process 1200 may include, at block 1220, rendering the set of neural primitives as a right-eye image based on a second viewing angle parameter. In some embodiments, the second viewing angle parameter may be associated with a right-eye viewing angle camera. In this way, the parallax characteristics in binocular stereo may be maintained.

[0133] Fig.13 FIG. 1 is a flowchart showing a process 1300 for iteratively calculating an equivalent monocular viewing angle parameter based on three viewing angle parameters according to some embodiments of the present invention. The process 1300 may be combined with the above Fig.10Step 1020 of method 1000 described above. Process 1300 can be combined with the above Figure 7 The description is related to determining the observation range of an equivalent monocular camera based on the observation range of a trinocular camera.

[0134] Process 1300 may include, at block 1310 , determining an intermediate viewing angle parameter based on the first viewing angle parameter and the second viewing angle parameter.

[0135] Process 1300 may include, at block 1320 , determining an equivalent monocular view parameter based on the intermediate view parameter and the third view parameter.

[0136] In some embodiments, the first perspective parameter, the second perspective parameter, and the third perspective parameter may be associated with three perspective cameras, respectively.

[0137] Fig.14 FIG. 14 is a flowchart showing a process 1400 for iteratively calculating equivalent monocular viewing angle parameters based on four viewing angle parameters according to some embodiments of the present invention. Fig.10 Step 1020 of method 1000 described above. Process 1400 can be combined with the above Figure 8 The description is related to determining the observation range of an equivalent monocular camera based on the observation range of a quadruple camera.

[0138] Process 1400 may include, at block 1410 , determining a first intermediate viewing angle parameter based on the first viewing angle parameter and the second viewing angle parameter.

[0139] Process 1400 may include, at block 1420 , determining a second intermediate viewing angle parameter based on the third viewing angle parameter and the fourth viewing angle parameter.

[0140] Process 1400 may include, at block 1430 , determining an equivalent monocular viewing angle parameter based on the first intermediate viewing angle parameter and the second intermediate viewing angle parameter.

[0141] In some embodiments, the first perspective parameter, the second perspective parameter, the third perspective parameter, and the fourth perspective parameter may be associated with four perspective cameras, respectively.

[0142] Fig.15 A flow chart illustrating a process 1500 for continuously performing neural rendering calculations for multiple scenes for multiple view cameras according to some embodiments of the present invention.

[0143] After starting, process 1500 may proceed to: At block 1510, a perspective of a scene to be rendered is selected.

[0144] Subsequently, process 1500 may proceed to: at block 1520, performing neural rendering associated with an equivalent monocular view of the scene captured by the camera. The process at block 1520 may include the above combined Figure 4 The neural rendering pipeline 420 described herein calls the computing resource 410 to perform the process, combined with Fig.10 The steps of the method 1000 described herein are combined with Figure 11-Figure 14 Describe the methods and steps of the process.

[0145] Subsequently, process 1500 may proceed to: at decision block 1530 , determining whether to stop rendering.

[0146] If so, process 1500 can end.

[0147] Otherwise, process 1500 may proceed to, at block 1540 , proceed to the perspective of the next scene.

[0148] Thereafter, process 1500 may continue execution from block 1510 .

[0149] Process 1500 may enable multiple view cameras to continuously perform neural rendering on multiple scenes.

[0150] According to another aspect of the present invention, a computer-readable storage medium is provided.

[0151] Fig.16 A block diagram of a computer-readable storage medium 1600 is shown according to some embodiments of the invention.

[0152] The computer readable storage medium 1600 may store a computer program 1650. When the computer program 1650 is executed by the processor, the computer program 1650 implements the above-mentioned Fig.10 The method described in 1000, combined with Fig.11 The process described in 1100, combined Fig.12 The process described in 1200, combined Fig.13 The process described in 1300, combined Fig.14 The process described in 1400, combined Fig.15 Process 1500 is described.

[0153] According to another aspect of the present invention, a computer program product is provided.

[0154] Fig.17 A block diagram of a computer program product 1700 is shown according to some embodiments of the invention.

[0155] The computer program product 1700 may include a computer program 1750. When the computer program 1750 is executed by a processor, the computer program 1750 implements the above-mentioned Fig.10 The method described in 1000, combined with Fig.11The process described in 1100, combined Fig.12 The process described in 1200, combined Fig.13 The process described in 1300, combined Fig.14 The process described in 1400, combined Fig.15 Process 1500 is described.

[0156] Embodiments of the present invention have been described with reference to the accompanying drawings. The embodiments are illustrative rather than restrictive.

Claims

1. A graphics processing device, comprising: Computing resources; as well as A neural rendering pipeline, wherein the neural rendering pipeline calls the computing resource to perform the following steps: Acquire a plurality of viewing angle parameters, wherein the plurality of viewing angle parameters are respectively associated with a plurality of viewing angle cameras, and the plurality of viewing angle cameras are configured to shoot the same scene at different viewing angles; Based on the multiple viewing angle parameters, calculating an equivalent monocular viewing angle parameter associated with the equivalent monocular camera; filtering a set of neural features within a viewing cone space associated with the equivalent monocular view parameters; Decoding the set of neural features into a set of neural graph elements; and The set of neural primitives is rendered into a plurality of images based on the plurality of viewing angle parameters respectively.

2. The graphics processing device according to claim 1, wherein: Each of the plurality of viewing angle parameters comprises at least a pose matrix and a projection matrix of a corresponding viewing angle camera among the plurality of viewing angle cameras; The equivalent monocular viewing angle parameters at least include the pose matrix and projection matrix of the equivalent monocular camera.

3. The graphics processing device according to claim 1, wherein: The neural rendering pipeline calls the computing resource to perform the following steps: The equivalent monocular viewing angle parameters associated with the equivalent monocular camera are calculated using a minimum fitting method.

4. The graphics processing device according to claim 1, wherein: The neural rendering pipeline calls the computing resource to perform the following steps: The viewing cone space associated with the equivalent monocular viewing angle parameter is determined by making each boundary of the viewing cone of the equivalent monocular camera parallel to each boundary of the viewing cone of the multiple viewing angle cameras, and making the observation range of the equivalent monocular camera not greater than the sum of the observation ranges of the multiple viewing angle cameras.

5. The graphics processing device according to claim 1, wherein: The neural rendering pipeline calls the computing resource to perform the following steps: The set of neural primitives is rendered into a plurality of images using sequential computation or fusion computation.

6. The graphics processing device according to claim 1, wherein: The neural rendering pipeline calls the computing resource to perform the following steps: The graphics processing device is caused to compute in parallel with at least one other device for graphics processing to render the set of neural primitives into a plurality of images.

7. The graphics processing device according to claim 1, wherein: The plurality of viewing angle parameters include a first viewing angle parameter and a second viewing angle parameter, wherein the first viewing angle parameter is associated with a left-eye viewing angle camera, and the second viewing angle parameter is associated with a right-eye viewing angle camera.

8. The graphics processing device according to claim 7, wherein: The neural rendering pipeline calls the computing resource to perform the following steps: Calculating an equivalent monocular viewing angle parameter associated with an equivalent monocular camera based on the first viewing angle parameter and the second viewing angle parameter; The set of neural primitives is rendered as a left-eye image based on the first viewing angle parameter, and the set of neural primitives is rendered as a right-eye image based on the second viewing angle parameter.

9. The graphics processing device according to claim 1, wherein: The multiple viewing angle parameters include at least three viewing angle parameters, and the at least three viewing angle parameters are respectively associated with at least three viewing angle cameras; The neural rendering pipeline calls the computing resource to perform the following steps: Based on the at least three viewing angle parameters, an equivalent monocular viewing angle parameter associated with the equivalent monocular camera is calculated in an iterative manner.

10. The graphics processing device according to claim 9, characterized in that: The at least three viewing angle parameters include a first viewing angle parameter, a second viewing angle parameter and a third viewing angle parameter, wherein the first viewing angle parameter, the second viewing angle parameter and the third viewing angle parameter are respectively associated with three viewing angle cameras; The neural rendering pipeline calls the computing resource to perform the following steps: Determining an intermediate viewing angle parameter based on the first viewing angle parameter and the second viewing angle parameter; and The equivalent monocular viewing angle parameter is determined based on the intermediate viewing angle parameter and the third viewing angle parameter.

11. The graphics processing device according to claim 9, characterized in that: The at least three viewing angle parameters include a first viewing angle parameter, a second viewing angle parameter, a third viewing angle parameter and a fourth viewing angle parameter, wherein the first viewing angle parameter, the second viewing angle parameter, the third viewing angle parameter and the fourth viewing angle parameter are respectively associated with four viewing angle cameras; The neural rendering pipeline calls the computing resource to perform the following steps: Determining a first intermediate viewing angle parameter based on the first viewing angle parameter and the second viewing angle parameter; Determining a second intermediate viewing angle parameter based on the third viewing angle parameter and the fourth viewing angle parameter; as well as The equivalent monocular viewing angle parameter is determined based on the first intermediate viewing angle parameter and the second intermediate viewing angle parameter.

12. A system for graphics processing, comprising: A graphics processing device as claimed in any one of claims 1 to 11; as well as A plurality of viewing angle cameras are coupled to the graphics processing device.

13. A method for graphics processing, comprising: Acquire a plurality of viewing angle parameters, wherein the plurality of viewing angle parameters are respectively associated with a plurality of viewing angle cameras, and the plurality of viewing angle cameras are configured to shoot the same scene at different viewing angles; Based on the multiple viewing angle parameters, calculating an equivalent monocular viewing angle parameter associated with the equivalent monocular camera; filtering a set of neural features within a viewing cone space associated with the equivalent monocular view parameters; Decoding the set of neural features into a set of neural graph elements; and The set of neural primitives is rendered into a plurality of images based on the plurality of viewing angle parameters respectively.

14. The method according to claim 13, characterized in that Each of the plurality of viewing angle parameters comprises at least a pose matrix and a projection matrix of a corresponding viewing angle camera among the plurality of viewing angle cameras; The equivalent monocular viewing angle parameters at least include the pose matrix and projection matrix of the equivalent monocular camera.

15. The method according to claim 13, characterized in that Based on the multiple viewing angle parameters, calculating the equivalent monocular viewing angle parameter associated with the equivalent monocular camera includes: using a minimum fitting method to calculate the equivalent monocular viewing angle parameter associated with the equivalent monocular camera.

16. The method according to claim 13, characterized in that The method further comprises: The viewing cone space associated with the equivalent monocular viewing angle parameter is determined by making each boundary of the viewing cone of the equivalent monocular camera parallel to each boundary of the viewing cone of the multiple viewing angle cameras, and making the observation range of the equivalent monocular camera not greater than the sum of the observation ranges of the multiple viewing angle cameras.

17. The method according to claim 13, characterized in that Rendering the set of neural primitives into a plurality of images based on the plurality of viewing angle parameters respectively includes: rendering the set of neural primitives into a plurality of images using sequential calculation or fusion calculation.

18. The method according to claim 13, characterized in that The multiple viewing angle parameters include a first viewing angle parameter and a second viewing angle parameter, wherein the first viewing angle parameter is associated with a left-eye viewing angle camera, and the second viewing angle parameter is associated with a right-eye viewing angle camera, wherein: Calculating the equivalent monocular viewing angle parameters associated with the equivalent monocular camera includes: Based on the first viewing angle parameter and the second viewing angle parameter, calculating an equivalent monocular viewing angle parameter associated with an equivalent monocular camera; and Rendering the set of neural primitives into a plurality of images based on the plurality of viewing angle parameters respectively comprises: Rendering the set of neural primitives as a left-eye image based on the first viewing angle parameter; and The set of neural primitives is rendered as a right-eye image based on the second viewing angle parameter.

19. The method of claim 13, wherein: The multiple viewing angle parameters include at least three viewing angle parameters, and the at least three viewing angle parameters are respectively associated with at least three viewing angle cameras, wherein: Calculating the equivalent monocular viewing angle parameters associated with the equivalent monocular camera includes: Based on the at least three viewing angle parameters, an equivalent monocular viewing angle parameter associated with the equivalent monocular camera is calculated in an iterative manner.

20. The method of claim 19, wherein: The at least three viewing angle parameters include a first viewing angle parameter, a second viewing angle parameter and a third viewing angle parameter, wherein the first viewing angle parameter, the second viewing angle parameter and the third viewing angle parameter are respectively associated with three viewing angle cameras, wherein: Calculating the equivalent monocular viewing angle parameters associated with the equivalent monocular camera includes: Determining an intermediate viewing angle parameter based on the first viewing angle parameter and the second viewing angle parameter; and The equivalent monocular viewing angle parameter is determined based on the intermediate viewing angle parameter and the third viewing angle parameter.

21. The method of claim 19, wherein: The at least three viewing angle parameters include a first viewing angle parameter, a second viewing angle parameter, a third viewing angle parameter and a fourth viewing angle parameter, wherein the first viewing angle parameter, the second viewing angle parameter, the third viewing angle parameter and the fourth viewing angle parameter are respectively associated with four viewing angle cameras, wherein: Calculating the equivalent monocular viewing angle parameters associated with the equivalent monocular camera includes: Determining a first intermediate viewing angle parameter based on the first viewing angle parameter and the second viewing angle parameter; Determining a second intermediate viewing angle parameter based on the third viewing angle parameter and the fourth viewing angle parameter; and The equivalent monocular viewing angle parameter is determined based on the first intermediate viewing angle parameter and the second intermediate viewing angle parameter.

22. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 13 to 21.

23. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 13 to 21.

Images