Panoramic image rendering quality evaluation method and related equipment

Abstract

The invention discloses a panoramic image rendering quality evaluation method and related equipment, and relates to the technical field of simulation scenes, and the method comprises the steps: constructing a three-dimensional Gaussian scattering model based on panoramic image data of a target scene; selecting a target viewpoint in the three-dimensional Gaussian scattering model, and generating a rendered image based on the target viewpoint; determining a rendering quality index at the target viewpoint based on a solid angle approximation method; determining a feature matching distance index of the rendered image and a target scene image according to a preset diffusion model evaluation condition; and obtaining a rendering quality evaluation result of the panoramic image data according to the rendering quality index and the feature matching distance index. The invention provides a method for quantitatively evaluating the rendering quality of a panoramic image during reconstruction based on a three-dimensional Gaussian scattering model. The semantic consistency and scene authenticity of a generated image and a real image are checked by utilizing a diffusion model evaluation condition.

Description

Technical Field

[0001] The present application relates to the field of simulation scene technology, and in particular to a method for evaluating panoramic image rendering quality and related equipment. Background Art

[0002] Currently, 3D scene reconstruction technology has become a key link in improving the application effects of digital twins, simulation testing and other fields. Existing 3D scene reconstruction technologies (such as methods based on point clouds, meshes or neural radiation fields) can generate basic 3D models, but they still have significant shortcomings in evaluating image rendering quality. For example, traditional solutions often rely on manual visual inspection or simple pixel-level difference calculations, resulting in highly subjective evaluation results and poor reproducibility.

[0003] Therefore, how to design a method to evaluate the rendering quality of panoramic images based on the 3D Gaussian Splatting (3DGS) scene reconstruction has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] The Summary of the Invention introduces a series of simplified concepts that will be further described in the Detailed Description of the Invention. The Summary of the Invention of this application is not intended to limit the key features and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.

[0005] In a first aspect, the present application proposes a method for evaluating panoramic image rendering quality, comprising:

[0006] Construct a three-dimensional Gaussian scattering model based on the panoramic image data of the target scene;

[0007] Selecting a target viewpoint in the three-dimensional Gaussian scattering model, and generating a rendered image based on the target viewpoint;

[0008] determining a rendering quality index at the target viewpoint based on a solid angle approximation method;

[0009] Determining a feature matching distance index between the rendered image and the target scene image according to a preset diffusion model evaluation condition;

[0010] A rendering quality evaluation result of the panoramic image data is obtained according to the rendering quality index and the feature matching distance index.

[0011] In a feasible implementation manner, before constructing the three-dimensional Gaussian scattering model based on the panoramic image data of the target scene, the method further includes:

[0012] Collecting panoramic video data of the target scene through a multi-lens fisheye camera;

[0013] Performing frame extraction processing on the panoramic video data to obtain an image frame sequence;

[0014] The image frame sequence is dedistorted based on the distortion parameters of the multi-lens fisheye lens camera to generate the panoramic image data.

[0015] In a feasible implementation manner, after collecting the panoramic video data of the target scene by using a multi-lens fisheye camera, the method further includes:

[0016] Rectangular projection and cubic map projection are performed on the panoramic video data.

[0017] In a feasible implementation manner, constructing a three-dimensional Gaussian scattering model based on the panoramic image data of the target scene includes:

[0018] extracting camera pose and sparse point cloud from the panoramic image data;

[0019] generating a set of Gaussian ellipsoids based on the camera pose and the sparse point cloud;

[0020] Constructing an initial three-dimensional Gaussian scattering model based on the Gaussian ellipsoid set;

[0021] Parameters within the Gaussian ellipsoid set are optimized until the image quality rendered by the initial three-dimensional Gaussian scattering model meets preset requirements, thereby obtaining the three-dimensional Gaussian scattering model.

[0022] In a feasible implementation manner, determining the rendering quality index at the target viewpoint based on a solid angle approximation method includes:

[0023] Discretizing the continuous closed surface of the three-dimensional Gaussian scattering model to obtain a plurality of surface elements;

[0024] Obtaining an approximate solid angle based on a normal vector of the surface element and a position of the target viewpoint;

[0025] The solid angle approximation is normalized to the rendering quality index in the range of 0 to 1.

[0026] In a feasible implementation manner, the rendering quality index is calculated using the following formula:

[0027]

[0028] Among them, Ψ(ρ) is the rendering quality index, and Φ(p) is the degree to which the viewpoint ρ is enclosed by the minimum envelope M in the scene.

[0029] In a feasible implementation manner, determining the feature matching distance index between the rendered image and the target scene image according to a preset diffusion model evaluation condition includes:

[0030] Inputting the rendered image and the target scene image into a target diffusion model respectively to extract feature distribution data of the rendered image and feature distribution data of the target scene image;

[0031] The feature matching distance index is obtained based on the preset diffusion model evaluation condition, the feature distribution data of the rendered image, and the image feature distribution data of the target scene.

[0032] In a second aspect, the present application proposes a panoramic image rendering quality evaluation system, which is applied to the panoramic image rendering quality evaluation method described in any of the above embodiments, comprising:

[0033] A scene reconstruction module is used to construct a three-dimensional Gaussian scattering model based on the panoramic image data of the target scene;

[0034] an image rendering module, configured to select a target viewpoint in the three-dimensional Gaussian scattering model and generate a rendered image based on the target viewpoint;

[0035] a rendering quality evaluation module, configured to determine a rendering quality index at the target viewpoint based on a solid angle approximation method;

[0036] A feature matching distance evaluation module is used to determine a feature matching distance index between the rendered image and the target scene image according to a preset diffusion model evaluation condition;

[0037] The image rendering quality evaluation module is used to obtain a rendering quality evaluation result of the panoramic image data according to the rendering quality index and the feature matching distance index.

[0038] In a third aspect, an electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the steps of the method for evaluating the panoramic image rendering quality as described in any one of the first aspects above when executing the computer program stored in the memory.

[0039] In a fourth aspect, the present application further proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for evaluating the panoramic image rendering quality of any one of the first aspects.

[0040] In summary, the present application proposes a method for evaluating the rendering quality of panoramic images. Based on a panoramic video dataset or handheld panoramic video acquisition data, the method uses the 3D Gaussian Splatting (3DGS) technology to reconstruct the three-dimensional scene, and evaluates the rendering quality and authenticity based on the scene. The Fréchet Inception Distance (FID) distance is used to verify the quality evaluation index. This method can achieve relatively standard improvements and progress in indicator data and visual quality, and can also be used as an evaluation of the rendering quality of the 3DGS model.

[0041] The evaluation method for panoramic image rendering quality proposed in this application, and other advantages, objectives and features of this application will be partially reflected in the following description, and will also be partially understood by technical personnel in this field through research and practice of this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present description. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0043] Figure 1 A flow chart of a method for evaluating panoramic image rendering quality provided in an embodiment of the present application;

[0044] Figure 2 A flowchart of panoramic video dataset processing and quality assessment provided in an embodiment of the present application;

[0045] Figure 3 A functional module diagram of a panoramic image rendering quality evaluation system provided in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of the structure of an electronic device for evaluating panoramic image rendering quality provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0048] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0049] In this article, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also include elements inherent to such process, method, article or equipment. In the absence of further restrictions, the elements defined by the statement "comprising a ..." do not exclude the presence of other identical elements in the process, method, article or equipment comprising the elements. The term "two or more" includes two or more than two cases.

[0050] See also Figure 1 , is a flow chart of a method for evaluating panoramic image rendering quality provided by an embodiment of the present application, which may specifically include:

[0051] S110 , constructing a three-dimensional Gaussian scattering model based on the panoramic image data of the target scene.

[0052] For example, since a panoramic camera can capture 360° scene data at any point in an external scene, and there will be no holes that cannot be rendered after the three-dimensional scene reconstruction, in this embodiment, the target scene is collected by a panoramic camera, and then panoramic image data is obtained through steps such as panoramic image projection and camera dedistortion. The camera posture and sparse point cloud in these panoramic image data are used to generate a three-dimensional Gaussian scattering model (3D Gaussian Splatting Model, 3DGS model) composed of Gaussian ellipsoids. Each Gaussian ellipsoid represents an element in the target scene and has geometric, optical and other properties, wherein the geometric properties include position (three-dimensional coordinates), covariance matrix (shape and direction), scale parameters (size), and the optical properties include transparency, eigenvectors (such as color, material), and background color. By using real captured image frames as training data to optimize the parameters of the Gaussian ellipsoid, the three-dimensional scene finally rendered by the three-dimensional Gaussian scattering model is made as consistent as possible with the real scene.

[0053] S120: Select a target viewpoint in the three-dimensional Gaussian scattering model, and generate a rendered image based on the target viewpoint.

[0054] For example, two points are randomly selected as target viewpoints in a three-dimensional Gaussian scattering model, and a corresponding rendered image can be generated by performing panoramic rendering based on the two points.

[0055] S130: Determine a rendering quality index at the target viewpoint based on a solid angle approximation method.

[0056] For example, the rendering quality decreases when approaching the edge of the Gaussian ellipsoid, and the solid angle approximation method can determine the rendering quality index at the target viewpoint from a quantitative perspective, and use it to measure the rendering quality at the target viewpoint.

[0057] S140: Determine a feature matching distance index between the rendered image and the target scene image according to a preset diffusion model evaluation condition.

[0058] Exemplarily, a preset diffusion model evaluation condition is used to measure the degree of feature matching between the rendered image and the target scene image.

[0059] S150 : Obtaining a rendering quality evaluation result of the panoramic image data according to the rendering quality index and the feature matching distance index.

[0060] Exemplarily, the rendering quality of the panoramic image data is comprehensively evaluated by combining the rendering quality index at the target viewpoint and the feature matching distance index between the rendered image and the target scene image.

[0061] In some examples, before constructing the three-dimensional Gaussian scattering model based on the panoramic image data of the target scene, the method further includes:

[0062] Collecting panoramic video data of the target scene through a multi-lens fisheye camera;

[0063] Performing frame extraction processing on the panoramic video data to obtain an image frame sequence;

[0064] The image frame sequence is dedistorted based on the distortion parameters of the multi-lens fisheye lens camera to generate the panoramic image data.

[0065] For example, a multi-lens fisheye lens camera (e.g., a 6-lens ultra-wide-angle fisheye lens) is used to capture a 6-lens panoramic video of the target scene. Key frames (image frames) are then extracted from the video to form an image frame sequence. The image frames are then geometrically corrected based on the multi-lens fisheye lens camera's distortion parameters (e.g., focal length, radial distortion parameters) to eliminate the effects of lens distortion and generate panoramic image data.

[0066] In some examples, after collecting the panoramic video data of the target scene using a multi-lens fisheye camera, the method further includes:

[0067] Rectangular projection and cubic map projection are performed on the panoramic video data.

[0068] For example, the present application uses two formats for panoramic images in panoramic video data, namely rectangular projection and cubemap projection. The two projection formats have different processing methods for the deformation caused by the projection of the acquired panoramic image. Rectangular projection is to project the panoramic image onto a sphere and unfold it, which can maintain the proportion but will cause the center area to be enlarged and the edge area to be deformed. Cubemap projection is to divide the panoramic image into six equal faces, and these faces are surrounded by a regular cube, and then the cube is projected onto the sphere to complete the conversion from cubemap projection to rectangular projection.

[0069] In some examples, constructing a three-dimensional Gaussian scattering model based on panoramic image data of a target scene includes:

[0070] extracting camera pose and sparse point cloud from the panoramic image data;

[0071] generating a set of Gaussian ellipsoids based on the camera pose and the sparse point cloud;

[0072] Constructing an initial three-dimensional Gaussian scattering model based on the Gaussian ellipsoid set;

[0073] Parameters within the Gaussian ellipsoid set are optimized until the image quality rendered by the initial three-dimensional Gaussian scattering model meets preset requirements, thereby obtaining the three-dimensional Gaussian scattering model.

[0074] Exemplarily, a sequence of de-distorted image frames is obtained from the panoramic image data, the camera pose (i.e. the position and orientation of the camera in the three-dimensional space) of each image frame is calculated by matching feature points in different image frames and combining the parameters of the camera itself. Then, based on the matched feature points and their corresponding three-dimensional coordinates, a sparse point cloud is generated to represent the preliminary three-dimensional structure of the target scene. Based on the sparse point cloud, a plurality of Gaussian ellipsoids are generated to obtain a Gaussian ellipsoid set, an initial three-dimensional Gaussian scattering model is constructed based on the Gaussian ellipsoid set, and then the initial three-dimensional Gaussian scattering model is continuously optimized by using the actually captured image frames as training data, so that the image quality rendered by the initial three-dimensional Gaussian scattering model meets the preset requirements, thereby obtaining a final available three-dimensional Gaussian scattering model.

[0075] Further, since the direction in which the user observes the target scene can be any direction, the rendering quality also needs to consider the rendering results of all perspectives in the optimization process, otherwise there will be a situation that some point positions and spatial positions appear holes or meaningless color blocks. Therefore, panoramic picture data or panoramic camera shooting needs to be selected in the process of selecting training data, so that the problem of lacking a scene in a certain direction can be effectively avoided. When close to the edge of the ellipsoid, the number of ellipsoids in that direction on the rendered image will obviously decrease or be missing, thereby causing the rendering quality to decrease and the reality to decrease. By selecting two points in the three-dimensional Gaussian scattering model as target viewpoints, panoramic image rendering is performed based on the two points. Assuming that the point with higher rendering quality is A and the point with lower rendering quality is B, a minimum envelope M can be found, which wraps A inside and B on the edge of M. In this way, the rendering quality can be judged by judging the topological relationship between the target viewpoint and the minimum envelope.

[0076] In some examples, the determining the rendering quality index at the target viewpoint based on the solid angle approximation method comprises:

[0077] Discretizing the continuous closed surface of the three-dimensional Gaussian scattering model to obtain a plurality of surface elements;

[0078] Based on the normal vector of the surface element and the position of the target viewpoint, a solid angle approximation value is obtained;

[0079] The solid angle approximation value is normalized to the rendering quality index in the range of 0 to 1.

[0080] Exemplarily, in mathematics, a continuous closed surface ∑ is subdivided to obtain surface elements σi∈∑, and the position of a viewpoint m on the surface is judged using a solid angle φ, and the specific formula is as follows:

[0081]

[0082] Among them, σi is the surface element, r i is the vector from the viewpoint ρ to the i-th surface element σi, n i is the unit normal vector of the i-th surface element σi.

[0083] During the rendering process of a 3D Gaussian scattering model, several elliptical light spots of varying opacity are sputtered, and the colors are synthesized according to the depth of the elliptical light spots. Ideally, the solid angle calculated by the above formula is obtained from the minimum enveloping volume. However, the minimum enveloping volume is difficult to calculate, so a Gaussian ellipsoid is generally used to approximate the solid angle. The approximate solid angle is normalized to a value between 0 and 1 to conveniently and intuitively represent the rendering quality index at the target viewpoint.

[0084] The spots of several ellipsoids on the camera plane may overlap, so the pixel value of each pixel reflects the combined coverage of multiple ellipsoids. Ellipsoids that obstruct the viewport from both sides only form a single elliptical spot when projected onto the camera plane. Therefore, when calculating the degree of obstruction using pixel values, two completely obstructed elliptical spots are not counted twice. The equivalent solid angle can be calculated from the coverage of the elliptical spots.

[0085] Let x,y be the pixel coordinates on the camera plane, then its pixel value p x The range of (x, y) mapped to (0, 1) can be regarded as the ellipsoid coverage of the pixel. Since pixels are square and the side length is 1, the area covered by the ellipsoid of each pixel can be estimated by the following formula.

[0086]

[0087] Next, we render the normals of each pixel on the surface. Same, but perspective Different, so it is necessary to calculate the coefficient c for different pixels f On (x,y), the formula for calculating the solid angle from the pixel color is rewritten as:

[0088]

[0089] Where l is the distance from viewpoint p to the rendering plane, which is half the resolution of the rendered image; Φ(p) is the total solid angle of a single rendered image to viewpoint p; σ(x,y) is the area covered by the ellipsoid for each pixel.

[0090] To evaluate the rendering quality at a specific point in the target scene, we need to consider the rendering results of all points in any direction and evaluate the overall rendering quality and effect. The rendering quality index Ψ(ρ) at any viewpoint ρ in the target scene is defined based on the solid angle.

[0091] In some examples, the rendering quality index is calculated by the following formula:

[0092]

[0093] wherein, Ψ(ρ) is the rendering quality index, the rendering quality index Ψ(ρ) can express the wrapping degree of the minimum envelope of the viewpoint p, that is, the scene wrapping degree, the index is distributed in the range of (0, 1), and the closer the result is to 1, the higher the rendering quality of the overall scene is.

[0094] In some examples, the feature matching distance index of the rendering image and the target scene image is determined according to the preset diffusion model evaluation condition, comprising:

[0095] The rendering image and the target scene image are respectively input into a target diffusion model to extract feature distribution data of the rendering image and feature distribution data of the target scene image.

[0096] Based on the preset diffusion model evaluation condition, the feature distribution data of the rendering image and the feature distribution data of the target scene image, the feature matching distance index is obtained.

[0097] For example, the rendering image and the target scene image (i.e., the real scene image) are respectively input into the target diffusion model (i.e., the pre-trained FID model) to extract the feature distribution data of the rendering image and the feature distribution data of the target scene image. The distance between the two sets of feature distribution data is calculated to obtain the FID distance index (i.e., the feature matching distance index), and if the FID distance index is less than a preset threshold, it indicates that the feature matching degree between the rendering image and the real scene image meets the requirements.

[0098] In summary, the evaluation method of the panoramic image rendering quality proposed in the present application uses the 3DGS technology to perform three-dimensional scene reconstruction based on panoramic video data sets or handheld panoramic video acquisition data, and performs rendering quality and reality evaluation based on the scene. The FID distance index is used to evaluate the rendering quality, so that the rendering quality can be improved and progressed in a relatively standard manner.

[0099] In a specific embodiment, as Figure 2As shown in the figure, first, relevant data is extracted from the panoramic video dataset, and then the position and posture information of the camera is determined. Then, this information is used to reconstruct the three-dimensional geometric structure, and the reconstructed model is rendered to generate a free-viewpoint panorama in the cubemap projection (CMP) format. Subsequently, the quality indicators of the generated panorama are calculated, and sequence frames are extracted from it. At the same time, sparse point cloud data is obtained through sparse reconstruction technology, and a free-viewpoint panorama in the rectangular projection (Equirectangular Projection, ERP) format is generated. Finally, a validation dataset is collected, the FID index of feature matching is calculated, and the various indicators obtained from the entire process are verified. This series of steps aims to comprehensively evaluate the quality of the panoramic video dataset and its performance in the three-dimensional reconstruction and rendering process.

[0100] It should be noted that the above embodiments are only the best examples and are not intended to limit the implementation of the present application.

[0101] Based on the same application concept, the embodiments of the present application also provide a panoramic image rendering quality evaluation system corresponding to the panoramic image rendering quality evaluation method provided in the above embodiment. Since the principle of solving the problem by the panoramic image rendering quality evaluation system in the embodiments of the present application is similar to the panoramic image rendering quality evaluation method in the above embodiment of the present application, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be repeated.

[0102] like Figure 3 As shown, Figure 3 This is a functional module diagram of a panoramic image rendering quality evaluation system provided by this application. The system includes:

[0103] A scene reconstruction module 21 is used to construct a three-dimensional Gaussian scattering model based on the panoramic image data of the target scene;

[0104] An image rendering module 22, configured to select a target viewpoint in the three-dimensional Gaussian scattering model and generate a rendered image based on the target viewpoint;

[0105] a rendering quality evaluation module 23, configured to determine a rendering quality index at the target viewpoint based on a solid angle approximation method;

[0106] a feature matching distance evaluation module 24 for determining a feature matching distance index between the rendered image and the target scene image according to a preset diffusion model evaluation condition;

[0107] The image rendering quality evaluation module 25 is configured to obtain a rendering quality evaluation result of the panoramic image data according to the rendering quality index and the feature matching distance index.

[0108] like Figure 4As shown, based on the same application concept, an embodiment of the present application also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored on the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, the steps of any of the above-mentioned methods for evaluating the quality of panoramic image rendering are implemented.

[0109] Since the electronic device introduced in this embodiment is a device used to implement a method for evaluating the quality of panoramic image rendering in an embodiment of the present application, based on the method introduced in the embodiment of the present application, technical personnel in this field can understand the specific implementation of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present application is not introduced in detail here. As long as the equipment used by technical personnel in this field to implement the method in the embodiment of the present application falls within the scope of protection of this application.

[0110] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0111] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0114] An embodiment of the present application further provides a computer program product, which includes computer software instructions. When the computer software instructions are executed on a processing device, the processing device executes the panoramic image rendering quality evaluation method process in the corresponding embodiment.

[0115] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0116] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0118] Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of these modules may be selected to achieve the purpose of this embodiment based on actual needs.

[0119] In addition, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules.

[0120] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0121] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for evaluating panoramic image rendering quality, characterized in that: include: Construct a three-dimensional Gaussian scattering model based on the panoramic image data of the target scene; Selecting a target viewpoint in the three-dimensional Gaussian scattering model, and generating a rendered image based on the target viewpoint; determining a rendering quality index at the target viewpoint based on a solid angle approximation method; Determining a feature matching distance index between the rendered image and the target scene image according to a preset diffusion model evaluation condition; A rendering quality evaluation result of the panoramic image data is obtained according to the rendering quality index and the feature matching distance index.

2. The method for evaluating panoramic image rendering quality according to claim 1, wherein: Before constructing the three-dimensional Gaussian scattering model based on the panoramic image data of the target scene, the method further includes: Collecting panoramic video data of the target scene through a multi-lens fisheye camera; Performing frame extraction processing on the panoramic video data to obtain an image frame sequence; The image frame sequence is dedistorted based on the distortion parameters of the multi-lens fisheye lens camera to generate the panoramic image data.

3. The method for evaluating panoramic image rendering quality according to claim 2, wherein: After the panoramic video acquisition data of the target scene is acquired by the multi-lens fisheye camera, the method further includes: Rectangular projection and cubic map projection are performed on the panoramic video data.

4. The method for evaluating panoramic image rendering quality according to claim 1, wherein: The constructing of a three-dimensional Gaussian scattering model based on the panoramic image data of the target scene includes: extracting camera pose and sparse point cloud from the panoramic image data; generating a set of Gaussian ellipsoids based on the camera pose and the sparse point cloud; Constructing an initial three-dimensional Gaussian scattering model based on the Gaussian ellipsoid set; Parameters within the Gaussian ellipsoid set are optimized until the image quality rendered by the initial three-dimensional Gaussian scattering model meets preset requirements, thereby obtaining the three-dimensional Gaussian scattering model.

5. The method for evaluating panoramic image rendering quality according to claim 1, wherein: The determining of the rendering quality index at the target viewpoint based on the solid angle approximation method includes: Discretizing the continuous closed surface of the three-dimensional Gaussian scattering model to obtain a plurality of surface elements; Obtaining an approximate solid angle based on a normal vector of the surface element and a position of the target viewpoint; The solid angle approximation is normalized to the rendering quality index in the range of 0 to 1.

6. The method for evaluating panoramic image rendering quality according to claim 5, wherein: The rendering quality index is calculated using the following formula: Among them, Ψ(ρ) is the rendering quality index, and Φ(p) is the degree to which the viewpoint ρ is enclosed by the minimum envelope M in the scene.

7. The method for evaluating panoramic image rendering quality according to claim 1, wherein: The determining of the feature matching distance index between the rendered image and the target scene image according to a preset diffusion model evaluation condition includes: Inputting the rendered image and the target scene image into a target diffusion model respectively to extract feature distribution data of the rendered image and feature distribution data of the target scene image; The feature matching distance index is obtained based on the preset diffusion model evaluation condition, the feature distribution data of the rendered image, and the image feature distribution data of the target scene.

8. A panoramic image rendering quality evaluation system, applied to the panoramic image rendering quality evaluation method according to any one of claims 1 to 7, characterized in that: include: A scene reconstruction module is used to construct a three-dimensional Gaussian scattering model based on the panoramic image data of the target scene; an image rendering module, configured to select a target viewpoint in the three-dimensional Gaussian scattering model and generate a rendered image based on the target viewpoint; a rendering quality evaluation module, configured to determine a rendering quality index at the target viewpoint based on a solid angle approximation method; A feature matching distance evaluation module is used to determine a feature matching distance index between the rendered image and the target scene image according to a preset diffusion model evaluation condition; The image rendering quality evaluation module is used to obtain a rendering quality evaluation result of the panoramic image data according to the rendering quality index and the feature matching distance index.

9. An electronic device comprising: A memory and a processor, wherein the processor is configured to implement the steps of the method for evaluating panoramic image rendering quality according to any one of claims 1 to 7 when executing a computer program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for evaluating panoramic image rendering quality according to any one of claims 1 to 7 are implemented.

Images