A 2-view naked-eye 3D image arrangement algorithm based on machine learning

Abstract

The present invention discloses a 2-view naked-eye 3D mapping algorithm based on machine learning, and specifically relates to the technical field of image data processing. The present invention selects two cameras of the same model to shoot multiple groups of left and right viewpoint images of different scenes and different shooting subjects, and performs image correction on the shot images after shooting is completed, which is beneficial to reducing the error of 3D reconstruction images, and inputs the left and right viewpoint images belonging to the same shot subject into a constructed 3D convolutional network model, and obtains the corresponding left and right viewpoint fusion disparity maps after model processing, and then converts the disparity value into a depth distance value, and calculates the 3D coordinates of the shot subject in the world coordinate system based on the disparity value, the depth distance value, the camera parameters and the principle of similar triangles to perform 3D reconstruction, which is beneficial to improving the geometric restoration degree, color restoration degree and 3D model depth information feedback accuracy of 3D reconstruction.

Description

Technical Field

[0001] The present invention relates to the field of image data processing technology, and more specifically, to a 2-viewpoint naked-eye 3D mapping algorithm based on machine learning. Background Art

[0002] With the development of new generation technologies, 3D reconstruction technology is increasingly being used in daily life and production, and the arrangement of reconstructed 3D images is also widely used in game development, film and television special effects, virtual reality (VR), augmented reality (AR), architectural visualization, industrial design and other fields.

[0003] The existing naked-eye 3D mapping algorithm obtains laser point cloud data by scanning objects through lidar, and then reconstructs the object in three dimensions. After the three-dimensional reconstruction is completed, the reconstructed three-dimensional image is aligned with the real scene and marked to complete the mapping.

[0004] However, the existing technology still has some shortcomings: there are differences in the images seen by the left and right eyes of a person, and the difference in the perspectives of the left and right eyes should be taken into account when performing naked-eye 3D mapping. Scanning objects with lidar to obtain laser point cloud data, and then performing three-dimensional reconstruction of the object after obtaining the laser point cloud data of the object can ensure that the spatial geometry of the object is restored more accurately, but it cannot ensure that the depth information, color restoration and texture details of the final reconstructed three-dimensional model are presented with high accuracy, and it should be improved. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a two-viewpoint naked-eye 3D mapping algorithm based on machine learning to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a two-viewpoint naked-eye 3D mapping algorithm based on machine learning, comprising the following steps:

[0007] S1: Use two cameras of the same model to shoot multiple groups of left and right viewpoint images of different scenes and different subjects, and perform image correction on the captured images after shooting;

[0008] S2: After the image correction is completed, it is determined whether the image quality meets the set standard. If it meets the set standard, the image storage setting is entered. If it does not meet the set standard, the image correction is continued until the image quality meets the set standard.

[0009] S3: Input the left and right viewpoint images belonging to the same photographed subject into the constructed 3D convolutional network model, and obtain the corresponding left and right viewpoint fusion disparity map after model processing, and then convert the disparity value into a depth distance value, and calculate the 3D coordinates of the photographed subject in the world coordinate system based on the disparity value, depth distance value, camera parameters and similar triangle principle to perform 3D reconstruction, and calculate the 3D reconstruction accuracy coefficient after the 3D reconstruction is completed;

[0010] S4: Based on the constructed 3D model, computer graphics technology is used to generate stereoscopic images of left and right eye perspectives, and then the generated stereoscopic image pair is optimized. After the naked eye 3D display technology is customized, the stereoscopic images of left and right eye perspectives are input into the naked eye 3D layout design model to generate 3D display layouts of different scenes;

[0011] S5: the number of points with consistent parallax of corresponding points in the left and right eye perspective stereo images of each object in the display scene, the number of points with inconsistent parallax of corresponding points, the number of objects that correctly reflect the scene depth information, the number of objects that incorrectly reflect the scene depth information, the area of ​​the scene where the colors of the original scene and the display scene are correctly restored, the area of ​​the scene where the colors of the original scene and the display scene are not correctly restored, the 3D reconstruction accuracy coefficient of each 3D reconstructed image in the display scene, the clarity compliance coefficient and matching accuracy coefficient of the outer surface texture details of each 3D reconstructed image in the display scene, and the image resolution and frame rate of the display scene;

[0012] S6: Calculate the display scene stereoscopic effect index, the display scene reality index and the display scene visual effect index based on the collected data;

[0013] S7: Calculate a comprehensive evaluation index of the three-dimensional arrangement effect based on the display scene stereoscopic effect index, the display scene authenticity index, and the display scene visual effect index.

[0014] Preferably, the step S1 calibrates the parameters of the two cameras used for shooting before shooting the dual-viewpoint images, and the camera parameters include focal length, aperture, shutter speed, viewing angle, white balance, color setting parameters, physical size of pixels, image center, coordinate axis tilt parameters, distortion parameters, rotation matrix, translation vector and external parameter matrix, and during the shooting process, the focal length, baseline distance, shooting height and shooting time of the two cameras are controlled to be consistent.

[0015] Preferably, the specific steps of determining whether the image quality meets the set standard in step S2 are as follows:

[0016] S21, collecting and processing the clarity coefficient Xqc, contrast parameter αbc, color distribution accuracy coefficient Xsc, noise parameter αzc, image deformation coefficient Xzc, structural similarity coefficient αgc and peak signal-to-noise ratio coefficient αfc of the captured image;

[0017] S22, retrieve the set standard image clarity coefficient Xqs, standard contrast parameter αbs, standard color distribution accuracy coefficient Xss, standard noise parameter αzs, standard image deformation coefficient Xzs, standard structural similarity coefficient αgs and standard peak signal-to-noise ratio coefficient αfs;

[0018] S23, respectively calculate the clarity standard coefficient YXq, contrast standard coefficient Yαb, color distribution standard coefficient YXs, noise standard coefficient Yαz, image deformation standard coefficient YXz, structural similarity standard coefficient Yαg, peak signal-to-noise ratio standard coefficient Yαf, the specific formula is as follows: , , , , , , ;

[0019] S24, when the value of any standard-reaching coefficient is 0, it is determined that the image quality does not meet the set standard, and the image correction is continued until the image quality meets the set standard;

[0020] S25. When the value of each compliance coefficient is not 0, it is determined that the image quality meets the set standard. The left and right viewpoint images belonging to the same scene and fixed shooting position that meet the set standard are marked as the same group of images, and the scene name, the name of the subject being photographed, and the shooting position number are used as the folder name of the group of images.

[0021] Preferably, the steps for obtaining the clarity coefficient, the color distribution accuracy coefficient and the image distortion coefficient are as follows:

[0022] S211. Calculate the two-dimensional discrete Fourier transform formula of the image: , where F(u,v) is the complex value of the frequency component (u,v) of the image in the frequency domain, f(x,y) is the pixel value of the image at the position (x,y) in the spatial domain, nh and nl are the number of rows and columns of image pixels respectively, and j is an imaginary unit, satisfying j 2 =-1, u and v are the frequency component indexes in the horizontal and vertical directions in the frequency domain respectively, and x and y are the horizontal and vertical coordinates of the image pixels respectively;

[0023] S212. Calculate the amplitude spectrum of the Fourier transform: , where Re[F(u,v)] and Im[F(u,v)] are the real and imaginary parts of the two-dimensional discrete Fourier transform formula respectively;

[0024] S213, define a high-frequency region D, and calculate the total energy Ez and average amplitude Ea of the high-frequency components in the high-frequency region. The specific formula is as follows: , , nD is the number of pixels in the high-frequency region;

[0025] S214, calculate the clarity coefficient Xq, the specific formula is: ;

[0026] S215, collecting the number of plane color areas with correct color distribution msa, the total number of plane color areas msb, the sum of the plane image areas Az of the parts of the image that are deformed, enlarged or shrunk, and the total theoretical plane image area Aw of the theoretical image that is not deformed;

[0027] S216, calculate the color distribution accuracy coefficient Xs, the specific formula is: ;

[0028] S217, calculate the image deformation coefficient Xz, the specific formula is: .

[0029] Preferably, the specific calculation formula of the 3D reconstruction accuracy coefficient Xwc in step S3 is: , where nw is the number of three-dimensional coordinate points, de and dsi are the preset distance error of the three-dimensional coordinate points and the actual distance error between the i-th reconstructed three-dimensional coordinate point (xci, yci, zci) and the i-th real three-dimensional coordinate point (xsi, ysi, zsi), respectively. , is the mathematical judgment model of de and dsi. Different values ​​of de and dsi correspond to different output values. .

[0030] Preferably, the specific calculation formula of the matching accuracy coefficient βpi of the outer surface texture details of the i-th 3D reconstructed image in the display scene collected in step S5 is: , Ayi and Axi are respectively the area of ​​the correct application region of the texture details of the outer surface of the i-th 3D reconstructed image and the area of ​​the incorrect application region of the texture details of the outer surface of the i-th 3D reconstructed image.

[0031] Preferably, the specific data processing process in step S6 is as follows:

[0032] S61, calculating the display scene stereoscopic effect index RLT, the specific formula is: , msy, msx, nt, nty, ntx are respectively the number of points with consistent parallax of corresponding points in the stereoscopic images of the left and right eye perspectives of the i-th object in the display scene, the number of points with inconsistent parallax of corresponding points, the number of objects in the display scene, the number of objects in the display scene that correctly reflect the scene depth information, and the number of objects that incorrectly reflect the scene depth information;

[0033] S62, calculating the display scene reality index RCZ, the specific formula is: , Ahz, Ahc, Xwci, βqi are respectively the scene area where the colors of the original scene and the displayed scene are correctly restored, the scene area where the colors of the original scene and the displayed scene are not correctly restored, the 3D reconstruction accuracy coefficient of the i-th 3D reconstructed image in the displayed scene, and the clarity compliance coefficient of the outer surface texture details of each 3D reconstructed image in the displayed scene;

[0034] S63, calculating the display scene visual effect index RSX, the specific formula is: , βb and βz are the display scene image resolution and frame rate respectively.

[0035] Preferably, in step S7, the specific calculation formula for calculating the comprehensive evaluation index RP of the three-dimensional arrangement effect based on the display scene stereoscopic effect index RLT, the display scene reality index RCZ and the display scene visual effect index RSX is: .

[0036] Technical effects and advantages of the present invention:

[0037] The present invention selects two cameras of the same model to shoot multiple groups of left and right viewpoint images of different scenes and different shooting subjects, performs image correction on the shot images after shooting is completed, takes into account the difference influence of left and right eye viewing angles, inputs the left and right viewpoint images belonging to the same shot subject into a constructed three-dimensional convolutional network model, obtains the corresponding left and right viewpoint fusion disparity map after model processing, then converts the disparity value into a depth distance value, and calculates the three-dimensional coordinates of the shot subject in the world coordinate system based on the disparity value, the depth distance value, the camera parameters and the principle of similar triangles to perform three-dimensional reconstruction, calculates the three-dimensional reconstruction accuracy coefficient after the three-dimensional reconstruction is completed, generates stereoscopic images of left and right eye viewing angles based on the constructed three-dimensional model by using computer graphics technology, and then optimizes the generated stereoscopic image pair, and inputs the stereoscopic images of left and right eye viewing angles into the naked eye 3D layout design model after custom selection of naked eye 3D display technology to generate three-dimensional display layouts of different scenes, which is beneficial to improving the depth information, color restoration and texture detail presentation accuracy of the reconstructed three-dimensional model, provides an image quality evaluation method, and provides data support for subsequent high-stereoscopic and high-realistic scene display effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a diagram of the method steps of the present invention.

[0039] Figure 2 It is a system structure block diagram of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] like Figure 1 This embodiment provides a two-viewpoint naked-eye 3D image arrangement algorithm based on machine learning, comprising the following steps:

[0042] S1: Use two cameras of the same model to shoot multiple groups of left and right viewpoint images of different scenes and different subjects, and perform image correction on the captured images after shooting;

[0043] Furthermore, the step S1 calibrates the parameters of the two cameras used for shooting before shooting the dual-viewpoint images, and the camera parameters include focal length, aperture, shutter speed, viewing angle, white balance, color setting parameters, physical size of pixels, image center, coordinate axis tilt parameters, distortion parameters, rotation matrix, translation vector and external parameter matrix. During the shooting process, the focal length, baseline distance, shooting height and shooting time of the two cameras are controlled to be consistent.

[0044] It should be specifically explained in this embodiment that image correction includes geometric correction, radiation correction, color correction, noise processing and enhancement processing, wherein geometric correction includes rotation, scaling, translation, affine transformation, perspective transformation and four-point transformation; radiation correction includes brightness adjustment, contrast adjustment and histogram equalization; color correction includes color balance, gamma value correction and color space conversion; noise processing includes filtering and denoising algorithms; enhancement processing includes edge enhancement, sharpening and contrast enhancement, and the above-mentioned image correction methods are all existing technologies and can be directly searched and applied through public literature, so they will not be elaborated here.

[0045] S2: After the image correction is completed, it is determined whether the image quality meets the set standard. If it meets the set standard, the image storage setting is entered. If it does not meet the set standard, the image correction is continued until the image quality meets the set standard.

[0046] Furthermore, the specific steps of determining whether the image quality meets the set standard in step S2 are as follows:

[0047] S21, collecting and processing the clarity coefficient Xqc, contrast parameter αbc, color distribution accuracy coefficient Xsc, noise parameter αzc, image deformation coefficient Xzc, structural similarity coefficient αgc and peak signal-to-noise ratio coefficient αfc of the captured image;

[0048] S22, retrieve the set standard image clarity coefficient Xqs, standard contrast parameter αbs, standard color distribution accuracy coefficient Xss, standard noise parameter αzs, standard image deformation coefficient Xzs, standard structural similarity coefficient αgs and standard peak signal-to-noise ratio coefficient αfs;

[0049] S23, respectively calculate the clarity standard coefficient YXq, contrast standard coefficient Yαb, color distribution standard coefficient YXs, noise standard coefficient Yαz, image deformation standard coefficient YXz, structural similarity standard coefficient Yαg, peak signal-to-noise ratio standard coefficient Yαf, the specific formula is as follows: , , , , , , ;

[0050] S24, when the value of any standard-reaching coefficient is 0, it is determined that the image quality does not meet the set standard, and the image correction is continued until the image quality meets the set standard;

[0051] S25. When the value of each compliance coefficient is not 0, it is determined that the image quality meets the set standard. The left and right viewpoint images belonging to the same scene and fixed shooting position that meet the set standard are marked as the same group of images, and the scene name, the name of the subject being photographed, and the shooting position number are used as the folder name of the group of images.

[0052] Furthermore, the steps for obtaining the clarity coefficient, the color distribution accuracy coefficient, and the image distortion coefficient are as follows:

[0053] S211. Calculate the two-dimensional discrete Fourier transform formula of the image: , where F(u,v) is the complex value of the frequency component (u,v) of the image in the frequency domain, f(x,y) is the pixel value of the image at the position (x,y) in the spatial domain, nh and nl are the number of rows and columns of image pixels respectively, and j is an imaginary unit, satisfying j 2 =-1, u and v are the frequency component indexes in the horizontal and vertical directions in the frequency domain respectively, and x and y are the horizontal and vertical coordinates of the image pixels respectively;

[0054] S212. Calculate the amplitude spectrum of the Fourier transform: , where Re[F(u,v)] and Im[F(u,v)] are the real and imaginary parts of the two-dimensional discrete Fourier transform formula respectively;

[0055] S213, define a high-frequency region D, and calculate the total energy Ez and average amplitude Ea of the high-frequency components in the high-frequency region. The specific formula is as follows: , , nD is the number of pixels in the high-frequency region;

[0056] S214, calculate the clarity coefficient Xq, the specific formula is: ;

[0057] S215, collecting the number of plane color areas with correct color distribution msa, the total number of plane color areas msb, the sum of the plane image areas Az of the parts of the image that are deformed, enlarged or shrunk, and the total theoretical plane image area Aw of the theoretical image that is not deformed;

[0058] S216, calculate the color distribution accuracy coefficient Xs, the specific formula is: ;

[0059] S217, calculate the image deformation coefficient Xz, the specific formula is: .

[0060] It should be specifically noted in this embodiment that the contrast parameters and noise parameters can be directly viewed using image processing software, and the structural similarity coefficient and peak signal-to-noise ratio coefficient are coefficients that can be directly calculated using the prior art and are not specifically calculated here. They are regarded as coefficients that can be directly collected.

[0061] S3: Input the left and right viewpoint images belonging to the same photographed subject into the constructed 3D convolutional network model, and obtain the corresponding left and right viewpoint fusion disparity map after model processing, and then convert the disparity value into a depth distance value, and calculate the 3D coordinates of the photographed subject in the world coordinate system based on the disparity value, depth distance value, camera parameters and similar triangle principle to perform 3D reconstruction, and calculate the 3D reconstruction accuracy coefficient after the 3D reconstruction is completed;

[0062] Furthermore, the specific calculation formula of the 3D reconstruction accuracy coefficient Xwc in step S3 is: , where nw is the number of three-dimensional coordinate points, de and dsi are the preset distance error of the three-dimensional coordinate points and the actual distance error between the i-th reconstructed three-dimensional coordinate point (xci, yci, zci) and the i-th real three-dimensional coordinate point (xsi, ysi, zsi), respectively. , is the mathematical judgment model of de and dsi. Different values ​​of de and dsi correspond to different output values. .

[0063] What needs to be specifically explained in this embodiment is that the model processing steps in step S3 include image feature extraction, image feature fusion, 3D convolution and disparity regression. The construction process of the three-dimensional convolutional network model used is the existing technology. The disparity value is converted into a depth distance value, and the three-dimensional coordinates of the photographed subject in the world coordinate system are calculated based on the disparity value, depth distance value, camera parameters and the principle of similar triangles. They can all be directly calculated by the existing technology and will not be elaborated here.

[0064] S4: Based on the constructed 3D model, computer graphics technology is used to generate stereoscopic images of left and right eye perspectives, and then the generated stereoscopic image pair is optimized. After the naked eye 3D display technology is customized, the stereoscopic images of left and right eye perspectives are input into the naked eye 3D layout design model to generate 3D display layouts of different scenes;

[0065] It should be specifically noted in this embodiment that the construction process of the naked-eye 3D layout design model is a prior art and will not be elaborated here in detail.

[0066] S5: the number of points with consistent parallax of corresponding points in the left and right eye perspective stereo images of each object in the display scene, the number of points with inconsistent parallax of corresponding points, the number of objects that correctly reflect the scene depth information, the number of objects that incorrectly reflect the scene depth information, the area of ​​the scene where the colors of the original scene and the display scene are correctly restored, the area of ​​the scene where the colors of the original scene and the display scene are not correctly restored, the 3D reconstruction accuracy coefficient of each 3D reconstructed image in the display scene, the clarity compliance coefficient and matching accuracy coefficient of the outer surface texture details of each 3D reconstructed image in the display scene, and the image resolution and frame rate of the display scene;

[0067] Furthermore, the specific calculation formula of the matching accuracy coefficient βpi of the outer surface texture details of the i-th 3D reconstructed image in the display scene collected in step S5 is: , Ayi and Axi are respectively the area of ​​the correct application region of the texture details of the outer surface of the i-th 3D reconstructed image and the area of ​​the incorrect application region of the texture details of the outer surface of the i-th 3D reconstructed image.

[0068] S6: Calculate the display scene stereoscopic effect index, the display scene reality index and the display scene visual effect index based on the collected data;

[0069] Further, the specific data processing process in step S6 is as follows:

[0070] S61, calculating the display scene stereoscopic effect index RLT, the specific formula is: , msy, msx, nt, nty, ntx are respectively the number of points with consistent parallax of corresponding points in the stereoscopic images of the left and right eye perspectives of the i-th object in the display scene, the number of points with inconsistent parallax of corresponding points, the number of objects in the display scene, the number of objects in the display scene that correctly reflect the scene depth information, and the number of objects that incorrectly reflect the scene depth information;

[0071] S62, calculating the display scene reality index RCZ, the specific formula is: , Ahz, Ahc, Xwci, βqi are respectively the scene area where the colors of the original scene and the displayed scene are correctly restored, the scene area where the colors of the original scene and the displayed scene are not correctly restored, the 3D reconstruction accuracy coefficient of the i-th 3D reconstructed image in the displayed scene, and the clarity compliance coefficient of the outer surface texture details of each 3D reconstructed image in the displayed scene;

[0072] What needs to be specifically explained in this embodiment is that the clarity coefficient of the outer surface texture details of each 3D reconstructed image in the display scene in this step is consistent with the calculation steps of calculating the clarity coefficient when determining whether the image quality meets the set standard in step S2. For the sake of distinction, βqi is used here to represent the clarity coefficient of the outer surface texture details of each 3D reconstructed image in the display scene.

[0073] S63, calculating the display scene visual effect index RSX, the specific formula is: , βb and βz are the display scene image resolution and frame rate respectively.

[0074] S7: Calculate a comprehensive evaluation index of the three-dimensional arrangement effect based on the display scene stereoscopic effect index, the display scene authenticity index, and the display scene visual effect index.

[0075] Furthermore, in step S7, the specific calculation formula for calculating the comprehensive evaluation index RP of the three-dimensional arrangement effect based on the display scene stereoscopic effect index RLT, the display scene authenticity index RCZ and the display scene visual effect index RSX is: .

[0076] It should be specifically noted in this embodiment that the set values ​​used are selected based on actual needs and are not limited to specific values ​​here.

[0077] like Figure 2The embodiment shown in the figure provides a 2-viewpoint naked-eye 3D mapping system based on machine learning, including an image acquisition module, an image correction module, an image quality compliance judgment module, a 3D reconstruction module, a naked-eye 3D mapping module, a display scene data acquisition module, a display scene data processing module, a mapping effect comprehensive evaluation module and a database. The image acquisition module, the image correction module, the image quality compliance judgment module, the 3D reconstruction module and the naked-eye 3D mapping module are connected in sequence, the 3D reconstruction module and the naked-eye 3D mapping module are connected to the display scene data acquisition module, the display scene data acquisition module, the display scene data processing module and the mapping effect comprehensive evaluation module are connected in sequence, and all modules in the system are connected to the database.

[0078] The image acquisition module uses two cameras of the same model to shoot multiple groups of left and right viewpoint images of different scenes and different shooting subjects;

[0079] The image correction module performs image correction on the captured image after the shooting is completed;

[0080] The image quality standard determination module determines whether the image quality meets the set standard after the image correction is completed. If the image quality meets the set standard, the image storage setting is entered; if the image quality does not meet the set standard, the image correction is continued until the image quality meets the set standard;

[0081] The 3D reconstruction module inputs the left and right viewpoint images of the same subject into the constructed 3D convolutional network model, obtains the corresponding left and right viewpoint fusion disparity map after model processing, then converts the disparity value into a depth distance value, and calculates the 3D coordinates of the subject in the world coordinate system based on the disparity value, the depth distance value, the camera parameters and the principle of similar triangles to perform 3D reconstruction, and calculates the 3D reconstruction accuracy coefficient after the 3D reconstruction is completed;

[0082] The naked-eye 3D layout module generates stereoscopic images of left and right eye perspectives based on the constructed three-dimensional model using computer graphics technology, and then optimizes the generated stereoscopic image pair. After customizing the naked-eye 3D display technology, the stereoscopic images of left and right eye perspectives are input into the naked-eye 3D layout design model to generate three-dimensional display layouts of different scenes.

[0083] The display scene data acquisition module acquires and generates the number of points with consistent parallax of corresponding points in the left-eye and right-eye perspective stereoscopic images of each object in the display scene, the number of points with inconsistent parallax of corresponding points, the number of objects that correctly reflect scene depth information, the number of objects that incorrectly reflect scene depth information, the area of ​​the scene region where the colors of the original scene and the display scene are correctly restored, the area of ​​the scene region where the colors of the original scene and the display scene are not correctly restored, the 3D reconstruction accuracy coefficient of each 3D reconstructed image in the display scene, the clarity compliance coefficient and matching accuracy coefficient of the outer surface texture details of each 3D reconstructed image in the display scene, and the display scene image resolution and frame rate;

[0084] The display scene data processing module calculates the display scene stereoscopic effect index, the display scene authenticity index and the display scene visual effect index respectively based on the collected data;

[0085] The layout effect comprehensive evaluation module calculates a three-dimensional layout effect comprehensive evaluation index based on a display scene stereo effect index, a display scene authenticity index, and a display scene visual effect index;

[0086] The database is used to store data information of all modules in the system.

[0087] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A two-view naked-eye 3D mapping algorithm based on machine learning, characterized by: The following steps are involved: S1: Use two cameras of the same model to shoot multiple groups of left and right viewpoint images of different scenes and different subjects, and perform image correction on the captured images after shooting; S2: After the image correction is completed, it is determined whether the image quality meets the set standard. If it meets the set standard, the image storage setting is entered. If it does not meet the set standard, the image correction is continued until the image quality meets the set standard. The specific steps of determining whether the image quality meets the set standard in step S2 are as follows: S21, collecting and processing the clarity coefficient Xqc, contrast parameter αbc, color distribution accuracy coefficient Xsc, noise parameter αzc, image deformation coefficient Xzc, structural similarity coefficient αgc and peak signal-to-noise ratio coefficient αfc of the captured image; S22, retrieve the set standard image clarity coefficient Xqs, standard contrast parameter αbs, standard color distribution accuracy coefficient Xss, standard noise parameter αzs, standard image deformation coefficient Xzs, standard structural similarity coefficient αgs and standard peak signal-to-noise ratio coefficient αfs; S23, respectively calculate the clarity standard coefficient YXq, contrast standard coefficient Yαb, color distribution standard coefficient YXs, noise standard coefficient Yαz, image deformation standard coefficient YXz, structural similarity standard coefficient Yαg, peak signal-to-noise ratio standard coefficient Yαf, the specific formula is as follows: , , , , , , ; S24, when the value of any standard-reaching coefficient is 0, it is determined that the image quality does not meet the set standard, and the image correction is continued until the image quality meets the set standard; S25, when the value of each compliance coefficient is not 0, it is determined that the image quality meets the set standard, and the left and right viewpoint images belonging to the same scene and fixed shooting position that meet the set standard are marked as the same group of images, and the scene name, the name of the subject being photographed, and the shooting position number are used as the folder name of the group of images; The steps for obtaining the clarity coefficient, color distribution accuracy coefficient, and image distortion coefficient are as follows: S211. Calculate the two-dimensional discrete Fourier transform formula of the image: , where F(u,v) is the complex value of the frequency component (u,v) of the image in the frequency domain, f(x,y) is the pixel value of the image at the position (x,y) in the spatial domain, nh and nl are the number of rows and columns of image pixels respectively, and j is an imaginary unit, satisfying j 2 =-1, u and v are the frequency component indexes in the horizontal and vertical directions in the frequency domain respectively, and x and y are the horizontal and vertical coordinates of the image pixels respectively; S212. Calculate the amplitude spectrum of the Fourier transform: , where Re[F(u,v)] and Im[F(u,v)] are the real and imaginary parts of the two-dimensional discrete Fourier transform formula respectively; S213, define a high-frequency region D, and calculate the total energy Ez and average amplitude Ea of the high-frequency components in the high-frequency region. The specific formula is as follows: , , nD is the number of pixels in the high-frequency region; S214, calculate the clarity coefficient Xq, the specific formula is: ; S215, collecting the number of plane color areas with correct color distribution msa, the total number of plane color areas msb, the sum of the plane image areas Az of the parts of the image that are deformed, enlarged or shrunk, and the total theoretical plane image area Aw of the theoretical image that is not deformed; S216, calculate the color distribution accuracy coefficient Xs, the specific formula is: ; S217, calculate the image deformation coefficient Xz, the specific formula is: ; S3: Input the left and right viewpoint images belonging to the same photographed subject into the constructed 3D convolutional network model, and obtain the corresponding left and right viewpoint fusion disparity map after model processing, and then convert the disparity value into a depth distance value, and calculate the 3D coordinates of the photographed subject in the world coordinate system based on the disparity value, depth distance value, camera parameters and similar triangle principle to perform 3D reconstruction, and calculate the 3D reconstruction accuracy coefficient after the 3D reconstruction is completed; The specific calculation formula of the 3D reconstruction accuracy coefficient Xwc in step S3 is: , where nw is the number of three-dimensional coordinate points, de and dsi are the preset distance error of the three-dimensional coordinate points and the actual distance error between the i-th reconstructed three-dimensional coordinate point (xci, yci, zci) and the i-th real three-dimensional coordinate point (xsi, ysi, zsi), respectively. , is the mathematical judgment model of de and dsi. Different values ​​of de and dsi correspond to different output values. ; S4: Based on the constructed 3D model, computer graphics technology is used to generate stereoscopic images of left and right eye perspectives, and then the generated stereoscopic image pair is optimized. After the naked eye 3D display technology is customized, the stereoscopic images of left and right eye perspectives are input into the naked eye 3D layout design model to generate 3D display layouts of different scenes; S5: the number of points with consistent parallax of corresponding points in the left and right eye perspective stereo images of each object in the display scene, the number of points with inconsistent parallax of corresponding points, the number of objects that correctly reflect the scene depth information, the number of objects that incorrectly reflect the scene depth information, the area of ​​the scene where the colors of the original scene and the display scene are correctly restored, the area of ​​the scene where the colors of the original scene and the display scene are not correctly restored, the 3D reconstruction accuracy coefficient of each 3D reconstructed image in the display scene, the clarity compliance coefficient and matching accuracy coefficient of the outer surface texture details of each 3D reconstructed image in the display scene, and the image resolution and frame rate of the display scene; S6: Calculate the display scene stereoscopic effect index, the display scene reality index and the display scene visual effect index based on the collected data; S7: Calculate a comprehensive evaluation index of the three-dimensional arrangement effect based on the display scene stereoscopic effect index, the display scene authenticity index, and the display scene visual effect index.

2. The two-view naked-eye 3D mapping algorithm based on machine learning according to claim 1, characterized in that: The step S1 calibrates the parameters of the two cameras used for shooting before shooting the dual-viewpoint images. The camera parameters include focal length, aperture, shutter speed, viewing angle, white balance, color setting parameters, physical size of pixels, image center, coordinate axis tilt parameters, distortion parameters, rotation matrix, translation vector and external parameter matrix. During the shooting process, the focal length, baseline distance, shooting height and shooting time of the two cameras are controlled to be consistent.

3. The two-view naked-eye 3D mapping algorithm based on machine learning according to claim 1, characterized in that: The specific calculation formula of the matching accuracy coefficient βpi of the outer surface texture details of the i-th 3D reconstructed image in the display scene collected in step S5 is: , Ayi and Axi are respectively the area of ​​the correct application region of the texture details of the outer surface of the i-th 3D reconstructed image and the area of ​​the incorrect application region of the texture details of the outer surface of the i-th 3D reconstructed image.

4. The two-view naked-eye 3D mapping algorithm based on machine learning according to claim 3, characterized in that: The specific data processing process in step S6 is as follows: S61, calculating the display scene stereoscopic effect index RLT, the specific formula is: , msy, msx, nt, nty, ntx are respectively the number of points with consistent parallax of corresponding points in the stereoscopic images of the left and right eye perspectives of the i-th object in the display scene, the number of points with inconsistent parallax of corresponding points, the number of objects in the display scene, the number of objects in the display scene that correctly reflect the scene depth information, and the number of objects that incorrectly reflect the scene depth information; S62, calculating the display scene reality index RCZ, the specific formula is: , Ahz, Ahc, Xwci, βqi are respectively the scene area where the colors of the original scene and the displayed scene are correctly restored, the scene area where the colors of the original scene and the displayed scene are not correctly restored, the 3D reconstruction accuracy coefficient of the i-th 3D reconstructed image in the displayed scene, and the clarity compliance coefficient of the outer surface texture details of each 3D reconstructed image in the displayed scene; S63, calculating the display scene visual effect index RSX, the specific formula is: , βb and βz are the display scene image resolution and frame rate respectively.

5. The two-viewpoint naked-eye 3D mapping algorithm based on machine learning according to claim 4, characterized in that: The specific calculation formula for calculating the comprehensive evaluation index RP of the three-dimensional arrangement effect based on the display scene stereoscopic effect index RLT, the display scene authenticity index RCZ and the display scene visual effect index RSX in step S7 is: .

Images