Integrated imaging three-dimensional display quality improving method

Abstract

The invention discloses an integrated imaging three-dimensional display quality improvement method, which comprises the following steps that: the quality of an acquired material image is low, which seriously restricts the quality of integrated imaging three-dimensional display; therefore, the material image needs to be analyzed; analyzing the collected scene image to obtain a quality characterization value; selecting a material image based on the quality characterization value; comparing the quality characterization value of the material image with a quality characterization threshold value, judging whether the image quality is qualified or not according to a comparison result, and if the image quality is not qualified, generating a repairing signal and marking the material image as a low-quality image; identifying the patch image based on the repair signal, and analyzing the matching degree of the patch image and the low-quality image to obtain a patch matching value; comparing the patch matching value with a patch matching threshold value, judging whether the matching degree of the patch image and the low-quality image is qualified or not according to a comparison result, generating a matching signal if the matching degree of the patch image and the low-quality image is qualified, and repairing the low-quality image by utilizing the corresponding patch image based on the matching signal.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional display, and in particular to a method for improving the quality of integrated imaging three-dimensional display. Background Art

[0002] With the continuous development of science and technology, integrated imaging 3D display technology has received widespread attention and application in many fields because it can provide a real and natural 3D visual experience. However, in practical applications, the image acquisition process is often affected by various factors. On the one hand, the performance limitation of the image acquisition equipment is a common cause of low image quality. On the other hand, the complexity of the acquisition environment can also have an adverse effect on the image quality. In addition, data loss or damage may occur during the transmission and storage of the image, further reducing the image quality. This seriously restricts the quality of integrated imaging 3D display. Therefore, how to effectively optimize and repair the image when the acquired image quality is low and improve the quality of integrated imaging 3D display has become a key issue that needs to be solved in this field.

[0003] The present invention first analyzes and screens the acquired images, and selects images that contain a large number of object types and a large number of objects with a high definition as material images for integrated imaging three-dimensional display; by analyzing the quality of the material images, it is determined whether the material images meet the quality requirements of the integrated imaging three-dimensional display, and if not, a suitable patch image is selected for repair; by analyzing the types, numbers and clarity of objects in the patch images, the matching degree between the patch images and the material images is determined, and the material images are repaired with patch images with a high matching degree; by reducing the impact of low image quality on the quality of integrated imaging three-dimensional display, the quality of integrated imaging three-dimensional display is improved. Summary of the invention

[0004] The object of the present invention is to provide a method for improving the quality of integrated imaging three-dimensional display to solve at least one of the above-mentioned problems in the prior art.

[0005] In a first aspect, the present invention provides a method for improving the quality of integrated imaging three-dimensional display, comprising the following steps:

[0006] Step 1: A group of scene images are collected through a camera array, and the collected scene images are analyzed to obtain a quality representation value; the quality representation values ​​of all images are compared, and the image with the largest quality representation value in the group of images is used as a material image for integrated imaging three-dimensional display;

[0007] Step 2: Obtain a quality representation value of the material image, compare the quality representation value of the material image with a quality representation threshold, and determine whether the image quality is qualified according to the comparison result. If it is unqualified, generate a repair signal and mark the corresponding image as a low-quality image;

[0008] Step 3: Based on the patching signal, identify the patch image, analyze the matching degree between the patch image and the low-quality image, and obtain the patch matching value;

[0009] Step 4: Compare the patch matching value with the patch matching threshold, and judge whether the matching degree between the patch image and the low-quality image is qualified according to the comparison result. If so, generate a matching signal, and based on the matching signal, use the corresponding patch image to repair the low-quality image.

[0010] In a second aspect, the present invention provides an integrated imaging three-dimensional display quality improvement system, including the following modules:

[0011] Material image acquisition module: Collect a set of scene images through a camera array, analyze the collected scene images to obtain a quality characterization value; compare the quality characterization values of all images, and use the image with the largest quality characterization value in this set of images as the material image for integrated imaging three-dimensional display;

[0012] Material image analysis module: Obtain the quality characterization value of the material image, compare the quality characterization value of the material image with the quality characterization threshold, and judge whether the image quality is qualified according to the comparison result. If not, generate a patching signal and mark the corresponding image as a low-quality image;

[0013] Matching image acquisition module: Based on the patching signal, identify the patch image, analyze the matching degree between the patch image and the low-quality image, and obtain the patch matching value;

[0014] Matching image analysis module: Compare the patch matching value with the patch matching threshold, and judge whether the matching degree between the patch image and the low-quality image is qualified according to the comparison result. If so, generate a matching signal, and based on the matching signal, use the corresponding patch image to repair the low-quality image.

[0015] Advantages of the present invention:

[0016] 1. The technical solution of the embodiment of the present invention is as follows: By analyzing the types, quantities, and clarity of objects included in the collected images, the quality of the collected images is obtained, and the image with the highest quality in the collected images is selected as the material image for integrated imaging three-dimensional display, and it is judged whether the quality of the material image meets the requirements; This ensures that the images used for three-dimensional display have high quality from the source, helps to present a clearer and more realistic three-dimensional scene, and brings a better visual experience to users.

[0017] 2. The technical solution of the embodiment of the present invention is as follows: By analyzing and comparing the quality of the material image, it is determined whether the quality of the material image meets the requirements. If the quality of the material image does not meet the requirements, the remaining images collected at the same time and same location as the material image are screened; by analyzing the matching degree between the remaining images and the material image, the most suitable image is selected for optimizing and repairing the material image; clarifying the quality standard of the material image helps to maintain the consistency and stability of the output image quality of the entire imaging system; by analyzing the matching degree between the remaining images and the material image, it is ensured that the image used to repair the material image highly conforms to the material image in terms of content and features, thereby maximizing the quality of the repaired image, making it closer to the high-quality image standard and improving the quality of integral imaging three-dimensional display. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0019] Figure 1 is the flowchart of the steps of the method for improving the quality of integral imaging three-dimensional display provided by the present invention;

[0020] Figure 2 is the flowchart of obtaining the quality characterization value of the method for improving the quality of integral imaging three-dimensional display provided in Embodiment 1 of the present invention;

[0021] Figure 3 is the flowchart of obtaining the patch matching value of the method for improving the quality of integral imaging three-dimensional display provided in Embodiment 2 of the present invention;

[0022] Figure 4 is the module schematic diagram of the method for improving the quality of integral imaging three-dimensional display provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] In order to enable those skilled in the art to better understand the solution of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0024] Embodiment 1

[0025] As Figure 1As shown in the figure, the method for improving the three-dimensional display quality of integral imaging provided by the embodiment of the present invention specifically includes the following steps:

[0026] Step 1: Collect a set of scene images through a camera array, analyze the collected scene images to obtain a quality characterization value; compare the quality characterization values of all images, and use the image with the largest quality characterization value in this set of images as the material image for integral imaging three-dimensional display;

[0027] Collecting a set of scene images through a camera array, if the collected scene images are not clear or some objects in the scene are lost in the images due to reasons such as angle deviation, etc., it will cause a decrease in the three-dimensional display quality of integral imaging; analyze the collected scene images, specifically;

[0028] Collect a set of scene images through a camera array, count all the object types of this scene that appear in this set of images to obtain the total number of object types; and obtain the total number of objects that appear in each image in this set of images, sum them up and take the average to obtain the average total number of objects;

[0029] Analyze the objects that appear in the image and the clarity of the objects that appear, and judge the quality of the image according to the analysis results, and then determine whether it is necessary to repair the image;

[0030] Analyze any one of the images in this set of images, specifically;

[0031] Count the number of object types included in the image to obtain the number of image object types; perform a ratio process on the number of image object types and the total number of object types to obtain the object type ratio;

[0032] Obtain the ratio of the total number of objects in the image to the average total number of objects to obtain the object quantity ratio;

[0033] Perform a weighted sum on the object type ratio and the object quantity ratio to obtain the object comprehensive ratio, denoted as WT;

[0034] Divide the image into several sub-regions of the same size, and analyze the image clarity of each sub-region;

[0035] Analyze each sub-region through an edge detection algorithm, and use the Sobel operator to calculate the gradient G x in the horizontal direction and the gradient G y in the vertical direction respectively, and use the formula to calculate the gradient magnitude G, and use the formula to obtain the gradient direction θ;

[0036] Based on any one sub-region; sum up the gradient magnitudes G of the pixel points in the sub-region and take the average to obtain the pixel gradient average value; compare the pixel gradient average value with the pixel gradient threshold. If the pixel gradient average value is less than the pixel gradient threshold, mark the corresponding sub-region as a blurred sub-region;

[0037] It should be noted that among all sub-regions, the edges of clear image sub-regions are sharp, the pixel values change violently, resulting in larger gradient magnitudes and higher average values; while the edges of blurred image sub-regions are gentle, with small gradient magnitudes and lower average values;

[0038] Count the number of blurred sub-regions, and process the ratio of the number of blurred sub-regions to the total number of sub-regions to obtain the blurred region ratio;

[0039] Obtain all the gradient magnitudes G in the blurred sub-region, and take the minimum value as the gradient reference value of the sub-region;

[0040] Obtain the gradient reference values of all blurred sub-regions of the corresponding image, sum them up and take the average to obtain the clarity reference value of the corresponding image; take the absolute value of the difference between the clarity reference value and the clarity threshold to obtain the reference difference, and process the ratio of the reference difference to the clarity threshold to obtain the clarity ratio;

[0041] Perform weighted summation on the clarity ratio and the blurred region ratio to obtain the comprehensive clarity ratio, denoted as MQ;

[0042] Perform data processing on the image object ratio WT and the comprehensive clarity ratio MQ, using the formula to obtain the quality characterization value BZ; where a1 and a2 are preset proportionality coefficients;

[0043] It should be noted that by analyzing whether the objects contained in the collected images are complete and the clarity of the collected images, the quality characterization value BZ is obtained; the larger the quality characterization value BZ, the higher the quality of the collected images, which is more conducive to improving the quality of integral imaging three-dimensional display;

[0044] Obtain the quality characterization values of all images, and take the image with the largest quality characterization value as the material image for integral imaging three-dimensional display;

[0045] Step 2: Obtain the quality characterization value of the material image, compare the quality characterization value of the material image with the quality characterization threshold, and judge whether the image quality is qualified according to the comparison result. If it is unqualified, generate a repair signal and mark the corresponding image as a low-quality image;

[0046] Compare the quality characterization value of the material image with the quality characterization threshold, and the specific process is as follows:

[0047] If the quality characterization value of the source image is greater than or equal to the quality characterization threshold, a qualified signal is generated; based on the qualified signal, no processing is performed on the image;

[0048] If the quality characterization value of the source image is less than the quality characterization threshold, a repair signal is generated; based on the repair signal, the source image is marked as a low-quality image, and the low-quality image is repaired;

[0049] The technical solution of the embodiment of the present invention is as follows: By analyzing the types, quantities, and sharpness of the objects included in the collected image, the quality of the collected image is obtained, and the image with the highest quality in the collected images is selected as the source image for integral imaging three-dimensional display, and it is determined whether the quality of the source image meets the requirements; This ensures that the images used for three-dimensional display have high quality from the source, which helps to present a clearer and more realistic three-dimensional scene and brings a better visual experience to users.

[0050] Embodiment 2

[0051] As Figure 2 shown, the method for improving the quality of integral imaging three-dimensional display provided by the embodiment of the present invention specifically includes the following steps:

[0052] Step 3: Based on the repair signal, identify the patch image, analyze the matching degree between the patch image and the low-quality image, and obtain the patch matching value;

[0053] Based on the repair signal, obtain the time and location information of the low-quality image taken. Based on the time and location information of the low-quality image taken, obtain the images in the image information library obtained at the same time and the same location, and record them as patch images;

[0054] Obtain the types of objects included in the blurred sub-region of the low-quality image, and mark the number of types of objects included in the blurred sub-region of the low-quality image as a;

[0055] Obtain the number of types of objects included in the patch image, and mark it as b;

[0056] Obtain the number of overlapping object types between the blurred sub-region of the low-quality image and the patch image, and mark it as c; The calculation formula is c = a ∩ b;

[0057] Calculate the type matching degree M of the types of objects included in the blurred sub-region of the low-quality image and the patch image, and the formula is M = c / a;

[0058] Obtain the object contour information included in the blurred sub-region of the low-quality image, and extract the contour information of the corresponding objects in the patch image for comparison;

[0059] Among the number c of overlapping object types, mark the objects included in the blurred sub-region of the low-quality image as Ni, where i = 1, 2,..., n; mark the corresponding objects in the patch image as Mi, where i = 1, 2,..., n;

[0060] Obtain the area of the object Ni in the blurred sub-region of the low-quality image and the area of the corresponding object Mi in the patch image, and calculate the overlapping area ratio Si, where Si = Ni / Mi;

[0061] It should be noted that if Si is close to 1, it is considered that the matching degree between Ni and Mi is high; if Si has a large difference, it is considered that the matching degree between Ni and Mi is low;

[0062] Calculate the area matching degree Ki between the object Ni in the blurred sub-region of the low-quality image and the corresponding object Mi in the patch image, using the formula where n is the number of overlapping object types; S is the overlapping area ratio threshold;

[0063] Perform weighted summation on the type matching degree M and the area matching degree Ki to obtain the image matching degree, denoted as PB;

[0064] Analyze whether each object in the number c of overlapping object types between the blurred sub-region of the low-quality image and the patch image is clear in the patch image. Specifically;

[0065] Calculate the object gradient magnitude G1 of each object in the patch image in the same way as calculating the gradient magnitude G of the sub-region of the low-quality image; compare the object gradient magnitude G1 with the mean value of the object gradient magnitudes. If the object gradient magnitude G1 is greater than or equal to the mean value of the object gradient magnitudes, it is determined that the object is clear in the patch image, denoted as a clear object; otherwise, it is determined that the object is blurred in the patch image, denoted as a blurred object d;

[0066] Perform ratio processing on the blurred object d and the number c of overlapping object types to obtain the proportion of blurred objects, denoted as MH;

[0067] Perform data processing on the image matching degree PB and the proportion of blurred objects MH, using the formula Obtain the patch matching value BA; where b1 and b2 are preset proportionality coefficients;

[0068] It should be noted that by analyzing the number of overlapping objects and the clarity of the overlapping objects between the low-quality image and the patch image, and the overlapping area of the corresponding objects between the low-quality image and the patch image, the patch matching value is obtained. The larger the patch matching value, the higher the matching degree between the patch image and the low-quality image;

[0069] Step 4: Compare the patch matching value with the patch matching threshold. According to the comparison result, determine whether the matching degree between the patch image and the low-quality image is qualified. If so, generate a matching signal. Based on the matching signal, use the corresponding patch image to repair the low-quality image;

[0070] Compare the patch matching value with the patch matching threshold. The specific process is as follows:

[0071] If the patch matching value is greater than or equal to the patch matching threshold, generate a matching signal, match the patch image with the low-quality image, and use the patch image to repair the low-quality image;

[0072] If the patch matching value is less than the patch matching threshold, generate a non-matching signal; mark the corresponding patch image as a non-matching patch image, and repeat the calculation of the patch matching value of the patch image until a patch image that matches the low-quality image is selected;

[0073] The technical solution of the embodiment of the present invention is: by analyzing the matching degree between the patch image and the low-quality image, select a suitable patch image to match the low-quality image, and use the matching patch image to repair the low-quality image to obtain a suitable image for integral imaging three-dimensional display; when the quality of the collected image is low, repairing the low-quality image can enable the image to obtain normal texture and color, thereby improving the clarity and integrity of the entire image and providing better-quality image materials for three-dimensional display; repairing the low-quality image can obtain a high-quality image required for integral imaging three-dimensional display, effectively improving the quality of integral imaging three-dimensional display.

[0074] Embodiment 3

[0075] As Figure 3 shown, the integral imaging three-dimensional display quality improvement system provided by the embodiment of the present invention specifically includes the following modules:

[0076] Material image acquisition module: Collect a set of scene images through a camera array, analyze the collected scene images to obtain a quality characterization value; compare the quality characterization values of all images, and use the image with the largest quality characterization value in this set of images as the material image for integral imaging three-dimensional display;

[0077] Material image analysis module: Obtain the quality characterization value of the material image, compare the quality characterization value of the material image with the quality characterization threshold, and determine whether the image quality is qualified according to the comparison result. If not, generate a repair signal and mark the corresponding image as a low-quality image;

[0078] Matching image acquisition module: Based on the repair signal, identify the patch image, analyze the matching degree between the patch image and the low-quality image, and obtain the patch matching value;

[0079] Matching image analysis module: Compare the patch matching value with the patch matching threshold, and judge whether the matching degree between the patch image and the low-quality image is qualified according to the comparison result. If so, generate a matching signal, and based on the matching signal, use the corresponding patch image to repair the low-quality image.

[0080] The above has described an embodiment of the present invention in detail, but the content described is only a preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the application of the present invention shall still fall within the scope covered by the patent of the present invention.

Claims

1. A method for improving the quality of integrated imaging three-dimensional display, characterized in that: The following steps are involved: Step 1: A group of scene images are collected through a camera array, and the collected scene images are analyzed to obtain a quality representation value; the quality representation values ​​of all images are compared, and the image with the largest quality representation value in the group of images is used as a material image for integrated imaging three-dimensional display; Step 2: Obtain a quality representation value of the material image, compare the quality representation value of the material image with a quality representation threshold, and determine whether the image quality is qualified according to the comparison result. If it is unqualified, generate a repair signal and mark the corresponding image as a low-quality image; Step 3: Based on the patch signal, the patch image is identified, the matching degree between the patch image and the low-quality image is analyzed, and the patch matching value is obtained; Step 4: Compare the patch matching value with the patch matching threshold, and determine whether the patch image matches the low-quality image based on the comparison result. If so, generate a matching signal, and based on the matching signal, use the corresponding patch image to repair the low-quality image.

2. The method for improving integrated imaging three-dimensional display quality according to claim 1, characterized in that: The quality characterization value is obtained in the following manner: The types and numbers of objects in the image are analyzed to obtain the comprehensive ratio of objects, which is marked as WT; The clarity of the image sub-region is analyzed to obtain the comprehensive clarity ratio, which is marked as MQ; The image object ratio WT and the clarity comprehensive ratio MQ are processed and the formula is used The quality characterization value BZ is obtained; wherein a1 and a2 are preset proportional coefficients.

3. The method for improving integrated imaging three-dimensional display quality according to claim 2, characterized in that: The object comprehensive ratio is obtained as follows: A set of scene images are collected by the camera array, and all types of objects appearing in the set of images are counted to obtain the total number of object types; and the total number of objects appearing in each image in the set of images is obtained, and the sum and average are taken to obtain the average total number of objects; Analyze any image in this set of images, specifically; The number of object types contained in the image is counted to obtain the number of image object types; the number of image object types is ratio processed with the total number of object types to obtain the object type ratio; The total number of objects in the acquired image is compared with the average total number of objects to obtain the object number ratio; The object type ratio and the object quantity ratio are weightedly summed to obtain the object comprehensive ratio WT.

4. The method for improving integrated imaging three-dimensional display quality according to claim 2, characterized in that: The method for obtaining the clarity comprehensive ratio is as follows: Analyze the gradient amplitude of the pixel points in the sub-area to obtain the fuzzy area ratio; Analyze and process the gradient reference value in the blurred sub-region to obtain the clarity ratio; The clarity ratio and the fuzzy area ratio are weightedly summed to obtain the comprehensive clarity ratio MQ.

5. The method for improving the quality of integrated imaging three-dimensional display according to claim 4, characterized in that: The fuzzy area ratio is obtained as follows: Divide the image into several sub-regions of equal size and analyze the image clarity of each sub-region; The pixel points in the sub-region are analyzed by edge detection algorithm, and the horizontal gradient G is calculated by Sobel operator. x and the vertical gradient G y , and use the formula Calculate the gradient amplitude G of the pixel point and use the formula Get the gradient direction θ; Based on any sub-region; sum and average the gradient amplitudes G of the pixels in the sub-region to obtain the pixel gradient mean; compare the pixel gradient mean with the pixel gradient threshold, if the pixel gradient mean is less than the pixel gradient threshold, the corresponding sub-region is recorded as a fuzzy sub-region; The number of fuzzy sub-regions is counted, and the ratio of the number of fuzzy sub-regions to the total number of sub-regions is processed to obtain the fuzzy region ratio.

6. The method for improving the quality of integrated imaging three-dimensional display according to claim 4, characterized in that: The method for obtaining the clarity ratio is as follows: Get all the gradient amplitudes G in the fuzzy sub-region, and take the minimum value as the gradient reference value of the sub-region; Obtain the gradient reference values ​​of all blurred sub-regions of the corresponding image, sum and average them to obtain the clarity reference value of the corresponding image; make a difference between the clarity reference value and the clarity threshold and take the absolute value to obtain the reference difference; perform ratio processing on the reference difference and the clarity threshold to obtain the clarity ratio.

7. The method for improving integrated imaging three-dimensional display quality according to claim 1, characterized in that: The patch matching value is obtained as follows: The object types contained in the blurred sub-region of the low-quality image and the object types contained in the patch image are analyzed to obtain the image matching degree, which is marked as PB; Analyze the blurred object d in the patch image and obtain the blurred object ratio, marked as MH; The image matching degree PB and the fuzzy object proportion MH are processed and the formula is used. The patch matching value BA is obtained, wherein b1 and b2 are preset proportional coefficients.

8. The method for improving the quality of integrated imaging three-dimensional display according to claim 7, characterized in that: The image matching degree is obtained in the following manner: The category matching degree M and the area matching degree Ki are weightedly summed to obtain the image matching degree, which is marked as PB; The process of obtaining the category matching degree M is as follows: Based on the patch signal, the time and location information of the shooting of the low-quality image is obtained. Based on the time and location information of the shooting of the low-quality image, an image obtained at the same time and the same location in the image information library is obtained and recorded as a patch image; Obtain the object types contained in the blurred sub-region of the low-quality image, and mark the number of object types contained in the blurred sub-region of the low-quality image as a; Get the number of object types contained in the patch image, marked as b; Get the number of overlapping object types between the blurred sub-region of the low-quality image and the patch image, marked as c; the calculation formula is c = a ∩ b; The category matching degree M between the blurred sub-region of the low-quality image and the object category contained in the patch image is calculated, and the formula is M=c / a.

9. The method for improving integrated imaging three-dimensional display quality according to claim 8, characterized in that: The area matching degree is obtained as follows: Obtain the object contour information contained in the blurred sub-region of the low-quality image, and extract the contour information of the corresponding object in the patch image for comparison; Among the number of overlapping object types c, the objects contained in the blurred sub-region of the low-quality image are marked as Ni, where i = 1, 2, ..., n; the corresponding objects in the patch image are marked as Mi, where i = 1, 2, ..., n; Get the area of ​​the blurred sub-region object Ni in the low-quality image and the area of ​​the corresponding object Mi in the patch image, and calculate the overlap area ratio Si, where Si = Ni / Mi; Calculate the area matching degree Ki between the blurred sub-region object Ni of the low-quality image and the corresponding object Mi in the patch image, using the formula Where n is the number of overlapping object types; S is the overlapping area ratio threshold.

10. The method for improving integrated imaging three-dimensional display quality according to claim 7, characterized in that: The method for obtaining the proportion of blurred objects is as follows: Calculate the object gradient amplitude G1 of each object in the patch image in the same way as calculating the gradient amplitude G of the low-quality image sub-region; compare the object gradient amplitude G1 with the object gradient amplitude mean, if the object gradient amplitude G1 is greater than or equal to the object gradient amplitude mean, then the object is determined to be clear in the patch image and recorded as a clear object; Otherwise, the object is judged to be blurred in the patch image and recorded as blurred object d; The blurred object d is ratioed with the number of overlapping object types c to obtain the blurred object ratio, which is marked as MH.

Images