A color difference detection method, device, apparatus and storage medium

Abstract

This application discloses a color difference detection method, apparatus, device, and storage medium. The method includes: acquiring images to be calibrated obtained by a light field camera taking pictures of a black and white calibration object with no filter and different color filters respectively; performing scale calibration processing on the images to be calibrated to obtain a color calibration matrix for each color; acquiring an image to be detected taken by the light field camera, in which an extended reality device displays an image of the color calibration object; and detecting the color difference of the extended reality device based on the color calibration matrix and the image to be detected for the corresponding color.

Description

Technical Field

[0001] This application relates to the field of optical technology, and in particular to a color difference detection method, apparatus, device, and storage medium. Background Technology

[0002] With the booming development of the Metaverse industry, the demand for Extended Reality (ER) devices, such as Virtual Reality (VR) devices or Augmented Reality (AR) devices, is becoming increasingly urgent. However, because ER devices use various lenses or polarizers for reverse optical design, referencing... Figure 1 As shown, the distance from the virtual imaging surface to the device's exit pupil may vary for different colored displayed images. Therefore, there is an urgent need to provide a method for detecting color differences in extended reality devices. Summary of the Invention

[0003] This application aims to provide a color difference detection method, apparatus, device, and storage medium.

[0004] The technical solution of this application is implemented as follows:

[0005] This application provides a color difference detection method, the method comprising:

[0006] The calibration images were obtained by taking pictures of the black and white calibration object with a light field camera with no filter and with different color filters.

[0007] The image to be calibrated is subjected to scale calibration to obtain a color calibration matrix for each color;

[0008] Acquire the image to be detected, captured by a light field camera and displayed on an extended reality device with an image of a color calibration object;

[0009] The color difference of the extended reality device is detected based on the color calibration matrix and the corresponding color image to be detected.

[0010] This application provides a color difference detection device, the device comprising:

[0011] The acquisition module is used to acquire the images of the black and white calibration object obtained by the light field camera when it takes pictures of the object with no filter and with different color filters respectively;

[0012] The processing module is used to perform scale calibration processing on the image to be calibrated to obtain a color calibration matrix for each color;

[0013] The acquisition module is also used to acquire the image to be detected captured by the light field camera and displayed by the extended reality device as an image of a color calibration object;

[0014] The processing module is also used to detect the color difference of the extended reality device based on the color calibration matrix and the corresponding color of the image to be detected.

[0015] This application provides a color difference detection device, the device comprising:

[0016] Memory, used to store color difference detection programs;

[0017] The processor is used to execute the color difference detection program stored in the memory to implement the color difference detection method described above.

[0018] This application provides a storage medium, characterized in that the storage medium stores one or more programs, which can be executed by one or more processors to implement the color difference detection method described above.

[0019] The color difference detection method, apparatus, device, and storage medium provided in this application acquire images to be calibrated obtained by a light field camera when shooting black and white calibration objects with and without filters of different colors; scale calibration is performed on the images to be calibrated to obtain a color calibration matrix for each color; since the light field camera contains optical lenses such as a main lens and microlens array, color difference problems are introduced, so by changing the filters of different colors of the light field camera and shooting black and white calibration objects with and without filters, multiple sets of images to be calibrated are obtained, and scale calibration is performed on the multiple sets of images to be calibrated to obtain a color calibration matrix for each color, thereby completing the color difference correction of the light field camera. In this way, when using a light field camera to detect color difference in extended reality devices, interference problems caused by the color difference of the light field camera itself are avoided. Furthermore, the system acquires a test image captured by a light field camera, showing the color calibration object displayed on the extended reality device. Based on the color calibration matrix and the test image with corresponding colors, the color difference of the extended reality device is detected. The light field camera is fixed by a fixing device to capture the color calibration object image displayed on the extended reality device's screen. After capturing the test image, i.e., the light field image, the depth information of the image displayed on the screen can be obtained from the light field image. This enables rapid detection of whether the virtual imaging surface of the extended reality device is qualified, with higher detection efficiency and a simple and easy-to-implement structure. Attached Figure Description

[0020] Figure 1 A schematic diagram illustrating the inconsistent distances between virtual imaging surfaces of different colors in extended reality devices provided for related technologies;

[0021] Figure 2A schematic flowchart of an optional color difference detection method provided in an embodiment of this application;

[0022] Figure 3 A schematic flowchart of an optional color difference detection method provided in an embodiment of this application;

[0023] Figure 4 A schematic flowchart of an optional color difference detection method provided in an embodiment of this application;

[0024] Figure 5 A schematic diagram illustrating the chromatic aberration calibration process of a light field camera provided in an embodiment of this application;

[0025] Figure 6 A schematic diagram of a black checkerboard pattern with different colored backgrounds provided for embodiments of this application;

[0026] Figure 7 A schematic diagram illustrating the positional relationship between the light field camera and the augmented reality device provided in an embodiment of this application;

[0027] Figure 8 A schematic flowchart of an optional color difference detection method provided in an embodiment of this application;

[0028] Figure 9 This is a schematic diagram of the structure of a color difference detection device provided in an embodiment of this application;

[0029] Figure 10 This is a schematic diagram of the structure of a color difference detection device provided in an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0031] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0032] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0033] Embodiments of this application provide a color difference detection method, applied to a color difference detection device, with reference to... Figure 2 As shown, the method includes the following steps:

[0034] Step 101: Obtain the images to be calibrated obtained by the light field camera when shooting black and white calibration objects with no filter and with different color filters.

[0035] In this embodiment, the filter is an optical device used to select the desired radiation band. A common characteristic of filters is that no filter can make the image of a celestial object brighter, because all filters absorb certain wavelengths, thus making the object darker. Different color filters include, but are not limited to, red, green, and blue filters. It should be noted that the light field camera can also photograph a black and white calibration object without installing a filter to obtain the image to be calibrated.

[0036] In this embodiment, the image to be calibrated is an image with a corresponding color background obtained by a light field camera when photographing a black and white calibration object with no filter and with different color filters. For example, when the light field camera photographs a black and white calibration object without a filter, the resulting image is a black dot array with a white background, i.e., a white calibration object image; when the light field camera photographs a black and white calibration object with a red filter, the resulting image is a black dot array with a red background, i.e., a red calibration object image; when the light field camera photographs a black and white calibration object with a green filter, the resulting image is a black dot array with a green background, i.e., a green calibration object image; and when the light field camera photographs a black and white calibration object with a blue filter, the resulting image is a black dot array with a blue background, i.e., a blue calibration object image.

[0037] In this embodiment, the calibration object is a calibration plate or alignment plate with specific dots or patterns. For example, the calibration object can be a calibration plate with an array of dots, or it can be a calibration plate with a checkerboard pattern. Calibration plates are widely used in machine vision, image measurement, photogrammetry, 3D reconstruction, and other fields. By photographing a plate with a fixed-spacing pattern array using a camera and performing calculations using a calibration algorithm, the geometric model of the camera can be obtained, thus achieving high-precision measurement and reconstruction results. The plate with the fixed-spacing pattern array is the calibration plate. It should be noted that the calibration object includes multiple dots, and the spacing between any two adjacent dots is equal. For example, the spacing between two adjacent dots can be 0.5 millimeters (mm). Here, by establishing a coordinate system with any dot on the calibration object as the origin, the spatial positions of all patterns, i.e., dots, on the calibration object can be determined.

[0038] In this embodiment, the black and white calibration object refers to an array of black dots with a white background.

[0039] In this embodiment, the black and white calibration object is located in the focal plane area of ​​the light field camera. By switching different color filters at the front end of the main lens of the light field camera, and without installing filters, multiple sets of images to be calibrated are obtained when the light field camera takes pictures of the corresponding black and white calibration object.

[0040] In other embodiments of this application, the light field camera may also directly photograph the calibration object with black dot arrays against different colored backgrounds without installing a filter, thereby obtaining multiple sets of images to be calibrated. This allows for direct use of a light field camera without filters to photograph the calibration object against different colored backgrounds, eliminating the need to use and replace filters, thus simplifying operation. Here, the color difference detection device can be the light field camera itself, or other devices with computing and processing capabilities; this application does not impose specific limitations on this.

[0041] Step 102: Perform scale calibration on the image to be calibrated to obtain the color calibration matrix for each color.

[0042] In this embodiment, the color calibration matrix is ​​used to map the pixel coordinates of the light field camera to spatial coordinates.

[0043] In this embodiment, the light field camera includes a main lens, a microlens array, and an image sensor. After scale calibration, the mapping relationship between the image to be calibrated and the spatial distance can be obtained, thus enabling the light field camera to perform three-dimensional spatial distance testing with a single shot.

[0044] In this embodiment, since the spatial positions of all calibration points on the black and white calibration object are known, and each dot in the dot array on each black and white calibration object will form a circle of confusion on the image to be calibrated, the diameter of the circle of confusion is obtained, and then the disparity value and pixel coordinates of the circle of confusion are calculated. According to the light field camera scale calibration model, the relationship between the three-dimensional coordinates in space corresponding to each color and the pixel coordinates and disparity value of the light field camera is fitted, that is, the color calibration matrix of each color is obtained, thereby completing the color difference correction of the light field camera.

[0045] It should be noted that, since light field cameras contain optical lenses such as a main lens and microlens arrays, chromatic aberration is inevitably introduced. To eliminate interference from the chromatic aberration of the light field camera itself when detecting extended reality devices in large-scale scenarios such as VR, automotive, and autonomous driving, chromatic aberration correction is necessary.

[0046] Step 103: Acquire the image to be detected, captured by the light field camera and displayed on the extended reality device with the image of the color calibration object.

[0047] In this embodiment of the application, the color marker refers to a black checkerboard pattern with a background color of the corresponding color. The color marker includes, but is not limited to, red markers, green markers, and blue markers.

[0048] In this embodiment of the application, the color calibration object image refers to an image of a color calibration object displayed on the display of the extended reality device.

[0049] In this embodiment of the application, the image to be detected includes an image of the virtual imaging surface of a color calibration object image displayed by an extended reality device.

[0050] In this application embodiment, Extended Reality (AR) is a collective term for technologies such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). AR technology is a relatively new technology that integrates real-world and virtual-world information. It simulates and overlays virtual information onto physical entities that are difficult to experience in the real world, effectively applying this virtual information within the real world. This process is perceptible to human senses, achieving a sensory experience that transcends reality. After the real environment and virtual objects overlap, they can coexist in the same scene and space. AR technology not only effectively reflects real-world content but also enables the display of virtual information, with these detailed contents complementing and overlapping each other. In visualized AR, the user needs to ensure that the real world and computer graphics overlap on top of a head-mounted display, allowing them to fully perceive the real world surrounding them. AR technology mainly incorporates new technologies and methods such as multimedia, 3D modeling, and scene fusion. The information content provided by AR differs significantly from the information content that humans can perceive.

[0051] In this embodiment of the application, the extended reality device is a device with computing processing capabilities. Extended reality devices include, but are not limited to, AR / VR head-mounted devices, such as AR / VR glasses and AR / VR helmets.

[0052] In this embodiment, after the color difference detection device performs scale calibration processing on the image to be calibrated to obtain the color calibration matrix for each color, the extended reality device displays images of different color calibration objects on its display, and uses a light field camera to capture the virtual imaging surface of the color calibration object images displayed by the extended reality device, thereby obtaining the image to be detected.

[0053] Step 104: Detect the color difference of the extended reality device based on the color calibration matrix and the corresponding color of the image to be detected.

[0054] In this embodiment, after the color difference detection device acquires the image to be detected captured by the light field camera and the extended reality device displays a color calibration object, it detects the color difference of the extended reality device based on the calibration matrix and the image to be detected.

[0055] The color difference detection method provided in this application acquires images of a black and white calibration object taken with a light field camera using both no filter and different color filters. The images are then subjected to scale calibration to obtain a color calibration matrix for each color. Since the light field camera contains optical lenses such as a main lens and microlens arrays, it introduces color difference issues. Therefore, by changing the color filters on the light field camera and taking images of the black and white calibration object without filters, multiple sets of calibration images are obtained. Scale calibration is then performed on each set of calibration images to obtain a color calibration matrix for each color, thus completing the color difference correction for the light field camera. In this way, when using a light field camera to detect color difference in extended reality devices, interference caused by the color difference of the light field camera itself is avoided. Furthermore, the system acquires a test image captured by a light field camera, showing the color calibration object displayed on the extended reality device. Based on the color calibration matrix and the test image with corresponding colors, the color difference of the extended reality device is detected. The light field camera is fixed by a fixing device to capture the color calibration object image displayed on the extended reality device's screen. After capturing the test image, i.e., the light field image, the depth information of the image displayed on the screen can be obtained from the light field image. This enables rapid detection of whether the virtual imaging surface of the extended reality device is qualified, with higher detection efficiency and a simple and easy-to-implement structure.

[0056] Embodiments of this application provide a color difference detection method, applied to a color difference detection device, with reference to... Figure 3 As shown, the method includes the following steps:

[0057] Step 201: Obtain the depth of field range of the light field camera, and determine the maximum and minimum distances between the light field camera and the black and white calibration object based on the depth of field range.

[0058] In this embodiment, the depth of field (DOF) refers to the range of distances in front of and behind the subject measured when a clear image can be obtained at the front edge of the lens or other imager of a light field camera. It should be noted that the aperture, lens, and distance from the focal plane to the subject are important factors affecting the depth of field.

[0059] In this embodiment of the application, the color difference detection device acquires the depth range of the light field camera and determines the maximum distance between the light field camera and the black and white calibration object based on the depth range. Of course, the minimum distance between the light field camera and the black and white calibration object can also be determined based on the depth range.

[0060] Step 202: Starting from the position where the light field camera is at its minimum distance from the black and white calibration object, move the light field camera to each position at its maximum distance from the black and white calibration object along the direction of the line connecting the light field camera and the black and white calibration object, with a moving step size, to obtain the images to be calibrated obtained by the light field camera at different positions when taking pictures of the black and white calibration object with no filter or different color filters.

[0061] In this embodiment, the light field camera can be moved manually, that is, by a staff member moving the light field camera to each position according to the moving step size; of course, the light field camera can also be mounted on a driving device, which is connected to the color difference detection device. The driving device is moved by control commands, thereby realizing the movement of the light field camera. It should be noted that the distance between the position of the moved light field camera and the position of the black and white calibration object is less than or equal to the maximum distance and greater than or equal to the minimum distance.

[0062] In other embodiments of this application, the black and white calibration object can also be moved. The black and white calibration object can be moved manually, i.e., by a worker moving the calibration object to each position according to a step size; alternatively, the black and white calibration object can be mounted on a driving device connected to a color difference detection device, and the movement of the driving device is controlled by control commands, thereby realizing the movement of the black and white calibration object. It should be noted that the distance between the moved position of the black and white calibration object and the position of the light field camera is less than or equal to the maximum distance and greater than or equal to the minimum distance.

[0063] In this embodiment, the color difference detection device acquires the depth-of-field range of the light field camera, and after determining the maximum and minimum distances between the light field camera and the black and white calibration object based on the depth-of-field range, it determines the initial positions of the light field camera and the black and white calibration object when the distance between them is the minimum distance based on the minimum distance. Further, in one case, the color difference detection device moves the light field camera with a step size along the direction of the line connecting the light field camera and the black and white calibration object, starting from the initial position of the light field camera, until it reaches the maximum distance between the light field camera and the black and white calibration object. During the movement of the light field camera, multiple positions of the light field camera are obtained, and when the light field camera is in different positions, the black and white calibration object is photographed with different color filters to obtain the image to be calibrated. It should be noted that the distance D1 between the position of the moved light field camera and the position of the black and white calibration object is greater than or equal to the minimum distance, and less than or equal to the maximum distance. In another scenario, the color difference detection device can also start from the initial position of the black and white calibration object and move the black and white calibration object along the direction of the line connecting the light field camera and the black and white calibration object in a step size until it moves to the maximum distance mentioned above between the black and white calibration object and the light field camera. During the movement of the black and white calibration object, multiple positions of the black and white calibration object are obtained, and the light field camera takes pictures of the black and white calibration object at different positions to obtain the image to be calibrated. It should be emphasized that the distance D2 between the position of the black and white calibration object after movement and the position of the light field camera is greater than or equal to the minimum distance and less than or equal to the maximum distance.

[0064] In other embodiments of this application, the image to be calibrated is obtained through... Figure 4 The steps shown are to be completed.

[0065] Step 221: Obtain the first image to be calibrated when the light field camera takes pictures of the black and white calibration object with different color filters.

[0066] In this embodiment, the different color filters include a red filter, a green filter, and a blue filter.

[0067] In this embodiment of the application, the color difference detection device acquires a first red image to be calibrated obtained by a light field camera when shooting a black and white calibration object with a red filter, a first green image to be calibrated obtained by shooting a black and white calibration object with a green filter, and a first blue image to be calibrated obtained by shooting a black and white calibration object with a blue filter. The first image to be calibrated includes a first red image to be calibrated, a first green image to be calibrated, and a first blue image to be calibrated.

[0068] In other embodiments of this application, the color difference detection device can also determine multiple different distances between the light field camera and the black and white calibration object, and acquire a first red image to be calibrated when the light field camera and the black and white calibration object are at different distances, a first green image to be calibrated when the black and white calibration object is photographed with a red filter, a first green image to be calibrated when the corresponding green calibration object is photographed with a green filter, and a first blue image to be calibrated when the corresponding blue calibration object is photographed with a blue filter, wherein the first image to be calibrated includes a first red image to be calibrated, a first green image to be calibrated, and a first blue image to be calibrated.

[0069] Step 222: Obtain the second image to be calibrated when the light field camera takes a picture of the black and white calibration object without a filter.

[0070] The image to be calibrated includes at least two images: a first image to be calibrated and a second image to be calibrated.

[0071] In this embodiment of the application, the color difference detection device acquires a white image to be calibrated obtained by a light field camera when shooting a black and white calibration object without a filter, wherein the second image to be calibrated includes the white image to be calibrated.

[0072] In other embodiments of this application, the color difference detection device can also determine multiple different distances between the light field camera and the black and white calibration object, and obtain a white image to be calibrated obtained by shooting the black and white calibration object without a filter when the light field camera and the black and white calibration object are at different distances.

[0073] In a feasible application scenario, refer to Figure 5 As shown, Figure 5 This illustrates the color difference calibration process of a light field camera. A black and white calibration object is located in the focal plane region of the light field camera. The position of the light field camera remains constant, but the distance between the camera and the calibration object is changed by moving the position of the calibration object. During the calibration process, by changing the position of the calibration object and using different filters (red, green, and blue) on the light field camera, three sets of first calibration images are obtained from the black and white calibration object. This allows for the creation of red, green, and blue calibration matrices, as well as a set of white calibration images from the black and white calibration object taken without filters. Furthermore, the color difference detection device performs color difference correction on the light field camera based on these three sets of first calibration images and the white calibration image.

[0074] Step 203: Calculate the devigation matrix and the sub-pixel center coordinate matrix of the microlens of the light field camera based on the image to be calibrated.

[0075] In this embodiment, the image to be calibrated is processed using a filter to remove noise from the light field white image, and non-maximum suppression is performed on the filtered image to be calibrated. Then, the local maximum value is taken based on the processed image, which is exactly the integer center of the microlens of the light field camera. Using the integer center of the microlens as the initial iteration value, the microlens arrangement grid is iteratively optimized to finally obtain the angle and spacing of the microlens arrangement and obtain the sub-pixel level microlens center.

[0076] Step 204: Based on the image to be calibrated, establish a color calibration matrix for each color from three-dimensional coordinates to parallax, and complete the scale calibration of the light field camera.

[0077] In this embodiment, a fixed black and white calibration object is used, while a light field camera is moved to different positions. At each position, the light field camera captures images of the black and white calibration object with and without filters, and with different color filters. Since the spatial positions of all calibration points on the black and white calibration object are known, and each dot calibration point on the black and white calibration object forms a circle of confusion on the image to be calibrated, the diameter of the circle of confusion is obtained, and then the disparity value and pixel coordinates of the circle of confusion are calculated. Based on the light field camera scale calibration model, the relationship between the three-dimensional coordinates in space corresponding to each color and the pixel coordinates and disparity value of the light field camera is fitted, thus obtaining the color calibration matrix for each color, thereby completing the color difference correction of the light field camera.

[0078] In one feasible application scenario, the images to be calibrated include red, green, and blue calibration object images. For calibration object images at the same distance, if the background color of the image to be calibrated corresponds to the color of the calibration matrix used, the distance between the calibration object and the lens (main lens) of the light field camera at that color can be calculated using the image to be calibrated. Ultimately, the distances between the calibration object and the camera lens corresponding to different colors will remain consistent. If the background color of the image to be calibrated does not correspond to the calibration matrix used, the distances between the calibration object and the camera lens calculated using the image to be calibrated for different background colors will be inconsistent.

[0079] In a feasible application scenario, referring to Table 1, which compares the measured distance and actual distance of a light field camera before and after chromatic aberration correction, the measured distance refers to the distance obtained from the image to be calibrated using the calibration matrix to calculate the calibration position. The actual distance refers to the distance between the actual black and white calibration object (such as the calibration plate) and the camera lens during the scale calibration process of the light field camera. All units are millimeters (mm). The table shows that before chromatic aberration correction, the measured distance obtained using the calibration matrix without filters varies depending on the color filter used during shooting, indicating that the light field camera has a chromatic aberration problem, with a maximum chromatic aberration of approximately 30 millimeters (mm). After chromatic aberration correction, the measured distances obtained by calculating the image to be calibrated using the corresponding calibration matrix for the corresponding color filter are very close, and the maximum chromatic aberration has been reduced to 3 mm. This demonstrates that the light field camera has achieved excellent chromatic aberration correction.

[0080]

[0081] Table 1

[0082] It should be noted that, regarding Table 1, the data in the row representing the actual camera distance refers to the distances between the calibration plate and the camera lens at the three positions: 815.5mm, 1135.5mm, and 1455.5mm. The data in the row representing the camera measurement distance obtained from the filterless calibration matrix refers to the camera measurement distances calculated using the filterless calibration matrix for images to be calibrated with no filter, red filter, green filter, and blue filter, respectively. The data in the row representing the camera measurement distance obtained from the red calibration matrix refers to the camera measurement distance calculated using the red filter calibration matrix for images to be calibrated with a red filter. The data in the row representing the camera measurement distance obtained from the green calibration matrix refers to the camera measurement distance calculated using the green filter calibration matrix for images to be calibrated with a green filter. The data in the row representing the camera measurement distance obtained from the blue calibration matrix refers to the camera measurement distance calculated using the blue filter calibration matrix for images to be calibrated with a blue filter. Furthermore, the three sets of data are derived from near depth of field, focal plane, and far depth of field, respectively, ensuring representativeness, eliminating randomness, and increasing the reliability of the experiment.

[0083] Step 205: With the extended reality device displaying a color calibration object image, capture the virtual imaging surface of the extended reality device using a light field camera to obtain the image to be detected.

[0084] In this embodiment, when color calibration objects are displayed on the extended reality device's screen, the color difference detection device captures the virtual imaging surface of the extended reality device using a light field camera to obtain the image to be detected. Here, to simulate virtual imaging surfaces with different background colors, such as... Figure 6 As shown, Figure 6 In this context, A represents a black checkerboard pattern with a red background. Figure 6 In this context, B represents a black checkerboard pattern against a green background. Figure 6 In the example, C represents a black checkerboard pattern with a blue background; furthermore, black checkerboard patterns with red, green, and blue backgrounds are imported into the extended reality device as display images.

[0085] In this embodiment of the application, when the extended reality device displays an image of a color calibration object, the light field camera captures the virtual imaging surface of the extended reality device, such as... Figure 7 As shown, Figure 7 This is a schematic diagram showing the relative positions of the light field camera and the augmented reality device. Keeping the light field camera and the augmented reality device relatively stationary, the image displayed on the augmented reality device is switched between black checkerboard patterns with red, green, and blue backgrounds, serving as color calibration images.

[0086] Step 206: Using the color calibration matrix of each color, calculate the measurement distance from the virtual imaging surface in the image to be detected of the corresponding color to the entrance pupil of the main lens of the light field camera.

[0087] In this embodiment of the application, when the light field camera captures the virtual imaging surface of the extended reality device, the entrance pupil of the main lens of the light field camera coincides with the exit pupil of the extended reality device. Therefore, the distance from the virtual imaging surface of the extended reality device to the entrance pupil of the main lens of the light field camera is the same as the distance from the virtual imaging surface of the extended reality device to the exit pupil of the extended reality device.

[0088] In this embodiment, the measurement distance from the virtual imaging surface in the image to be detected to the entrance pupil of the main lens of the light field camera is calculated using the color calibration matrix for each color. This includes: using the red calibration matrix to calculate the red measurement distance from the virtual imaging surface in the image to be detected to the entrance pupil of the main lens of the light field camera; using the green calibration matrix to calculate the green measurement distance from the virtual imaging surface in the image to be detected to the entrance pupil of the main lens of the light field camera; and using the blue calibration matrix to calculate the blue measurement distance from the virtual imaging surface in the image to be detected to the entrance pupil of the main lens of the light field camera. The measurement distance includes the red measurement distance, the green measurement distance, and the blue measurement distance.

[0089] Step 207: Calculate the difference between the maximum and minimum measurement distances among the multiple measurement distances.

[0090] In this embodiment of the application, the maximum and minimum measurement distances are selected from the red, green, and blue measurement distances, and the difference between the maximum and minimum measurement distances is calculated to obtain the measurement distance difference.

[0091] Step 208: Detect the color difference of the extended reality device based on the measured distance difference and the distance difference threshold.

[0092] In this embodiment of the application, the distance difference threshold can be a pre-set distance range with negligible color difference. For example, the distance difference threshold is 50mm. At this time, the maximum measurement distance and the minimum measurement distance are determined from multiple measurement distances, and the measurement distance difference obtained by subtracting the minimum measurement distance from the maximum measurement distance is obtained. Further, it is determined whether the measurement distance difference is greater than the distance difference threshold. This application does not impose specific restrictions on this.

[0093] In this embodiment, if the measured distance difference is less than or equal to a distance difference threshold, it is determined that the extended reality device does not have a significant color difference. That is, the maximum and minimum measured distances are selected from the red, green, and blue measured distances, and the difference between the maximum and minimum measured distances is calculated. If the measured distance difference is less than or equal to the distance difference threshold, the color difference detection device can determine that the extended reality device does not have a significant color difference. Furthermore, based on the detection result, the color difference detection device can generate and output a prompt message to indicate whether the extended reality device has a significant color difference.

[0094] If the difference in measured distances exceeds a distance difference threshold, a significant color difference is detected in the extended reality device. Specifically, the maximum and minimum measured distances are selected from the red, green, and blue measured distances. The difference between these two distances is calculated. If this difference exceeds the distance difference threshold, the color difference detection device can confirm that the extended reality device has a significant color difference issue. Furthermore, based on the detection result, the color difference detection device can generate and output a prompt message to indicate whether the extended reality device has a significant color difference problem.

[0095] In a feasible application scenario, referring to Table 2, which shows the measured distances obtained after a light field camera captures the virtual imaging surface of a VR headset, it can be seen from Table 2 that if the light field camera does not perform color difference correction (the measured distance obtained without a filter calibration matrix indicates that the light field camera has not performed color difference correction), the maximum difference between the maximum measured distance (e.g., 1705.8) and the minimum measured distance (e.g., 1094.4) is 611.4 mm. If the light field camera performs color difference correction (the measured distance obtained using the corresponding color calibration matrix indicates that the light field camera has performed color difference correction), the maximum difference between the maximum measured distance (e.g., 1723.0) and the minimum measured distance (e.g., 1112.3) is 610.7 mm. Moreover, the measured distances of the red and green background virtual imaging surfaces are relatively close, while the measured distance of the blue background has the largest difference, indicating that this VR headset has a significant color difference problem.

[0096]

[0097] Table 2

[0098] It should be noted that, for Table 2, the data in the row containing the measurement distance obtained from the filterless calibration matrix refers to the measurement distances obtained by calculating the VR glasses' white, red, green, and blue background checkerboard virtual imaging surfaces using the "filterless calibration matrix" respectively; the data in the row containing the measurement distance obtained from the red calibration matrix refers to the measurement distances obtained by calculating the VR glasses' red background checkerboard virtual imaging surfaces using the "red filter calibration matrix"; the data in the row containing the measurement distance obtained from the green calibration matrix refers to the measurement distances obtained by calculating the VR glasses' green background checkerboard virtual imaging surfaces using the "green filter calibration matrix"; and the data in the row containing the measurement distance obtained from the blue calibration matrix refers to the measurement distances obtained by calculating the VR glasses' blue background checkerboard virtual imaging surfaces using the "blue filter calibration matrix".

[0099] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0100] To further understand the application scenarios of this application, embodiments of this application provide a color difference detection method, applied to a color difference detection device, with reference to... Figure 8 As shown, the method includes the following steps:

[0101] Step 301: With red, green, and blue filters installed on the lens of the light field camera, and with and without filters installed, based on the depth of field range of the light field camera, acquire four sets of images to be calibrated, formed by the black and white dot array captured by the light field camera at each position.

[0102] Here, an optical field camera without a filter captures a black-and-white dot array at each position, resulting in a white image to be calibrated; an optical field camera with a red filter captures a black-and-white dot array at each position, resulting in a red image to be calibrated; an optical field camera with a green filter captures a black-and-white dot array at each position, resulting in a green image to be calibrated; and an optical field camera with a blue filter captures a black-and-white dot array at each position, resulting in a blue image to be calibrated.

[0103] Step 302: Perform scale calibration calculations on each group of images to be calibrated to obtain the red calibration matrix, green calibration matrix, and blue calibration matrix, thereby completing the chromatic aberration correction of the light field camera.

[0104] Here, at each position during light field camera calibration, three sets of images to be calibrated are captured by changing the red, green, and blue filters, resulting in three sets of calibration files (corresponding to the calibration matrix mentioned above). This enables chromatic aberration correction for the light field camera. For calibration boards at the same distance, when the background color of the calibration board corresponds to the calibration file used, the distance between that color calibration board and the camera lens can be calculated. Ultimately, the distances between different color calibration boards and the camera lens will remain consistent. However, if the background color of the calibration board does not correspond to the calibration file used, the calculated distances between different color calibration boards and the camera lens will be inconsistent.

[0105] Step 303: Switch the displayed image of the extended reality device to a black checkerboard pattern with a red background, a black checkerboard pattern with a green background, and a black checkerboard pattern with a blue background, respectively.

[0106] Step 304: Use a light field camera to capture the virtual imaging surface of the extended reality device.

[0107] Step 305: Calculate the distance from the virtual imaging surface to the lens entrance pupil using the calibration matrix of the corresponding color, and obtain three sets of measurement distances.

[0108] Step 306: Are the three measured distances consistent?

[0109] In this embodiment of the application, if the three measured distances are consistent, step 307 is executed; if the three measured distances are inconsistent, step 308 is executed.

[0110] Step 307: Confirm that there is no obvious color difference in the extended reality device.

[0111] Step 308 determines that the extended reality device has a significant color difference.

[0112] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0113] Embodiments of this application provide a color difference detection device, which can be used to implement... Figures 2 to 4 A corresponding embodiment provides a color difference detection method, referring to... Figure 9 As shown, the color difference detection device 9 includes:

[0114] The acquisition module 401 is used to acquire the images to be calibrated obtained by the light field camera when shooting black and white calibration objects with no filter and with different color filters respectively;

[0115] The processing module 402 is used to perform scale calibration processing on the image to be calibrated to obtain the color calibration matrix for each color.

[0116] The acquisition module 401 is also used to acquire the image to be detected captured by the light field camera and displayed on the extended reality device as an image of a color calibration object;

[0117] The processing module 402 is also used to detect the color difference of the extended reality device based on the color calibration matrix and the corresponding color of the image to be detected.

[0118] In some embodiments of this application, the acquisition module 401 is further configured to acquire the depth range of the light field camera and determine the maximum and minimum distances between the light field camera and the black and white calibration object based on the depth range; the processing module 402 is further configured to take the position where the light field camera is at its minimum distance from the black and white calibration object as the starting position, and move the light field camera to each position at the maximum distance from the black and white calibration object along the direction of the line connecting the light field camera and the black and white calibration object with a moving step size, so as to obtain the images to be calibrated obtained by the light field camera at different positions when taking pictures of the black and white calibration object with no filter or different color filters respectively.

[0119] In some embodiments of this application, the acquisition module 401 is further configured to acquire a first image to be calibrated obtained by the light field camera when shooting a black and white calibration object with different color filters; and to acquire a second image to be calibrated obtained by the light field camera when shooting a black and white calibration object without a filter, wherein the image to be calibrated includes at least two first images to be calibrated and a second image to be calibrated.

[0120] In some embodiments of this application, the processing module 402 is further configured to calculate the devigation matrix and the microlens subpixel-level center coordinate matrix of the light field camera based on the image to be calibrated; and to establish a color calibration matrix for each color from the three-dimensional coordinates to the parallax based on the image to be calibrated, thereby completing the scale calibration of the light field camera.

[0121] In some embodiments of this application, the acquisition module 401 is further configured to capture the virtual imaging surface of the extended reality device using a light field camera to obtain the image to be detected when the extended reality device displays an image of a color calibration object.

[0122] In some embodiments of this application, the processing module 402 is further configured to use the color calibration matrix of each color to calculate the measurement distance from the virtual imaging surface in the image to be detected of the corresponding color to the entrance pupil of the main lens of the light field camera; calculate the maximum measurement distance minus the minimum measurement distance among multiple measurement distances to obtain the measurement distance difference; and detect the color difference of the extended reality device based on the measurement distance difference and the distance difference threshold.

[0123] In some embodiments of this application, the processing module 402 is further configured to determine that the extended reality device has no obvious color difference if the measured distance difference is less than or equal to the distance difference threshold; and to determine that the extended reality device has an obvious color difference if the measured distance difference is greater than the distance difference threshold.

[0124] Based on the foregoing embodiments, this color difference detection device can be used for Figures 2 to 4 A corresponding embodiment provides a color difference detection method, referring to... Figure 10 As shown, the color difference detection device 10 ( Figure 10 Color difference detection equipment 10 and Figure 9 The color difference detection device 9 in the middle includes: a processor 501 and a memory 502, wherein:

[0125] Memory 502 is used to store the color difference detection program;

[0126] Processor 501 is used to execute the color difference detection program stored in memory 502, and to perform the following steps:

[0127] The calibration images were obtained by taking pictures of the black and white calibration object with a light field camera with no filter and with different color filters.

[0128] The image to be calibrated is scaled to obtain the color calibration matrix for each color.

[0129] Acquire the image to be detected, captured by a light field camera and displayed on an extended reality device with an image of a color calibration object;

[0130] The color difference of the extended reality device is detected based on the color calibration matrix and the corresponding color of the image to be detected.

[0131] In other embodiments of this application, processor 501 is used to execute a color difference detection program stored in memory 502 to perform the following steps:

[0132] Obtain the depth of field range of the light field camera, and determine the maximum and minimum distances between the light field camera and the black and white calibration object based on the depth of field range. Taking the position where the light field camera is at its minimum distance from the black and white calibration object as the starting position, move the light field camera to each position at the maximum distance from the black and white calibration object along the direction of the line connecting the light field camera and the black and white calibration object with a moving step size, to obtain the calibration images obtained by the light field camera at different positions when taking pictures of the black and white calibration object with no filter or different color filters.

[0133] In other embodiments of this application, processor 501 is used to execute a color difference detection program stored in memory 502 to perform the following steps:

[0134] Acquire a first image to be calibrated when the light field camera takes pictures of the black and white calibration object with different color filters; acquire a second image to be calibrated when the light field camera takes pictures of the black and white calibration object without a filter, wherein the images to be calibrated include at least two first images to be calibrated and two second images to be calibrated.

[0135] In other embodiments of this application, processor 501 is used to execute a color difference detection program stored in memory 502 to perform the following steps:

[0136] The devigation matrix and the sub-pixel center coordinate matrix of the microlens of the light field camera are calculated based on the image to be calibrated. Based on the image to be calibrated, a color calibration matrix for each color from the three-dimensional coordinates to the parallax is established to complete the scale calibration of the light field camera.

[0137] In other embodiments of this application, processor 501 is used to execute a color difference detection program stored in memory 502 to perform the following steps:

[0138] When an extended reality device displays an image of a color calibration object, a light field camera is used to capture the virtual imaging surface of the extended reality device to obtain the image to be detected.

[0139] In other embodiments of this application, processor 501 is used to execute a color difference detection program stored in memory 502 to perform the following steps:

[0140] Using the color calibration matrix for each color, the measurement distance from the virtual imaging surface to the entrance pupil of the main lens of the light field camera in the image to be detected for the corresponding color is calculated; the maximum measurement distance is subtracted from the minimum measurement distance among multiple measurement distances to obtain the measurement distance difference; based on the measurement distance difference and the distance difference threshold, the color difference of the extended reality device is detected.

[0141] In other embodiments of this application, processor 501 is used to execute a color difference detection program stored in memory 502 to perform the following steps:

[0142] If the measured distance difference is less than or equal to the distance difference threshold, it is determined that the extended reality device has no obvious color difference; if the measured distance difference is greater than the distance difference threshold, it is determined that the extended reality device has an obvious color difference.

[0143] Embodiments of this application provide a computer storage medium storing one or more programs, which can be executed by one or more processors to achieve... Figures 2 to 4 A color difference detection method is provided in the corresponding embodiment.

[0144] It should be noted that the aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; it can also be various terminals that include one or any combination of the above-mentioned memory, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0145] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0146] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0147] Furthermore, in the various embodiments of this application, all functional units can be integrated into one processing module, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0149] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0150] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0151] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A color difference detection method characterized by, The method comprises: obtaining a to-be-calibrated image obtained by a light field camera when shooting a black-and-white calibration object with no filter and different color filters respectively; performing scale calibration processing on the to-be-calibrated image to obtain a color calibration matrix of each color; if an extended display device displays a color calibration object image, obtaining a virtual imaging surface of the extended reality device shot by the light field camera to obtain a to-be-detected image; detecting a chromatic aberration of the extended reality device based on the color calibration matrix and the to-be-detected image of the corresponding color; wherein the detecting the chromatic aberration of the extended reality device based on the color calibration matrix and the to-be-detected image of the corresponding color comprises: calculating a measurement distance from the virtual imaging surface to a main lens entrance pupil of the light field camera in the to-be-detected image of the corresponding color by using the color calibration matrix of each color; calculating a maximum measurement distance minus a minimum measurement distance in a plurality of measurement distances to obtain a measurement distance difference value; detecting the chromatic aberration of the extended reality device based on the measurement distance difference value and a distance difference value threshold.

2. The color difference detection method according to claim 1, characterized by, The obtaining of the to-be-calibrated image obtained by the light field camera when shooting the black-and-white calibration object with no filter and different color filters respectively comprises: obtaining a depth of field range of the light field camera, and determining a maximum distance and a minimum distance between the light field camera and the black-and-white calibration object based on the depth of field range; taking a position where the light field camera is at the minimum distance from the black-and-white calibration object as a starting position, moving the light field camera to each position at the maximum distance from the black-and-white calibration object along a direction of a line connecting the light field camera and the black-and-white calibration object with a moving step, and obtaining the to-be-calibrated image obtained by the light field camera when shooting the black-and-white calibration object with no filter or the different color filters respectively at different positions.

3. The color difference detection method according to claim 1 or 2, characterized by, The obtaining of the to-be-calibrated image obtained by the light field camera when shooting the black-and-white calibration object with no filter and different color filters respectively comprises: obtaining a first to-be-calibrated image obtained by the light field camera when shooting the black-and-white calibration object with the different color filters respectively; obtaining a second to-be-calibrated image obtained by the light field camera when shooting the black-and-white calibration object with no filter, the to-be-calibrated image comprising at least two first to-be-calibrated images and the second to-be-calibrated image.

4. The color difference detection method according to claim 1, characterized by, The performing of the scale calibration processing on the to-be-calibrated image to obtain the color calibration matrix of each color comprises: calculating a de-vignetting matrix and a micro-lens sub-pixel level center coordinate matrix of the light field camera according to the to-be-calibrated image; establishing the color calibration matrix of each color from three-dimensional coordinates to parallax based on the to-be-calibrated image, and completing scale calibration of the light field camera.

5. The color difference detection method according to claim 1, characterized by, The detecting of the chromatic aberration of the extended reality device based on the measurement distance difference value and the distance difference value threshold comprises: if the measurement distance difference value is less than or equal to the distance difference value threshold, determining that no obvious chromatic aberration of the extended reality device is detected; if the measurement distance difference value is greater than the distance difference value threshold, determining that obvious chromatic aberration of the extended reality device exists.

6. A color difference detection device characterized by comprising: The device comprises: An acquisition module is configured to acquire to-be-calibrated images obtained by a light field camera when the light field camera captures a black-and-white calibration object with no filter and different color filters respectively; A processing module is configured to perform scale calibration processing on the to-be-calibrated images to obtain a color calibration matrix of each color; The acquisition module is further configured to acquire a to-be-detected image obtained by the light field camera capturing a virtual imaging surface of the extended reality device if the extended display device displays an image of a colored calibration object; The processing module is further configured to detect a color difference of the extended reality device based on the color calibration matrix and the to-be-detected image of the corresponding color; The processing module is further configured to calculate a measurement distance from the virtual imaging surface to a main lens entrance pupil of the light field camera in the to-be-detected image of the corresponding color by using the color calibration matrix of each color; calculate a measurement distance difference by subtracting a minimum measurement distance from a maximum measurement distance in a plurality of measurement distances; and detect the color difference of the extended reality device based on the measurement distance difference and a distance difference threshold.

7. A color difference detecting apparatus characterized by comprising: The device comprises: a memory configured to store a color difference detection program; a processor configured to execute the color difference detection program stored in the memory to implement the color difference detection method in any one of claims 1 to 5.

8. A storage medium, characterized by The storage medium stores one or more programs, and the one or more programs are executable by one or more processors to implement the color difference detection method in any one of claims 1 to 5. The storage medium stores one or more programs, and the one or more programs are executable by one or more processors to implement the color difference detection method in any one of claims 1 to 5.

Images